Paper Title: Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-Bénard convection.



Article Title]Multi-agent reinforcement learning for the control of three-dimensional Rayleigh–Bénard convection

[1]Joel Vasanthjvasanth@kth.se,

2]Jean Rabault


1]Francisco Alcántara-Ávila


3]Mikael Mortensen

[1]Ricardo Vinuesarvinuesa@mech.kth.se

*[1]FLOW, Engineering Mechanics, KTH Royal Institute of Technology, Stockholm,  Sweden

[2]Independent Researcher, Oslo, Norway

[3]Department of Mathematics, University of Oslo, Oslo, Norway


Deep reinforcement learning (DRL) has found application in numerous use-cases pertaining to flow control. Multi-agent RL (MARL), a variant of DRL, has shown to be more effective than single-agent RL in controlling flows exhibiting locality and translational invariance. We present, for the first time, an implementation of MARL-based control of three-dimensional Rayleigh–Bénard convection (RBC). Control is executed by modifying the temperature distribution along the bottom wall divided into multiple control segments, each of which acts as an independent agent. Two regimes of RBC are considered at Rayleigh numbers Ra=500 and 750. Evaluation of the learned control policy reveals a reduction in convection intensity by 23.5% and 8.7% at Ra=500 and 750, respectively. The MARL controller converts irregularly shaped convective patterns to regular straight rolls with lower convection that resemble flow in a relatively more stable regime. We draw comparisons with proportional control at both Ra and show that MARL is able to outperform the proportional controller. The learned control strategy is complex, featuring different non-linear segment-wise actuator delays and actuation magnitudes. We also perform successful evaluations on a larger domain than used for training, demonstrating that the invariant property of MARL allows direct transfer of the learnt policy.




[
    *
    August 5, 2024
==================





§ INTRODUCTION

Reinforcement learning (RL) is a data-driven approach to the problem of goal-directed sequential decision making. The RL framework involves an `agent', which is its decision-making component, that learns a mapping from states to actions, called the `policy', via a series of trial-and-error interactions with the environment. A reward signal, which is a scalar measure of performance of the environment in relation to the pre-defined goal, is fed back to the agent and is used in the optimization. The state S_t, action A_t and reward R_t form the three fundamental channels of communication between the agent and the environment. Each state is assigned a value V(s) which is the expected total reward that can be obtained by that state in the long run, i.e., V(s)=[ Σ_k=0^∞γ^k R_t+k+1 | s ] <cit.>. The goal of the RL problem, then, is to maximise expected cumulative rewards or `returns'. In Deep RL (DRL), function approximators such as neural networks are used to parameterise either the policy, the value function, or both, which are updated during training via interaction with the environment in order to maximise the returns. Various solution methods have been developed to perform this optimisation, namely: (i) value-based methods such as Q-learning method <cit.>, where the value function is approximated and actions are chosen based on the approximated value function, (ii) policy-based methods <cit.>, such as the policy gradient method, where the optimal policy is learned directly, and (iii) actor-critic methods such as the proximal policy optimisation (PPO) method <cit.>, which leverages the advantages of both methods. During training, interaction between the agent and the environment occurs on an episode basis, where an `episode' is a single trajectory of the environment for a given fixed duration. Training is performed through trial-and-error that involves a stochastic exploration of the environment's states, i.e., selecting actions other than those dictated by the latest updated policy, which can improve the estimation of the state values. After training, the agent is evaluated on the environment in a deterministic manner according to the optimal policy, without exploration. 

DRL is well adapted to perform control of complex, high-dimensional, nonlinear systems. A successful application that contributed to its recent popularity is in gaming, where neural network-based agents are trained to play and win computer games such as Atari <cit.>, Go <cit.> and Starcraft <cit.>. DRL has also been applied to industrial control problems <cit.>, biomechanical optimization problems <cit.>, and recently, a number of classical active flow control (AFC) problems <cit.>. Compared to traditional control methods that are used in AFC, such as state-space methods and adjoint methods <cit.>, DRL does not require information about the governing equations or insight into the full state of the system. Even though there are a number of other techniques that aim at performing similar tasks as DRL <cit.>, DRL has been found to be particularly flexible and effective in the recent years. While the trial-and-error learning method used in DRL can make it more data-hungry and computationally expensive to train, once trained, predicting the next action from the learned approximation to the optimal policy is computationally cheap. This, in conjunction with the fact that DRL can be adapted to large state spaces, makes it a viable option for real-world control applications, such as AFC. 

As a consequence, DRL for AFC has been the focus of much work over the last few years. One of the first uses of DRL in AFC was for the stabilisation of the vortex street in the flow past a two-dimensional (2D) cylinder at a low Reynolds number (Re) of 100 <cit.>. Here, the Reynolds number is given as Re=ŨH̃/ν̃, where the symbols Ũ, H̃ and ν̃ are the mean flow velocity, characteristic length scale of the system and kinematic viscosity respectively, and tilde ·̃ represents dimensional quantities. With two synthetic jets as the actuators, the drag was reduced by 8% and the learned control strategy was found to reduce pressure drop across the cylinder by increasing the area of the separated region. Following this study, DRL was applied to different configurations of the 2D cylinder. In <cit.>, four synthetic jets were used as DRL controllers for a wider Re range, Re=100-400, and a maximum drag reduction of ≈ 38% was achieved. Drag reduction was observed for unseen values of Re as well, showing the robustness of the DRL control process. The work by <cit.> extends these studies to weakly turbulent conditions at Re=1000, showing a reduction in drag by 30% along with a reduction of turbulent fluctuations. <cit.> shows that a transition in the actuation strategy is obtained when the Reynolds number is further increased and turbulent stresses play a dominating role. <cit.> and <cit.> use a pair of smaller downstream rotating cylinders as actuators with a combined objective of drag reduction over the main cylinder and also to reduction of power loss due to cylinder rotation. More recently, an application to three-dimensional (3D) cylinders was studied in the Re range of 100-400 <cit.>, where again, DRL successfully showed a reduction of drag of ≈ 10% for Re=400. A number of studies have also considered questions related to the optimization of the full DRL control setup, including probe placement <cit.> and the use of time history from the state probes <cit.>. Moreover, applications related to vortex induced vibration <cit.>, or hydrodynamic stealth <cit.>, have also attracted interest. Aside from the cylinder, other applications include the control of chaotic flows such as the flow arising from the Kuramoto–Sivashinsky (KS) equation <cit.>, drag reduction in 3D channel flows <cit.>, suppression of the instability of a one-dimensional (1D) falling liquid film <cit.>, suppression of 2D Rayleigh–Bénard convection (RBC) <cit.> and other practical problems <cit.>. We refer the interested reader to some comprehensive reviews of DRL applied to flow control <cit.> for more detail. This rapid development has been made possible largely thanks to both the availability of open source DRL frameworks that make it easy to deploy advanced DRL algorithms <cit.>, and open source computational fluid dynamics (CFD) tools <cit.>. These are being increasingly merged into turn-key frameworks that make it easy to start applying DRL to CFD simulations <cit.>. 

In relation to the works described above, several associated challenges have been solved over the last few years. For example, <cit.> have leveraged a multi-environment DRL framework where several numerical simulations run in parallel, each providing its own set of S_t, A_t and R_t to a common agent, therefore scaling up the number of experiences. With their approach, they showed that speedups in training are obtained proportional to the number of simultaneous environments used. The study by <cit.> introduced a multi-agent RL (MARL) framework for the control of flow systems with distributed inputs and outputs. MARL was since then repeatedly proved to be effective in problems where the dynamics of the system are translationally invariant in space and multiple actuators are required to control the system, i.e., the control dimensionality is large. With a single-agent RL (SARL) framework, this large dimensionality would result in the curse of dimensionality on the action dimension making learning impractical. In the MARL framework, each agent is defined by separate streams of states, actions and rewards, each unique to a single agent. Moreover, critically, all agents share a common parameterisation, i.e., the same neural network weights. Each agent observes a small subdomain of the full domain, which we define as a `pseudo-environment' (Sec. <ref>), and is thus responsible for only a subset of the total number of control actions, thereby overcoming the curse of dimensionality. The use of a common agent parameterisation is justified by the fact that the governing equations (and thus the qualitative features of the flow dynamics) are invariant over all pseudo-environments, and experience can be shared among MARL agents. Following <cit.>, recent works that employ MARL and have demonstrated its effectiveness are <cit.> for the control of 3D turbulent channels, <cit.> to reduce drag over a 3D cylinder, <cit.> for turbulence closure modelling and <cit.> for the control of 2D RBC, to name a few. 

The present work is an extension to 3D of the previous application of MARL to RBC in 2D presented by <cit.>. The RBC phenomenon is a fundamental problem widely studied academically in the field of thermally driven flow instabilities <cit.>, which is also found in a range of natural phenomena and industrial applications. In our configuration, the domain consists of a fluid layer of height H̃ bounded by two rigid walls, where the temperature at the bottom wall T̃_H is maintained at a constant value higher than the top wall temperature T̃_C. This forms an adverse temperature gradient ΔT̃/H̃=( T̃_H-T̃_C)/H̃, which at a certain large enough value, destabilises the system. The instability occurs due to competing driving and damping effects, the driving effect being the vertical adverse thermal gradient and the damping effect being viscosity <cit.>. The stability is governed by a non-dimensional number, the Rayleigh number , which is the ratio of the time scales of heat transport by conduction and by convection. The Rayleigh number is expressed as =g̃α̃_̃T̃ΔT̃H̃^3/(ν̃κ̃), where g̃ stands for the acceleration due to gravity, α̃_̃T̃ is the thermal expansion coefficient and κ̃ is the thermal diffusivity. At a certain critical =_c, instability occurs via a supercritical bifurcation from the no-motion purely conductive state to the convective state <cit.>. At large >_c, the heat transfer from the bottom wall to the top wall is dominated by convection. Another useful dimensionless number is the Nusselt number Ν, which is defined as the ratio of the combined convective and conductive heat flux to the heat flux due to conduction alone. The Nusselt number is given as Ν = 1+√() ⟨ vT ⟩_x,y,z, where  is the Prandtl number (Pr=ν̃/κ̃), v and T are non-dimensional vertical velocity and temperature and ⟨·⟩_x,y,z represents averaging over the entire domain. The manner of non-dimensionalisation is detailed in Sec. <ref>. 

Several approaches have been used to suppress the magnitude of convection in RBC systems. Passive flow control techniques <cit.> have shown only minor improvements in delaying the onset of convection (i.e., increasing the Ra_c). Active flow control techniques have proven to be more successful. Several AFC techniques involve modulation of the bottom wall temperature distribution <cit.>. 

The first application of DRL to RBC was presented by <cit.> in a 2D domain using the single-agent RL framework. They studied the control of RBC in a square domain confined by no-slip walls with the lateral walls being adiabatic, top wall being isothermal, and temperature control modulation imposed on the bottom wall. The _c at which convection occurs was reduced as the DRL controller converted a single-cell configuration in the uncontrolled system to a vertically stacked double cell configuration, reducing the effective . In a following study by <cit.>, the MARL framework was used on a domain with periodic boundary conditions on the lateral walls, featuring two convection cells in the uncontrolled state. Given that MARL is advantageous for problems with distributed inputs and outputs for which invariants are observed, a superior performance compared to SARL was demonstrated, in terms of faster learning. MARL achieved a 22.7% reduction in Ν, employing a control strategy that coalesces the two convection cells into a single-cell state, thus reducing the convection intensity. By extension to <cit.> and <cit.>, in the present study, the RBC system is three-dimensional. Therefore, the control dimensionality (number of actuators) and state space dimensionality is even larger than in 2D, and the dynamics are more complex than in 2D. This further motivates the need for the MARL framework for the execution of control in 3D. As far as the authors' knowledge of the past literature goes, the present study is the first application of MARL to 3D RBC. 

In particular, the main contributions of the present work are the following:

    
  * We introduce the MARL framework for 3D RBC. In 3D, the control dimensionality is multiplicatively larger, therefore, the use of multiple local agents corresponding to a MARL setup is much preferred to the SARL setup. Two cases of Rayleigh numbers,  and , are tested.
    
  * We show that MARL is successful in reducing convection at both  regimes. The MARL controller drives the system from an uncontrolled state with irregular unsteady convection rolls, to a controlled state with regularly arranged steady convection rolls. The regularly arranged convection rolls are similar to the shape of convection structures found at a lower  than  and . Given that the  is representative of the regime of instability <cit.>, with flows at lower  being relatively less unstable than flows at higher , we make it evident that the MARL-controlled states at  and  resemble flow from a less unstable regime, i.e., a regime at lower .
    
  * We demonstrate how MARL can exploit translational invariances to obtain transferable strategies, by testing a pre-trained agent trained at  on a domain of larger size. We show that MARL reduces convection on the larger-size domain as well, without any additional tuning or modification of the agent.
    
  * We apply classical proportional control and compare the performance of the MARL controller against the proportional controller. The MARL controller is shown to provide a larger reduction in Ν than the proportional controller in both  cases studied.


The rest of the paper is organised as follows. Sec. <ref> provides a detailed description and formulation of the RBC problem (Sec. <ref>), the numerical methods used for the CFD (Sec. <ref>), and a description of the DRL control (Sec. <ref>) and proportional control (Sec. <ref>) methods. In the results section, Sec. <ref>, the uncontrolled states are described in Sec. <ref>, and results from the training are discussed in <ref>. The evaluation of the trained agent in deterministic mode is provided in Sec. <ref> and results of deterministic evaluation in a larger domain are provided in Sec. <ref>. Proportional control is discussed in Sec. <ref>. We conclude the paper in Sec. <ref>. Multimedia (video) files are referred to in the caption of various figures in the text and information about these is provided in the `Supplementary Information' section in Appendix <ref>. We release the full code used in this study as open source materials, and the relevant information is provided in Appendix <ref>.




§ FORMULATION AND METHODOLOGY



 §.§ RBC Formulation

The formulation of the RBC problem closely follows that of <cit.>. The governing equations are the Navier–Stokes equations under the Bousinessq approximation, commonly adopted for convection problems. The Bousinessq approximation states that the density variation is only important in the gravitational term and neglects it in the rest of the equation. The continuity equation then reduces to the incompressible form, which further simplifies the viscous terms in the momentum equation. The viscosity is also assumed to be a constant. The equations are then made dimensionless by the domain height H̃, reference velocity Ũ_ref=√(g̃α̃_TΔT̃H̃) and the temperature difference ΔT̃ = T̃_H - T̃_C between the bottom and top walls. Note that dimensional quantities are indicated with the tilde ·̃. The variable g̃ stands for the acceleration due to gravity, and α̃_T is the thermal expansion coefficient. Upon non-dimensionalisation, the coefficients can be refactored in terms of two non-dimensional constants, i.e., the Rayleigh number =g̃α̃_̃T̃ΔT̃H̃^3/(ν̃κ̃) and Prandtl number =ν̃/κ̃. The variable ν̃ is the kinematic viscosity and κ̃ is the thermal diffusivity. Denoting the unit vectors in the x, y and z directions as 𝐢, 𝐣 and 𝐤 respectively, the governing equations are <cit.>:



    ∇·u = 0,


    ∂u/∂ t + (u·∇)u = -∇ p + √(/)∇^2 u + T 𝐣,


    ∂ T/∂ t + u·∇ T = 1/√(Ra)∇^2 T.


The velocity vector 𝐮(x,y,z,t) comprises of the three velocity components 𝐮=u𝐢+v𝐣+w𝐤, and T(x,y,z,t) is the temperature. The computational domain is cuboidal in shape with coordinates x and z representing horizontal directions and y representing the vertical direction. The variable t represents time. The length L of the domain is measured along x and z and the height H along y. The domain extends in the horizontal direction as x, z∈[0,L] and in the vertical direction as y∈[-1,1]. Note that in this work, L=4π and H=2. A schematic representation of the domain is shown in Fig. <ref>. To set up the temperature gradient, the top wall is at all times maintained at a spatially uniform temperature of 1, i.e., T_C=T(x,y=1,z,t)=1. The bottom wall is allowed to have a spatial distribution, i.e., T_H(x,z,t)=T(x,y=-1,z,t)≠constant, but we enforce that the spatial mean, i.e., ⟨ T_H(x,z,t) ⟩_x,z=T_H,0, is equal at all time to 2. The operator ⟨·⟩ represents averaging in space. Note that T_H is also spatially uniform during the uncontrolled simulations, i.e., T_H(x,z,t)=2, and it is only during the control simulations that the constant mean temperature constraint is enforced at the bottom wall. For the initial condition, a constant temperature gradient is provided as the temperature field, and quiescent flow for the velocity field. Periodic boundary conditions for velocity and temperature are provided at the lateral boundaries, while a no-slip boundary condition is prescribed at the top and bottom walls, along with the isothermal T_C=1 boundary at the top wall and constant T_H,0=2 at the bottom wall. 



A useful dimensionless number that can be used to quantify the magnitude of convection is the Nusselt number Ν. The Ν is defined as the ratio between the combined heat fluxes due to convection and conduction, and the heat flux due to conduction alone <cit.>:

    Ν(t) = 1+√() ⟨ vT ⟩_x,y,z.





 §.§ Numerical Method for the CFD

Following <cit.>, the governing equations (<ref>) along with boundary conditions are implemented with a spectral Galerkin method using a technique developed by <cit.> for direct numerical simulations of turbulent channel flows. In this method the pressure is eliminated and the 3D Navier–Stokes equations are basically reduced to the continuity equation, a fourth-order equation for the wall-normal velocity component and a second-order equation for the wall-normal vorticity component. Adding the energy equation, the four scalar equations that are solved are Eqs. (<ref>), (<ref>), as well as:

    ∂∇^2 v/∂ t   = ∂^2 H_x/∂ x ∂ y + ∂^2 H_z/∂ z ∂ y - ∂^2 H_y/∂ x^2 -∂^2 H_y/∂ z^2  + √(/)∇^4 v  + ∂^2 T/∂ x^2 + ∂^2 T/∂ z^2, 
    ∂ g/∂ t   = √(/)∇^2 g + ∂ H_x/∂ z - ∂ H_z/∂ x,

where H = (u·∇) u is the convection vector and g=(∇×u) ·𝐣 is the normal vorticity component. The initial condition for the temperature is a linear gradient in the y-direction with the bottom and top wall temperatures maintained at 2 and 1 respectively. Equation (<ref>) is implemented with the four boundary conditions v(x, ± 1, z, t) = v'(x, ± 1, z, t) = 0, where the first two are due to no slip, whereas the two latter follow from the continuity equation. Equation (<ref>) is derived by taking the curl of (<ref>), and the y-component g is subsequently solved with g(x, ± 1, z, t)=0 (no slip). Periodic boundary conditions in the x and z directions are applied between the lateral boundaries, and this is implemented through the choice of the numerical elements and basis functions, as described below. 

The four scalar equations (<ref>), (<ref>), (<ref>) and (<ref>) are implemented using a highly-accurate spectral Galerkin discretization in space <cit.> and a third-order implicit/explicit (IMEX) Runge-Kutta method <cit.> for the temporal integration. The Galerkin method makes use of tensor product basis functions constructed from Chebyshev polynomials for the wall-normal y direction and Fourier exponentials for the periodic directions x and z. The boundary conditions in both directions are built into the basis functions and as such enforced exactly. For the wall-normal direction this requires the use of composite Chebyshev polynomials <cit.>, whereas the Fourier exponentials for the x-direction are already periodic. Since the continuity equation cannot be used to find u for Fourier wavenumber 0, we solve for this wavenumber the momentum equation in the x direction. All other unknowns are closed through Eqs. (<ref>), (<ref>), (<ref>) and (<ref>). The convection terms H and u·∇ T are computed in physical space after expanding the number of collocation points by a factor of 3/2 in order to avoid aliasing. For H we use the rotational form H = -u× (∇×u), with the remaining 1/2 ∇u·u absorbed by the pressure, and for temperature we use the divergence form u·∇ T = ∇·uT. 

The code is implemented using the open-source spectral Galerkin framework `shenfun' <cit.>, where the equations can be automatically discretized through a high-level scripting language closely resembling the Mathematics. The Navier–Stokes solver has been verified by reproducing the growth of the most unstable eigenmode of the Orr–Sommerfeld equations over long time integrations. The Navier–Stokes and Rayleigh–Bénard solvers are distributed as part of the shenfun software, and there is a demonstration guide published in the documentation <cit.>, which also provides a much more detailed description of the numerical method. Links are provided in Appendix <ref>, and we refer the reader interested in all the technicalities of the numerical solver implementation to the resources detailed therein. 

The observation probes are distributed over the domain as a uniform mesh. For purposes of illustration, probes over four randomly selected segments (to avoid overcrowding) are shown in black dots in Fig. <ref>. There are 32 × 8 × 32 total probe-mesh points in the x, y and z directions, respectively. The number of quadrature points in the solution approximation used by the spectral Galerkin solver is equal to the number of Galerkin modes, which is 32 × 16 × 32. The resulting solution is spatially continuous, and the observations are then evaluated from these global, continuous spectral Galerkin functions at the uniform mesh probe locations. These observations are used in the DRL-based control methodology which we describe next.




 §.§ DRL Control Methodology

The numerical simulations is solved with the methods described in the previous section from the constant temperature gradient initial condition up to a time instant when the time-averaged Ν, , becomes statistically constant and the RBC flow structures are fully-developed. By `fully-developed', we mean that the flow variables u, v, w and T (and by consequence, Ν) are statistically stationary. Note that the overbar denotes averaging with respect to time. The  averaged over the last half of the total baseline simulation time is denoted as , which is the reference Ν used in the reward computation, as will be described later in this section. We refer to these uncontrolled initial simulations as the `baseline', since states at the final instants of these simulations serve as initial conditions for the control. For , the time duration of the baseline is 400 time units and for  it is 5000. 

The control is applied to the RBC system as temperature actuations on the bottom wall boundary at y=-1. The bottom wall boundary is divided into 8× 8 square segments all of which are of equal size of π/2 ×π/2 in the x and z directions. The RBC system with the bottom wall segments together form the DRL environment. As mentioned in Sec. <ref>, DRL occurs via interactions between three fundamental channels of communication between the environment and the agent: the state S_t, action A_t and reward R_t at time t. As per the SARL framework, S_t comprises all the physical quantities measured at observation probes distributed within the domain, R_t is a measure based on the performance of the system, and A_t are the set of outputs from the agent which are control actuations. In the case of the RBC system S_t can, for example, be values of [T, u, v, w] at the observation probes, R_t can be a function of Ν and A_t the set of 64 temperature actuations T_act applied on each segment. 

In the following, we use a MARL setup as illustrated in Fig. <ref>. In MARL, N_ag multiple agents are each assigned to different local sections of the entire domain, which we refer to as `pseudo-environments'. We define a pseudo-environment as the block of the domain directly above each individual segment. Each agent is associated with its own stream of state, action and reward from its corresponding pseudo-environment which we denote as s^(j)_t, a^(j)_t and r^(j)_t respectively, j=1,⋯, N_ag being the agent index. Here, note the modification in notation from uppercase letters to lowercase letters: uppercase representing states, actions and rewards corresponding to the entire domain as per the SARL framework, and lowercase representing states, actions and rewards specific to each pseudo-environment as per the MARL framework. We represent segment-specific values with the superscript j in parenthesis. Note that the r^(j)_t is a `weighted' local reward, which is a combination of a `local' and `global' Ν as described below. This is based on the fact that the RBC environment is invariant in the horizontal directions x and z and so maximising a local reward can correspond to maximising the global reward in a cumulative manner. The state, action and reward tuples from each pseudo-environment and then from each timestep are batched together consecutively up to a size corresponding to the batch size, before an agent update is performed.

A key aspect of the MARL implementation that exploits translational invariance is that all the trainable weights in the agent parameterisation are shared, i.e., all agents, defined by their separate streams of state, action and reward, share the same neural network. This is an effective way to share experience across the domain and alleviate the curse of dimensionality over the control space dimension, as previously highlighted in <cit.>. 




The states s^(j)_t from each pseudo-environment are obtained from a set of 4× 8× 4 = 128 observation probes distributed with even spacing in the j^th pseudo-environment. A representation of this is shown in Fig. <ref> within the RBC domain in four randomly selected pseudo-environments. Four physical quantities: temperature and three velocity components [T, u, v, w] are observed from each probe in each pseudo-environment. A scaling or shifting is performed to ensure that the state values are roughly between -1 and 1 and not too small. The velocities are scaled as u_s=1.5× u, v_s=1.5× v and w_s=1.5× w and the temperature is shifted as T_s=T-1.8, where subscript s stands for `scaled' or `shifted'. The scaling factors used are based on the range of values observed in the baseline so that the scaled variables are roughly within the range [0,1]. These 512 quantities (4 variables measured at 128 observation probes) are flattened to form the local state vector s_t^(j)=[T_s, u_s, v_s, w_s]^(j). 


The goal of any DRL algorithm is formulated as a maximisation problem, i.e., to maximise the reward. In the MARL framework, the definition of reward for the j^th pseudo-environment is slightly modified: it incorporates two Ν values, one calculated within the j^th MARL pseudo-environment called the local Nusselt number Ν^(j), and one calculated over the entire domain called the global Ν. The global Ν is calculated using Eq. (<ref>). The local Ν, Ν^(j), is given as: 

    Ν^(j)(t) = 1+√() ⟨ vT ⟩_x^(j),y,z^(j),

where x^(j) and z^(j) are the horizontal coordinate intervals corresponding to the j^th pseudo-environment, i.e., in the j^th pseudo-environment. The global Ν is the part of the reward that accounts for the main goal of the agent, to minimise the convection over the entire channel. This incentivizes each pseudo-environment to improve the global flow state. The local Ν provides information to the agent about the local effects of the actions on the flow. This allows for more reward granularity to the agent during training. Given that the RBC control goal is to minimise Ν, we define the total reward for the j^th pseudo-environment to be a weighted reward r^(j)_t proportional to the negative of a weighted sum of the global and local Ν, defined as: 

    r^(j)_t =  - [βΝ^(j)(t) + (1-β)Ν(t)],

where  is the reference value equal to  with time-averaging performed over the last half of the total baseline simulation time and the factor β=0.0015 is a weighting factor determining the relative weight of the local and global Ν contained in the reward. From Eq. (<ref>), a lower local or global Ν results in a higher reward. The value of β appears low, but this is due to the difference in geometric width used to compute the local and global Ν's. We choose β such that the local Ν accounts for about 10% contribution to the total reward.


The output from the agents are actions a_t^(j) in the range between -1 and 1, each agent generating a single action for its corresponding control segment. These actions are then transformed into temperature actuations T^(j)_act applied to each bottom area using a shifting and normalisation process following:


    a'^(j)_t    = a^(j)_t - 1/N_ag∑_j=1^N_ag a^(j)_t, 
    
        K    = max( 1, max_j| a'^(j)_t | ), 
    
        T^(j)_act   = T_H,0 + Λa'^(j)_t/K.


In the above, each raw action is first subtracted by the mean over actions for all segments (i.e., for each MARL agent) to form action perturbations a'^(j)_t. The value K is a factor to ensure that action perturbations larger than 1 are scaled down before being converted into temperatures. The factor Λ controls the magnitude of the temperature actuations. The above equations ensure that the mean temperature at the bottom boundary is kept at a constant value of T_H,0=2. This is the constant mean temperature constraint as described in Sec. <ref>. The RBC flow regime is characterised by the  which depends on T_H,0, and thus changing T_H,0 would modify this regime. Thus, the constant mean temperature constraint is to ensure that despite the application of the control values, the regime of the instability remains unmodified and the system is controlled within this regime. 

The states, rewards and actions described above are implemented within the framework of the `Tensorforce' library <cit.>. Tensorforce contains several optimisation algorithms for reinforcement learning. In the present work, the widely used policy gradient method, proximal policy optimisation (PPO), is employed to train the agent. PPO is an actor-critic method where the actor approximates the policy distribution and the critic approximates the value function of the current state <cit.>. Both the policy and the critic are represented by neural networks.  We make a distinction in terminology between two types of episodes used in the MARL framework: `CFD episodes' and `MARL episodes'. A CFD episode is a simulation of the entire RBC domain for 200 actuations. Each actuation in a CFD episode comprises N_ag actions applied locally by each MARL pseudo-environment. Hence, each pseudo-environment applies 200 actuations in one CFD episode. We define 200 actuations applied by a single pseudo-environment as one MARL episode. For N_ag pseudo-environments, one CFD episode equals N_ag MARL episodes. All results in this paper are plotted with respect to CFD episodes only. Therefore, hereafter in the text and figures, the usage of the term `episode' refers to `CFD episode', unless otherwise specified. A set of other training hyperparameters used in this study is provided in Table <ref>. 

In the present work, 200 and 400 total episodes are simulated for  and  respectively, and each actuation in any given episode is applied for a duration of 15 timesteps. In order to ensure that the agent is also trained on states from longer simulations, we ensure that 20% of the episodes start from the end of the previous episode, and the remaining 80% starts from the end state of the baseline. In this way, the system also explores trajectories spanning a longer time across successive episodes durations, and thus the agent observes more states, including states representative of the long-term behavior of the system. We use a batch size of 3 CFD episodes (i.e., 3×N_ag=3× 64=192 MARL episodes). We found that with lower batch sizes, training is largely unstable. We explain this observation as follows. Neural networks learn best from data that is uncorrelated. Since data from adjacent MARL environments is strongly (spatially) correlated, a batch size large enough must be chosen such that independent trajectories can be sampled during training. On the other hand, a batch size too large would make training sluggish. Other agent hyperparameters and CFD parameters are provided in Table <ref>. Training is performed with a discount factor of γ=0.99. For details of the algorithmic implementation of MARL, we refer the interested reader to <cit.>. We follow the same code implementation therein.





 §.§ Proportional Control Methodology

We also employ a proportional controller in order to compare the performance of the DRL-based controller described in the previous section, for  and . For proportional control, actuations proportional to the temperature perturbations are applied on the bottom wall segments. The size of the bottom wall segments and number of segments are the same as described in Sec. <ref>. In order to compute the magnitude of the temperature perturbations during instability, we require a reference temperature quantity from a stable state devoid of any RB instability. For this, we consider a low =100, corresponding to a stable regime that features a no-motion purely conductive state with a constant temperature gradient. The temperature at the mid-plane y=0 is measured, and we denote the temperature averaged over the entire mid-plane as T_ref,100=⟨ T(x,y=0,z,t) ⟩_x,z,t. We use T_ref,100 as the reference temperature. Next, for  and , at each time instant of the simulation, the temperature perturbation T' is measured as

    T'(x,z,t)=T_ref,100 - T(x,y=0,z,t).

Note that the value of T' at the portion of the mid-plane directly above each control segment is used as the observation. For the j^th segment, the observation is T'(x^(j),z^(j),t). A control action T_p^(j) proportional to -T'(x^(j),z^(j),t) is then applied on the j^th segment on the bottom wall as: 

    T_p^(j)=-K_p ⟨ T'(x^(j),z^(j),t)  ⟩_x^(j), z^(j).
 

For the constant of proportionality K_p, a range of values is tested in both  and . For , the K_p range is from 0.1 to 1.0 in increments of 0.1. For , the range of K_p is 0.1 to 2.0 is increments of 0.1. A wider range is used for  since it is more unstable than  and may require a larger amplitude of control. The response of the system for each individual K_p is simulated with the initial condition being the end state of the baselines, until a controlled state is reached.








§ RESULTS AND DISCUSSION




 §.§ Baselines




Figure <ref> shows the evolution of Ν in the baseline simulations of each of the cases  and . The value of the Ν begins at 1.0, indicating the initial condition, i.e., the no-motion state where the heat transfer is due to conduction only. This state is unstable and the convective state sets in accompanied by a rise in Ν. Then, Ν stabilises at a given value or oscillates around a mean value, after which the baseline simulation is stopped. 

For , five simulations were performed with the same parameters. The simulations all result in topologically similar solutions with similar Ν values. We show a single solution in Fig. <ref>(a). For , similarly, five simulations of the baseline were performed with identical simulation parameters. However, in contrast to , three different end-states are reached owing to the system chaoticity and stochasticity at , some of which are oscillatory and some of which are steady. Since  is higher, this state is more unstable and dynamically rich compared to . We classify these baselines into three classes, class 1 to class 3, based on the magnitudes of Ν after the transients have died away. The value of  is the obtained after time averaging Ν over the last half of the baseline time. Class 1 corresponds to run 4, which shows a value of =1.379, class 2 corresponds to run 2 with a lower steady  of 1.348, and class 3 corresponds to simulations 1, 3 and 5 characterised by oscillatory Ν values with =1.31. For , all of the baselines 1–5 are used to initialize the episodes while training the DRL agent, each episode starting from a randomly selected baseline.  The resulting  for  is 1.268 and is used as  in Eq. (<ref>). For classes 1–3 of , we use the average of all three  obtained at each of the three classes, obtaining a value of =1.346.  



In Fig. <ref>, we plot the shapes of the convection rolls at the final time step of the baselines. Although training for  is performed with all 5 baseline runs, for brevity, Fig. <ref>(b) shows the topology for class 3 (run 5) only, since this is the baseline from which the deterministic agent evaluation and proportional control cases are simulated. The topologies of the remaining baseline runs for  are shown in Fig. <ref> of Appendix <ref>. At the warmest regions, the v velocities are positive and maximum indicating upward flow, and in the coolest regions, v is negative and minimum indicating flow towards the bottom wall. We define the length scale of the convection rolls as the distance between the warm and cold regions adjacent to each other at points where the temperatures are the maximum and minimum, respectively (i.e., v is maximum and minimum). In both cases, the convection rolls have a length scale of approximately π, an observation in line with results of linear stability analysis <cit.>. Note that videos of the temperature field evolution for these baselines are provided in Online Resource 1 for  and Online Resource 2 for . Visible from the fields, all baseline shapes for  and 750 are irregularly shaped, with no particular spatial pattern. Class 2 and class 3 baselines for  exhibit spatial oscillations about a mean shape.



 §.§ Training


Figure <ref> shows the training curves overlaid with their moving averages, with the horizontal axis being CFD episodes. The plotted values are the Ν averaged over all time steps in each episode. The moving average curves are averaged over a moving window of 20 episodes. In both cases  and , the mean Ν reduces to values below the  in the baseline and then remains within a range of values. To measure the final Ν after training, the plotted Ν values are averaged over the last  episodes. We denote this final Ν after training as ⟨Ν⟩_, with ⟨·⟩_ representing averaging over the last  episodes. Thus, in each case, we can quantify the extent of reduction of convection by calculating the percentage decrease of Ν from the reference value  using the following formula:


    Percentage reduction of Ν=-⟨Ν⟩_/-1× 100%.


Note that the denominator is -1 and not simply . This is because the minimum possible Ν in the RBC system is 1 (Eq. (<ref>) with no convection, i.e., v=0) and so the percentage reduction calculation is offset by 1. For , =50, (i.e., averaging over episodes 150 to 200) and ⟨Ν⟩_=1.209. From the baseline =1.268, this gives is a percentage Ν reduction of 22%. 

In Fig. <ref>, we show the Ν evolution during three episodes in different stages of the training. At episode 25, there is not much improvement compared to . However, it is noticeable that the different actuations make the Ν variation more noisy compared to the baseline. At episode 40 and more so at episode 193, a significant improvement is observed. In both cases, the Ν trajectories first drop to a low value between t=400-450, and then remain low. In episode 193, the drop to a low Ν occurs sooner (t=400-410) and to a much lower value compared to episode 40 (t=400-450). This shows that as the training proceeds, the agent learns a more optimal combination of actions to reduce convection more rapidly in the first few timesteps of control, and ensures that the system sustains this controlled state. 



In the training curve at  in Fig. <ref>(b), similar to , the DRL agent reduces the Ν, showing that a control strategy is learned. However, given that the  case is more unstable than  and five baselines are used during training, the phase space to be explored by the agent is much larger than for , and so learning requires more steps, i.e., a larger number of episodes are required before a significant drop in Ν is observed. The Ν begins to drop at episode 100, compared to episode ≈ 20 for . After a gradual decrease in Ν from episode 100–250, it saturates after episode 250. Choosing =150,  ⟨Ν⟩_=1.302 giving a percentage Ν reduction of 12.7%. 

In Fig. <ref>, we plot the Ν evolution during three episodes during the training at . Similar to the  case, the progress to a more effective control policy is observed for higher episodes. Also similar to , during initial times t<5020, the drop in Ν from its initial value is quicker (i.e., a more negative slope) in later episodes compared to earlier ones. 

In order to understand the nature of the control policy learned by the agent, we need to investigate the deterministic evaluations of the trained agent, which we show in the next section.







 §.§ Deterministic Evaluation

In this section, we get an insight into the control method learned by the agent. In each case  and , we load the agent saved at the final episodes of the training, i.e., episode 200 for  and episode 400 for . The loaded agent is used in the evaluation mode, by choosing the most probable action for each observed state in a deterministic manner, in contrast to the training mode, when the agent explored the phase space by using exploration on the actions. In order to compute the percentage reduction in Ν during the deterministic runs, we use the following formula: 

    Percentage reduction of Ν=-Ν_C/-1× 100%,

where Ν_C=^Δ t_C is the Ν of the controlled state calculated by averaging the Ν over some portion (Δ t_C) of the time during which control is executed. In order to justify the offset by 1 in the denominator, the same reasoning is used as in Eq. (<ref>).




  §.§.§ 

First, we look at the evaluation for  in Fig. <ref>. Control is executed from t=400. Within the first few timesteps, the Ν drops to a low value. Following a period of transients, the system stabilises into a final controlled state from t=1300-1900. Thus, Δ t_C=600 and following Eq. (<ref>), Ν_C=1.205. This corresponds to a Ν reduction of 23.5%. 

Next, we show the actions alongside the observations in a few segments in Fig. <ref> (b)-(e), to get an idea of the learned policy. Here, segments 1, 20, 40 and 56 are shown for time instants during the final controlled state. In each segment, we compare control temperatures T_act^(j) (Eq. (<ref>)) with temperature observations T_obs^(j), which are temperatures in the mid-plane y=0 in the j^th pseudo-environment averaged in the horizontal directions, i.e., T_obs^(j)=⟨ T(x^(j),y=0,z^(j),t) ⟩_x^(j), z^(j). We also plot in dashed lines the mid-plane temperature at a stable state if there was no convection, which equals 1.5. Noticeably, the control strategy is non-trivial and there exists a complex non-linear relation between the averaged temperature above a segment and the bottom wall temperature chosen by the DRL controller. In particular, we observe:


    
  * Control actuations disproportionate to observations: although T_obs^(j) are roughly sinusoidal with time, the variation of T_act^(j) does not follow the same pattern, but is instead composed of extended periods of actuations below T_H,0=2 and short peaks of warmer actuations above T_H,0. Also, in segments 1 and 20, as T_obs increases above 1.5, T_act decreases, similar to opposition control. However, T_act remains at a low value even when T_obs begins to drop. In segments 40 and 56, T_obs and T_act have the same variation pattern: when T_obs decreases below (increases above) the mean, then T_act increases above its mean as well. Thus, the learned control strategy features actuations that are disproportionate to the observations in some segments. This is distinct from an opposition control strategy, where the actuations follow a pattern of variation opposite to the observations, uniformly across segments.
    
  * Differences in actuator delays per segment: It is also visible from segments 1 and 20 that the T_act begins to increase after a short delay in T_obs increasing. This duration of delay is different across different segments. In the case of opposition control with delays, T_act^(j) would follow the exact opposite pattern of variation as T_obs^(j) across segments, and there would be no difference in delays from segment to segment. Moreover in opposition control, the delays in each segment are required to be tuned before the optimal control state is achieved, while in DRL control, these delays are implicitly learned in the control policy. 





Next, we show in Fig. <ref> how the flow structure corresponding to  is modified by the DRL control compared with that of the baseline (previously shown in Fig. <ref>). During the transient phase (t=400-1300), the DRL controller breaks down the convection rolls in the baseline state into smaller localised blobs (Fig. <ref>(a)). Then, in the final controlled state beyond t=1300, the structure is rearranged to non-intersecting diagonally running convection rolls (Fig. <ref>(c)). A sequence of spatial actuation patterns (two of which are shown in Fig. <ref>(b) and Fig. <ref>(d)) is executed by the agent in order to modify the flow topology and bring the system to this controlled state. This sequence of actuations is a feature of the learned control policy. 

The regular convection structure of the controlled state is representative of RBC at a value of  lower than . To demonstrate this, we perform an additional baseline simulation at Ra=400 and plot its Ν and temperature field in in Fig. <ref>. Ra=400 corresponds to a less unstable regime, featuring a smaller convection amplitude compared to , as evident from Fig. <ref>(a). Further, the temperature field at instability (Fig. <ref>(b)) shows the regular pattern of diagonal convection rolls. It can thus be stated that the DRL controller modifies the trajectory of the system so that it resembles states observed in more stable regimes.







  §.§.§ 

We now study the deterministic runs at . We perform evaluations on all the three baseline classes as described in Sec. <ref>, using the latest saved agent from the training of . The results are plotted in Fig. <ref>. In Fig. <ref>(a), we see that in all three classes, the agent reduces the Ν to a lower value from the respective baselines. For classes 1 and 2, a controlled state is reached within approximately 200 time units. For class 3, a controlled state is reached within 1000 time units, i.e. at time 6000, after a period of transients. In order to calculate the percentage reduction in Ν, the formula in Eq. (<ref>) is used with the  values being the Ν averaged over the last half of the baselines for each of the classes. As mentioned in Sec. <ref>, this is 1.379, 1.348 and 1.310 respectively for classes 1, 2 and 3. From Fig. <ref>(a) we calculate the Ν_C reached in each class as 1.312, 1.281 and 1.283 respectively.  Therefore, for classes 1, 2 and 3, the agent achieves a percentage of Ν reduction of 17.7%, 19.3% and 8.7%, respectively. Given that the  agent is trained on five baselines, the ability to control all classes of baselines (a wider range of states) is an indicator of the robustness of the DRL agent.



To better understand the learned policy, we observe the plots in <ref>(c)–(f), where four segments are chosen at random to analyse the action chosen based on the observed states. Opposition control-like relationships are visible in segments 1 and 20, where from t=6000-7000, high temperature observations at the mid-plane cause the corresponding actuations to be lower than T_H,0=2. However, in segments 40 and 56, an opposite trend is visible. In segment 40 at t=6500, a decrease in T_obs is accompanied by a decrease in the actuation temperatures as well, and in segment 56 at t=6500, an increase in T_obs is accompanied by an increase in T_act. Similar to , the delays between the increase/decrease of T_obs and increase/decrease of T_act are segment-dependent and not consistent across segments, which differentiates the agent-learned control policy from a simple opposition control policy. The qualitative results of the learned control strategy are similar to that observed in . The fact that such differences in the policies are observed, while the MARL setup implies that the DRL agent is shared across segments, indicates that complex non-linear strategies that depend on the state in a complex fashion have been chosen by the agent. 



Next, we plot the temperature fields of  during control in Fig. <ref> for the class 3 baseline. As is visible from the plot, the agent breaks down the convection cells in the baseline (previously shown in Fig. <ref>(b)) into smaller cells, reshaping the structure of the convection to straight rolls running diagonally across the domain. The final controlled state achieved by the agent resembles a convection pattern at a lower, less unstable =400 (Fig. <ref>).




 §.§ Proportional Control


We compare the control policy learned by the DRL agent with a proportional controller applied to the Rayleigh–Bénard system as described in Sec. <ref>. In Figs. <ref> and <ref>, we show the response of the system to proportional control at  and , respectively. In each case, the simulation is performed starting from the end state of the baseline. For , the class 3 baseline is used, since this baseline was used in the deterministic run in Sec. <ref>.  In Figs. <ref>(a) and <ref>(a), we only show the Ν curves corresponding to the K_p values with the best control performance. In Figs. <ref>(b) and <ref>(b), the  averaged over the last half of the total control time duration is plotted across all K_p. In both Figs. <ref> and <ref>, the mean Ν of the DRL-controlled state, Ν_C is also shown for comparison. 



The reduction in Ν for proportional control is computed using Eq. (<ref>). In Fig. <ref>, K_p=0.6 yields a controlled state with Ν_C=1.215, corresponding to a percent reduction in Ν of 19.8%. This is calculated from the  for  equal to 1.268. A drop in the averaged Ν is observed as the K_p is increased up to 0.6, with either constant Ν, or low-amplitude fluctuations. As K_p is increased beyond 0.6, overshooting occurs: the large control actuations cause the system to become further unstable, and the mean Ν increases. Compared to DRL-based control in Fig. <ref>(a), the proportional control strategy is sub-optimal even at the best K_p, since DRL-based control reduces the value of Ν to a lower value than the proportional control at K_p=0.6. This result suggests that with DRL, a control policy more sophisticated than a simple proportional control law is discovered. As discussed in Sec. <ref>, this corresponds to additional learned features of the control law such as a range of different actuator delays across segments and actuations disproportionate to temperature observations, corresponding to non-linear DRL control laws. 



The temperature field at the final controlled state at K_p=0.6 for  is also plotted in Fig. <ref>. In contrast to the DRL-controlled end state, the convection rolls are not straight and parallel, and do not have uniform thickness, indicating that proportional control is not able to cause the same topological modifications obtained by the non-linear DRL controller.



At , as is visible from Fig. <ref>(a), the K_p corresponding to the largest Ν reduction is 1.1. Similar to , the controlled-state mean Ν reduces linearly with K_p up to K_p=1.1 with a low-amplitude fluctuation about the mean Ν at each K_p, and beyond 1.1, the mean Ν increases, i.e., overshooting occurs, and the system becomes unstable. A controlled state similar to the DRL-controlled state, with straight parallel cells running diagonally past the domain is observed in Fig. <ref>. The Ν_C at the controlled state achieved by the DRL agent is 1.283, which is slightly lower than that of the proportional controller at K_p=1.1, which is 1.289. The  of the class 3 baseline is 1.31 (Sec. <ref>). The DRL controller achieves a percentage Ν reduction of 8.7%, and with proportional control, a 6.8% reduction is achieved. 



Although the Ν reduction at  using DRL and proportional control are comparable, it is noteworthy that the agent at  is trained using three different baselines, and learns an effective control policy for all baselines during a single training run. The agent at  is thus more capable to generalise to a wider range of states, highlighting its robustness. In the case of a proportional controller, an efficient tuning of K_p must be carried out for each of the three baseline classes until an optimum Ν reduction is observed.





 §.§ Deterministic evaluation in a larger domain

One of the core advantages of MARL compared to SARL is its ability to reduce the state and action space dimensionality by exploiting spatial invariances in the system. This also allows to easily transfer a MARL agent trained on a given domain size to another domain size: as long as the underlying physics are similar, the policy can be applied to a domain of different dimensions by adapting the number of MARL pseudo-environments deployed. In this section, we perform a deterministic run of the agent trained on the 4π-width and -length domain at  on a larger domain of size 8π. The number of quadrature points for the CFD solver is doubled in the horizontal directions from 32× 16 × 32 to 64× 16× 64, and the number of control segments is also doubled in each horizontal direction from 8× 8 control segments in the 4π domain to 16× 16 control segments in the 8π domain. In this way, the size of the input state to each agent is kept constant and each agent observes the same segment size as before, i.e., a segment size of π/2 ×π/2. 



We first show the Ν and temperature field of the baseline state in the mid-plane of the 8π domain in Fig. <ref>. Several curved convection rolls are observed, and the Ν stabilises at a  (time-averaged over the last half of the baseline time, 5000 to 10000) of 1.246. In Fig. <ref> we show the evolution of the Ν of the baseline state and controlled states. By applying the 4π domain-trained agent, the Ν reduces and reaches a mean value of Ν_C=1.211 in the controlled state. Here, Ν_C is computed from t=11500 to the end of the simulation. Using Eq. (<ref>), this is a Ν reduction of 14.2%. Next, we show the flow topology at the mid-plane of the final controlled state in Fig. <ref>. From the plot, it is visible that the baseline convection rolls are broken and reconfigured to a shape with diagonally running convection rolls that are nearly straight. This pattern is qualitatively similar to the end control states reached by the deterministic evaluation of the agent in  and  in figures <ref>(b) and <ref>(b), respectively, and is topologically closer to the configuration observed at lower  (Fig. <ref>). 







§ CONCLUSION


In the present study, we apply control using deep reinforcement learning (DRL) on a 3D Rayleigh–Bénard convection (RBC) system, as an extension of the work in 2D by <cit.>. We use the multi-agent reinforcement learning (MARL) framework that involves several agents, each with its own pseudo-environment. Each agent has its own stream of state observations and rewards from its corresponding local portion of the domain, and applies an independent control action. MARL is especially beneficial for physical systems such as RBC whose dynamics exhibit spatial translational invariance, and allows to overcome the curse of dimensionality, which is otherwise making trial-and-error learning of distributed output control laws computationally intractable. Compared to the 2D case in <cit.>, the larger control dimensionality (number of actuators required), the larger state space dimensionality and the more complex flow dynamics in the 3D case motivate the need for MARL-based control in 3D. This is, to the best of our knowledge, the first time that DRL control is applied to the 3D RBC system. 

The RBC computational domain size is 4π× 2 × 4π, and the bottom wall is divided into 8× 8 square segments of size 0.5π× 0.5π, each of which is assigned to a single MARL agent that applies a temperature value across the segment as its control actuation. Training is performed at  and .  With MARL, control of the RBC system is successfully achieved. During training, the training curves show a drop in Ν by 22% for  and 12.7% for . Both  and  are trained with each episode starting from the end state of the baseline for 80% of the total episodes and the remaining 20% starts from the end of the previous episode. In this way, the system also explores trajectories spanning a longer time across successive episodes durations, and thus the agent observes more states, including states representative of the long-term behavior of the system. The case of  uses three different classes of baselines with different convective patterns, and hence the trained agent is more robust than if trained using a single baseline. During deterministic evaluation, the agent trained at  achieves 23.5% reduction in Ν, and the agent trained at  achieves 17.7%, 19.3% and 8.7% Ν reduction on baselines of classes 1, 2 and 3, respectively. The fact that the agent is able to control all three classes of baselines at  is an indicator of its robustness. The structure of the convection in the domain is modified upon control from a spatially irregular pattern to straight convection rolls that run diagonally across the domain. The structure of the convection rolls in the controlled state is topologically similar to that of a RBC system at a lower =400. 

Comparisons have also been made with proportional control (PC) using a range of coefficients of proportionality K_p. Results show that the DRL-based control outperforms PC. At , PC achieves a 19.8% Ν reduction and at , PC achieves a 6.8% reduction, while DRL-based control achieves 23.5% and 8.7%, respectively. Using DRL-based control, in a single training run, the agent learns a more complex control policy than PC, which includes actuator delays and magnitudes of control actions that are different across different segments. As the MARL strategy is adopted in this study, we also show that an agent trained on the 4π domain can be extended in application to control the RBC system at  in a domain of a larger size, i.e., 8π-domain. Extending control to larger domains highlights the advantage that MARL can exploit translational invariances of the RBC system at . 

These results open opportunities for a number of future investigations. Various configurations of agents can be adopted for the control of RBC at larger domains and larger . For example, instead of a single agent having as input the states corresponding to a single segment, a set of multiple neighbouring state segments can be assigned as the input state to a single agent. The input states would then contain information from a larger window of the domain that is influenced by the action imposed in the control segment. We are also working on investigating how the results presented here can be extended to higher  cases, in which convection is both stronger and more chaotic.



Acknowledgements 
Part of the deep-learning-model training was enabled by resources provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS). Ricardo Vinuesa acknowledges financial support from ERC grant no. `2021-CoG-101043998, DEEPCONTROL'. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them. 



§ DECLARATIONS


Conflict of Interest 
The authors declare no conflict of interest.

Author Contributions 
Conceptualization: Ricardo Vinuesa, Jean Rabault, Joel Vasanth, Francisco Alcántara-Ávila; Methodology: Ricardo Vinuesa, Jean Rabault, Joel Vasanth, Francisco Alcántara-Ávila; Formal analysis and investigation: Joel Vasanth; Software: Joel Vasanth, Francisco Alcántara-Ávila and Mikael Mortensen;  Writing - original draft preparation: Joel Vasanth; Writing - review and editing: Joel Vasanth, Jean Rabault, Francisco Alcántara-Ávila, Mikael Mortensen, Ricardo Vinuesa; Funding acquisition: Ricardo Vinuesa; Resources: Ricardo Vinuesa; Supervision: Jean Rabault, Ricardo Vinuesa, Francisco Alcántara-Ávila. All authors read and approved the final manuscript.

Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.






§ SUPPLEMENTARY INFORMATION


Several supplementary files are provided along with this article. These are video animations of temperature-field evolutions from simulation results corresponding to various figures in the main text. An exhaustive list of the video filenames with their captions is provided below. The reader is advised to refer to this list when `Online Resource' is mentioned in the main text. Note that `Online Resource' is abbreviated as `OR' below. 


    
  * OR 1: `'. Mid-plane temperature field evolution of the baseline of . Corresponding figure - Fig. <ref>(a).
    
  * OR 2: `'. Mid-plane temperature field evolution of the class 3 baseline of  (run 5 in Fig. <ref>(b)).
    
  * OR 3: `'. Mid-plane and bottom wall temperature fields during the evolution of the RBC system under action of the MARL controller. At time 1300, a more organised regular spatial arrangement of convection rolls is observed, characterised by straight diagonal running cells. Corresponding figure - Fig. <ref>.
    
  * OR 4: `'. Evolution of the baseline at Ra=400. Time units shown correspond to those in Fig. <ref>(a).
    
  * OR 5: `'. Deterministic evaluation of agent trained at . Irregular baseline shape broken down and convection structure becomes regular, i.e., straight diagonal cells at time 5928. Corresponding figure - Fig. <ref>
    
  * OR 6: `'. Proportional control for  at K_p=0.6.  Corresponding figure - Fig. <ref>.
    
  * OR 7: `'. Proportional control for  at K_p=1.1.  Corresponding figure - Fig. <ref>.
    
  * OR 8: `'. Baseline evolution for  in a domain of length 8π. Corresponding figure - Fig. <ref>.
    
  * OR 9: `'. MARL agent trained on domain of length 4π used in deterministic evaluation mode for the control of RBC in larger domain of length 8π. Corresponding figure - Fig. <ref>.




§ OPEN SOURCE CODE RELEASE ON GITHUB

All the codes, scripts, and post-processing tools used in this work are made available on Github together with readmes and user instructions. Reasonable user support will be provided through the issue tracker of the corresponding Github repository. The url for the repository is <https://github.com/KTH-FlowAI/DRL_MARL_RayleighBenard3D_Control>. The shenfun CFD case setup is described in great details, including all the numerical implementation considerations, elements chosen, and code walk-through, on the shenfun RBC documentation page: <https://shenfun.readthedocs.io/en/latest/rayleighbenard.html>. Note that the conventions used in the present paper and in this documentation page are slightly different: while the present paper uses x as the horizontal (“wall parallel") description and y as the vertical (“wall normal") direction, the documentation page uses the opposite conventions. This has no influence on the CFD results per se. The code uses the same conventions as the shenfun documentation, and only the present paper uses separate conventions, for simplicity of the writing and conformity with the literature.




§ CONVECTION STRUCTURE FOR  BASELINES


In this appendix, we show the temperature fields for the baselines for  (not shown in Fig. <ref>(b)) as described in Sec. <ref>.













69

#1ISBN #1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1<https://doi.org/#1>et al.#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1#1<><#>1#1#1#1#1#1#1#1#1#1#1#1#1#1PreBibitemsHook


[Alnæs et al.2015]alnaes2015fenics

Alnæs, M.,
Blechta, J.,
Hake, J.,
Johansson, A.,
Kehlet, B.,
Logg, A.,
Richardson, C.,
Ring, J.,
Rognes, M.E.,
Wells, G.N.:
The fenics project version 1.5.
Archive of numerical software
3(100)
(2015)




[Ascher et al.1997]ascher97

Ascher, U.M.,
Ruuth, S.J.,
Spiteri, R.J.:
Implicit-explicit runge-kutta methods for time-dependent partial differential equations.
Applied Numerical Mathematics
25(2),
151–167
(1997)
10.1016/S0168-9274(97)00056-1




[Beintema et al.2020]beintema2020controlling

Beintema, G.,
Corbetta, A.,
Biferale, L.,
Toschi, F.:
Controlling Rayleigh–Bénard convection via reinforcement learning.
Journal of Turbulence
21(9-10),
585–605
(2020)




[Bergé and Dubois1984]bergedubois1984

Bergé, P.,
Dubois, M.:
Rayleigh-Bénard convection.
Contemporary Physics
25(6),
535–582
(1984)
10.1080/00107518408210730




[Bae and Koumoutsakos2022]Bae2022

Bae, H.J.,
Koumoutsakos, P.:
Scientific multi-agent reinforcement learning for wall-models of turbulent flows.
Nature Communications
13(1),
1443
(2022)
10.1038/s41467-022-28957-7




[Brunton and Noack2015]brunton2015closed

Brunton, S.L.,
Noack, B.R.:
Closed-loop turbulence control: Progress and challenges.
Applied Mechanics Reviews
67(5),
050801
(2015)




[Brunton et al.2020]brunton2020machine

Brunton, S.L.,
Noack, B.R.,
Koumoutsakos, P.:
Machine learning for fluid mechanics.
Annual review of fluid mechanics
52(1),
477–508
(2020)




[Belus et al.2019]belus2019exploiting

Belus, V.,
Rabault, J.,
Viquerat, J.,
Che, Z.,
Hachem, E.,
Reglade, U.:
Exploiting locality and translational invariance to design effective deep reinforcement learning control of the 1-dimensional unstable falling liquid film.
AIP Advances
9(12),
125014
(2019)




[Bucci et al.2019]bucci2019control

Bucci, M.A.,
Semeraro, O.,
Allauzen, A.,
Wisniewski, G.,
Cordier, L.,
Mathelin, L.:
Control of chaotic systems by deep reinforcement learning.
Proceedings of the Royal Society A
475(2231),
20190351
(2019)




[Chevalier2002]chevalier2002adjoint

Chevalier, M.:
Adjoint based control and optimization of aerodynamic flows.
PhD thesis,
Mekanik
(2002)




[Carbo et al.2014]car14

Carbo, R.M.,
Smith, R.W.M.,
Poese, M.E.:
A computational model for the dynamic stabilization of Rayleigh–Bénard convection in a cubic cavity.
The Journal of the Acoustical Society of America
135(2),
654–68
(2014)




[Chen et al.2023]chen2023deep

Chen, W.,
Wang, Q.,
Yan, L.,
Hu, G.,
Noack, B.R.:
Deep reinforcement learning-based active flow control of vortex-induced vibration of a square cylinder.
Physics of Fluids
35(5)
(2023)




[Degrave et al.2022]degrave2022magnetic

Degrave, J.,
Felici, F.,
Buchli, J.,
Neunert, M.,
Tracey, B.,
Carpanese, F.,
Ewalds, T.,
Hafner, R.,
Abdolmaleki, A.,
Las Casas, D., :
Magnetic control of tokamak plasmas through deep reinforcement learning.
Nature
602(7897),
414–419
(2022)




[Drazin and Reid2004]Drazin_Reid_2004

Drazin, P.G.,
Reid, W.H.:
Thermal Instability.
Cambridge Mathematical Library,
pp. 32–68.
Cambridge University Press,
London
(2004)




[Font et al.2024]font2024active

Font, B.,
Alcántara-Ávila, F.,
Rabault, J.,
Vinuesa, R.,
Lehmkuhl, O.:
Active flow control of a turbulent separation bubble through deep reinforcement learning.
In: Journal of Physics: Conference Series,
vol. 2753,
p. 012022
(2024).
IOP Publishing




[Fan et al.2020]fan2020reinforcement

Fan, D.,
Yang, L.,
Wang, Z.,
Triantafyllou, M.S.,
Karniadakis, G.E.:
Reinforcement learning for bluff body active flow control in experiments and simulations.
Proceedings of the National Academy of Sciences
117(42),
26091–26098
(2020)




[Getling1998]get98

Getling, A.V.:
Rayleigh-benard Convection: Structures And Dynamics.
World Scientific Publishing Co Pte Ltd,
London
(1998)




[Guadarrama et al.2018]TFAgents

Guadarrama, S.,
Korattikara, A.,
Ramirez, O.,
Castro, P.,
Holly, E.,
Fishman, S.,
Wang, K.,
Gonina, E.,
Wu, N.,
Kokiopoulou, E.,
Sbaiz, L.,
Smith, J.,
Bartók, G.,
Berent, J.,
Harris, C.,
Vanhoucke, V.,
Brevdo, E.:
TF-Agents: A library for Reinforcement Learning in TensorFlow
(2018).
<https://github.com/tensorflow/agents>




[Guastoni et al.2023]Guastoni2023deep

Guastoni, L.,
Rabault, J.,
Schlatter, P.,
Azizpour, H.,
Vinuesa, R.:
Deep reinforcement learning for turbulent drag reduction in channel flows.
The European Physical Journal E
46(4),
27
(2023)
10.1140/epje/s10189-023-00285-8




[Garnier et al.2021]garnier2021review

Garnier, P.,
Viquerat, J.,
Rabault, J.,
Larcher, A.,
Kuhnle, A.,
Hachem, E.:
A review on deep reinforcement learning for fluid mechanics.
Computers & Fluids
225,
104973
(2021)




[Howle1997]how97a

Howle, L.E.:
Active control of Rayleigh–Bénard convection.
Physics of Fluids
9,
1861
(1997)




[Jasak et al.2007]jasak2007openfoam

Jasak, H.,
Jemcov, A.,
Tukovic, Z., :
Openfoam: A C++ library for complex physics simulations.
In: International Workshop on Coupled Methods in Numerical Dynamics,
vol. 1000,
pp. 1–20
(2007).
Dubrovnik, Croatia)




[Kelly1992]kel92

Kelly, R.E.:
Stabilization of Rayleigh–Bénard convection by means of a slow nonplanar oscillatory flow.
Physics of Fluids A: Fluid Dynamics
4,
647
(1992)




[Kim et al.1987]kim_moin_moser_1987

Kim, J.,
Moin, P.,
Moser, R.:
Turbulence statistics in fully developed channel flow at low Reynolds number.
Journal of Fluid Mechanics
177,
133–166
(1987)
10.1017/S0022112087000892




[Kuhnle et al.2017a]kuhnle2017tensorforce

Kuhnle, A.,
Schaarschmidt, M.,
Fricke, K.:
Tensorforce: a tensorflow library for applied reinforcement learning.
Web page
9
(2017)




[Kuhnle et al.2017b]tensorforce

Kuhnle, A.,
Schaarschmidt, M.,
Fricke, K.:
Tensorforce: a TensorFlow library for applied reinforcement learning.
Web page
(2017).
<https://github.com/tensorforce/tensorforce>




[Li and Zhang2022]li2022reinforcement

Li, J.,
Zhang, M.:
Reinforcement-learning-based control of confined cylinder wakes with stability analyses.
Journal of Fluid Mechanics
932,
44
(2022)




[Mnih et al.2013]mnih2013playing

Mnih, V.,
Kavukcuoglu, K.,
Silver, D.,
Graves, A.,
Antonoglou, I.,
Wierstra, D.,
Riedmiller, M.:
Playing atari with deep reinforcement learning.
arXiv preprint arXiv:1312.5602
(2013)




[Mortensen2018a]mortensen_joss

Mortensen, M.:
Shenfun: High performance spectral Galerkin computing platform.
Journal of Open Source Software
3(31),
1071
(2018)
10.21105/joss.01071




[Mortensen2018b]shenfun

Mortensen, M.:
Shenfun's documentation : https://shenfun.readthedocs.io.
Web page
(2018).
<https://shenfun.readthedocs.io/en/latest/index.html>




[Novati et al.2021]Novati2021

Novati, G.,
Laroussilhe, H.L.,
Koumoutsakos, P.:
Automating turbulence modelling by multi-agent reinforcement learning.
Nature Machine Intelligence
3(1),
87–96
(2021)
10.1038/s42256-020-00272-0




[Or and Speyer2003]or03

Or, A.C.,
Speyer, J.L.:
Active suppression of finite-amplitude Rayleigh–Bénard convection.
Journal of Fluid Mechanics
483,
111–128
(2003)




[Paris et al.2021]paris2021robust

Paris, R.,
Beneddine, S.,
Dandois, J.:
Robust flow control and optimal sensor placement using deep reinforcement learning.
Journal of Fluid Mechanics
913,
25
(2021)




[Paris et al.2023]paris2023reinforcement

Paris, R.,
Beneddine, S.,
Dandois, J.:
Reinforcement-learning-based actuator selection method for active flow control.
Journal of Fluid Mechanics
955,
8
(2023)




[Pino et al.2023]pino2023comparative

Pino, F.,
Schena, L.,
Rabault, J.,
Mendez, M.A.:
Comparative analysis of machine learning methods for active flow control.
Journal of Fluid Mechanics
958,
39
(2023)




[Pandey et al.2018]pan18

Pandey, A.,
Scheel, J.D.,
Schumacher, J.:
Turbulent superstructures in Rayleigh-Bénard convection.
Nature Communications
9,
2118
(2018)




[Rabault and Kuhnle2019]rabault2019accelerating

Rabault, J.,
Kuhnle, A.:
Accelerating deep reinforcement learning strategies of flow control through a multi-environment approach.
Physics of Fluids
31(9),
094105
(2019)




[Rabault et al.2019]rabault2019artificial

Rabault, J.,
Kuchta, M.,
Jensen, A.,
Réglade, U.,
Cerardi, N.:
Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control.
Journal of fluid mechanics
865,
281–302
(2019)




[Ren et al.2021]ren2021applying

Ren, F.,
Rabault, J.,
Tang, H.:
Applying deep reinforcement learning to active flow control in weakly turbulent conditions.
Physics of Fluids
33(3),
037121
(2021)




[Rabault et al.2020]rabault2020deep

Rabault, J.,
Ren, F.,
Zhang, W.,
Tang, H.,
Xu, H.:
Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization.
Journal of Hydrodynamics
32,
234–246
(2020)




[Ren et al.2019]ren2019active

Ren, F.,
Wang, C.,
Tang, H.:
Active control of vortex-induced vibration of a circular cylinder using machine learning.
Physics of Fluids
31(9),
093601
(2019)




[Ren et al.2021]ren2021bluff

Ren, F.,
Wang, C.,
Tang, H.:
Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth.
Physics of Fluids
33(9),
093602
(2021)




[Remillieux et al.2007]rem07

Remillieux, M.C.,
Zhao, H.,
Bau, H.H.:
Suppression of Rayleigh-Bénard convection with proportional-derivative controller.
Physics of Fluids
19,
017102
(2007)




[Ren et al.2024]ren2024enhancing

Ren, F.,
Zhang, F.,
Zhu, Y.,
Wang, Z.,
Zhao, F.:
Enhancing heat transfer from a circular cylinder undergoing vortex induced vibration based on reinforcement learning.
Applied Thermal Engineering
236,
121919
(2024)




[Suárez et al.2023]suarez2023afc

Suárez, P.,
Alcántara-Ávila, F.,
Miró, A.,
Rabault, J.,
Font, B.,
Lehmkuhl, O.,
Vinuesa, R.:
Active flow control for three-dimensional cylinders through deep reinforcement learning
(2023)
https://arxiv.org/abs/2309.02462arXiv:2309.02462




[Singer and Bau1991]sin91

Singer, J.,
Bau, H.H.:
Active control of convection.
Physics of Fluids A: Fluid Dynamics
3,
2859
(1991)




[Sutton and Barto2018]sutton2018reinforcement

Sutton, R.S.,
Barto, A.G.:
Reinforcement Learning: An Introduction (second Edition).
MIT press,
Cambridge, Massachussets
(2018)




[Swaminathan et al.2018]swa18

Swaminathan, A.,
Garrett, S.L.,
Poese, M.E.,
Smith, R.W.M.:
Dynamic stabilization of the Rayleigh-Bénard instability by acceleration modulation.
The Journal of the Acoustical Society of America
144,
2334
(2018)




[Silver et al.2018]silver2018alphago

Silver, D.,
Hubert, T.,
Schrittwieser, J.,
Antonoglou, I.,
Lai, M.,
Guez, A.,
Lanctot, M.,
Sifre, L.,
Kumaran, D.,
Graepel, T., :
A general reinforcement learning algorithm that masters chess, shogi, and go through self-play.
Science
362(6419),
1140–1144
(2018)




[Silver et al.2014]silver2014deterministic

Silver, D.,
Lever, G.,
Heess, N.,
Degris, T.,
Wierstra, D.,
Riedmiller, M.:
Deterministic policy gradient algorithms.
In: International Conference on Machine Learning,
pp. 387–395
(2014)




[Sonoda et al.2022]sonoda2022reinforcement

Sonoda, T.,
Liu, Z.,
Itoh, T.,
Hasegawa, Y.:
Reinforcement learning of control strategies for reducing skin friction drag in a fully developed channel flow.
arXiv preprint arXiv:2206.15355
(2022)




[Shen et al.2011]shenbook

Shen, J.,
Tang, T.,
Wang, L.-L.:
Spectral Methods - Algorithms, Analysis and Applications.
Springer,
Heidelberg
(2011)




[Schulman et al.2017]schulman2017proximal

Schulman, J.,
Wolski, F.,
Dhariwal, P.,
Radford, A.,
Klimov, O.:
Proximal policy optimization algorithms.
arXiv preprint arXiv:1707.06347
(2017)




[Suárez et al.2024]suarez2024active

Suárez, P.,
Álcantara-Ávila, F.,
Miró, A.,
Rabault, J.,
Font, B.,
Lehmkuhl, O.,
Vinuesa, R.:
Active flow control for drag reduction through multi-agent reinforcement learning on a turbulent cylinder at Re_D= 3900
(2024).
<https://arxiv.org/abs/2405.17655>




[Tang and Bau1993]tan93a

Tang, J.,
Bau, H.H.:
Stabilization of the no-motion state in Rayleigh-bénard convection through the use of feedback control.
Phys. Rev. Lett.
70,
1795–1798
(1993)
10.1103/PhysRevLett.70.1795




[Tang et al.2020]tang2020robust

Tang, H.,
Rabault, J.,
Kuhnle, A.,
Wang, Y.,
Wang, T.:
Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning.
Physics of Fluids
32(5),
053605
(2020)




[Vinyals et al.2019]vinyals2019grandmaster

Vinyals, O.,
Babuschkin, I.,
Czarnecki, W.M.,
Mathieu, M.,
Dudzik, A.,
Chung, J.,
Choi, D.H.,
Powell, R.,
Ewalds, T.,
Georgiev, P., :
Grandmaster level in starcraft II using multi-agent reinforcement learning.
Nature
575(7782),
350–354
(2019)




[Van Hasselt et al.2016]van2016deep

Van Hasselt, H.,
Guez, A.,
Silver, D.:
Deep reinforcement learning with double q-learning.
In: Proceedings of the AAAI Conference on Artificial Intelligence,
vol. 30
(2016)




[Vinuesa et al.2022]vinuesa2022flow

Vinuesa, R.,
Lehmkuhl, O.,
Lozano-Durán, A.,
Rabault, J.:
Flow control in wings and discovery of novel approaches via deep reinforcement learning.
Fluids
7(2),
62
(2022)




[Verma et al.2018]VermaEtAl2018

Verma, S.,
Novati, G.,
Koumoutsakos, P.:
Efficient collective swimming by harnessing vortices through deep reinforcement learning.
Proceedings of the National Academy of Sciences of the United States of America
115(23),
5849–5854
(2018)
10.1073/pnas.1800923115 .
Accessed 2023-01-03




[Vignon et al.2023]vignon2023review

Vignon, C.,
Rabault, J.,
Vinuesa, R.:
Recent advances in applying deep reinforcement learning for flow control: Perspectives and future directions.
Physics of Fluids
35(3),
031301
(2023)
10.1063/5.0143913




[Vignon et al.2023]vignon2023marlrbc

Vignon, C.,
Rabault, J.,
Vasanth, J.,
Alcántara-Ávila, F.,
Mortensen, M.,
Vinuesa, R.:
Effective control of two-dimensional Rayleigh–Bénard convection: Invariant multi-agent reinforcement learning is all you need.
Physics of Fluids
35(6),
065146
(2023)
10.1063/5.0153181




[Varela et al.2022]varela2022deep

Varela, P.,
Suárez, P.,
Alcántara-Ávila, F.,
Miró, A.,
Rabault, J.,
Font, B.,
García-Cuevas, L.M.,
Lehmkuhl, O.,
Vinuesa, R.:
Deep reinforcement learning for flow control exploits different physics for increasing Reynolds number regimes.
In: Actuators,
vol. 11,
p. 359
(2022).
MDPI




[Wang et al.1992]wan92

Wang, Y.Z.,
Singer, J.,
Bau, H.H.:
Controlling chaos in a thermal convection loop.
J. Fluid Mech.
237,
479
(1992)




[Wang et al.2022]wang2022drlinfluids

Wang, Q.,
Yan, L.,
Hu, G.,
Li, C.,
Xiao, Y.,
Xiong, H.,
Rabault, J.,
Noack, B.R.:
Drlinfluids: An open-source python platform of coupling deep reinforcement learning and openfoam.
Physics of Fluids
34(8)
(2022)




[Wang et al.2024]wang2024dynamic

Wang, Q.,
Yan, L.,
Hu, G.,
Chen, W.,
Rabault, J.,
Noack, B.R.:
Dynamic feature-based deep reinforcement learning for flow control of circular cylinder with sparse surface pressure sensing.
Journal of Fluid Mechanics
988,
4
(2024)




[Xu and Zhang2023]xu2023reinforcement

Xu, D.,
Zhang, M.:
Reinforcement-learning-based control of convectively unstable flows.
Journal of Fluid Mechanics
954,
37
(2023)




[Xu et al.2020]xu2020active

Xu, H.,
Zhang, W.,
Deng, J.,
Rabault, J.:
Active flow control with rotating cylinders by an artificial neural network trained by deep reinforcement learning.
Journal of Hydrodynamics
32(2),
254–258
(2020)




[Yan et al.2023]yan2023stabilizing

Yan, L.,
Li, Y.,
Hu, G.,
Chen, W.-l.,
Zhong, W.,
Noack, B.R.:
Stabilizing the square cylinder wake using deep reinforcement learning for different jet locations.
Physics of Fluids
35(11)
(2023)









 Paper Title: A phase-field model for wet snow metamorphism.





§ ABSTRACT 
The microstructure of snow determines its fundamental properties such as the mechanical strength, reflectivity, or the thermo-hydraulic properties.
Snow undergoes continuous microstructural changes due to local gradients in temperature, humidity or curvature, in a process known as snow metamorphism.
In this work, we focus on wet snow metamorphism, which occurs when temperature is close to the melting point and involves phase transitions amongst liquid water, water vapor, and solid ice. 
We propose a pore-scale phase-field model that simultaneously captures the three relevant phase-change phenomena: sublimation (deposition), evaporation (condensation), and melting (solidification).
The phase-field formulation allows one to track the temperature evolution amongst the three phases and the water vapor concentration in the air. 
Our three-phase model recovers the corresponding two-phase transition model when one phase is not present in the system. 2D simulations of the model unveils the impact of humidity and temperature on the dynamics of wet snow metamorphism at the pore scale. We also explore the role of liquid melt content in controlling the dynamics of snow metamorphism in contrast to the dry regime, before percolation onsets. 
The model can be readily extended to incorporate two-phase flow and may be the basis for investigating other problems involving water phase transitions in a vapor-solid-liquid system such as airplane icing or thermal spray coating.








§ INTRODUCTION

Snow and firn are heterogenous porous material composed of two components, water and air, distributed amongst three phases: solid ice, liquid water, and air with water vapor. The microstructure of snow and firn dictates their mechanical strength, reflectivity, and thermo-hydraulic properties, which govern important processes such as snow avalanches, snowpack and glacial hydrology, radar remote sensing, and the performance of snow vehicles <cit.>.

However, these properties constantly evolve because snow is a thermodynamically active material that undergoes continuous microstructural changes caused by phase transitions between ice, liquid water, and vapor, in a process known as snow metamorphism <cit.>. Ultimately, snow metamorphism leads to a denser snow composed of coarser and rounder ice grains. Based on whether liquid water is present, snow metamorphism is considered in two types —dry and wet. Colbeck and other authors published a series of pioneer papers in the 70's and 80's <cit.> that derived the theoretical foundations to understand the mechanisms driving each type of snow metamorphism.

Dry snow metamorphism <cit.> results from sublimation and deposition and is driven by the transport of water vapor between regions with different vapor pressure, i.e., regions with different temperature and/or grain curvature, according to the Gibbs-Thomson condition <cit.>. 
On the other side, wet snow metamorphism occurs when the temperature is close to the melting point. The presence of liquid water fundamentally alters the mechanisms of metamorphism by increasing the thermodynamic activity of snow and strengthening the mechanical and thermal connection between ice grains through capillary bridging <cit.>.
Thus, compared to dry metamorphism, wet snow metamorphism displays accelerated coarsening and densification.

Colbeck proposed two different regimes for wet metamorphism depending on the liquid water content: the pendular and the funicular regimes <cit.>. 
In the funicular regime, the liquid water content is high so that the water completely surrounds the ice grains.
In the pendular regime, the liquid water content is low and most of the ice surface is in contact with air.
Thus, the pendular regime displays lower heat flow, grain growth, and snow densification, and larger capillary forces and grain-to-grain mechanical strength, compared to the funicular regime.
In both cases, vapor kinetics plays an important role <cit.>. While Colbeck's work provided mechanistic understanding of snow metamorphism,  it focused on theoretical snow configurations under ideal environmental conditions.

Subsequent experimental works used X-ray microtomography (microCT) to study the evolution of the dry snow microstructure in realistic scenarios under isothermal  conditions <cit.> and temperature gradients <cit.>. Many authors leveraged the microCT data to perform a comprehensive analysis of local crystal growth laws <cit.> and validate mean field models <cit.>.

The first experiments of wet metamorphism focused on fully saturated snow <cit.>.
Improved experimental techniques enable studies of wet metamorphism on samples with low liquid water content <cit.>. These works analyzed snow wetness and grain size and derived empirical laws for the snow properties under wet conditions <cit.>. However, more advanced techniques are still being developed to robustly visualize liquid water within snow samples, such as the use of hyperspectral imaging to map out liquid water content in snow <cit.>.


Mathematical models may complement experimental development to provide insights into the wet snow metamorphism process. To this end, wet snow is modeled as a three-phase system composed of liquid water, vapor and solid ice, and thus can be grouped with other class problems such as snowflake growth and water drop icing in a humidity-controlled atmosphere. In this broader group of literature, some have proposed simplified 1D model to describe the role of humidity on the evolution dynamics <cit.>.
More elaborated 2D continuum models resorted to the phase-field method <cit.> or the level-set method <cit.>, which have been successful in reproducing the phase transitions between two phases of water, i.e., solidification <cit.>, sublimation <cit.>, and evaporation <cit.>. However, most of these continuum models are limited to the two-phase regime, and do not simultaneously capture the transitions amongst the three water phases; thus, these models do not address the wet snow metamorphism problem.


Here, we propose a phase-field model that describes the non-equilibrium evolution of the three-phase mixture composed of liquid water, water vapor and solid ice.
The modeling choice is inspired by recent applications of phase-field methods in porous media problems with phase transitions, including the modeling of microstructural evolution of porous materials and multicomponent fluid mixtures <cit.>, phase transitions during two-phase flow in porous media<cit.> and crystallization flow in porous media <cit.>. The three-phase model proposed in this work reduces to the classic two-phase model in the absence of a third phase, and the resulting two-phase models readily capture the Gibbs-Thomson conditions. 
The model allows us to analyze the pore-scale dynamics of that dictates wet snow metamorphism.  Currently, the model does not account for granular compaction, melt film imbibition or melt percolation. Rather, the primary focus of the model is to elucidate the role of thermodynamic-driven phase transitions on microstructure evolution of a quiescent wet snow mixture. 



§ A PHASE-FIELD MODEL FOR WET SNOW MIXTURE 


The phase-field method <cit.> is a mathematical technique that is based on a free energy description of multi-phase mixtures, and is well-suited for problems with moving interfaces. Within the phase-field framework, one defines a dimensionless phase field variable, denoted ϕ_l ∈ [0,1], which represents the volume fraction of the phase l at any given point in the domain. The phase variable smoothly transitions across the interface between two distinct phases (Fig. <ref>). With the appropriate evolution equations, the moving boundary problem is reformulated in a fixed domain, which avoids most of the numerical issues caused by the moving interfaces.

Here, we propose a non-variational phase-field model that captures the phase transitions between ice, liquid water, and water vapor, i.e., solidification, sublimation, evaporation, and the opposite transitions. For simplicity, from here on we denote the liquid water as water and the water vapor as vapor. We assume that the air is at the atmospheric pressure. We assume that ice and water have equal density for the phase evolution equations, which introduces small mass conservation error. Although fluid flow is not considered in the current model, future work that incorporates fluid flow and unequal density may mitigate the mass conservation issue.

We note that in the spirit of numerical robustness and physical consistency, a  variational phase-field formulation <cit.> is often more desirable to ensure non-negative entropy production and the convergence towards equilibrium <cit.>. However, due to the close proximity in the density of the ice and liquid water phase, we find that formulating a free energy for the wet snow system based on the density field introduces numerical singularities in the formulation. 








 §.§ Model variables

Our model unknowns are the phase field variables for ice ϕ_i(,t), water ϕ_w(,t), and air ϕ_a(,t) phases (see Fig. <ref>), as well as variables for the temperature T(,t) (units ^∘C) and the vapor density ρ_v(,t) (units kg/m^3). All variables are defined point-wise in the problem domain Ω (Fig. <ref>). 
Note that even though temperature and vapor density are not phase field variables, here we leverage the formulation to specify phase-dependent temperature evolution (see Section <ref>), and to localize the vapor dynamics onto the air phase (see Section <ref>). 







 §.§ Model equations


In this section, we will introduce the five partial differential equations that describe the co-evolution of our model variables.



  §.§.§ The phase evolution equations


The basic form of the phase evolution equations for the three phase field variables are expressed as:


    ∂ϕ_i/∂ t =  -M_0/Σ_iδℱ^tri(ϕ)/δϕ_i  - α_solϕ_i^2ϕ_w^2T-T_melt/L_sol/c_p,w + α_subϕ_i^2ϕ_a^2ρ_v-ρ_vs^I(T)/ρ_i +θ_nucl(ϕ,T), 
    
            ∂ϕ_w/∂ t = -M_0/Σ_wδℱ^tri(ϕ)/δϕ_w + α_solϕ_i^2ϕ_w^2T-T_melt/L_sol/c_p,w +  α_evaϕ_w^2ϕ_a^2ρ_v-ρ_vs^W(T)/ρ_w -θ_nucl(ϕ,T), 
    
            ∂ϕ_a/∂ t = -M_0/Σ_aδℱ^tri(ϕ)/δϕ_a - α_subϕ_i^2ϕ_a^2ρ_v-ρ_vs^I(T)/ρ_i - α_evaϕ_w^2ϕ_a^2ρ_v-ρ_vs^W(T)/ρ_w,


where δ·/δ· denotes the variational derivative and we use the notation ϕ={ϕ_i,ϕ_w,ϕ_a}. Note that Eqs.(<ref>)-(<ref>) account for total mass conservation by assuming equal density of ice and water (ρ_i=ρ_w).


The first terms in the right-hand side of these three equations capture the Allen-Cahn kinetics of phase-field variables <cit.>, where M_0 is the phase-dependent mobility defined below in Sect. <ref> and  ℱ^tri is the classical energy functional of a ternary mixture <cit.>:


    ℱ^tri(ϕ)=∫( 3/ε F^tri(ϕ) + 3/2ε[Σ_i(ϕ_i)^2 +Σ_w(ϕ_w)^2 +Σ_a(ϕ_a)^2 ] +β(1-ϕ_i-ϕ_w-ϕ_a) )Ω.


The parameter ε represents the width of the phase-field interface and β is the Lagrange multiplier (see Section <ref>). The function F^tri is the triple-well potential, defined as: 


    F^tri(ϕ)=1/2Σ_iϕ_i^2(1-ϕ_i)^2+1/2Σ_wϕ_w^2(1-ϕ_w)^2+1/2Σ_aϕ_a^2(1-ϕ_a)^2 +Λϕ_i^2ϕ_w^2ϕ_a^2,


where the Λ-term improves the dynamic consistency of the model<cit.> to mitigate spurious effect along phase-phase boundaries. 
The parameters Σ_i, Σ_w, and Σ_a are related to the surface tensions (σ_lm) between phases l and m such that 


    Σ_i = σ_iw +σ_ia -σ_wa,  Σ_w = σ_iw +σ_wa -σ_ia,   Σ_a = σ_wa +σ_ia -σ_iw.








We extend the Allen-Cahn kinetics with the α-terms in Eqs. (<ref>) to (<ref>), which account for the different phase transitions. In particular, the α_sol term accounts for solidification or melting, the α_sub term accounts for sublimation or deposition, and the α_eva term accounts for evaporation or condensation. 

The estimation of α's parameter values is described in the Supporting Information (Section <ref> and Table <ref>).

The parameters L_sol, c_p,w, ρ_w and ρ_i are the solidification latent heat, water specific heat capacity, water density, and ice density, respectively.

We follow <cit.> and express the saturated vapor density of ice (ρ_vs^I) and water (ρ_vs^W) as

    ρ_vs^J(T)=ρ_a 0.62P_vs^J(T)/P_a-P_vs^J(T),


where the index J stands for I and W, ρ_a is the dry air density, P_a is the atmospheric pressure (units Pa), and P_vs^J is the saturated vapor pressure of phase J (units Pa). The saturated vapor pressure of ice and water are defined as <cit.>


    P_vs^I(T)=exp( ∑_j=0^4 k_j(T+T_0)^j-1 + k_5 ln(T+T_0)),
       P_vs^W(T)=exp( ∑_j=0^6 g_j(T+T_0)^j-2 + g_7 ln(T+T_0)),


respectively, where T_0=273.15, and the values of the parameters k_j and g_j are listed in Table <ref> in the Supporting Information.




The θ_nucl terms in Eqs. (<ref>)–(<ref>) account for ice and water nucleation, which are necessary in order to trigger the nucleation of a third phase when only two phases exist initially. In particular, here we account for the nucleation of water when T increases above T_melt in an ice-air system and the nucleation of ice when T decreases below the ice nucleation temperature T_nucl in a water-air system.

We propose a phenomenological nucleation term

    θ_nucl(ϕ,T) = α_nuclM_0ϕ_a^2 [ϕ_w^2 N_i(T)-ϕ_i^2 N_w(T)],


where α_nucl is a parameter controlling the strength of the nucleation and the ice and water nucleation functions N_i(T) and N_w(T), respectively, are expressed as

    N_i(T) = 0.5-0.5 tanh[20(T-T_nucl+0.1)],     N_w(T) = 0.5+0.5 tanh[20(T-T_melt-0.1)].

Note that N_i becomes active when T<T_nucl, which allows the presence of undercooled water in the range T_nucl<T<T_melt.




According to Eq. (<ref>), ice nucleation occurs on the water-air  interface when T<T_nucl (note the term ϕ_a^2ϕ_w^2 multiplying N_i).

Once ice nucleation advances, the water-air interface splits into the water-ice and ice-air interfaces, resulting in ϕ_a^2ϕ_w^2=0 and thus θ_nucl stays zero even if T<T_nucl.
Water nucleation exhibits an analogous behavior.

Note that θ_nucl only needs to be active until the system enters into the spinodal decomposition region of the ternary mixture.







  §.§.§ Energy conservation equation
 

Here, we formulate conservation of energy in terms of temperature, expressed as


    ρ(ϕ)c_p(ϕ)∂ T/∂ t = ÷[ K(ϕ) T ] + ρ(ϕ)L_sol(∂ϕ_a/∂ t + ∂ϕ_i/∂ t) -ρ(ϕ)L_sub∂ϕ_a/∂ t.


Eq. (<ref>) accounts for thermal diffusion and the latent heat released and/or absorbed during the different phase transitions. L_sub is the sublimation latent heat. The latent heat of evaporation is then captured through L_eva=L_sub-L_sol. The phase-dependent density (ρ), specific heat capacity (c_p), and thermal conductivity (K) are defined as:

    ρ(ϕ)=∑_l=i,w,aϕ_lρ_l;    
        c_p(ϕ)=∑_l=i,w,aϕ_l c_p,l;   
        K(ϕ)=∑_l=i,w,aϕ_l K_l,

where ρ_l, c_p,l, and K_l are the density, specific heat capacity, and thermal conductivity, respectively, of phase l. We assume that ρ_l, c_p,l, and K_l are constant in time. Note that here we violate our primary assumption that ρ_i=ρ_w used in Eqs. (<ref>)-(<ref>), which incurs mass conservation error; instead, we consider different density for the ice and water phase for the energy equation in order to adopt the correct heat capacity for each phase. Although doing so does not eliminate errors in energy conservation due to errors in mass conservation, it improves the thermal consistency of the model.












  §.§.§ Mass conservation of vapor
 

The vapor mass conservation equation is expressed as 


    ∂(ϕ_aρ_v)/∂ t=÷[ ϕ_a D_v(T)ρ_v ] + ρ_SE∂ϕ_a/∂ t,


which accounts for vapor diffusion in the air and mass transfer of water during sublimation and evaporation (and the opposite transitions). 
Here, we formulate a phase-dependent density ρ_SE(ϕ) such that ρ_SE=ρ_i in case of sublimation and ρ_SE=ρ_w in case of evaporation. The specific form for ρ_SE(ϕ) is discussed and provided in Eq. (<ref>) of Section <ref>. 

D_v is the vapor diffusion coefficient, whose dependence on T (in ) follows the expression: <cit.>


    D_v(T)=D_v0(T+T_0/T_0)^1.81,

where D_v0 is the vapor diffusion coefficient at the freezing point and T_0=273.15.






 §.§ Phase-dependent functions
 
The model equations introduced above rely on a few parameters that are phase-dependent. For material property parameters such as ρ, c_p, K, we simply use phase-weighted averages as shown in Eq. (<ref>). However, parameters such as the mobility function M_0 in Eqs. (<ref>)–(<ref>) and ρ_SE in Eq. (<ref>) are related to phase change dynamics at phase-phase boundaries or triple junction regions, and thus require more careful mathematical formulation. Here we resort to the ternary diagram to illustrate these modeling choices.

A ternary diagram is the common tool to represent three-phase systems. 
In a ternary diagram, the vertex l represents the point ϕ_l=1, while the opposite side of that vertex represents the points ϕ_l=0 (for l={i,w,a}). The interior of the diagram represents points where the three phases coexist. Thus, any valid {ϕ_i,ϕ_w,ϕ_a} configuration is represented by a point in the ternary diagram. In particular, evaporation takes place on the side ϕ_i=0, sublimation on the side ϕ_w=0, and solidification on the side ϕ_a=0.

At the boundaries of the ternary diagram (i.e. ϕ_a=0, ϕ_w=0, and ϕ_i=0), the mobility function M_0 must take the values M_sol, M_sub, and M_eva that correspond to the kinetics of solidification, sublimation and evaporation, respectively. The exact expressions for these parameters are provided in Eqs. (<ref>), (<ref>), and (<ref>) of the Supporting Information. 

Within the interior region of the ternary diagram, however, M_0 needs to be interpolated based on M_sol, M_sub, and M_eva. A common choice is the cubic interpolation, which presents issues for two reasons<cit.>: (1) the triple junction is a region, not a point, where the three phases coexist; (2) the values of M_sol, M_sub, and M_eva range several orders of magnitude in this region (see Table <ref> in the Supporting Information).
Thus, a cubic interpolation of M_0 would result in large gradients in mobility within the interior of the ternary diagram, leading to severe numerical challenges. 
In addition, a non-unique value of M_0 within the triple junction is not physically consistent and result in unrealistic behavior of the triple junction.

To avoid these two issues, we propose a piecewise constant interpolation of M_0, similar to that proposed in <cit.>, where M_0 takes a constant value near each of the three sides and in the interior region of the ternary diagram:


    M_0(ϕ)={[                                      M_sol if ϕ_a<=ϕ_i,  ϕ_a<=ϕ_w,   and  ϕ_a < 0.01,;                                      M_eva  if ϕ_i<ϕ_a,  ϕ_i<=ϕ_w,   and  ϕ_i < 0.01,;                                      M_sub   if ϕ_w<ϕ_a,  ϕ_w<ϕ_i,   and  ϕ_w < 0.01,;                                       M_av                                 otherwise, ].


where we take M_av as the geometric mean (i.e., M_av=√(M_subM_solM_eva)). We find that Eq. (<ref>), compared to a cubic interpolation, improves the numerical efficiency and allows us to simulate triple junctions while respecting the actual kinetics of the different phase transitions. 

The density ρ_SE must take the value ρ_i in case of sublimation and ρ_w in case of evaporation (i.e., ϕ_w=0 and  ϕ_i=0, respectively, in the ternary diagram). The value of ρ_SE along the side ϕ_a=0 is irrelevant because ρ_SE∂ϕ_a/∂ t (see Eq. (<ref>)) is zero on that side. We propose a cubic interpolation between the sides ϕ_w=0 and ϕ_i=0 in the ternary diagram, such that the derivatives of ρ_SE(ϕ) are zero on the sides ϕ_w=0 and ϕ_i=0. We define the function ρ_SE as


    ρ_SE(ϕ)={[ ρ_i+ρ_w/2+ρ_w-ρ_i/4x_SE(3-x_SE^2)                        if ϕ_a≠ 1,;                         ρ_i+ρ_w/2                       if ϕ_a = 1, ]. 
          with   x_SE=1-ϕ_a-2ϕ_i/1-ϕ_a.


Equation (<ref>) displays a discontinuity at ϕ_a=1. From a physical and numerical point of view, this discontinuity does not represent an issue because the term ρ_SE∂ϕ_a/∂ t is  zero if ϕ_a=1.
Nevertheless, we implement a regularization of Eq. (<ref>) (see Section <ref> in the Supporting Information).



 §.§ The Gibbs-Thomson condition

The Gibbs-Thomson effect describes the deviations in the equilibrium chemical potential at a two-phase boundary due to the curvature of the interface. Such effect readily applies to the three types of interface considered in wet snow and influences both the kinetics and equilibrium conditions experienced by curved interfaces. 
In Section <ref> of the Supporting Information, we show that our model is equivalent to the two-phase models for solidification, sublimation, and evaporation when only two phases are present, and the resulting reduced model equations recover the Gibbs-Thomson for two-phase boundaries under certain conditions. In particular, our model reproduces, under certain parameter regimes, the kinetics defined by the Gibbs-Thomson condition for ice-liquid interfaces:

    T-T_melt/L_sol/c_p,w=-d_solχ-β_solv_n,

where T is the interface temperature, β_sol is the kinetic attachment coefficient, d_sol is the capillary length, χ is the curvature of the interface (positive for spherical ice grains), and v_n is the normal velocity of the interface (positive for ice growth).
Note that, at thermodynamic equilibrium (i.e., v_n=0), Eq. (<ref>) simplifies to the classic Gibbs-Thomson equation:

    T^equil=T_melt-L_sold_sol/c_p,wχ,

which states that the actual equilibrium temperature for solidification is lower than T_melt=0^∘C for spherical ice grains. Similarly, our model captures, under certain parameter regimes, the kinetics defined by the Gibbs-Thomson condition for air-ice interfaces:

    ρ_v-ρ_vs^I(T)/ρ_vs^I(T)=d_subχ+β_subv_n,

and for air-liquid interfaces:

    ρ_v-ρ_vs^W(T)/ρ_vs^W(T)=d_evaχ+β_evav_n,

where β_sub and d_sub are the Gibbs-Thomson coefficients for sublimation and β_eva and d_eva are the Gibbs-Thomson coefficients for evaporation. The model's versatility in capturing the Gibbs-Thomson conditions for all three types of two-phase boundaries rely on the phase-dependent formulation of M_0 as discussed in Section <ref>.

Note that because we do not assume equal Σ_α's, spurious phase (nonphysical third phase) may appear along phase-phase boundaries, leading to inaccuracies in the equilibrium interface solution. We mitigate this issue using the Λ-term in the free energy formulation (Eq. (<ref>)). In addition, we note that the Gibbs-Thomson condition is only recovered in phase-field models under certain parameter conditions <cit.> or mathematical constructions through asymptotic analysis <cit.>. Our current formulation does not entail the mathematical additions needed to recover the Gibbs-Thomson condition for all interfaces for all parameter regimes; however, under certain parameter regimes, the model recovers these conditions. More details are provided in Section <ref> and Supplementary Information.



§ NUMERICAL IMPLEMENTATION
 

We first apply the phase constraint ϕ_w=1-ϕ_i-ϕ_a to Eqs. (<ref>)–(<ref>), which allows us to compute the Lagrange multiplier β in Eq. (<ref>) and eliminate one phase evolution equation:

    β = 1/1/Σ_a+1/Σ_w+1/Σ_i3/ε( 1/Σ_i∂ F^tri/∂ϕ_i + 1/Σ_w∂ F^tri/∂ϕ_w + 1/Σ_a∂ F^tri/∂ϕ_a)

The resulting system of four coupled partial differential equations are:


    ∂ϕ_i/∂ t       =         - 3 M_0(ϕ)/εΣ_T[ (Σ_w+Σ_a)∂ F^tri/∂ϕ_i -Σ_a∂ F^tri/∂ϕ_w - Σ_w∂ F^tri/∂ϕ_a] + 3M_0(ϕ)ε∇^2ϕ_i 
                - α_solϕ_i^2ϕ_w^2T-T_melt/L_sol/c_p,w + α_subϕ_i^2ϕ_a^2ρ_v-ρ_vs^I(T)/ρ_i
               +α_nuclM_0(ϕ)ϕ_a^2 [ϕ_w^2 N_i(T)-ϕ_i^2 N_w(T)],
    ∂ϕ_a/∂ t      =        - 3 M_0(ϕ)/εΣ_T[ -Σ_w∂ F^tri/∂ϕ_i -Σ_i∂ F^tri/∂ϕ_w + (Σ_i+Σ_w)∂ F^tri/∂ϕ_a] + 3M_0(ϕ)ε∇^2ϕ_a 
                - α_subϕ_i^2ϕ_a^2ρ_v-ρ_vs^I(T)/ρ_i - α_evaϕ_w^2ϕ_a^2ρ_v-ρ_vs^W(T)/ρ_w, 
    1/ξ_Tρ(ϕ)c_p(ϕ)∂ T/∂ t      =      ÷[ K(ϕ) T ] + ρ(ϕ)L_sol(∂ϕ_a/∂ t + ∂ϕ_i/∂ t) -ρ(ϕ)L_sub∂ϕ_a/∂ t,
    1/ξ_v∂(ϕ_aρ_v)/∂ t      =      ÷[ ϕ_a D_v(T)ρ_v ] + ρ_SE(ϕ) ∂ϕ_a/∂ t,


where F^tri, ρ_SE and M_0 are defined in Eqs. (<ref>), (<ref>), and (<ref>), respectively. 
The saturated vapor densities ρ^I_vs and ρ^W_vs are defined in Eqs. (<ref>)–(<ref>).
The parameter Σ_T=Σ_iΣ_w+Σ_iΣ_a+Σ_wΣ_a, where Σ_l's are defined in Eq. (<ref>).
The nucleation functions N_i and N_w are defined in Eq. (<ref>).
The phase-dependent functions ρ, c_p, and K are defined through Eq. (<ref>) and D_v follows Eq. (<ref>). The time scaling parameters ξ_T and ξ_v are introduced in the next section (Section <ref>).
The rest of the model parameters are listed in Tables <ref> and <ref> in the Supporting Information.




 §.§ Temporal scaling

Wet snow metamorphism involves multiple coupled processes at different time scales. Assuming a characteristic length scale of L=10^-4m, the characteristic times for different processes can range from ∼ 10^-4 s for vapor diffusion to ∼ 10^4 s for sublimation (see Section <ref> in the Supporting Information for detailed calculations). 
A monolithic numerical solver requires the use of time steps that resolve the fastest process (i.e., vapor diffusion at ∼ 10^-4 s ), which would significantly increase the computational cost. Here, we leverage the procedure explained in <cit.> to speed up the simulations. This approach leverages the fact that T and ρ_v are quasi-steady compared to the phase transition kinetics. 
The procedure consists of multiplying the right-hand side of the T and ρ_v equations (Eqs. (<ref>) and (<ref>), respectively) by a time-scale factor (ξ_T and ξ_v respectively), so that the process of vapor and thermal diffusion is numerically slowed down by orders of magnitude while still maintaining a quasi-steady state in T and ρ_v. 
This approach enables the use of larger time steps without introducing noticeable errors in the simulations. In this work, we take ξ_v=10^-3 and ξ_T=1 or ξ_T=10^-2 depending on whether or not solidification occurs in any parts of the domain.
More details on this procedure are described in Section <ref> of the Supporting Information.





 §.§ Kinetic parameters constraints

In order to accurately capture the interface kinetics set by the Gibbs-Thomson conditions, certain model parameters need to be constrained by the interface physics. To this end, the relations between the Gibbs-Thomson parameters (β_j,d_j) and our model parameters (M_j,α_j) (for j={sol,sub,eva}; see Section <ref>) can be formulated from asymptotic analysis <cit.> but are only valid under certain conditions. In particular, we note that these asymptotic analysis do not readily apply to the wet snow problem as they are originally developed for compositional problems with symmetric <cit.> and asymmetric diffusivities <cit.>. Solutions using anti-trapping current have been devised for the thermal problems, but only with symmetric diffusivities <cit.>. Thus, in order to robustly capture the Gibbs-Thomson condition under all parameter regimes for the wet snow problem, further mathematical additions remain to be developed.

We note, however, in the earlier and similar work on dry snow metamorphism by <cit.>, it is shown that if we impose ε<Dβ', where D are the different thermal or vapor diffusivities and β' is the scaled kinetic coefficient, then no anti-trapping current is needed to correctly recover the sharp-interface limit. In particular, for the wet snow problem, this corresponds to the following sets of constraints: 

    ε < β_solK_w/ρ_w c_p,w≈ 10^-5m;  (
    for water-ice interface)
       ε < β_subρ_vs/ρ_iK_a/ρ_a c_p,a≈ 10^-7m;  (
    for air-ice interface)
       ε < β_evaρ_vs/ρ_iK_w/ρ_w c_p,w≈ 10^-8m.  (
    for air-water interface)

These conditions impose a restriction on ε, which represents the diffusive interface width at phase-phase boundaries. The value of ε dictates the spatial discretization of our model, which must be fine enough to resolve the interface width with sufficient numerical elements. Given that the most restrictive conditions of the above requires ε<10^-8 m, this severely restricts the domain size we can simulate. In order to speed up the simulations, we consider a value ε<10^-6 m, which allows us to use a coarser spatial discretization. As a result, our simulation are less accurate in capturing interface kinetics along the air-ice and air-water interfaces. However, considering the phase change kinetics involving the air phase is slow compared to the solidification/melting processes along the water-ice interface, we suspect the overall error on wet snow processes is small. 




 §.§ Numerical methods


We perform both 1D and 2D simulations of our model equations using the finite element method. In particular, we use a uniform mesh composed of bilinear (linear in 1D) basis functions. For the time integration, we use a semi-implicit algorithm based on the generalized-α method <cit.>, where we treat implicitly all the terms except the function M_0 and the parameter ξ_T, which are constant during each time step. We use the Newton-Raphson method with adaptive time stepping scheme to solve the resulting nonlinear system. To avoid singularities and numerical issues we regularize the functions ρ, c_p, K, and ρ_SE (see Section <ref> in the Supporting Information). 


We impose no-flux boundary conditions for the phase variables and Dirichlet boundary condition for the temperature.  The vapor boundary condition is either Dirichlet or no-flux depending on specific case studies. In most simulations, we initialize the vapor density with the saturated vapor density for ice at the prescribed temperature.









§ RESULTS



 §.§ Directional solidification and nucleation of a third phase

In this section, we demonstrate the ability of our model to capture directional solidification as imposed by a temperature gradient, and the nucleation of a third phase in a system that is initially composed of two phases only. In particular, we present three 1D simulations in Fig. <ref> that show (a) the complete freezing of water in an initial ice-water-air system (Fig. <ref>A), (b) water nucleation in an initial ice-air system (Fig. <ref>B), and (c) ice nucleation in an initial water-air system (Fig. <ref>C).  To perform these 1D simulations, we impose fixed temperatures and no-flux boundary condition for the vapor phase on the two ends of the domain, which measures 1 in length and is discretized with 2000 elements.



In the freezing simulation (Fig. <ref>A), the initially imposed temperature gradient results in the directional solidification of water towards the air phase. At each time step, the temperature distribution appears piecewise linear, which confirms the quasi-steady behavior of T as discussed in Section <ref>.
Although the ρ_v dynamics are also quasi-steady, its distribution is not always linear (see t=27 and 27.5 in Fig. <ref>A). This nonlinear response is primarily triggered when a new ice-air interface is established upon complete freezing (∼27.5), and a new equilibrium is being established to satisfy the saturated vapor density in the presence of ice, prescribed by Eq. (<ref>).





The nucleation simulations demonstrate significantly more nonlinear dynamics. In the water nucleation simulation (Fig. <ref>B), we impose 0 and 6 on the left and right boundaries of the domain respectively to induce melting at the initial ice-air interface. In the ice nucleation simulation (Fig. <ref>C), the water-air system is initially at -2 uniformly and we consider the ice nucleation temperature T_nucl=-1. 
Ice nucleation onsets, preferentially towards the water phase, well before the temperature on the water-air interface increases to T_nucl  (Fig. <ref>C, t=0.2). At t=0.3, when T>T_nucl on the interface, the ice phase has fully developed. 
Both types of nucleation described here are highly energetic events, leading to rapid changes in temperature and saturated vapor density at the nucleation region. 
As a result, the T and ρ_v profiles are no longer linear, in contrast to the directional freezing problem (Fig. <ref>A). 
We note that here, nucleation only occurs at the phase-phase interface due to its mathematical formulation in Eq. (<ref>).





We note that the phase field variables maintain the tanh-profile (quasi-equilibrium profile) during the freezing simulation (insets of Fig. <ref>A), but such profiles are temporarily lost during the nucleation simulations (insets Figs. <ref>B, C). 
In the latter cases, the loss of the tanh-profile is due to changes in mobility M_0 (Eq. (<ref>)) during the dynamic nucleation events, and could result in small errors in curvature calculations and thus influence the kinetics of phase change through the Gibbs-Thomson condition. Nevertheless, the tanh-profile is recovered shortly after nucleation (e.g., at t=1 in Figs. <ref>B, C); thus the undesired loss of the tanh-profile is temporary (occurs only during nucleation) and for this reason, we assume its impact on the overall kinetics of the problem is negligible.




 §.§ Simulation  of dry snow metamorphism in 2D

Under thermal conditions when liquid water phase is absent (ϕ_w=0), our model reduces to the two-phase model for dry snow metamorphism  first proposed in <cit.>:

    ∂ϕ_i/∂ t   = -3 M_0( 1/εϕ_i(1-ϕ_i)(1-2ϕ_i)   - ε∇^2ϕ_i ) + α_subϕ_i^2ϕ_a^2ρ_v-ρ_vs^I(T)/ρ_i
    1/ξ_Tρ(ϕ)c_p(ϕ)∂ T/∂ t   = ÷[ K(ϕ) T ]  -ρ(ϕ)L_sub∂ϕ_a/∂ t,
    1/ξ_v∂(ϕ_aρ_v)/∂ t   =÷[ ϕ_a D_v(T)ρ_v ] + ρ_SE∂ϕ_a/∂ t,

with ∂ϕ_a/∂ t=-∂ϕ_i/∂ t. 
Here, we show a 2D example of the dry metamorphism model using a simple snow geometry composed of four circular ice grains in contact (see t_0 in Fig. <ref>A).
The domain measures 0.2×0.2^2 and is meshed with 400× 400 elements.
We impose a fixed temperature T_B=-5 and no-flux of ρ_v on all boundaries of the domain. Over the 90 minute time span of the simulation, we plot the time evolution of the ice phase ϕ_i and vapor concentration ρ_v (Fig. <ref>A), along with the T (Fig. <ref>B) and the total ice interface length S_i (Figs. <ref>C and D).
Also known as coarsening, dry snow metamorphism under isothermal condition is a curvature-driven process, which minimizes the total interfacial length and the curvature of the system. The process penalizes regions of high curvature and is driven by vapor mass transfer between regions of different curvatures <cit.>.
Our simulation readily illustrates the above mechanism. We observe that the ice grains quickly sinter (before t_1=40) and the coarsening process gradually slows down, as measured by the decay in slope in Figs. <ref>C and D. Accompanying these geometric changes are the dynamics in T distribution due to latent heat generation in regions of higher curvature (see Fig. <ref>B). 
The simulation also captures the entrapment of an air bubble during the coarsening process.
At steady state (e.g., t=3 in Fig. <ref>D), the vapor concentration inside the bubble is set by its curvature according to the Gibbs-Thomson condition (Eq. (<ref>)).





 §.§ Simulations of wet snow dynamics in 2D


In this section, we perform a suite of numerical simulations of our full model to investigate wet snow dynamics as it undergoes phase transition. Due to high computational cost of the model, here we only focus on small sized domains (∼mm) in 2D.




  §.§.§ Influence of vapor concentration


In this part, we study the impact of vapor concentration on the kinetics of melt generation when the initially dry material is subjected to melting. To achieve this, we consider the same four-grain geometry as the initial dry snow simulation in Fig. <ref>, we impose a boundary temperature of T_B=1 and run two simulations with different vapor concentrations on the boundary, denoted ρ_v,B. In particular, we impose an undersaturated vapor density, ρ_v,B=0.8ρ^I_vs(1), in the first simulation (Fig. <ref>A) and a saturated vapor density, ρ_v,B=ρ^I_vs(1), in the second one (Fig. <ref>B).

We quantify these two simulations by plotting the time evolution of the spatially averaged temperature T, volume of ice phase V_i and water phase V_w in Figs. <ref>A and C. The results illustrate that, when the initial vapor phase is undersaturated, melt nucleation is delayed and the kinetics of melt generation is slower (Fig. <ref>C). We explain this from the perspectives of thermal balance. Because both sublimation and melt generation requires energy input, an unsaturated vapor phase will reduce the amount of energy available for melting. This is evident when we observe that the average temperature decrease is stronger in the unsaturated vapor case (Fig. <ref>A, dashed line). 
This lower temperature also causes a delay in water nucleation (Figs. <ref>A, Figs. <ref>C). We also compare the final snapshots of the two simulations, taken at different times (t=45 for the first and t=16 for the second simulation) but both correspond to a state when ice no longer exists (Fig. <ref>B). It is evident that there is significantly less melt water in the first simulation, which is caused by the significant sublimation that claims part of the ice in order to saturate the vapor phase.





Finally, we want to remark that while water nucleation occurs in many parts of the ice interface, melting into the ice phase only advances in some regions of the interface because the process is limited by the thermal energy influx (Figs. <ref> and <ref>). In particular, we observe that melting usually starts at the grain contact regions because larger curvatures induce higher temperatures, as already shown in the case with dry snow (Fig. <ref>B).







  §.§.§ Influence of temperature

In this part, we study the impact of temperature on wet snow evolution. Using the same initial dry snow geometry (Fig. <ref>), here we fix the vapor concentration on the boundary such that ρ_v,B=ρ^I_vs(1) and consider two different boundary temperature T_B's, namely, T_B=1 (Fig. <ref>A) and T_B=2 (Fig. <ref>B).
We further quantify these two simulations with the time evolution of V_i and V_w (Fig. <ref>C) and the ice-water interface length S_iw (Fig. <ref>D). As expected, we observe that melting is faster at a higher boundary temperature (Fig. <ref>C). 
In particular, our results illustrate that a higher temperature accelerates melting by promoting melt nucleation and thus increases the amount of ice-water interface available for melt advancing (Fig. <ref>D). In particular, we observe that a film of nucleated water completely surrounds the ice phase at the higher temperature (Fig. <ref>B), while water film only appears at some regions of the ice interface at the lower temperature(Fig. <ref>A). 
After nucleation, we observe that melting only advances at the three grain contact locations for low T_B, while melting progresses in a few more locations, some away from grain contacts, at high T_B (Fig. <ref>B).

Finally, upon complete melting, the two simulations display a small difference in V_w (Fig. <ref>C, at t=10). This difference is caused by the imposed T and ρ_v boundary conditions. Since we impose ρ_v,B=ρ^I_vs(1) for both simulations, vapor concentration on the boundary is out of equilibrium for the case of T_B=2. This induces a vapor outflux through the boundary which is reflected in final value of V_w via sublimation/evaporation. 







  §.§.§ Influence of liquid water content




In this section, we study the influence of the liquid water content (LWC) on wet snow metamorphism. We consider a simplified snow geometry composed of circular ice grains in a domain of 0.2×0.2^2 meshed with 800× 800 elements (Fig. <ref>A, t=0s).  We run three simulations with different initial LWC (LWC_0) that correspond to dry (LWC_0=0), partially wet (LWC_0=0.15), and fully wet (LWC_0=0.46) snow. The initial ice phase distribution is the same for all the three cases. For the partially wet snow, we impose a water film of 2.6 thickness uniformly surrounding the ice grains. For the fully wet snow, we fill the pore space entirely with the liquid phase. We adopt the parameter value ε=2×10^-7m. To keep the system in quasi-isothermal conditions, we impose a fixed temperature T_B=T_melt and no-flux in terms of vapor density ρ_v on all boundaries.


The simulations results (Figs. <ref> and <ref>) illustrate the influence of LWC_0 on the detailed dynamics of metamorphism. 
In particular, when comparing against dry snow, we find that partially wet snow evolves towards larger and fewer ice clusters faster (e.g., LWC_0=0 and LWC_0=0.15 at t=10 in Fig. <ref>A). While this accelerated coarsening in partially wet snow is visually apparent, plot of the normalized interface length S_a does not readily capture this difference (Fig. <ref>B, black and red lines). Note that S_a in the partially wet snow accounts for the interface of the combined phase ice and water (ϕ_i+ϕ_w). Meanwhile, the coarsening speed of the fully wet snow is much faster, as captured visually in Fig. <ref>A and also in terms of the normalized ice interface length S_i (Fig. <ref>B, blue line).


Wet snow also experiences a broader range of dynamic temperatures than dry snow during metamorphism (Fig. <ref>). This may be explained by the fact that in wet snow, the thermal fluctuations are dominated by the latent heat released/absorbed during freezing/melting process, which is orders of magnitude faster than the process of deposition/sublimation in dry snow (see Table <ref> in the Supporting Information). The temperature distribution also reflects the Gibbs-Thomson effect: because the equilibrium temperature (T^equil, Eq. (<ref>)) decreases with increasing ice grain curvature, smaller ice grains require colder temperature (T^equil<T_melt) to be stable (Fig. <ref>). Because we impose a temperature T_melt at the boundaries, which is higher than the equilibrium temperature for curved grains, we find that all ice grains eventually melt away in the case of fully saturated snow (Figs. <ref>A, C, LWC=0.46). Note that here, if we initialize the simulation with concave grain shapes, this will lead to T^equil>T_melt and complete freezing of the water in the case of fully wet snow. 

A key characteristic of dry snow metamorphism is the reduction in microstructure surface area while maintaining an almost constant ice volume (Fig. <ref>B-C, black curves). Interestingly, we find that for the partially wet snow case investigated here, the volume of the air phase is also roughly constant throughout the simulation (not shown here). We attribute this to the fact that water storage capacity of the air is limited (the initial vapor concentration is already close to the saturated vapor density). Nevertheless, we do observe a small mass exchange between the ice and water phases for partially wet snow, as evidenced by the changes in V_i for LWC_0=0.15 in Fig. <ref>C.






















§ NUMERICAL CHALLENGES AND MODEL IMPROVEMENTS





We have so far presented a series of 1D and 2D simulations of wet snow in domains of limited sizes (∼ 0.2mm). The reason we do not explore larger problems in 3D is due to the high computational cost associated with these simulations. This limits our ability to compare with experimental studies, which are always in 3D and done on samples that are ∼cm in size (e.g., in <cit.>). Here we summarize these computational challenges as well as points for model improvements in future work.

In order to accurately capture the kinetics defined by the Gibbs-Thomson conditions, our model requires that the numerical interface width parameter ε<10^-8m (see Section <ref>). The consequence of this requirement is the need for a rather fine numerical mesh that resolves such interface, making it computationally expensive to simulate larger problems. An adaptive mesh refinement algorithm, rather than the uniform mesh used in this work, would allow one to run mm- and cm-scale simulations in reasonable computational times. Additionally, mathematical additions to our model based on an asymptotic analysis for the thermal problem with asymmetric diffusivities could further improve robustness of our model at the interface.

Another cause for high computational cost is the need to resolve the kinetics of different processes. In this work, we resort to a monolithic scheme, where our time step size is limited by the fastest process. We have alleviated this issue partially using the time-scaling strategy explained in Section <ref>. Another way to speed up time integration is to neglect the phase transitions related to the vapor phase during melting/freezing scenarios. The errors introduced with this approach would be small as long as melting/freezing occurs for short time intervals compared to the total time of interest.

Finally, our model is currently not suited for studying the dynamics of the triple junction —the region where the air, liquid and ice phases contact— when the system is near the triple point. 
At the triple point, defined here as {T,ρ_v}={T_melt,ρ^I_vs(T_melt)}, the triple junction is expected to display an equilibrium configuration defined by the surface tensions in Eq. (<ref>). In our model, an initially non-equilibrium triple junction configuration will evolve towards equilibrium through the co-movement of the water-air, air-ice and water-ice interfaces. However, because each interface moves at different speed, as defined by the phase-dependent mobility (Eq. (<ref>)), a true equilibrium may not be possible. Instead, the model produces oscillations around the triple junction that impede the achievement of the equilibrium configuration <cit.>. 
Far away from the triple point, phase change kinetics around the triple junction dominate over the surface tension kinetics and these oscillations do not appear.


For future work, we will focus on more realistic representation of the nucleation process that includes randomized Gaussian noise <cit.>, which would produce a more realistic behavior of the wet snow metamorphism problem. The current model also does not consider the density difference between the solid ice and liquid water phase, which would introduce small mass conservation error (the model is volume conserved). Our model might be expanded based on <cit.> to account for mass conservation during melting/freezing, but further research is needed.




§ CONCLUSIONS

We propose a non-variational phase-field model for wet snow metamorphism.
The model accounts for the ice, water, and air phases, along with the temperature and vapor dynamics, and captures the actual kinetics of solidification, sublimation, and evaporation (and the opposite transitions). 
To the best of our knowledge, this is the first phase-field model that simultaneously reproduces the different phase transitions of water.

The model results unveil the intimate coupling amongst the various transport and phase change processes involved in wet snow metamorphism. Our results show that local humidity conditions affect snow melting and melt refreezing.

The results also reveal the differences between dry and wet snow metamorphism when water flow is not considered: partially and fully wet snow experience larger thermal fluctuations due to the dominant freeze/melt processes and the coarsening rate is higher for fully wet snow compared to dry snow as expected. However, our observations are based on numerically generated partial wet snow where a liquid film of constant thickness is imposed over all ice grain surfaces. We recognize that this way of initializing the simulation may not represent actual liquid film distribution in wet snow and thus may result in unrealistic dynamics. In addition, because we do not consider thin film flow and other capillary-driven processes of the liquid phase, our model may be missing key fluid mechanical processes that influence how wet snow evolves. 


The current results focus on 1D and 2D problems, due to the high computational cost of the model equations. Future modification, extension and application of this model in 3D may be used to quantitatively analyze the evolution of the snow pore structure during wet snow metamorphism, which dictates the thermo-mechanical and hydraulic properties of the snow at larger scales. The proposed modeling framework has proven useful to study wet snow metamorphism and may be the basis to investigate other mm- or cm-scale problems involving water phase transitions such as, e.g., the freezing of a water droplet on a surface <cit.>, the human-induced thermo-mechanical changes of icy planetary surfaces, or water spray cooling.





The authors acknowledge the partial support from the Resnick Sustainability Institute at California Institute of Technology, the National Science Foundation under Grant No. EAR-2243631 and the ACS Petroleum Research Fund Doctoral New Investigator Grant No. 66867-DNI9. The authors also acknowledge insightful discussions with Dr. Quirine Krol.












§ EQUIVALENCE TO MODELS OF TWO-PHASE TRANSITION


In this section we show that our wet snow metamorphism model is equivalent to the two-phase models for solidification, sublimation, and evaporation when only two phases are present in the system. 
Only for this section, we assume that θ_nucl=0 (see Eqs. (<ref>)–(<ref>) in the main text).

To prove the equivalency, we first need to show that a two-phase system remains as a two-phase system and the non-existing phase does not appear in Ω.
It is trivial to show that the phase-fields equations in our model, Eqs. (<ref>)–(<ref>) in the main text, have the property ϕ_l(,t)=0 ∀ t if ϕ_l(,0)=0 and θ_nucl=0, which proves this first condition.

The second condition constitutes the actual equivalence, which we analyze for each phase transition in the following paragraphs.



 §.§ Solidification


Solidification and melting occurs when ϕ_a=0.
We can rewrite the ϕ_i evolution equation (Eq. (<ref>) in the main text) imposing ϕ_a=0 as


    ∂ϕ_i/∂ t = -3 M_0( 1/εϕ_i(1-ϕ_i)(1-2ϕ_i)   - ε∇^2ϕ_i ) - α_solϕ_i^2ϕ_w^2T-T_melt/L_sol/c_p,w.


The Lagrange multiplier in our formulation guarantees that ϕ_i+ϕ_w=0, which implies that ∂ϕ_w/∂ t=-∂ϕ_i/∂ t.

Imposing ϕ_a=0, the temperature evolution equation (Eq. (<ref>) in the main text) is expressed as:


    ρ(ϕ)c_p(ϕ)∂ T/∂ t = ÷[ K(ϕ) T ] + ρ(ϕ)L_sol∂ϕ_i/∂ t.


We can disregard the vapor density equation since there is no air phase.

Equations (<ref>) and (<ref>) constitute a two-phase model for solidification (and melting) known as the generalized Stefan problem <cit.>. 
Under certain parameter regimes, this model reproduces the kinetics defined by the Gibbs-Thomson condition for the ice-liquid water interface, which reads


    T-T_melt/L_sol/c_p,w=-d_solχ-β_solv_n,


where T is the interface temperature, β_sol is the kinetic attachment coefficient, d_sol is the capillary length, χ is the curvature of the interface (positive for spherical ice grains), and v_n is the normal velocity of the interface (positive for ice growth). Karma and Rappel <cit.> showed that the two-phase model captures the Gibbs-Thomson condition kinetics when (i) ε→ 0 and (ii) the model parameters M_0 and α_sol follow the relations

    M_0=M_sol=ε/3τ_sol ,   α_sol = λ_sol/τ_sol,   
        d_sol=a_1ε/λ_sol,   β_sol=a_1(τ_sol/ελ_sol - a_2ε/D^*_iw),


where a_1≈ 5, a_2≈0.1581 [The a_1 and a_2 values are computed by following the same procedure as in <cit.>, applied to Eqs. (<ref>) and (<ref>).], and D^*_iw≈ (D_i+D_w)/2 represents thermal diffusivity, such that D_j=K_j/(ρ_j c_p,j) for j={i,w}. 
However, we note that the above relations assume equal thermal diffusivity around the freezing/melting interface, which is not true for ice and water in our case. Asymptotic analysis assuming asymmetric diffusivities have been conducted for compositional problems <cit.>, but the same type of analysis for the thermal problems remain to be developed. Nevertheless, <cit.> has shown that the phase-field solidification model recovers the sharp-interface solution if ε < β_solD^*_iw≈ 10^-5m, which is true in the simulations presented in this paper.



 §.§ Sublimation


Sublimation and deposition occurs when ϕ_w=0.
After imposing ϕ_w=0, Eqs. (<ref>), (<ref>), and (<ref>) in the main text are expressed as


    ∂ϕ_i/∂ t = -3 M_0( 1/εϕ_i(1-ϕ_i)(1-2ϕ_i)   - ε∇^2ϕ_i ) + α_subϕ_i^2ϕ_a^2ρ_v-ρ_vs^I(T)/ρ_i
       ρ(ϕ)c_p(ϕ)∂ T/∂ t = ÷[ K(ϕ) T ]  -ρ(ϕ)L_sub∂ϕ_a/∂ t,
       ∂(ϕ_aρ_v)/∂ t=÷[ ϕ_a D_v(T)ρ_v ] + ρ_SE∂ϕ_a/∂ t,


with ∂ϕ_a/∂ t=-∂ϕ_i/∂ t. 

These equations constitute the two-phase sublimation model <cit.> and capture the kinetics defined by the Gibbs-Thomson condition


    ρ_v-ρ_vs^I(T)/ρ_vs^I(T)=d_subχ+β_subv_n,


as long as the parameters M_0, α_sub, and ρ_SE follow the relations <cit.>:

    M_0=M_sub=ε/3τ_sub ,   α_sub = λ_sub/τ_sub,   ρ_SE=ρ_i,
       d_subρ_vs^I/ρ_i=a_1ε/λ_sub,   β_subρ_vs^I/ρ_i=a_1(τ_sub/ελ_sub - a_2ε/D^*_ia- a_2ε/D_v0),


where β_sub and d_sub are the Gibbs-Thomson coefficients for sublimation and D^*_ia represents the average thermal diffusivity of ice and air. Again, we note that the above relations assume equal thermal diffusivity around the interface, which is not true for ice and air in our case. Asymptotic analysis assuming for this exact problem remains to be developed. Meanwhile, <cit.> has shown that the above phase-field model recovers the sharp-interface solution if ε < β_subρ_vs^I/ρ_iD^*_ia and ε < β_subρ_vs^I/ρ_iD_v0. These two conditions yield the requirement that ε < 10^-7m, which we do not satisfy in the simulations presented. Thus, our numerical results do not strictly recover the Gibbs-Thomson condition for air-ice interfaces.



 §.§ Evaporation


The derivation for the evaporation case (ϕ_i=0) is analogous to the sublimation case. The final two-phase model for evaporation (without flow) is analogous to the 2-phase sublimation model (Eqs. (<ref>)–(<ref>)), where the evaporation latent heat is expressed as (L_eva=L_sub-L_sol). Here, we simply list the final relations between the model parameters M_0, α_eva, ρ_SE, and the Gibbs-Thomson coefficients  β_eva and d_eva, which read


    M_0=M_eva=ε/3τ_eva ,   α_eva = λ_eva/τ_eva,   ρ_SE=ρ_w,
       d_evaρ_vs^W/ρ_w=a_1ε/λ_eva,   β_evaρ_vs^W/ρ_w=a_1(τ_eva/ελ_eva - a_2ε/D^*_wa- a_2ε/D_v0),


where D^*_wa represents the average thermal diffusivity of water and air. We note again that the above relations assume equal thermal diffusivity around the interface, which is not true for water and air in our case. Asymptotic analysis assuming for this exact problem remains to be developed. Meanwhile, <cit.> has shown that the above phase-field model recovers the sharp-interface solution if ε < β_evaρ_vs^W/ρ_wD^*_wa and ε < β_evaρ_vs^I/ρ_wD_v0. These two conditions yield the requirement that ε < 10^-8m, which we do not satisfy in the simulations presented. Thus, our numerical results do not strictly recover the Gibbs-Thomson condition for air-water interfaces.







§ ADDITIONAL DETAILS FOR NUMERICAL IMPLEMENTATION



 §.§ Determining temporal scaling coefficients

Wet snow metamorphism involves processes at different time scales.

The characteristic times for solidification (t_sol), evaporation (t_eva), sublimation (t_sub), thermal diffusion (t_T,i, t_T,w, and t_T,a), and vapor diffusion (t_v), assuming a characteristic length scale of L=10^-4m, are

    t_sol≡ L/v_n,sol∼ 1 s,   t_sub≡ L/v_n,sub∼ 10^4 s,   t_eva≡ L/v_n,eva∼ 10^3 s,   t_T,i≡ L^2/D_i ∼ 10^-2 s, 
        t_T,w≡ L^2/D_w ∼ 10^-1 s,   t_T,a≡ L^2/D_a ∼ 10^-3 s,  t_v ≡ L^2/D_v0∼ 10^-4 s,

where thermal diffusivities are calculated as D_l=K_l/(ρ_l c_p,l), for phase l, and we consider typical interface velocities v_n,j, for j={sol,sub,eva} (see Table <ref>).

These values show that sublimation and evaporation are several orders of magnitude slower than solidification and any diffusion process. 
A monolithic numerical solver requires the use of time steps that capture the lower characteristic time (i.e., t_v), which would significantly increase the computational times. 

Here, we leverage the procedure explained in <cit.> to speed up the simulations. 
This approach leverages the fact that T and ρ_v exhibit a quasi-steady state compared to the phase transition kinetics. 
The procedure consists of multiplying the right-hand side of the T and ρ_v equations (Eqs. (<ref>) and (<ref>) in the main text, respectively) by a time-scale factor, so that the characteristic times for vapor and thermal diffusion increase some orders of magnitude while T and ρ_v keep displaying a quasi-steady state.
This approach enables the use of larger time steps without introducing noticeable errors in the simulations.

We multiply the right-hand side of the ρ_v equation by the time-scale factor ξ_v<1, which provokes an increase of the vapor diffusion characteristic time to t^*_v=t_v/ξ_v.
As long as t^*_v does not exceed t_sol, t_sub, and t_eva,

vapor concentration displays a quasi-steady state and the overall problem kinetics is not affected. This condition leads to ξ_v>10^-4.

For the T equation we consider the largest thermal diffusion characteristic time, t_T,w.
In Eq. (<ref>) we observe that t_T,w∼ t_sol, while t_T,w< t_sub and t_T,w< t_eva. Thus, T displays a quasi-steady state when solidification does not occur. 
We follow the same procedure and multiply the right-hand side of the T equation by the time-scale factor ξ_T<1 when solidification does not take place. 
As long as the new characteristic time t^*_T,w=t_T,w/ξ_T does not exceed t_sub and t_eva, temperature displays a quasi-steady state and the problem kinetics is not affected. This condition leads to ξ_T>10^-4.
In this work, we take ξ_v=10^-3 and ξ_T=1 or ξ_T=10^-2 depending on whether or not solidification occurs in any parts of the domain. 





 §.§ Kinetic parameters constraints on grid resolution


If we assume equal diffusivity for all phases, then the relations between the Gibbs-Thomson parameters (β_j,d_j) and our model parameters (M_j,α_j) (for j={sol,sub,eva}) can be derived from asymptotic analysis <cit.> and are valid under the following conditions:


    1. χε<1,     2. ε v_n,j/D_k<1,      3. λ_j u_j<1,      4. τ_j v_n,j / ε <1,


for k={i,w,a,v0}, where ε represents the interface width, D_k is the thermal or vapor diffusivity coefficient, v_n,j is the normal velocity of the interface, λ_j and τ_j are parameters defined in Section <ref> in the Supporting Information, and u_j is the phase-change factor that multiplies the terms α_jϕ_l^2ϕ_m^2 in the phase-field equations (for {l,m}={i,w,a}; see Eqs. (<ref>) and  (<ref>) in the main text).
Condition 1 imposes an interface width smaller than the curvature radius,
condition 2 guarantees that thermal and vapor diffusion are faster than the interface motion, and
conditions 3 and 4 ensure the preservation of a tanh-profile (i.e., quasi-equilibrium profile) across the interface. We use these conditions to guide the choice of ε in our model; however, we recognize that these conditions are only valid for equal diffusivity problems, which is not true for the wet snow problem.

These conditions impose a restriction on ε, which represents the numerical interface width and is an adjustable model parameter. 




To find a valid range for ε, we first estimate the Gibbs-Thomson parameters β_j and d_j from the literature <cit.> (see Table <ref>). 
Note that β_j may range several orders of magnitude depending on ambient conditions and growth type <cit.>.

We next estimate the typical values of u_j and v_n,j (Table <ref>), based on T and ρ_v distributions observed during the different phase transitions. 
We use the Gibbs-Thomson conditions defined in Section <ref> of the Supporting Information to relate u_j and v_n,j, under the assumption that the curvature χ≈0.

In order to express λ_j and τ_j as a function of β_j and d_j (see Eqs. (<ref>), (<ref>), and (<ref>)), we assume ρ^I_vs/ρ_i=ρ^W_vs/ρ_w= 5×10^-6, which is the approximate value at the freezing point.
For condition 1, we consider typical ice curvatures χ∼ 10^4 m^-1.
With this information, we solve for ε in Eq. (<ref>).
We indicate the less favorable valid range of ε in Table <ref>.

We find that condition 3 is the most restrictive, such that ε<10^-8 m. 

In order to speed up the simulations, we consider a value of d_sol, d_eva and d_sub one or two orders of magnitude higher than its actual value (see bottom rows of Table <ref>).
This is because the T- and ρ_v-terms in the Gibbs-Thomson conditions (Eqs. (<ref>) and (<ref>)) dominate the interface motion compared to the curvature terms (d_j) for χ∼ 10^4 m^-1 in most cases.

Doing so, the ε constraint in Eq. (<ref>) relaxes to ε<10^-6 m, which allows us to use a coarser spatial discretization, whereas the wet snow metamorphism kinetics is not significantly affected. 

If compared against experiments, deviations in our modeled kinetics from the observed kinetics could be attributed to this numerical treatment and, hence, the actual values of 
d_sol, d_eva and d_sub must be considered.



 §.§ Parameter values


We list the value of the parameters used in our model at the bottom of Table <ref> and in Table <ref>.
For simplicity, we assume that the values of these parameters are constant.
The values of the parameters k_j and g_j in the saturated vapor pressures are listed in Table <ref>.
The parameter ξ_T is defined as


    ξ_T={[             1 if  Γ_iw > ϵ,;         10^-2   if  Γ_iw≤ϵ, ].


where Γ_iw is the length (in 2D) of the ice-water interface and the parameter ϵ is close to zero. In particular, we compute the interface as Γ_iw=∫_Ωϕ_i^2ϕ_w^2 dx and we take ϵ=2× 10^-4ε in 1D and ϵ=2× 10^-6ε√(S_Ω) in 2D, where S_Ω is the area of Ω. 









 §.§ Initial and boundary conditions


For the phase fields and the temperature we consider the boundary conditions


    ϕ_i·=0,   ϕ_a·=0,   T=T_B(,t)  on ∂Ω,


where ∂Ω is the boundary of Ω and the boundary temperature T_B may take different values on each boundary and may evolve in time.
For the vapor concentration, we consider a fixed concentration on the boundary or we impose no-flux through the boundary, depending on each simulation, which are expressed as


    ρ_v=ρ_v,B  or  ρ_v·=0  on ∂Ω,


where ρ_v,B is a constant value.

The initial conditions of our problem may be written as


    ϕ_l(,0)=0.5(1+tanh(d_l()/2ε)),   T(,0)=T_0(),   ρ_v(,0)=ρ^I_vs(T_0),


where l={i,a}, d_l is the signed distance to the interface of phase ϕ_l and T_0() is a linear function consistent with the temperature boundary condition T_B. 
The distance d_l depends on the geometry considered in each simulation. 




 §.§ Spatial, time discretization and variable regularization


We use the Finite Element Method to solve our model. In particular, we use a uniform mesh composed of bilinear (linear in 1D) basis functions. 

We compute the weak form by multiplying the model equations (Eqs. (<ref>)–(<ref>) in the main text) by weighting functions, integrating over Ω, and integrating by parts taking into account the boundary conditions defined in Eqs. (<ref>) and (<ref>).
To obtain the Galerkin form, we substitute the problem unknowns and the weighting functions by the basis functions.

The phase field variables may take values ϕ_l<0 and ϕ_l>1 during transient states, with l={i,w,a}.
To avoid singularities and numerical issues we regularize the functions ρ, c_p, K, and ρ_SE: 
We restrict the value of ϕ_l between 0 and 1 only when we compute the functions ρ, c_p, and K.

For the regularization of ρ_SE we
(1) implement an additional cubic interpolation locally around ϕ_a=0 which smooths out the discontinuity at that point, and
(2) extend ρ_SE off the ternary diagram constant in the normal direction to the diagram sides.




 Paper Title: Developing a Model-Consistent Reduced-Dimensionality training approach to quantify and reduce epistemic uncertainty in separated flows.





























plain  









 
  
 










    

       






















§ 
 
  .1em
  










   
 
[table]position=top 


unlist
  
    
    
    
    
    
    
  

  




=1











1,*]Minghan Chu
[1]Mechanics and Materials Engineering Department, Queen's University, Kingston, K7L 3N6, ON, Canada


[*]Corresponding author: 

  


            

Developing a Model-Consistent Reduced-Dimensionality training approach to quantify and reduce epistemic uncertainty in separated flows
    [
    
======================================================================================================================================



§ ABSTRACT 
This proposed work introduces a data-assimilation-assisted approach to train neural networks, aimed at effectively reducing epistemic uncertainty in state estimates of separated flows. This method, referred to as model-consistent training, ensures that input features are derived directly from physics-based models, such as Reynolds Averaged Navier Stokes (RANS) turbulence models, to accurately represent the current state of the flow. Autoencoders have been selected for this task due to their capability to capture essential information from large datasets, making them particularly suitable for handling high-dimensional data with numerous discretization points in both spatial and temporal dimensions. This innovative approach integrates the ensemble Kalman method to enhance the training process, providing a robust framework for improving model accuracy and performance in turbulent flow predictions.











































§ INTRODUCTION

Simulations of unsteady separated flows are routinely used in turbomachinery engineering applications. Reynolds Averaged Navier Stokes (RANS) turbulence models serve as the accepted model for these simulations. While the simplifications in RANS models make them computationally inexpensive, they also introduce significant epistemic uncertainty, referred to as structural uncertainty, into the Reynolds stresses in RANS formulations <cit.>. This predictive uncertainty leads to limitations in the engineering design and need to be estimated for reliable and optimal designs. Currently, the Eigenspace Perturbation Method (EPM) <cit.> is the only approach to predict these uncertainties. However, due to its purely physics-based nature and the uniform perturbations to the Reynolds stress tensor, this method often results in unrealistically large uncertainty bounds, leading to overly conservative designs.

Over the past few years, data-driven methods have been rapidly adopted in fluid mechanics, thanks to the availability of large high-fidelity datasets from direct numerical simulations and experiments <cit.>. Data-driven methods have emerged as a promising approach used to develop closure models dedicated to RANS and large eddy simulation (LES) models <cit.>. Most of these studies adopt a priori approach to augment a RANS model, where the neural network is trained offline without modifying the underlying source code of the RANS equations. The concept of priori training minimizes the loss (discrepancy) between the output of a neural network and high-fidelity DNS using the back-propagation method <cit.>. After the training process, the trained model is coupled with the RANS or LES solver for prediction. Although the training process is nonintrusive (i.e., without involving the RANS or LES solver and constrained directly by extracting physics from DNS), the priori training is notorious for the inconsistency between training and prediction <cit.>. This inconsistency may cause unexpected errors in prediction due to the issue of ill-conditioning <cit.>. According to <cit.>, even replacing RANS-based fields with exact turbulence fields from DNS did not result in satisfactory predictive outcomes. Additionally, the priori methods require a full field of DNS data, which is the key limitation due to the unavailability of DNS data for real-world applications. 

To address this problem, the input features need to be generated from the RANS or LES predictions in a manner consistent with the loss function used for training. This is referred to as `model-consistent training' <cit.>. For model-consistent training, the loss is minimized using field inversion approaches, such as 1) deterministic (exact) gradient-based methods (e.g., adjoint-based methods) <cit.> and 2) probabilistic Bayesian-based methods (e.g., ensemble Kalman methods) <cit.>. Adjoint-based methods require accurate access to the full field of variables and gradients. However, notable drawbacks include <cit.>: 1) significant complexity in implementation (e.g., deriving and coding adjoint equations can be mathematically and computationally challenging), and 2) time consumption due to the need for high accuracy and exact gradient computation. As an alternative to adjoint-based methods, a recent study by <cit.> uses ensemble-based methods to approximate derivatives. The ensemble-based gradient method is nonintrusive and can treat the PDEs as a black box. This capability enables ensemble-based methods to address complex, non-differentiable, or noisy systems effectively. The drawback is ensemble based gradient method is less accurate than the exact adjoint-based method <cit.>. To improve accuracy the ensemble Kalman methods can be adapted to directly update the weights and biases of the neural network based on observed data.  The covariance matrix guides the update process, implicitly incorporating both first-order and second-order gradient information to accelerate convergence and improve the accuracy of the parameter updates. Neural network augmentation with the ensemble Kalman method is scarcely studied. Very recently, the study by <cit.> proposed a method to train a supervised neural network using the ensemble Kalman method. This method utilizes ensemble predictions to estimate the covariance matrix, representing uncertainties in the state estimates.

Given the recent rapid development in computational power, high-dimensional data with a vast number of discretization points in both spatial and temporal directions are increasingly being utilized. Neural networks capable of handling these high-dimensional dynamical systems with large and complex datasets are attracting significant attention. Unsupervised autoencoders have emerged as a powerful tool for managing high-dimensional nonlinear dynamical systems due to their ability to capture essential information from large datasets. By compressing input data into a lower-dimensional latent space, autoencoders focus on the most critical patterns and features. However, studies on employing autoencoders in model-consistent training are still very limited.


Recent research has demonstrated the effectiveness of autoencoders in various applications. For instance, autoencoders have been used to distinguish the linear and nonlinear responses of airfoil pressure distribution to changes in the angle of attack <cit.>. Furthermore, it has been shown that autoencoders can capture useful information on prediction confidence even in the absence of ground truth data <cit.>.

Autoencoders can be trained using the ensemble Kalman method to update their weights and biases, combining the strengths of autoencoders in dimensionality reduction and feature learning with the robust state estimation capabilities of the ensemble Kalman method. However, integrating autoencoders with the ensemble Kalman method in flow-physics problems is still in its infancy and requires further investigation.

Given this context, the primary goal of the proposed work is to develop a model-consistent reduced-dimensionality training method to effectively reduce epistemic uncertainty in the state estimates of separated flows. To achieve this goal, we propose three research objectives:



0em
    
    
  * Develop a model-consistent training framework that trains a reduced-dimensionality deep-learning model by incorporating observed data through a data assimilation method.
    
  * Assess the performance of the trained neural network by evaluating the degree of uncertainty reduction and improvements in prediction accuracy.
    
  * Inform management decision-making to lead to better design, overall performance, and safety of machinery.




§ OBJECTIVE

The overarching goal of this proposed work is to develop a rigorous ensemble-Kalman-autoencoder training framework. This framework aims to adjust the neural network parameters to better fit observed data, minimizing the discrepancies between predictions and actual observations, thereby improving overall model performance in quantifying and reducing epistemic uncertainty. In this proposed work, autoencoders will be chosen as the machine learning model to be trained through the ensemble Kalman method for the following three practical advantages:


    
  * By incorporating observed data and estimating the distribution of weights and biases, ensemble Kalman method provides a robust way to train the autoencoder, potentially leading to better generalization and performance.
    
  * Ensemble Kalman method can effectively handle uncertainties in the parameter estimates, making the training process more resilient to noise and variations in the data.
    
  * The sample-based nature of ensemble Kalman method allows it to handle non-Gaussian and nonlinear dynamics, which are often present in complex data distributions encountered in autoencoder applications.










The proposed work will develop an ensemble-Kalman-autoencoder training framework that leverages machine learning predictions, physics knowledge, and observations to yield accurate estimations of epistemic uncertainty. This framework will be tested for estimating quantities in turbomachinery separated flows, such as the separation location, drag coefficient, lift coefficient, and pressure coefficient. The three research objectives of the proposed work are as follows:


     
  * Development of an ensemble-Kalman-autoencoder framework. 
     
  * Assess the performance in accurately capturing the behavior of turbomachinery separated flows, including separation location, drag coefficient, and pressure coefficient. Evaluate the framework's impact on uncertainty quantification capability and convergence speed.
     
     
  * Enhance confidence in decision-making by improving the understanding of uncertainty quantification.
     











§ APPROACH


Problem Formulation: To inform decision-making for better design, I propose a framework that fuse autoencoders, RANS formulations, and observations. The method to quantify and reduce the structural uncertainties would eventually guide the design of optimal turbomachines (i.e., risk management, cost savings, and safety of machinery process) in routine engineering applications for management-relevant decisions. 



The Reynolds stress tensor can be decomposed into eigenvalues (λ_ij), eigenvectors (v_ij), and the amplitude (k). These components are not directly measurable and must be indirectly inferred from observed data. Therefore, they are referred to as decision-level states.

The proposed ensemble-Kalman-autoencoder training framework includes several key components:


  
  * The state vector 𝐗= (α_o, 𝐤, λ_𝐢𝐣, 𝐯_𝐢𝐣), which represents the estimated observed state (i.e., α_o) and hidden state variables (i.e.,  𝐤, λ_𝐢𝐣, 𝐯_𝐢𝐣).
  
  * A forward model that includes model uncertainties. Here, the forward model includes RANS equations and an autoencoder-based neural network with uncertainties arising from various sources, including data volume, data quality, learning rate, batch size, and the regularization coefficient. These uncertainties are propagated into the predicted Reynolds stress field (i.e.,𝐤, λ_𝐢𝐣, 𝐯_𝐢𝐣).
  
  * High-fidelity observations with artificially added noise/error covariance matrix ϵ.


The observation vector is denoted 𝐙_𝐝 (where the subscript `d' stands for `data'), which can include state observations such as velocity or pressure drag. The ensemble-Kalman-autoencoder training can be formulated as a minimization problem of appropriate loss function 𝐋. I will use an ensemble Kalman method for neural network training in the proposed work, considering the error covariance matrices as described in <cit.> for implementing the fusion algorithms. The fusion algorithms solve a state estimation problem, which can naturally be approached using Bayesian inference <cit.>. The observations enter the ensemble-Kalman-autoencoder problem via the likelihood f(𝐙_𝐝|𝐗), where f is the probability density function (pdf). Importantly, the likelihood f(𝐙_𝐝|𝐗) reflects the pdf of observed errors f(ϵ), where ϵ is the matrix containing observed errors, referred to as the error covariance matrix. Since I plan to use high-fidelity DNS observations, the values in the ϵ matrix should be negligibly small. Specifying ϵ is a key step in quantifying the effect of observed uncertainties in obtaining the estimated state 𝐗.

Technical Innovation: In the proposed work, the observed data serves as constraints during training of the neural network.  This allows for a more general and accurate estimation of 𝐗 via autoencoders  (Obj. 1, Task 1.1 and 1.2). Another key innovation is that the autoencoders will be optimized by adding a regularization term in the loss function, incorporating the observed data (Obj. 1, Task 1.3). This ultimately enables a probabilistic prediction of 𝐗 in space and time, which is necessary to inform management decision-making, leading to better design, overall performance, and safety of the machinery (Obj. 2).








 §.§ Methodology





  §.§.§ Objective 1: Development of an ensemble-Kalman-autoencoder framework


This objective will be carried out by completing the following three tasks.

Task 1.1: Learn a turbulence model represented by autoencoders

This proposed work will integrate an autoencoder with an ensemble Kalman framework, incorporating the observed data as a regularization term in the loss function (see Task 1.2). The ensemble Kalman framework assimilates the available observed data to infer the missing states of a system. In this work, an autoencoder will be trained to represent the Reynolds stress tensor (as modeled by a closure model), and the model parameters (weight vector w) of the autoencoder will be inferred through the ensemble Kalman approach. The procedure for model learning through the ensemble Kalman method consists of the following steps:”


    
  * The initial weights (w) for the autoencoder will be sampled from baseline distribution. (see Figure <ref> (a)). If we assume these parameters are independent and identically distributed (i.i.d), the initial weights can be defined using a Gaussian distribution 𝒩(w_0, σ^2) with mean w_0 and variance σ^2. 
    
  * The autoencoder model will be used to represent the deviatoric part of Reynolds stress τ. It consists of the hidden states of k, λ_ij, v_ij (see Figure <ref> (b)). 
    
  * The Reynolds stress term (τ) appears in the momentum equation and can be solved using a RANS solver with a given velocity field (see Figure <ref> (c)). The initial velocity field will be obtained using a baseline turbulence model. 
    
  * Using the RANS solver, the current predicted observed state is calculated. With the Kalman gain and the new observed data, the updated weights of the autoencoders can be determined (see Figure <ref> (d)).
    
  * The Reynolds stress term is updated by updating the weights through the statistical analysis of comparing the predicted observed quantities (e.g., velocity field at specified locations or the drag from integrating the surface pressure) with observed data (e.g. sparse velocity observations, e.g., see Figure <ref>) using the ensemble Kalman method (see Figure <ref> (e)).  












Task 1.2: Loss function in training the autoencoder uisng an ensemble Kalman framework

The goal of training is to minimize the loss L = L(x, g(f(x))), where f and g are nonlinear functions representing the encoder and decoder, respectively. The encoder maps the input x to a latent representation z (i.e., z=f(x)), and the decoder maps the latent representation back to the reconstructed input x̂ (i.e., x̂ = g(z) or x̂ = g(f(x))). The loss function can be a mean squared error (MSE) (i.e., x-x̂^2=(x-x̂)^2) or even a mean absolute error (MAE) (i.e., x-x̂=(x-x̂)). Therefore, reconstruction loss can be defined as:
 
 
    L(x, g(f(x)))=x-x̂^2=x-g(f(x))^2


 or

 
    L(x, g(f(x)))=x-x̂=x-g(f(x))


To incorporate observed data, I will introduce a regularization term based on the observed data. This term penalizes the difference between the predicted observed quantity y and the actual observation Z. In this context, y represents the observed quantity (e.g., velocity field or drag), and Z represents the available observed data (e.g., DNS data). The observation loss can be defined as:


    Observation  Loss = y-Z^2


The total loss function combines the reconstruction loss and the observation loss. The observation loss acts as a regularization term that incorporates the observed data:


    Total  Loss = L(x, g(f(x))) + λy-Z^2
.

Here, λ is a regularization parameter that controls the trade-off between the reconstruction loss and the observation loss.




 


















































Task 1.3: Creating observations 

The proposed ensemble-Kalman-autoencoder framework indirectly assimilates observations to infer the hidden Reynolds stress field, requiring only measurable flow quantities such as sparse observations of mean velocities or integral quantities like drag and lift (e.g., see Figure <ref>). This approach eliminates the need to measure the full-field Reynolds stresses. To date, the role of measurement noise has often been investigated by adding artificial noise (e.g., uniformly sampled noise) to noise-free data <cit.>. The error matrix ϵ for the high-fidelity DNS data is expected to be small. In this work, the observed DNS data are obtained from in-house DNS data generated by a former PhD student from Queen's University. The noise values in the error matrix ϵ are negligibly small and will be constructed by adding Gaussian random noise (i.e., 𝒩(Z, σ_z) and ϵ = (0, σ_z^2)).





  §.§.§ Objective 2: Inform management decision-making to enhance design effectiveness

This objective outlines how integrating autoencoders with an ensemble Kalman method can enhance our understanding of uncertainty quantification, improve design, and ensure safety. The ensemble-Kalman-autoencoder algorithm developed in Objective 1 will inform management decision-making. The proposed work aims to consider both prediction accuracy and the degree of uncertainty reduction for the quantities of interest in management decisions. By simultaneously quantify these two metrics, we can decide when the decision time point will be to implement management or mitigation measures, and also what effective measure(s) to choose. 

The level of accuracy is assessed by comparing the predicted mean with reference data. As shown in Figure <ref> (a), the posterior distributions at four different locations closely approximate the truth compared to the prior distributions. In addition to the mean, the uncertainty range indicates the confidence level that designers should place in the predicted mean. For instance, Figure <ref> (a) illustrates a trend where sample distributions shrink in size compared to those in the prior distributions, signaling an increased confidence in the posterior distributions.

With a clear understanding of uncertainties, managers can make more informed decisions based on posterior probability distributions rather than solely deterministic predictions. This approach aids in optimal resource allocation by identifying where to focus efforts and resources to effectively mitigate risks. For instance, in Figure <ref> (a), the posterior distribution at the far-right location of the geometry deviates significantly from the truth with a high degree of uncertainty. Recognizing this pattern can prompt designers to prioritize model redesign for flow simulation in that specific area of the computational domain. Robust designs are achievable only when designers comprehend the range and nature of uncertainties, allowing them to create systems that perform reliably under diverse conditions, not just ideal or expected scenarios. Understanding uncertainties also facilitates setting appropriate safety margins, ensuring designs remain safe even when faced with unexpected conditions.



§ PRELIMINARY TESTS AND FUTURE WORK





I have tested and validated the concept using the canonical case of flow over periodic hills. The periodic hill flow features a recirculation zone formed by forced separation, a strong mean flow curvature due to the domain geometry, and a shear layer that is not aligned with the overall flow direction. I used the open-source platform OpenFOAM <cit.> to solve a RANS-based k-ω model. First, the baseline RANS was solved using the built-in  solver for incompressible, steady-state turbulent flow simulations. As initial conditions, the generated baseline predictions were updated using the iterative Ensemble Kalman filter (IEnKF) method. The second forward RANS solver, , was coupled with the IEnKF framework to compute velocity with a given Reynolds stress field. Therefore, the  solver does not need to solve the equations for turbulence quantities, as the Reynolds stress field is reconstructed using Karhunen–Loeve (KL) expansions <cit.>. The combined OpenFOAM solvers and IEnKF methods are implemented on the open-source DAFI platform by Xiao et al. <cit.>. The data-assimilation framework assimilates the observations obtained from the marked locations, as shown in Figure <ref>, to infer both the observed velocity field and the hidden Reynolds stress field.
 


 

Figure <ref> shows the predictions for the hidden state of eddy viscosity for flow over a periodic hill. For the sake of brevity, the case setup is omitted in this work; please refer to the paper by Xiao et al. for details <cit.>. From Figures (a) and (b), it is evident that the  method generally updates the posterior eddy viscosity profiles towards the true values, compared to its prior. However, an exception is observed within the recirculation region at the rightmost side of the geometry. 

Future work will develop a new ensemble-Kalman assisted approach that employs autoencoders to model the deviatoric part of the Reynolds stress tensor. Open-source software such as DAFI can serve as the foundation for implementations, for instance, by replacing the KL expansion with autoencoders. The deviatoric part of the Reynolds stress tensor can be expressed as τ_ij^dev = 2k ∑_n=1^N c^(n) T_ij^(n), where c^(n) are scalar coefficient functions that need to be determined, T_ij^(n) are the tensor basis functions, and N is the number of tensor basis functions used <cit.>.















































§ CONCLUSIONS

The proposed work focuses on the development of a novel data-assimilation-assisted approach, integrating the ensemble Kalman method with autoencoders to address complex flow problems characterized by high degrees of turbulence in separated regions. By incorporating observed data as a regularization term in the loss function, this method aims to adjust neural network parameters to better fit actual observations, thereby minimizing discrepancies between predictions and real-world data. This approach leverages machine learning predictions, physics knowledge, and observations to yield accurate estimations of epistemic uncertainty.

The ensemble Kalman method’s robustness in handling uncertainties, its ability to work with non-Gaussian and nonlinear dynamics, and its sample-based nature make it a powerful tool for training autoencoders. This integration provides several practical advantages, including improved generalization, resilience to noise, and enhanced performance in capturing the behavior of complex turbulent flows.

The proposed framework will be rigorously tested on turbomachinery separated flows to estimate quantities such as separation location, drag coefficient, lift coefficient, and pressure coefficient. This work’s research objectives include developing the ensemble-Kalman-autoencoder framework, assessing its performance in accurately capturing flow behaviors and its impact on uncertainty quantification and convergence speed, and enhancing confidence in decision-making by improving the understanding of uncertainty quantification.

Through this innovative approach, the proposed work aims to significantly advance the field of turbulence modeling, offering a robust method to quantify and reduce epistemic uncertainty. This, in turn, will inform management decisions and enhance design effectiveness, ensuring that systems perform reliably under diverse and unpredictable conditions.


























 
*equationsection 
 
*figuresection 
 
*tablesection 








 Paper Title: A dynamical mechanism rooted in the Lorenz system for giant negative and positive mobility of a memory-driven active particle.



APS/123-QED












]A dynamical mechanism rooted in the Lorenz system for giant negative and positive mobility of a memory-driven active particle

rahil.valani@physics.ox.ac.uk

^1Rudolf Peierls Centre for Theoretical Physics, Parks Road, University of Oxford, OX1 3PU, United Kingdom
^2Ecole Nationale Supérieure de Génie Mathématique et Modélisation (ENSGMM), Université Nationale des Sciences, Technologies, Ingénierie et Mathématiques, Abomey, Republique du Bénin



             

§ ABSTRACT 

On applying a small bias force, non-equilibrium systems may respond in paradoxical ways such as with giant negative mobility (GNM) – a large net drift opposite to the applied bias, or giant positive mobility (GPM) – an anomalously large drift in the same direction as the applied bias. 


We consider an idealized model of a memory-driven active particle inspired from experiments with walking/superwalking droplets that takes the form of the celebrated Lorenz system. By adding a small bias force to this Lorenz model, we uncover a mechanism for simultaneous emergence of GNM and GPM in the parameter space. Within the chaotic sea of the parameter space, a symmetric pair of coexisting asymmetric limit cycles separate and migrate under applied bias force, resulting in anomalous transport behaviors that are sensitive to the particle's memory. This provides a general dynamical mechanism for the emergence of anomalous transport behaviors for self-driven particles described by low-dimensional autonomous chaotic systems.



[
    Bruno S. Dandogbessi^2
    August 5, 2024
==========================



When a small bias force is applied to a system at equilibrium, it responds with a net drift in the direction of the applied bias. However, non-equilibrium systems can respond in counterintuitive ways to a small applied bias. A peculiar anomalous transport behavior that defies intuition is negative mobility, where a system responds with a net drift in the direction opposite to the applied bias force. Negative mobility has been observed in experiments of both classical as well as quantum systems, such as electrons in GaAs quantum wells <cit.>, transport through semiconductor superlattices <cit.> and particles moving in structured microfluidic channels <cit.>. Theoretically and numerically, negative mobility and related anomalous transport behaviors have been extensively investigated in various minimal models of a passive inertial particle, either deterministic or stochastic, with typical features of dissipation, spatially periodic potential and external time-periodic driving <cit.>. These studies have identified different mechanisms that give rise to negative mobility in these externally-driven non-autonomous systems, and the effect is attributed to at least one of the following ingredients: noise-induced fluctuations, chaotic attractors and periodic limit cycle attractors. Investigation of negative mobility in self-driven active systems has been relatively scarce but it is increasing with a recent surge of interest in active matter <cit.>. 

In this letter, we consider a simple model of a memory-driven active particle based on the experimental system of walking/superwalking droplets <cit.>, that takes the form of the celebrated Lorenz dynamical system <cit.>. We uncover a dynamical mechanism for simultaneous emergence of giant negative mobility (GNM) and giant positive mobility (GPM) rooted in the Lorenz system [Although existence of negative mobility has been reported by the first author for the Lorenz model in a previous work <cit.>, an underlying mechanism was not investigated]. 




 
A vertically vibrating liquid bath can support bouncing and walking droplets of the same liquid on its free surface <cit.>. Each impact in the periodic bouncing of the droplet generates a localized standing wave that decays in time. The droplet interacts with these self-generated waves on subsequent bounces to propel itself horizontally resulting in walking and superwalking droplets (see Fig. <ref>(a)). Such droplets have been shown to mimic hydrodynamic analogs of several quantum systems <cit.>. Three key features to note about this system - (i) the droplet and its underlying waves co-exist as a wave-particle entity; without the droplet, surface waves decay completely. (ii) There is memory in the system since at large amplitudes of bath vibrations, the droplet-generated waves decay very slowly in time. Hence the droplet motion is guided by the complex dynamical landscape sculpted by the history of waves along its trajectory. (iii) The droplet is self-driven or active <cit.> since the droplet locally extracts energy from the vibrating bath and converts it into persistent horizontal walking motion. A commonly used theoretical model that captures the key features of a droplet's walking dynamics is the stroboscopic model <cit.>. In this model, the vertical periodic bouncing motion of the droplet is averaged over and one gets a trajectory equation for the horizontal walking motion. As shown schematically in Fig. <ref>(b), we consider a simple setup where the droplet can move only in one horizontal dimension, resulting in the following trajectory equation of motion (in dimensionless units) <cit.>

    ẍ_d + ẋ_d = F_self + F_bias ,




where x_d is the position of the particle and an overdot denotes a time derivative. The left-hand-side of Eq. (<ref>) is composed of an inertial term ẍ_d and a dissipative force term proportional to particle velocity ẋ_d. The first term on the right-hand-side is the wave-memory force on the particle due to its self-generated waves. At each instant of time, the particle generates waves W(x) centered at the particle location and decaying exponentially in time. By integrating over the history of the particle's trajectory, one can calculate the overall wave field generated by a linear superposition of the individual waves. The wave-memory force is then proportional to the gradient of this overall wave field at the particle location and can be written as

    F_self=-R ∫_-∞^t W'( (x_d(t)-x_d(s)) ) e^-(t-s)/τ ds.

Here, the parameter R is a dimensionless wave-amplitude of particle-generated waves and the parameter τ is a dimensionless wave decay rate and hence determines the memory of the system (see <cit.> and <cit.> for more details on the parameters). The second term on the right hand side is the external bias force applied to the particle and we take it to be a constant force i.e. F_bias=F. In experiments, droplet-generated waves have features of both spatial oscillations and spatial decay (see Fig. <ref>(a)). However, if we only retain spatial oscillations and consider an idealized cosine waveform i.e. W(x)=cos(x), then this still captures the key qualitative features of walking dynamics <cit.> and significantly simplifies the system [In terms of the experimental system of walking/superwalking droplets, although in this letter we have only investigated an idealized wave form W(x)=cos(x) that does not include spatial decay, a modified wave form with weak spatial Gaussian decay also show a similar mechanism for GNM and GPM in a smaller region of the parameter space (see Supplemental Material).] by allowing us to map the integro-differential equation in Eq. (<ref>) onto the following system of three nonlinear ordinary differential equations <cit.> (see <cit.> for a derivation):

    Ẋ   =Y-X+F, 
    Ẏ   =-1/τY+XZ, 
    Ż   =R-1/τZ-XY.

Here, X=ẋ_d is the particle velocity whereas Y and Z are related to the memory forcing with Y=R ∫_-∞^tsin(x_d(t)-x_d(s)) e^-(t-s)/τ ds, as the wave-memory force on the particle, and Z=R∫_-∞^tcos(x_d(t)-x_d(s)) e^-(t-s)/τ ds, as the wave field height at the particle location. With no applied bias force, F=0, the system in Eq. (<ref>) is equivalent to the celebrated Lorenz system <cit.> (see Supplemental Material). Hence, the motion of our memory-driven active particle can be directly mapped onto the dynamics of the Lorenz system. We numerically solve the dynamical system of Eq. (<ref>) in MATLAB using the inbuilt ode45 solver.




With the above correspondence between the particle's trajectory equation and the Lorenz system, we can map the attractors and phase-space features of the Lorenz system onto the motion and trajectories of the active particle <cit.>. We proceed by reviewing different attractors realized in the Lorenz model <cit.> of Eq. (<ref>) as a function of the memory parameter τ and map them onto the different types of trajectories for the particle. For small values of τ, there is only one equilibrium point of the system at (X,Y,Z)=(0,0,0) which is stable. This corresponds to a (Type I) non-walking state for the particle since its velocity is ẋ_d=X=0 (see Fig. <ref>(a)). With increasing τ, the Lorenz system undergoes a pitchfork bifurcation where the non-walking state becomes unstable and a symmetric pair of stable equilibrium points emerge, (X,Y,Z)=(±√(R-1/τ^2),±√(R-1/τ^2),1/τ); this corresponds to steady walking motion (Type II) of the particle to right/left (see Fig. <ref>(b)). Further increase in τ leads to a complex set of bifurcations in the Lorenz system <cit.> and eventually chaos emerges at large τ with a strange attractor in the phase space. Phase-space dynamics on the Lorenz chaotic attractor corresponds to a  (Type III) chaotic walk <cit.> for the active particle where it unpredictably switches between left and right moving states as the phase space trajectory jump between left and right “wing" of the Lorenz attractor (see Fig. <ref>(c)). 

 


It is common for periodic windows to exist within a chaotic regime of parameter space. Since the Lorenz system obeys a symmetry i.e. the system is symmetric about a rotation of 180 degrees around the Z axis, it restricts the types of periodic limit cycles that can exist. Two types of limit cycles can be found in the Lorenz system: (i) a symmetric limit cycle about the Z axis which would correspond to a back-and-forth oscillation for the particle with no net drift, and (ii) a symmetric pair of asymmetric limit cycles, as shown in Fig. <ref>(d), that corresponds to the particle undergoing back-and-forth oscillations along with a net drift either towards the left or right (Type IV). A parameter space plot showing the time-averaged particle velocity ⟨ẋ_d ⟩ = ⟨ X ⟩ in the (R,τ) space is shown in Fig. <ref>(a) which helps identify these oscillating states that have a net drift. Region I corresponds to non-walking state (X=0), Region II is steady walking (X=±√(R-1/τ^2)), Region III is mostly filled with chaotic walks with no net drift (⟨ X ⟩ = 0). Of particular interest is Region IV corresponding to bands of net particle drift. The mixture of positive (blue) and negative (red) drift states along this band indicates the sensitivity of these states to small changes in parameter values. 

Now under the application of a small bias force of F=0.05, the corresponding parameter space plot in Fig. <ref>(a) transforms to Fig. <ref>(b). We clearly see that the band corresponding to Region IV splits into two in the parameter space, one corresponding to the asymmetric limit cycle with a net drift opposite to the applied bias (red) and the other corresponding to the asymmetric limit cycle with a net drift in the direction of the applied bias (blue). We identify the former as the GNM band while the latter the GPM band. Further, on these separated bands, all initial conditions lead to the same limit cycle attractor, unlike the F=0 band where the basin of attraction of left/right drifting states in the initial-condition space are intricately mixed. After complete separation, the two bands migrate away from each other with increasing small bias F such that the GNM band moves below the F=0 band (black dashed curve) while the GPM band migrates above the F=0 band. 

We note that this band separation mechanism of coexisting asymmetric limit cycles is closely related to the separation mechanism of phase-locked periodic attractors reported by <cit.> and <cit.> for negative mobility of a passive particle in a periodic potential with external time-periodic driving. However, in contrast, the negative and positive mobility bands completely separate and migrate in the parameter space of our Lorenz model, whereas the two bands only partially separate in the system of <cit.> along with the negative mobility band shrinking and positive mobility band growing in the parameter space. 

To demonstrate more clearly how this separation of bands manifests in GNM and GPM for our memory-driven active particle, let us consider the system at two different memory parameter τ values for a fixed R, one value below the region IV band with lower memory τ=6.9 and the other above the band τ=12.2 with higher memory. If we fixed the system parameter at these values, and investigate how the average velocity (both time and ensemble averaged) varies as a function of the bias force, we get the results shown in Fig. <ref>(a). For lower memory (red curve), the average velocity shows a strong dip of GNM near F=0.05. This is because, the GNM band after separation, migrates through the parameter space with increasing F and passes through this point in parameter space near F=0.05. Similarly, for the higher memory value (blue curve), the average velocity shows a strong spike near F=0.05 due to the passing of the GPM band. Figure <ref>(b) further shows the sensitivity of these behaviors as a function of the system memory τ. With no bias force (gray curve), there is no net mobility of the particle (when ensemble averaged) at all values of memory τ. However, with a small applied bias, the splitting of the region IV band results in GNM regions for lower memory and GPM regions for higher memory. This highlights the sensitivity of anomalous transport behaviors to memory of our system under small applied bias. 

In summary, we have shown a dynamical mechanism rooted in the celebrated Lorenz system model for the generation of giant negative mobility and giant positive mobility for a memory-driven active particle under a small applied bias. The degenerate pair of asymmetric limit cycles existing for the same parameter values completely separate and migrate in the parameter space upon small applied bias, resulting in the emergence of memory-dependent GNM and GPM for the active particle. We believe this dynamical mechanism to be general to many low-dimensional chaotic systems and it might be at play in generating anomalous transport behaviors for deterministic active particles. Further, such mechanisms may be used to tune the internal parameters of the system in order to control a particle's transport behaviors, or in sorting of active particles with different intrinsic properties.
Acknowledgments. This research was supported by the Leverhulme Trust [Grant No. LIP-2020-014]. 

Some of the numerical results were computed using the Hydra cluster in the Department of Physics at the University of Oxford.






 Paper Title: Line tension from dual-geometry sessile droplet measurements: Combining contact angle size dependence data for axisymmetric and cylindrical droplets to determine the line tension.




Line tension from dual-geometry sessile droplet measurements]Line tension from dual-geometry sessile droplet measurements: Combining contact angle size dependence data for axisymmetric and cylindrical droplets to determine the line tension


d.tatyanenko@spbu.ru



Department of Statistical Physics, Faculty of Physics, Saint Petersburg State University
  7–9 Universitetskaya nab., St. Petersburg, 199034, Russia



§ ABSTRACT 
We perform a thermodynamic analysis of various contributions to the size dependence of the contact angle cosine for axisymmetric and cylindrical sessile droplets. This shows that a widely used method to determine the line tension from the slope of the contact angle cosine dependence on the three-phase contact line curvature (for axisymmetric droplets) provides a certain combination of the line tension, adsorptions at the three interfaces, and the macroscopic contact angle. To extract the leading-order contact-line-related contribution and determine the line tension, we propose a simple technique using contact angle size dependences for axisymmetric and cylindrical droplets at the same conditions.




[
    Konstantin D. Apitsin
    August 5, 2024
=========================






§ INTRODUCTION


The contact angle of a large sessile droplet can be found from the well-known classical Young equation <cit.>

    cos =  -

with σ the surface tension of a corresponding interface,  the macroscopic contact angle; α, β, and γ mark the liquid, gaseous and solid phases, respectively, and double Greek superscripts mark the corresponding interfaces. The surface tensions here are thermodynamic (defined as the surface excesses of the grand potential per unit area of the interface). For small droplets, the thermodynamic line tension κ (defined as the line excess of the grand potential per unit length of the three-phase contact line <cit.>) affects the contact angle <cit.>:

    cos =  -  - κ/.

Here θ is the contact angle, r is the radius of the three-phase contact line, subscript “a” refers to axisymmetric droplet (see Fig. <ref>(a)); the line tension κ is implied to be constant. Eq. (<ref>) is usually called the modified Young equation.

Subtracting the modified Young equation (<ref>) from the classical  one (<ref>) and dividing the both parts of the resulting equation by , one usually arrives at <cit.>

    cos - cos = κ/.

This equation is widely used for contact-angle-based measurements of the line tension in laboratory experiments (see, e.g., <cit.>) and computer modeling (e.g., molecular dynamics simulations <cit.>). According to Eq. (<ref>), the slope of the graph cos vs 1/ must be equal to - κ/. The surface tension  is typically measured or calculated separately.

However, some authors have noted that Eq. (<ref>) is generally not correct <cit.>, since it assumes the corresponding surface tensions in Eqs. (<ref>) and  (<ref>) to be equal, whereas they correspond to different equilibrium sessile droplet sizes, which implies different values of the chemical potential(s). There are arguments that adsorption at the solid–liquid interface and its effect on the surface tension  affect cos in the same manner and with a similar magnitude as it is measured in experiments <cit.>, so the line tension itself is not necessary to explain the dependence of cos on 1/. Consideration with use of an interface displacement  model shows <cit.> that the contributions of the line tension and the adsorption at the solid–gas interface (and its effect on the surface tension ) to the  dependence of cos on 1/ can be comparable in the presence of a precursor film. Some authors <cit.> suppose only  to depend on the droplet size and Eq. (<ref>) to be correct, but the slope of the graph cos vs 1/ to give a quantity not equal to - κ/.  Sometimes the resulting “line tension” is called the apparent line tension <cit.>. General thermodynamic analysis shows <cit.> that the value of this slope has contributions both from the line tension and the adsorptions at all three interfaces (see also <cit.> for similar arguments).
             

How to “extract” the line-tension contribution and determine the value of the line tension from the measurements or calculations of cosθ vs 1/r? Thermodynamic analysis shows <cit.> that, for cylindrical droplets [long liquid “channels”, or “filaments”, or “ridges” attached to the substrate surface, see Fig. <ref>(b)], there is no κ/r term in the equation corresponding to (<ref>). It means that dependence of the contact angle  on the size of a cylindrical droplet (, its half-width  on the substrate) is governed by dependences of the surface tensions and the droplet size on the chemical potential(s). This suggests the idea to compare or combine data of size dependence of cosθ for axisymmetric and cylindrical droplets at the same conditions to exclude the surface-tension-related contributions and determine the line tension.

In section <ref>, we present a thermodynamic analysis of various contributions to droplet size dependence of the contact angle for both axisymmetric and cylindrical droplets in a system with a single-component fluid and an ideal smooth homogeneous solid substrate in absence of external fields. Based on it, in section <ref> we propose a simple technique that allows to determine the line tension from a combination of contact angle size dependences for axisymmetric and cylindrical droplets in a manner similar to use of Eq. (<ref>).

Cylindrical droplets are often used in modeling, e.g., in molecular dynamics (MD) simulations <cit.> due to convenient use of periodic boundary conditions along their longitudinal axes and due to expected independence of the contact angle of (or a weak dependence on) the droplet width. However, they are hardly available for experimental measurements due to Plateau–Rayleigh-type instability <cit.>. Thus, the technique we propose is mainly oriented to modeling, especially with methods not allowing for direct calculation of the line excess of the grand potential (e.g., MD simulations). The dimensions of such simulated axisymmetric and cylindrical droplets, i.e., R, r, and θ, can be “measured”, and these data can be used to determine the line tension.





§ THERMODYNAMICS AND CONTACT ANGLES





 §.§ Equilibrium conditions for sessile droplets

For thermodynamic description of a small sessile droplet on a partially wettable substrate, we consider a materially open system of a fixed volume including the droplet itself, the substrate, and the surrounding gas (vapor). The liquid and gaseous phases are supposed to be a single-component fluid at a given values of chemical potential μ and temperature T (that supposed to be constant).  The grand thermodynamic potential of the system decomposes into bulk, surface, and line parts:

    Ω =    - p^α - p^β + ω^γ V^γ
        +   +  +  + κ L

with V the volume, A the area of the interface, L the length of the three-phase contact line, p the pressure in the bulk phase, ω the bulk density of the grand thermodynamic potential (in fluid phases equals -p). Other quantities as well as notations of phases and interfaces has been explained above.

Let us consider two types of droplet geometry: (1) axisymmetric droplet [shaped as a spherical segment in our “macroscopic” thermodynamic description, see Fig. <ref>(a)] and (2) a fixed-length piece of a cylindrical droplet [shaped as a segment of a circular cylinder in our “macroscopic” thermodynamic description, see Fig. <ref>(b)]. In both geometries, the cross-section of the droplet in our “macroscopic” thermodynamic description is a circular segment, therefore,

    r = R sinθ.

Thus, the shape and size of the droplet can be described by two independent variables, e.g., (R, r), (R, θ) or (r, θ).

In both geometries, choosing the  dividing surface to be the surface of tension, one can obtain equilibrium conditions for sessile droplets by expressing all the volumes, areas and the length of the three-phase contact line in (<ref>), and solving one of the following sets of two equations


    ∂Ω(R,r,T,μ)/∂ R = 0     and     ∂Ω(R,r,T,μ)/∂ r = 0,
       ∂Ω(R,θ,T,μ)/∂ R = 0    and     ∂Ω(R,θ,T,μ)/∂θ = 0, 
       ∂Ω(r,θ,T,μ)/∂ r = 0    and     ∂Ω(r,θ,T,μ)/∂θ = 0,


and then taking into account the geometric relation (<ref>). Since T and μ are constant, the pressures and the surface tensions in (<ref>) will be constant at such variations (for , this also requires / R = 0 which is true namely for the surface of tension chosen as the  dividing surface).

The droplet-size-dependent surface tensions  and  for spherical and cylindrical interfaces can generally differ, though their difference is expected to be very small at the same value of μ (see subsection <ref>).


Let us chose R and θ as independent variables describing the sessile droplet. Introducing non-variable quantities for the total volume occupied by the fluid ≡ V^α + V^β and the total area of the substrate ≡ + and using relation (<ref>), one can express the volumes, areas, and the contact line length for an axisymmetric droplet as

    = π^3/3 (2 + cos)(1 - cos)^2,
       = 2πsin, 
     = 2π^2 (1 - cos),
       = π^2 sin^2 ,
     =  - ,
       =  - .


Decomposition (<ref>) of the grand potential then takes form

    (,) = - ( p^α - p^β) π^3/3 (2 + cos)(1 - cos)^2 
        + 2 π^2 (1 - cos) + ( - ) π^2 sin^2 
        + 2πsin - p^β - .

Application of the equilibrium conditions (<ref>) gives

    [t][8.5cm]
    - π^2 ( p^α - p^β) (2 + cos) (1 - cos)^2 
     + 4 π (1 - cos) + 2 π( - ) sin^2 
     + 2 πsin + 2 π∂/∂sin^2  = 0,
    
          - π^3  ( p^α - p^β) sin^3  + 2 π^2 sin + 2 πcos
        + 2 π^2 (  - ) sincos + 2 π^2 ∂/∂sincos = 0.

To exclude the pressure difference p^α - p^β, we multiply the first equation by (2 π)^-1sin^2   (1 - cos)^-2 and then subtract the second equation multiplied by (2 + cos)(2 π^2 sin)^-1 <cit.>. After some trigonometric simplifications and substitution of the result into one of the above equilibrium conditions, we arrive at the Laplace equation and the generalized Young equation as equilibrium conditions for an axisymmetric sessile droplet <cit.>

    p^α - p^β = 2/, 
    cos =  -  - / - /


For a cylindrical droplet,

    = ^2/4 (2  - sin 2),
     = ,
       = sin,
     =  - ,
       =  - ,

and the length  of the three-phase contact line does not depend on  and  [see Fig. <ref>(b)] and supposed to be constant. Contact angle  here is in radians. Decomposition (<ref>) of the grand potential then  takes form

    (,)    = - ( p^α - p^β) ^2/4  (2  - sin 2) + 
        + ( - ) sin +  - p^β - .

Application of the equilibrium conditions (<ref>) gives

    [t][8.1cm]
    - ( p^α - p^β)  ( - sincos) + 
     + ( - ) sin + ∂/∂sin = 0,
    
       [t][8.1cm]
    - ( p^α - p^β) ^2 sin^2  + 
      +  ( - ) cos + ∂/∂cos = 0.

Here we have employed trigonometric equalities  sin 2 = 2 sincos and 1 - cos 2 = 2 sin^2. To obtain a simpler equation for p^α - p^β, we multiply the first equation by cos and subtract the second equation multiplied by sin. After some simplifications and substitution of the result into one of the above equilibrium conditions, we arrive at the Laplace equation and the generalized Young equation as equilibrium conditions for an cylindrical sessile droplet, slightly different from Eqs. (<ref>) and (<ref>):

    p^α - p^β = /, 
    cos =  -  - /.


The absence of the coefficient 2 in the Laplace equation  (<ref>) for the cylindrical droplet can be explained by the fact that its  interface is curved only along one direction, whereas the spherical droplet has equal curvatures along two directions. The line-tension terms in the generalized Young equations (<ref>) and (<ref>) originate from variations of the line term κ L in (<ref>). For an axisymmetric droplet, = 2π and ()/ = 2π + 2π / giving the two terms / + / in Eq. (<ref>). For a cylindrical droplet,  does not depend on  and ()/ =  / giving the only term / in Eq. (<ref>).

For both geometries, we suppose the line tension to depend not only on T and μ, but also on the droplet size. This dependence is usually overlooked and, in this case, Eq. (<ref>) transforms into the modified Young equation (<ref>), whereas Eq. (<ref>) transforms into formally classical Young equation (<ref>) (with surface tensions, however, depending on the chemical potential, i.e. on the equilibrium droplet size, that makes the contact angle  different from , see subsection <ref>).

We suppose the contact line radius (half-width) r to be the only appropriate size variable the line tension can be considered to depend on. It is not clear for the derivation we used in this paper, and some authors suppose the line tension to depend also on the contact angle θ <cit.>, corresponding “notional” derivatives also appear in the work <cit.>. However, a variational derivation of Eqs. (<ref>), (<ref>), (<ref>), and (<ref>), which considers all three variables R, r, and θ to be independent in some sense, gives an argument in favor of r as the mathematically appropriate size variable for κ in both geometries <cit.>.

Simple dropping the size-dependent term κ/ r in the generalized Young equations (<ref>) and (<ref>) make them inexact (even if κ is supposed to depend on the droplet size). It can be seen in a model system if the line tension is calculated directly <cit.>. Also, the generalized Young equation is the simplest way to calculate κ/ r if θ and r are found and the surface and line tensions (the latter for axisymmetric droplet only) are calculated directly <cit.>.




 §.§ Size dependence of contact angle and relations between pressures, surface tensions, droplet size and chemical potential


Comparing the generalized Young equation (<ref>) or (<ref>) with the classical Young equation (<ref>), one should take into account that these two equations are written for droplets corresponding to different values of the chemical potential μ. The classical Young equation corresponds to a macroscopically large (strictly speaking, infinite) drop, i.e. liquid–gas bulk phase coexistence at the binodal. Let us mark the values of the quantities corresponding to the binodal with subscript “∞”. The surface tensions at the binodal then should be denoted as , , and , and the classical Young equation (<ref>) should be rewritten as

    cos =  - .


Subtracting the generalized Young equations (<ref>) and
(<ref>) from  Eq. (<ref>) we obtain, for an axisymmetric droplet,

    cos - cos = δ + / + / r
    and a similar equation for a cylindrical droplet:cos - cos = δ + /.

Here δ≡ - with ≡ - and ≡ -. To make these equations more similar to Eq. (<ref>), let us introduce δ≡ -. Then, Eqs. (<ref>) and (<ref>) can be rewritten as

    ( cos - cos ) = δ + δcos + / + /, 
        ( cos - cos) = δ + δcos + /.


Now we see, that, in the exact counterpart of Eq. (<ref>) for the difference cos - cosθ, we have 4 terms for an axisymmetric droplet (instead of 1) and 3 terms for a cylindrical droplet (instead of 0). How do they depend on the droplet size [the radius (half-width) of the droplet on the substrate]?

For the  liquid–gas interface, let us write down the isothermal Gibbs adsorption equations:

    = - μ,

where  is the adsorption (the surface excess of matter per unit surface area) at the  interface for corresponding geometry. Since the surface of tension is chosen as the dividing surface in both geometries, these equations do not contain (/ R)  R terms.

For each of the solid–fluid interfaces, similar equations can be written. Generally, they are more complicated (see, e.g., Sec. 6.3 in <cit.> and Sec. 3.4.3 in <cit.>), but in some simple cases (e.g., for non-deformable substrate and such a choice of the solid–fluid dividing surfaces that the amounts of matter of the immobile component of solid do not change in the solid phase at any change of μ) they take <cit.> the same simple form as (<ref>), so we can write down

    ≡(  - ) = (  - ) μ.

Integrating Eqs. (<ref>) and (<ref>) from the value  at the binodal to the current value of μ, we obtain, introducing δμ≡μ -,

    δ = - ∫_^μ(μ') μ' ≃ - δμ,  
    δ = ∫_^μ(  - ) μ' ≃(  - ) δμ.

Since () = () =, the asymptotic expression in (<ref>) is the same for spherical and cylindrical surfaces at the same value of μ, but the exact integral is generally not if (μ) ≠(μ). Thus,  and  generally differ, but at least in the second order in δμ:

    -  = δ - δ = ((δμ)^2).


The last asymptotic expressions in Eqs. (<ref>) and (<ref>) are valid for small values of δμ. How is this quantity related to the droplet size? This can be estimated using the Laplace equation (<ref>) or (<ref>) for the axisymmetric or cylindrical droplet, respectively. Applying the isothermal Gibbs–Duhem equations p = n μ to liquid and gaseous phases, respectively, we obtain p^α - p^β≃δμ with  the number density of molecules in α liquid phase. Here we have neglected the compressibility of the liquid (≃) and the number density of molecules  in the gas (≪). With use of the Laplace equations (<ref>) and (<ref>), and geometric relation (<ref>), we obtain

    2/ = 2sin/≃δμ, 
    / = sin/≃δμ.

We now see that δμ∝ 1/R = (1/r), i.e., small values of δμ correspond to large droplets, and δμ→ 0 expectedly corresponds to R →∞ and r →∞.

Thus, for the correction terms in Eqs. (<ref>) and (<ref>) we have

    δ≃2sin/(  - )  = (δμ) = (1/r), 
    δ≃ - 2sin/ = (δμ) = (1/r), 
    /≃/ = (δμ) = (1/r), where = at μ = (→∞) is the line tension of the macroscopic (infinite) droplet, andδ≃sin/(  - ) = (δμ) = (1/r), 
    δ≃ - sin/ = (δμ) = (1/r).


As for the terms κ/ r, it can be shown <cit.> that they are ( (δμ)^2 ) = ( 1/r^2 ). For sufficiently large droplets, sinθ can be approximated with sin, with a relative error (1/r), that makes Eqs. (<ref>), (<ref>), (<ref>), and (<ref>) valid with sin replaced by sin. The same applies to cos in the correction terms δcosθ in Eqs. (<ref>) and (<ref>).

Taking this into account, we can now write down the linear in 1/r approximations for Eq. (<ref>) and (<ref>):

    cos - cos≃/, 
    cos - cos≃/,




where

    ≡ + 2sin/(  -  - cos) 
    and   ≡sin/(  -  - cos)

are the quantities sometimes called the apparent line tensions <cit.> of axisymmetric and cylindrical droplets, respectively (see below).

Eqs. (<ref>) and (<ref>) show that the dependence of cos on 1/, even in the first order in 1/, is determined not only by the line tension, but also by the adsorptions at the three interfaces. In fact, Eq. (<ref>) should be treated as Eq. (<ref>) often is. This means that the slope of the graph cos vs 1/ at 1/→ 0 does not give the line tension itself but rather a combination of the line tension, the surface tension of the droplet free surface, adsorptions at the three interfaces, and the macroscopic contact angle. Some authors use the term “apparent line tension” meaning the value of the line tension calculated from the dependence of cosθ on 1/r (denoted further as cosθ (1/r)) at 1/r → 0 as if Eq. (<ref>) were the correct linear approximation. Then Eq. (<ref>) gives the value of the apparent line tension of axisymmetric droplets. More interesting, Eqs. (<ref>) and (<ref>) show that, for cylindrical droplets, cos also depends on 1/ in the first order in 1/. This has been found in simulations <cit.> and attributed to size dependence of the surface tension  expressed using the Tolman length <cit.> ≈/.  As it is seen from Eq. (<ref>), dependences of other surface tensions on the droplet size via the chemical potential could also made their contributions to the apparent line tension of cylindrical droplets.




§ LINE TENSION FROM DUAL-GEOMETRY CONTACT ANGLE DEPENDENCES ON DROPLET SIZE


One can easily notice that the apparent line tension (<ref>) of cylindrical droplets, proportional to the slope of the cos(1/)  graph at 1/→ 0, is exactly half of the second, adsorption-related, term in the apparent line tension (<ref>) of axisymmetric droplets, proportional to the slope of the cos(1/) graph at 1/→ 0. Thus, combining data for dependences cos(1/) and cos(1/), one can determine the line tension of a macroscopic droplet as

    =  - 2.


Using this relation, we have calculated the line tension  of a macroscopic droplet from the apparent line tensions  and  of sessile droplets simulated by Kanduč et al. <cit.> using data from Fig. 2(c) in <cit.>. As one can see in Fig. <ref>, the values of  calculated with Eq. (<ref>) are in good agreement with ones obtained in <cit.> by taking into account Tolman length <cit.> and “line tension stiffness” <cit.> corrections. It is clearly seen, that the values of  and  are comparable in these simulations that makes  visibly different from . In contrast, simulations of Isaiev et al. <cit.> show very weak dependence of cos(1/) comparing to cos(1/) (see Fig. 4 therein). This corresponds to ≪ and, thus, ≈.

             

Eqs. (<ref>)–(<ref>) suggest another idea. With dual-geometry sessile droplet measurements, it is not necessary to use the classical Young equation at all. Subtracting the generalized Young equation for axisymmetric droplets (<ref>)  from the one for cylindrical droplets (<ref>) we obtain

    cos - cos = / + / - /.

Taking into account Eqs. (<ref>), (<ref>) with subsequent comments and κ/ r = ( 1/^2 ), the first-order approximation to the combined equation (<ref>) will be

    cos - cos≃/.

Here dependences of  on μ and  and of  on μ are no longer taken into account since they give corrections ( 1/^2 ).

All the quantities in Eqs. (<ref>) and (<ref>) must be taken at the same values of the chemical potential and temperature. The chemical potential is often unknown in simulations or measurements. But, according to Eqs. (<ref>) and (<ref>) and taking into account Eq. (<ref>), the same value of δμ corresponds to the relation ≃/2 or, in the same order in δμ, to ≃/2. The former relation is expectedly more accurate, since it relies only on the Laplace equations (<ref>) and (<ref>), whereas the latter also uses approximation sin≃sin.

Thus, to determine the line tension from the measured (in the experiment or modeling) dependences of the cosθ on 1/r for both axisymmetric and cylindrical droplets at the same conditions, it is sufficient to calculate the difference

    cos|_ = /2 - cos|_  or  cos|_ = /2 - cos|_

and take a linear fit of it vs 1/ at sufficiently large values of . The slope of this graph will give /:

    cos|_ = /2 - cos|_≃/, 
    cos|_ = /2 - cos|_≃/.


An additional advantage of this technique is that the graph of the difference cos - cos vs 1/ must start from a known point (0 at 1/ = 0), thus the slope of the linear graph remains the only fitting parameter. When plotting cos(1/) and cos(1/) independently, each of them must be approximated by a straight line starting from an unknown point cos(0), which must be the same for axisymmetric and cylindrical droplets (and this, in principle, should be somehow taken into account to reduce the number of fitting parameters from 4 to 3). Unlike experimental measurements, droplet simulations require more computational resources as the droplet size increases, so approaching droplet sizes whose contact angles is macroscopic can be resource demanding. However, macroscopic contact angle , can be found, e.g., using the classical Young equation (<ref>) and calculating  and  in three simulations of  planar interfaces <cit.>.


Experiments with nanosized droplets suggest another possible advantage of using dual-geometry techniques to determine the line tension. For nanosized droplets, the graph cos vs 1/ can be fairly linear but this linear fit may work well only for small droplets and give a false value of cos at extrapolation down to 1/→ 0 (see Fig. 5 in <cit.>) and the slope that does not correspond to asymptotic Eq. (<ref>). In many simulations, the droplets are also nanosized, so the same problem can be with simulated droplets, both axisymmetric and cylindrical, giving false values of  and . If Eq. (<ref>) is then used, this would ultimately give a wrong value of . However, not only the slope would be wrong in such situation, but, most probably, the extrapolated values of cos would be different for cos vs 1/ and cos vs 1/ indicating an inconsistency. With use of the technique we proposed, such a situation would appear as a set of points of the quantity (<ref>) vs 1/ that do not fit a straight line starting from (0,0). Both such indicators would simply mean that the droplets are not large enough for the asymptotic Eqs. (<ref>), (<ref>), and (<ref>) to work and, thus, cannot be used to determine the contact angle  and the line tension  of macroscopic droplets.




§ OVERVIEW OF RESULTS AND DISCUSSION


Thermodynamic analysis we have performed shows that droplet size corrections to the contact angle cosine for axisymmetric sessile droplets attributed often to the line tension only, are determined generally by the line tension , its dependence on the droplet size (directly on the droplet radius  on the surface and via the chemical potential) and also by dependences of all three surface tensions on the droplet size (via the chemical potential). The same applies to corrections to the contact angle cosine for cylindrical sessile droplets, except for the line tension term / that is absent in the generalized Young equation (<ref>) for cylindrical droplets.

In the first order in 1/ (<ref>), this gives the slope of cos vs 1/ to be proportional not to the line tension  of macroscopic droplets [as it is widely believed according to Eq. (<ref>)] but rather to the so called “apparent line tension” , a sum of this line tension  and a combination of adsorptions at the three interfaces, the number density of molecules in the liquid phase and the macroscopic contact angle (<ref>).

For cylindrical droplets, the contact angle does also depend on the droplet half-width  on the surface already in the first order in 1/ (<ref>), giving the apparent line tension . It is determined by the same combination of the adsorptions, the number density of molecules in the liquid phase and the macroscopic contact angle (<ref>) but lacks the numerical coefficient 2 compared to the axisymmetric case (<ref>).

This gives a possibility to determine the line tension  from  and  found in dual-geometry measurements (e.g., for droplets simulated at the same conditions, i.e. at the same temperature with the same interaction potentials, etc.) as (<ref>). It was illustrated with calculations of  from  and  of MD-simulated droplets <cit.> (see Fig. <ref>).

We have also proposed a technique for  dual-geometry measurements based on calculation of the slope of the difference cos - cos at the same value of the chemical potential vs 1/. It can be implemented as a calculation of this difference (<ref>) at = /2 or = /2. The slope will equal / with  the surface tension of the planar liquid–gas interface. This technique has some advantages discussed in section <ref>.

The Laplace and the generalized Young equations in both geometries, as well as the Gibbs adsorption equation (<ref>) for the liquid–gas interface, were obtained and considered for the surface of tension as the liquid–gas dividing surface. At an arbitrary choice of this dividing surface, all these equations are more complicated <cit.>. Even though most researchers use these equations in their simple forms, the real choice of the dividing surface is often different or even not specified explicitly. In simulations, it can be done, e.g., by a boundary detection algorithm, and different algorithms can give <cit.> visibly different dependences of cosθ vs 1/r. This can be considered as an additional source of errors that need to be estimated.

As for the solid–liquid dividing surfaces, we have used the analogue of the Gibbs adsorption equation (<ref>) for the difference ≡ - written under some conditions that are not necessarily met at an arbitrary choice of these dividing surfaces. However, if the same choice is made in both geometries, these quantities will still cancel, both with use of Eq. (<ref>) and in the proposed technique where Eq. (<ref>) will remain valid. In contrast, the line tensions  and  (including their macroscopic value ) do generally depend on the choice of the solid–fluid dividing surfaces. For any specific choice, the dual-geometry approach should work, giving the value corresponding to chosen dividing surfaces.

Finally, let us discuss if the proposed technique can be easily adapted to determine the size-dependent line tension  and/or  (not only their limit  at r →∞). Eq. (<ref>) does not contain solid–fluid surface tensions, but it contains the surface tensions . They can be calculated separately for free spherical/cylindrical droplets of the same radius  of the liquid–gas surface as for the sessile droplet of the same type of geometry (thus, at the same value of chemical potential). However, since (μ,) and (μ,) are different functions, the expression / - / on the right hand side of Eq. (<ref>) is still ( (δμ)^2 ) = ( 1/^2 ) and, therefore, cannot be neglected even in the second order in 1/. Thus, Eq. (<ref>) can hardly be used to obtain more detailed information on the line tension from dual-geometry measurements of cosθ(1/r).



The authors thank  Volkov and  Polovinkin for valuable discussions and comments on the manuscript. The research has been carried out with financial support of the Russian Science Foundation, grant no. 22-13-00151, https://rscf.ru/en/project/22-13-00151/https://rscf.ru/en/project/22-13-00151/






 Paper Title: Predicting nonlinear-flow regions in highly heterogeneous porous media using adaptive constitutive laws and neural networks.





















 



Dodowhile







./fig/








margin=1.5in,includefoot,footskip=30pt


5pt 










§ 
0.5em#1[]


 §.§ 
[runin]0.5em#1[.]


  §.§.§ 
[runin]0.5em#1[.]


  
[runin]0.5em#1[.—]
*

§ 
0pt0.80.6
*

 §.§ 
0pt0.60.4
*

  §.§.§ 
0pt0.40.4
*

  
0pt0.20.2





equationsection


itemsep=-0.2em,topsep=0.3em

itemsep=-0.2em,topsep=0.3em





mystyle 
    0.3em 
    0.3em 
     
     
     
    . 
    .5em 
    #1 #2 (#3) 

plain
thmTheorem[section]
res[thm]Result
rem[thm]Remark
lem[thm]Lemma
cor[thm]Corollary
prop[thm]Proposition
hyp[thm]Hypothesis
assu[thm]Assumption
pb[thm]Problem
defn[thm]Definition
notn[thm]Notation
que[thm]Question
conv[thm]Convention
ex[thm]Example

namedthmstyle 
         
	 
	 
	 
	 
	 
	  
        
namedthmstyle

*genericthm
namedthm[1]

  
  
#1#2currentlabel#2
   






showonlyrefs
inter-unit-product=





















 
 
  |.   
  
⇀ 



 



 


∇
∇
÷∇·
∇·
Δ
 









 
 
 






R 
N 
𝒫 
M 
C 
ℬ 
S 


ε


ḍ
∫_^d
∫_^d×^d


d
D
∂




0
x
y
z
u
v
w
m
a
b
e
d
X
Y
Z
D
H
J
g
f
n
ν
τ
F
K
G
I
φ
Λ
Θ
λ
ϕ
L
V
B
U
D
X
∂


ℱ 
ℰ 
𝕀id 
ℋ
ℌ
𝒟 
G 
𝕂
𝔻
𝕂^-1




C[1]>m#1
L[1]>m#1





∇
  |.   













compat=1.18
matrix




shapes.geometric, arrows

rounded rect = [rectangle, 
rounded corners, 
minimum width=1cm, 
minimum height=1cm,
text centered, 
text width=2.5cm, 
draw=black, 
fill=red!30]

trap = [trapezium, 
trapezium stretches=false,
trapezium left angle=70, 
trapezium right angle=110, 
minimum width=1cm, 
minimum height=1cm, 
text width=2.5cm, 
text centered, 
draw=black, 
fill=blue!30]

rect = [rectangle, 
minimum width=1cm, 
minimum height=1cm, 
text centered, 
text width=2.5cm, 
draw=black, 
fill=orange!30]

arrow = [thick, ->, >=stealth]













Predicting nonlinear-flow regions in highly heterogeneous porous media using adaptive constitutive laws
 and neural networks
    Chiara Giovannini^1 ^2 ^3 Alessio Fumagalli^1 Francesco Saverio Patacchini^2
 August 5, 2024
    ^1 Department of Mathematics, Politecnico di Milano, p.za Leonardo da Vinci 32, 20133 Milano, Italy

^2 IFP Energies nouvelles, 1 et 4 avenue de Bois-Préau, 92852 Rueil-Malmaison, France

^3 Department of Engineering Mechanics, KTH Royal Institute of Technology, Brinellvägen 8, 114 28 Stockholm, Sweden

====================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
In a porous medium featuring heterogeneous permeabilities, a wide range of fluid velocities may be recorded, so that significant inertial and frictional effects may arise in high-speed regions. In such parts, the link between pressure gradient and velocity is typically made via Darcy's law, which may fail to account for these effects; instead, the Darcy–Forchheimer law, which introduces a nonlinear term, may be more adequate. Applying the Darcy–Forchheimer law globally in the domain is very costly numerically and, rather, should only be done where strictly necessary. The question of finding a prori the subdomain where to restrict the use of the Darcy–Forchheimer law was recently answered in <cit.> by using an adaptive model: given a threshold on the flow’s velocity, the model locally selects the more appropriate law as it is being solved. At the end of the resolution, each mesh cell is flagged as being in the Darcy or Darcy–Forchheimer subdomain. Still, this model is nonlinear itself and thus relatively expensive to run. In this paper, to accelerate the subdivision of the domain into low- and high-speed regions, we instead exploit the adaptive model from <cit.> to generate partitioning data given an array of different input parameters, such as boundary conditions and inertial coefficients, and then train neural networks on these data classifying each mesh cell as Darcy or not. Two test cases are studied to illustrate the results, where cost functions, parity plots, precision-recall plots and receiver operating characteristic curves are analyzed.


Keywords: porous medium, heterogeneous medium, adaptive constitutive law, neural network













§ INTRODUCTION



When modeling fluid flow in highly heterogeneous porous media, characterized by irregularly distributed permeability profiles, such as they may arise  when studying underground storage and management of CO2, geothermal energy, nuclear waste or freshwater <cit.>, Darcy's law, which is widespread in this context, may be challenged as noted in many studies <cit.>. Indeed, Darcy's law being a linear relation between fluid velocity (or, more precisely, seepage flux) and pressure gradient, it ignores inertial and frictional effects possibly arising in highly permeable areas, such as macropores, fractures or conduits, where high speeds may develop. Theoretical studies <cit.> provide admissible corrections in this particular setting, which consist in adding a nonlinear term to Darcy's law. The choice we make here is the so-called Darcy–Forchheimer law (cf. later for a generalization of it), which has the form

    - p = ( 1 + c_F√(k)/μ) ν/k ,

where  is the seepage flux, μ and ν are, respectively, the fluid's dynamic and kinematic viscosities, k is the medium's permeability and c_F is a dimensionless empirical inertial term which we refer to as the Forchheimer coefficient. The notation · stands for the Euclidean norm. Note that, for simplicity, we decide here to focus on scalar permeabilities, although tensor generalizations of this law are available <cit.>.

The Darcy–Forchheimer law is more accurate than Darcy's law thanks to its nonlinear term, but it is also more costly to solve numerically because of this same nonlinearity. Therefore, it may not be reasonable resource-wise to solve it everywhere in the domain, but, rather, it may be better to select appropriately a partition of the domain into two subdomains, where in one subdomain Darcy's law is still solved thanks to a slow flow and in the other the Darcy–Forchheimer law is solved because of a fast regime that does not allow to neglect inertial and frictional terms. To implement this, we choose to determine the partition a priori and then, once it is known, to use domain-decomposition techniques <cit.> and thus obtain a numerical strategy which is faster than a global Darcy–Forchheimer approach but still more precise than the global Darcy counterpart. A-posteriori strategies are also envisageable <cit.> but not investigated here. In this paper, we focus on the first step of this methodology, that is, the a-priori calculation of the domain partition, the domain-decomposition step being left for future examination.

One possible way of achieving a-priori knowledge of the Darcy and Darcy–Forchheimer regions may be found in the adaptive model presented in <cit.> and further tested numerically in <cit.>. There, the authors study a constitutive law that, given a fixed threshold on the magnitude of the seepage flux, automatically and iteratively selects Darcy's law or the Darcy–Forchheimer law: if the local flux has smaller magnitude than the threshold, Darcy's law is selected, otherwise the Darcy–Forchheimer law is. As a result of the simulation, one obtains a classification of each mesh cell as belonging to either the Darcy or the Darcy–Forchheimer subdomain. However, although the authors show that such a model is well posed, it is discontinuous at the transitions between the laws, making it rather unsuitable for numerical resolution via a classical, explicit fixed-point algorithm. Instead, in <cit.>, they propose a smoothed version of this adaptive model, where, thanks to a convolution, the law is regularized around the transitions. As a result, this regularized model yields approximations of the sought-after partition but is better suited for numerical purposes. The regularized model is, nevertheless, nonlinear and thus still expensive to run.

To circumvent this, we propose here to combine the regularized model, already available from <cit.>, with machine-learning techniques to provide a fast way of obtaining the partition. The regularized model is used to generate data with varying parameters, such as the Forchheimer coefficient and the boundary conditions and other physical parameters, and then neural networks are trained on these data. The networks output classifications of the mesh cells, by deciding for each cell whether it is Darcy or not. The validation of the networks is carried out by analyzing cost functions, parity plots, precision-recall plots and receiver operating characteristic curves. The codes for the regularized model and the neural networks are written in Python and available at <cit.>, while the underlying discretization and equation resolution are performed using the open source Python simulation tool PorePy <cit.>. 

We start by giving the details of the physical framework and equations at stake and by deriving the threshold on the magnitude of the seepage flux (Section <ref>); we then describe our overall machine-learning methodology and the two test cases we apply it to, the first coming from <cit.> and applicable to landfill management and the second extracted from the SPE10 benchmark reservoir scenario <cit.> (Section <ref>); and, finally, we discuss the results on these test cases (Section <ref>). In Appendix <ref>, we recall and discuss the basics of deep learning which we use in our methodology.










§ PHYSICAL FRAMEWORK



We position ourselves within the modeling context established by <cit.>; to ensure comprehensive understanding, we provide a summary here. To begin, let us elucidate some notations. We denote by Ω⊂^d, d∈, the porous medium and assume it is open, bounded, and possesses a Lipschitz boundary ∂Ω. The outward normal unit vector of ∂Ω is written n. Additionally, we use R_+ = [0, ∞) and N_0 = N∪{0}.

The unknowns in the problems discussed throughout this paper are the fluid’s pressure p: Ω→R and the seepage flux : Ω→R^d. The seepage flux is defined by = ρϕV, where ϕ is the medium’s porosity, and ρ and V are the fluid’s density and velocity, respectively. Furthermore, we use μ, ν and k to represent the fluid’s dynamic viscosity and kinematic viscosity and the medium’s scalar permeability, respectively. We assume that ρ, μ, ν, k Ω→ (0, ∞) and ϕΩ→ (0,1) are space-dependent knowns of the problem.



 §.§ Underlying model equations



In the porous material under consideration, we wish to solve the time-independent problem given by the conservations of mass and momentum. The former conservation is expressed as

    = q   in Ω;

here q Ω→R is a known fluid mass source. For the latter conservation, we choose what is often called the generalized Forchheimer (GF) law <cit.>:

    -  p + f = ( 1 + c_F(√(k)/μ)^m ^m ) ν/k ,

where fΩ→^d is a known vector of external body forces, such as gravity, c_FΩ→ (0, ∞) is the Forchheimer coefficient (also referred to as the Ergün number in the literature <cit.>, containing information on the porosity and tortuosity of the medium <cit.>) and m ∈[1,∞) may be called the Forchheimer exponent. When m=1, we recover the Darcy–Forchheimer (DF) law, as already discussed; qualitatively, Darcy's linear law is retrieved by imposing m=0 or c_F≡ 0. Here, · stands for the Euclidean norm in ^d. The conservation of momentum is also referred to as the constitutive law or seepage law.

With this pair of equations, we associate the following boundary conditions:

    ·n = u_0    on Σ_v,
    
            p = p_0    on Σ_p,

where Σ_v,Σ_p⊂∂Ω are relatively open in ∂Ω (i.e., Σ_v and Σ_p are each the intersection of an open subset of ^d with ∂Ω) and satisfy ∂Ω = Σ_v∪Σ_p and Σ_v∩Σ_p = ∅. Here, u_0Σ_v→ and p_0Σ_p→ are functions setting the conditions on the boundary for the flux and pressure.

Equations (<ref>), (<ref>) and (<ref>) form our model of interest. The underlying theoretical framework, including the appropriate functional spaces and weak formulations, are described in detail in <cit.>.



 §.§ Adaptive model


We can discern two distinct components within the GF law in (<ref>): a linear component given by (ν/k) and a nonlinear one represented by c_F (k/μ)^m u^m (ν/k). When the velocity, or seepage flux, is relatively low, addressing the nonlinear part becomes unnecessary and may be safely disregarded; this amounts to the ubiquitous Darcy law. Conversely, when the velocity is relatively high, the nonlinear part should be taken into account if one wants to study the inertial and frictional aspects of the flow; in this case, m≥ 1 and, as already mentioned, the particularly relevant case m=1 yields the well known DF law. 

In practice, one could always treat the nonlinear component since it adds precision to Darcy's law in any range of velocities. However, because its benefits are extremely limited in low-speed regions, i.e., in most part, of the medium, the gain in precision in these parts of the domain does not outweigh the numerical cost brought by the nonlinearity. It is thus reasonable only to solve the nonlinear model where strictly necessary, that is, where the flux magnitude is above a certain threshold and then keep Darcy's model where the magnitude is below this same threshold. The derivation of this a-priori threshold is given in Section <ref> below.

Denoting by u̅ the threshold, the dichotomy arising from this approach can be formulated as

    - p + f = 
        ν/k    if < u,
    ( 1 + c_F(√(k)/μ)^m ^m ) ν/k    if > u.

We refer to this law as the adaptive law, which, combined with (<ref>) and (<ref>) constitutes the adaptive model. The terminology stems from the fact that this model adapts naturally the constitutive law considered depending on the local Euclidean norm of the flux magnitude. What happens at the transition set { = u̅} is a delicate matter which we do not delve into here. Indeed, it requires the introduction of a multivalued reformulation of the adaptive law which does not enter the qualitative scope of the present discussion; we refer the reader to <cit.> for the complete details of the theory. The well-posedness of such an adaptive model is obtained by a straightforward generalization of <cit.> to space-dependent physical parameters μ, ν, k and c_F, which we further assume to be bounded below and above by strictly positive constants in Ω.

Note that the adaptive law (<ref>) can be rewritten as

    - p + f = ( 1_{ < u̅} + c_F(√(k)/μ)^m ^m 1_{ > u̅}) ν/k ,

where 1_A stands for the indicator function of set A and { < u̅} (respectively, { > u̅}) is a short-hand notation for {∈Ω() < u̅} (respectively, {∈Ω() > u̅}).



 §.§ Flux threshold



We now explain how to choose a priori the flux threshold, and then use this opportunity to define formally the Darcy and GF subdomains and introduce the Reynolds and Forchheimer numbers, which we shall use later. These notions are discussed in details for even more general seepage laws in <cit.>.



  §.§.§ Threshold derivation


Given a flux , the forces - p_D + f and - p_GF + f resulting from the Darcy and GF laws are, respectively, deducted from the following constitutive laws:

    - p_D + f = ν/k ,
    
        - p_GF + f = ( 1 + c_F(√(k)/μ)^m ^m ) ν/k .

Subtracting the equations in (<ref>) and using the second one as reference, we arrive at the following definition of local error d[] Ω→ [0,1), which can be found in <cit.> for the case m=1:

    d[] = c_F(√(k)/μ)^m ^m/1 + c_F(√(k)/μ)^m ^m.


Note that there exists a function ΔΩ×_+ → [0,1) such that, for all ∈Ω, there holds

    d[]() = Δ(, ()),

and Δ(, ·) is an increasing bijection. Then, let δ∈[0,1) and deduce that there is a unique wΩ→_+ so that Δ(, w()) = δ for all ∈Ω. We can then define the flux threshold u̅:

    u̅ = inf_Ω w ∈_+.

Then, using (<ref>) and (<ref>) and inverting Δ, one finds

    w = (δ/1 - δ)^1/mμ/c_F^1/m√(k),

which yields

    u̅ = (δ/1 - δ)^1/minf_Ωμ/c_F^1/m√(k).

In particular, when m = 1, i.e., the reference law is DF, we get

    u̅ = δ/1 - δinf_Ωμ/c_F√(k).




  §.§.§ Subdomains

Given again a flux , an error tolerance δ∈[0, 1) and the corresponding flux threshold u̅, we define the Darcy and GF subdomains as

    Ω_D[] = {∈Ω() < u̅}  and  Ω_GF[] = {∈Ω() > u̅},

which we may respectively refer to as the slow and fast subdomains.

If ∈Ω_D[], we know that solving at  Darcy's law (cf. first equation in (<ref>)) does not generate a local error greater than δ relative to solving at  the GF law (cf. second equation). Indeed, suppose ∈Ω_D[]; then, recalling (<ref>), (<ref>) and (<ref>), we get

    () < u̅≤ w(),

so that 

    d[]() = Δ(, ()) < Δ(, w()) = δ.

Hence, by solving Darcy's law in Ω_δ[] and the GF law in Ω_GF[], we make sure that the local error stays everywhere below δ.



  §.§.§ Reynolds and Forchheimer numbers

Given a flux , we define the Reynolds number Re[]Ω→_+ by

    Re[] = √(k)/μ.

Thanks to this notation, together with the notions of subdomains given in the previous section, given an error tolerance, we may rewrite the adaptive law (<ref>) as

    - p + f = ( 1_Ω_D[] + c_FRe[]^m 1_Ω_GF[]) ν/k .


We also define the mth Forchheimer number Fo_m[]Ω→_+ by

    Fo_m[] = c_F^1/m√(k)/μ = c_F^1/mRe[].

The dimensionless number Fo[]^m quantifies the weight of the nonlinear term in the GF law relative to 1, i.e., relative to the linear term. The first Forchheimer number, with m=1, resulting from considering the DF law as reference, is simply referred to as the Forchheimer number in the literature <cit.> and denoted by Fo[]. Given an error tolerance δ∈[0,1), one can further rewrite (<ref>) as

    - p + f = ( 1_Ω_D[] + Fo_m[]^m 1_Ω_GF[]) ν/k .

Moreover, the local error in (<ref>) can be reformulated as

    d[] = Fo_m[]^m/1 + Fo_m[]^m.

Also note that if at ∈Ω we have Fo_m[]()>(δ/(1-δ))^1/m, then ∈Ω_DF[]; thus, the value (δ/(1-δ))^1/m may be seen as a critical Forchheimer number above which nonlinear effects are surely nonnegligeable.




 §.§ Regularized model


The adaptive law, found in its different forms in (<ref>), (<ref>), (<ref>) and (<ref>), is discontinuous in the flux variable. Indeed, across the transition zone {∈Ω() = u̅}, the law jumps from Darcy's to the GF law. This discontinuity is not well suited for numerical resolution, especially when using a fixed-point algorithm. To circumvent this, the authors in <cit.> propose to use the variational, or energetic, formulation of the adaptive model to regularize the law via a convolution of the underlying energy functional. For completeness, we describe the main idea below, although we refer the reader to <cit.> for the details and the rigorous framework.

As just mentioned, the regularization of the adaptive law is based on an energetic consideration. Indeed, it is shown there that a pair (, p) is solution to the adaptive model (<ref>)-(<ref>)-(<ref>) if and only if it is a saddle point of the energy functional ℰ defined by

    ℰ(φ, ψ) = ∫_Ωf·φ - ∫_Ω q ψ + ∫_Σ_v u_0 ψ - ∫_Ωψ·φ - 𝒟(φ),

where

    𝒟(φ) = 1/2∫_Ω( 1_Ω_D[φ] + 2/m+2Fo_m[φ]^m 1_Ω_GF[φ]) ν/k φ^2.

The term 𝒟 is called the dissipation and is minimized on a certain restricted function space by a certain component of the solution  (cf. <cit.>). The discontinuity of the adaptive law is reflected in the discontinuity of the integrand of 𝒟 with respect to the variable φ. In fact, 𝒟 is not differentiable and merely has a subdifferential, which is equal to the right-hand side of the adaptive law in (<ref>). To make the dissipation smooth, its integrand is convolved with a Gaussian kernel G_ε of mean 0 and standard deviation some fixed ε>0. The convolution is applied on the space of flux magnitudes and yields the regularized dissipation 𝒟_ε, defined as

    𝒟_ε(φ) = 1/2∫_Ω G_ε * [ ( 1_Ω_D[φ] + 2/m+2Fo_m[φ]^m 1_Ω_GF[φ]) ν/k φ^2 ],

and the corresponding energy ℰ_ε, given by

    ℰ_ε(φ, ψ) = ∫_Ωf·φ - ∫_Ω q ψ + ∫_Σ_v u_0 ψ - ∫_Ωψ·φ - 𝒟_ε(φ).

The regularized model then consists in finding a pair (_ε, p_ε) which is a saddle point of ℰ_ε. Equivalently, after differentiation of the regularized dissipation, it consists in solving the model given by (<ref>) and (<ref>) together with the following law:

    - p_ε + f = G_ε' * [ ( 1_Ω_D[_ε] + 2/m+2Fo_m[_ε]^m 1_Ω_GF[_ε]) ν/k _ε^2 ] _ε,

where G_ε' is the derivative of the Gaussian kernel G_ε. The smooth model (<ref>)-(<ref>)-(<ref>) is well posed thanks to <cit.> and we decide to use Raviart–Thomas finite elements and an explicit fixed-point algorithm to generate data on which to train our neural networks (cf. Sections <ref> and <ref>).

Of course, when solving the regularized model as opposed to the adaptive one, we obtain a different partition into slow and fast subdomains, so that Ω_D[] differs from Ω_D[_ε] and so does Ω_GF[] from Ω_GF[_ε]. Nevertheless, thanks to the weak convergence result <cit.>, we expect that, although this has not yet been proved, the regularized subdomains are not far from those stemming from the adaptive model for a reasonably small choice of ε, such as ε = 0.1, which is the value we take for the numerical experiments below.









§ METHODOLOGY AND TEST CASES


In our pursuit of an a-priori partitioning of the domain into Darcy and Generalized Forchheimer (GF) zones, we leverage neural networks to predict the categorization of cells within a given mesh. Specifically, we employ a binary classification scheme, where 0 (or positive) denotes a GF cell and 1 (or negative) a Darcy cell. 



 §.§ Overview of methodology

For details on the fundamental notions of deep learning used to set up our neural networks, tune hyperparameters and validate performance, we refer the reader to Appendix  <ref>. To summarize, for each test case (cf. Section <ref>), having chosen a mesh and a set of input features, such as boundary conditions, our methodology is based on generating a dataset using the codebase <cit.>, which uses the regularized model described in Section <ref> to obtain a domain partition into Darcy and GF regions, on training a dense feed-forward neural network with two or three hidden layers and cross-entropy as cost function, on using cross-validation and recall to tune the underlying hyperparameters, and on evaluating the train and test results with recall, precision and various performance plots as metrics. Figure <ref> provides a flowchart of this methodology, where the ordering of the above steps is specified, as well as the train-test split ratio, the number of cross-validation folds, and the value of the  parameter from the regularized model.





 §.§ Test cases


We apply the above methodology to two test cases, whose results are given in Section <ref> below. Both are two-dimensional, the former coming from an application in landfill management <cit.>, the second taken from Layer 35 of the SPE10 benchmark reservoir scenario <cit.>. We present here the way the data are generated for these test cases as well as the set of hyperparameters optimized.



  §.§.§ Case 1: landfill management 


Landfills serve as critical repositories for waste disposal, necessitating efficient management strategies to mitigate environmental impact. Within this context, understanding the dynamics of rainwater infiltration is pivotal. In this study, we focus on the case where the influx is at the upper boundary acting like rainwater permeating soil. We impose zero-flux conditions on the left and right boundaries, and hydrostatic pressure at the bottom. In Figure <ref>, the vertical axis is aligned with the gravitational force, and we notice the presence of highly permeable channels, or macropores, of which we study two possible configurations: one has two channels traversing the whole domain vertically, exactly as in <cit.>, and the other features a network of seven intersecting channels.





  
Data generation
In this configuration, our dataset comprises s s_train + s_test = 2352 examples, each characterized by a 12-element input vector and a corresponding 2500-element output vector, n_0=12 being the number of input features and k=2500 the mesh size (cf. Figure <ref> for the notation). The input vector includes the influx at the top boundary (u_0, in □; cf. Section <ref>), the Forchheimer coefficient (c_F, dimensionless; cf. Section <ref>), the Forchheimer exponent (m, dimensionless; cf. Section <ref>), the error tolerance (δ, dimensionless; cf. Section <ref>), and the number of channels in the domain (2 or 7) and their associated porosity values. To translate porosities into permeabilities, we use the Kozeny–Carman formula, given by

    K = K_refϕ^3(1-ϕ_ref)^2/ϕ_ref^3(1-ϕ)^2,

where ϕ_ref and K_ref are respectively the reference porosity and reference permeability, which are calculated based on the grain-size distribution WT1 from the nuclear power plant Würgassen <cit.>. The calculation of these parameters is detailed in <cit.> and yields ϕ_ref = 0.35 and K_ref = 1.01e-9□.

To generate diverse values for the input features, we employ specific probability distributions. For the influx, we use a normal distribution with mean, standard deviation and cut-off interval given by the following values:

    μ_u_0 = 0.0105,   σ_u_0 = 0.0035,   I_u_0 = [0.001, 0.1].

Similarly, the Forchheimer coefficient follows a normal distribution with

    μ_c_F = 0.55,   σ_c_F = 0.4,   I_c_F = [0.1, 0.9].

These ranges are inspired by values in <cit.>. The Forchheimer exponent is too generated from a normal distribution with

    μ_m = 1,   σ_m = 1.5,   I_m = [1, 4].

We restrict the interval to m ≥ 1 which corresponds to the Darcy–Forchheimer seepage law (cf. Section <ref>). The error tolerance δ is selected from a uniform distribution within the interval [0.01, 0.25].

Seven values are randomly selected for each parameter. These values are then clipped, resulting in six distinct c_F, seven δ, seven u_0, and four m. Subsequently, we compute the product with every possible combination of these values. The dataset is further diversified by using either 2 or 7 channels, effectively doubling its size, yielding the 2532 examples. For the porosity values in each channel, we generate a uniformly distributed vector of 7× 1176 values ranging from 0.9 to 0.99; from this vector, we select either the first 2× 1176 values for the two-channel case or the entire vector for the seven-channel case.

By adopting these distributions, our aim is to align the dataset with the patterns and insights highlighted in <cit.> and ensure a meaningful and contextually relevant dataset for experimentation. Using these values, we input data into the code from <cit.>, generating a 2500-cell output vector for each input vector, i.e., an output of size k× s, representing the Darcy and GF binary mesh values.



  
Hyperparameters
Using cross-validation with 5 folds based on recall as metric, we tune the learning rate in the range [0.001, 0.1] and the number of nodes per hidden layer across the two networks given in Table <ref>.





  §.§.§ Case 2: Layer 35 of SPE10


Layer 35 of the SPE10 model plays a key role in reservoir simulation due to the intricate interplay between injection and production dynamics, and due to the highly permeable paths it features. In the context of our investigation, this horizontal layer is characterized by a single injection well through which water is pumped at the center of the domain, along with four production wells at the corners (cf. Figure <ref>). From the known depth of Layer 35, namely, 21.336 meters, we make the assumption of hydrostatic pressure on the entire boundary for this particular scenario, although no-flux boundary conditions could also be considered.





  
Data generation
Here, our dataset encompasses s=3888 examples, each characterized by a 4-element input vector and a corresponding 13200-element output vector, where n_0=4 is the number of input features and k=13200 is the mesh size (cf. Figure <ref>). Similar to the preceding case, the input vector incorporates the Forchheimer coefficient, the Forchheimer exponent and the error tolerance. Additionally, the parameter Q (measured in ) is introduced, corresponding to an external source (cf. Section <ref>) and used to assign values to distinct injection and production wells within the mesh.
Common values are retained for the shared parameters across cases; for Q, a normal distribution is employed, characterized by

    μ_Q = 100,   σ_Q = 30,    I_Q = [10, 300].


Nine values are randomly selected for each parameter. These values are then clipped, resulting in eight distinct c_F, nine δ, nine Q, and six m. Subsequently, we compute every possible combination of these values, reaching the 3888 examples.



  
Hyperparameters
Using again cross-validation with 5 folds and recall as performance metric, we tune the learning rate in the range [0.001, 0.1] and the number of hidden layers and nodes across the two networks given in Table <ref>.



§ RESULTS



We discuss now the results for the two test cases described in Section <ref>. 



 §.§ Case 1: landfill management 





  §.§.§ Hyperparameter tuning


Among the networks listed in Table <ref>, we highlight in yellow the one with highest recall, which is considered our main metric of interest (cf. Appendix <ref>). In particular, the table illustrates the argument from Appendix <ref>, namely, that the error rate has little discriminative power in our context: with the exception of networks employing a learning rate of 0.1, the variations in error rate between different networks are almost negligible. 



The percentage difference between the training cost and the validation cost (%cost) is defined as

    %cost = | c_val - c_train|/c_train,

where c_val and c_train are the costs of the validation and training, respectively. This metric measures the potential for overfitting and underfitting, and, despite already using cross-validation to mitigate overfitting, it provides an additional layer of control. Elevated values of %cost suggest overfitting or underfitting tendencies. For instance, in the first row of the second part of Table <ref>, a notable disparity in %cost coincides with notably low precision and recall values, indicative of suboptimal model performance maybe attributable to overfitting or underfitting.

Another metric in Table  <ref> is the variance on the recall. It is calculated by measuring the variance among the five recall values obtained from each fold. This analysis ensures that a high average recall value is not driven by an outlier, but reflects consistent performance across all folds. By examining the variance, we test whether the recall values are consistently high, indicating robust model performance, or whether they vary significantly, suggesting sensitivity to specific subsets of data. 
To compare these variances to the average recall values, we actually look at the standard deviation, which in the highlighted case is 0.08609. This value, compared to the average 0.8444, seems reasonable and further validates the reliability and consistency of the performance of our highlighted neural network.


From Table <ref>, it is evident that when the learning rate is set to 0.1, we encounter notably low precision and recall values. Examination of the cost function in this case yields Figure <ref> across all five folds for the two combinations of hidden layers and nodes. The cost's behavior appears to be nonconvergent, oscillating. 



During cross-validation, we use as threshold 10^-6 for the early-stopping criterion mentioned in Appendix <ref>.





































    
    

    







  §.§.§ Exploring selected network architectures




  
Training
Using the highlighted network in Table <ref>, the training is conducted on the entire training set. Figure <ref> displays the trend of the cost function during training and testing with a different test set (generated using the feature distributions in (<ref>)). Notably, the expected behavior is observed, where the function steadily decreases as the number of iterations increases. Eventually, the function stabilizes, indicating convergence. The similar behavior of the cost function in both training and testing suggests that overfitting is not occurring. In this instance, a threshold of 10^-8 is used for the early-stopping criterion.





  
Testing
In Table <ref>, the network's performance on the test set is presented. We specifically opt for a test set comprising 5% of the total dataset, for a total of 117 samples. Notably, the recall value demonstrates a valuable performance, reaching 92%. Conversely, the precision metric appears lower at 80%. However, given our emphasis on recall over precision, this performance seems satisfactory.



Figure <ref> illustrates that by adjusting the classification threshold, such as to 0.75, a higher recall is obtained. Nevertheless, this adjustment inevitably entails a trade-off with precision; the decision regarding the threshold is thus left to the user. The ROC curve in Figure <ref> showcases an exemplary characteristic, summarized by the notably high AUC value of 0.99662. We recall that an AUC of 1 signifies perfect classification, wherein all positive instances supersede negative instances in score ranking across all thresholds. Additionally, Figure <ref> presents the confusion matrix at a classification threshold of 0.5. The majority of cells are correctly classified. Notably, the number of false negatives (1224) is lower than that of false positives (3720), which aligns with the desired outcome. The parity plot shown in Figure <ref> shows that the test examples closely align with the bisector within the first quadrant. This indicates a near equivalence between the predicted GF cells and the actual ones. While this does not explicitly indicate the accuracy of the predictions in terms of position in the mesh, it nonetheless provides a valuable insight.



  
Testing on different test set
To conduct further verification, we test the network using a dataset stemming from different parameter distributions. 
We use normal distributions for the influx, the Forchheimer coefficient and the Forchheimer exponent with means, standard deviations and cut-off intervals given by:

    μ_u_0 = 0.02,   σ_u_0 = 0.0035,   I_u_0 = [0.00007, 0.05];
    μ_c_F = 0.35,   σ_c_F = 0.45,   I_c_F = [0.05, 0.95];
    μ_m = 2,   σ_m = 1.4,   I_m = [1, 4].

The error tolerance δ is selected from a uniform distribution within the interval [0.008, 0.28]. We then test the network by changing one distribution at a time and, finally, by changing all distributions simultaneously; see results in Table <ref>. There, we note that the metrics exhibit similar values to the test conducted previously (cf. Table <ref>), indicating that performance is consistent over slightly perturbed test sets, although we note a slight degradation in performance when the distribution for m is changed, which may be due to the lower variability of this feature in the training set, which in turn leads the network to have more difficulty generalizing.







  
Visualization
The visualization of the results is done via two test examples in Figure <ref>, accompanied by the corresponding input values in Table <ref>. There, n_chan corresponds to the number of channels and ϕ_i, i∈{1,…,}, to the porosities in each channel. Analysis of Figures <ref>–<ref> reveals instances where the network misclassifies, mostly predicting 'GF' instead of 'Darcy', but this is in line with our objectives since it avoids numerical errors, despite an increased computation time.







 §.§ Case 2: Layer 35 of SPE10





  §.§.§ Hyperparameter tuning


Among the listed networks in Table <ref>, the one highlighted in yellow has highest recall which, as detailed in Appendix <ref>, serves as our primary metric. This network demonstrates remarkable qualities, displaying high recall and precision, and low error rate with respect to the others. However, it is essential to keep in mind the context: in this scenario, we are predict 13200 cells for each example, with the majority being Darcy, hence we expect lower error rate compared to the first case study of Section <ref>. Another noteworthy aspect is the computational time, which is significantly higher than that of the first case (cf. Table <ref>), although this is understandable given the absence of GPU or parallel processing and the large volume of data to be predicted because of the mesh size.



As for the previous case study, we look at the behavior of the cost function in the cases with learning rate equal to 0.1, even if from Table <ref> the results for precision and recall seem quite similar to the other cases. From Figure <ref>, we can observe that the values of the cost functions also fluctuate, but the fluctuations tend to be smaller than in Figure <ref>, remaining consistently below 1 in amplitude in most cases. It is possible that delaying our early stopping may lead to convergence, despite the current appearance suggesting otherwise. In fact, to mitigate the computational cost without significantly compromising performance, we implement a 10^-5 threshold for the early-stopping criterion. This adjustment aims to speed up the calculation process without overly impacting the metrics.





  §.§.§ Exploring selected network architectures




  
Training
Using the network configuration shown in Table <ref>, training is performed on the entire training set. The trends of the cost function during training and testing are illustrated in Figure <ref>. The expected behaviour is observed with a constant decrease of the function as the number of iterations increases. Eventually, the function stabilizes, indicating convergence. The cost function in both training and testing behaves similarly indicating that there is no overfitting. In this scenario, an early stopping criterion with a threshold of 10^-8 is used.





  
Testing
Table <ref> shows the performance of the network on the test set, specially chosen to include 5% of the entire dataset, with a total of 194 samples. The recall value obtained stands at 99%, underlining the robustness of the network.



Both the precision-recall curve (Figure <ref>) and the ROC curve (Figure <ref>) present great characteristics, further confirming the results obtained. In particular, the AUC value is extremely close to 1, estimated at 0.99995. Furthermore, the confusion matrix (Figure <ref>) shows that the false negatives amount to only 579 instances out of the total number of cells, indicating an very efficient classification process. Also, in the parity plot illustrated in Figure <ref>, the plotted sample points closely adhere to the bisector within the first quadrant. Like in Case 1, this indicates a near equivalence between the predicted GF cells and the actual ones. However, it does not provide us with insights into their spatial distribution within the mesh.





  
Visualization
Figure <ref> gives the results for two test examples with the corresponding input values given in Table <ref>. The predictions demonstrate a high level of accuracy, beating that for the previous test case (cf. Figure <ref>), which is probably attributable to a larger dataset and reduced variance; with fewer inputs resulting in less disparate outputs, the performance of the network is significantly improved.








§ CONCLUSION


Given the obtained results, which help us locate the Darcy and generalized Forchheimer areas in a heterogeneous porous medium, the natural next step would be, as already mentioned in the introduction, to implement domain-decomposition methods. Overall, we would then have a complete approach partitioning a priori the domain into fast- and slow-flow regions and then solving the resulting heterogeneous model on a decomposed domain. The outcome is a method which is more precise than a global Darcy model and computationally faster than a global Forchheimer model; this, however, still remains to be studied. This comparative study would further quantify the advancements achieved through our partitioning strategy.

Turning now to the results presented in the present paper, we see a significant disparity in behavior between the SPE10 case (cf. Section <ref>) and the landfill management scenario (cf. Section <ref>), prompting further investigation of potential improvements. The datasets themselves have significant disparities. The landfill dataset, characterized by its smaller size and greater variance, comprises 12 inputs, each demonstrating considerable diversity. In fact, scenarios with only 2 channels feature 5 inputs at zero, while in scenarios with 7 channels, all porosity inputs approximate unity. As a result, these variations yield substantial disparities in the output images, particularly in the distribution of Darcy and generalized Forchheimer areas. Consequently, the networks struggle to generalize the behavior of the inputs effectively. In contrast, in the case of Layer 35 of the SPE10 scenario, with only 4 more homogeneous inputs and outputs, coupled with a larger sample size, the network adeptly generalizes the behaviors.

To resolve these disparities and improve performance, several strategies can be pursued. First, one may expect that enriching and diversifying the dataset for the landfill model should prove useful. We thus conducted cross-validation using various datasets and sample sizes. The findings, given in Table <ref>, reveal a persistent trend indicating that variations in dataset size exert a negligible influence on network performance. This observation underscores the suitability of the selected dataset size for our analytical purposes, suggesting that further expansion of the dataset would not yield benefits. Notably, despite the modest dataset size, the network demonstrates robust performance and effectively captures the underlying output patterns, which shows its efficacy in learning and generalizing from limited data. 




Second, training separate networks for the cases with 2 or 7 channels could be beneficial to Case 1, given the manageable computational overhead due to the smaller output size compared to Layer 35 of the SPE10 case given the coarser mesh (2500 cells vs. 13200 cells). Additionally, integrating dropout regularization during training may strengthen the robustness of the model. While incorporating L^2 regularization into the cost function typically helps mitigate overfitting and improve generalization, Figures <ref> and <ref> show that overfitting does not appear to be an issue in our case. Overall, these approaches offer promising avenues for enhancing our network performance and bridging the observed gap between the two scenarios.

Third, a more extensive hyperparametric tuning than that given here would be necessary to further validate our approach. Indeed, the presented search for optimal hyperparameters is rather narrow and searching for more hidden layers, nodes and optimizing algorithms would improve the validity of our results. 

Fourth, a notable enhancement could be obtained by switching from NumPy <cit.> to PyTorch <cit.> or Tensorflow <cit.>. NumPy is a powerful library for numerical computations, but PyTorch and Tensorflow are prevalent libraries in the deep-learning community which offer additional benefits specifically tailored for building and training neural networks, such as GPU acceleration and automatic differentiation.

Finally, a possible direction to take to generalize our model would be to introduce a permeability tensor into the model, instead of using a scalar. This is not trivial as there is no general consensus on the exact form the generalized Forchheimer law should take in this case; possible formulations are given in <cit.>. This would introduce anisotropies, particularly relevant to three-dimensional scenarios, and testing our neural-network approach in this case would be a significant improvement.

All in all, although there is still work to arrive at a complete understanding and validation of our approach, the results obtained so far are promising. They provide a solid foundation on which to build further refinements and investigations.



§ AKNOWLEDGEMENTS

This paper was made possible thanks to the support of IFP Energies nouvelles, the research laboratory where this work was conducted. The present research is part of the activities of “Dipartimento di Eccellenza 2023-2027”, Italian Minister of University and Research (MUR), grant Dipartimento di Eccellenza 2023-2027.


§ FUNDAMENTALS OF OUR NEURAL NETWORKS



Neural networks are particularly suited for recognizing intricate patterns and extracting essential features. Moreover, their generalizing power to new data makes them a particularly attractive tool for our purpose, which is to predict a vector representing mesh cells that exhibit common patterns <cit.>. We use feed-forward neural networks with dense layers implemented using the Numpy library <cit.> following a methodology outlined in the course <cit.> and using a class-based architecture to improve modularity and maintainability.

We delve below into the various components of our neural networks.




 §.§ Datasets


The datasets employed in this investigation are constructed using the codebase provided by <cit.>, after suitable modification so as to align them with the specific requirements of our study. 

Each analytical scenario we test in this paper involves generating an input vector for each instance, or example, within the dataset and generating a respective binary output vector. Each slot in the output vector represents a cell within our mesh. Upon obtaining each dataset, we divide it into the following sets:

    
  * a training set consisting of examples for tuning hyperparameters, such as number of hidden units and learning rate, and for training the network;
    
  * a test set for measuring generalization performance.

This division is crucial for assessing how well our model generalizes to new, previously unseen data. In particular, in the cross-validation step performed to optimize the hyperparameters, the training set is further subdivided into folds, each in turn playing the role of a validation set (cf. Appendix <ref>). The cross-validation is essential, as tuning hyperparameters on the entire training set is not viable due to prior exposure and may lead to overfitting, and the test set is reserved for reporting the final performance and is usable only once <cit.>.

A fundamental pre-processing step common to both test cases is the normalization of input data with respect to the training set. This enhances the model's ability to learn patterns and generalize across diverse instances within the dataset.



 §.§ Architecture




  §.§.§ Dense layers

We decide to exploit classical dense layers, also known as fully connected layers. These layers exhibit simplicity and flexibility, each neuron establishing deep connections with and receiving input from every neuron in its preceding layer, facilitated through matrix-vector multiplication. 
Broadly speaking, a multilayer perceptron, that is, a dense feed-forward neural network, comprises L+1 layers (with the input considered as Layer 0) and includes L-1 hidden layers, with the output layer being the Lth layer. Each layer can have varying numbers of neurons, namely, n_l where l ranges from 0 to L.



In our specific implementation,  input data is represented by a matrix with dimensions (n_0, s), where n_0 is the number of inputs and s is the size of the dataset. The neural network's operation can be dissected into two phases: the forward propagation and the backward propagation (also known as backpropagation).

During the forward propagation, matrix-vector multiplication is applied to calculate Z, the input to the activation function, which is a mathematical operation that may introduce nonlinearity to the network to learn complex patterns in the data. The dimensions of Z are (n_l, s), where l denotes the current layer. The computation is expressed as

    Z^(l) = W^(l) A^(l-1) + b^(l).

Here, W^(l) is the weight matrix of dimensions (n_l, n_l-1), b^(l) the bias vector with dimensions (n_l, 1), and A^(l-1) the activations from the previous layer or input data, with dimensions (n_l-1, s). The resulting matrix Z^(l) serves as the input to the activation function a, shaping the network's output according to A^(l) = a(Z^(l)). 



  §.§.§ Activation functions


Our neural network employs commonly used activation functions, specifically the Rectified Linear Unit (ReLU) and the sigmoid activation function. The ReLU activation function ReLu→ is defined as

    ReLU(Z) = max(0,Z),

and it is used in all hidden layers. One of the key advantages of ReLU is its computational efficiency compared to other activation functions like the sigmoid or the hyperbolic tangent, as it involves simple element-wise operations. Additionally, ReLU helps mitigate the vanishing gradient problem, occurring when the gradients calculated during backpropagation become extremely small as they propagate backwards through the network layers, eventually causing the gradients effectively to vanish. By producing zero outputs for negative inputs and transmitting positive inputs unaltered, ReLU ensures that it avoids saturation for positive inputs, thus enabling continuous gradient flow during backpropagation. This characteristic distinguishes ReLU from functions such as sigmoid or tanh, which are prone to gradient vanishing. Consequently, ReLU facilitates the information propagation across network layers during backpropagation, thereby promoting expedited convergence and safeguarding against neuron saturation <cit.>.
The output layer, responsible for binary output, employs the sigmoid activation function σ→(0,1):

    σ(Z) = 1/1+ e^-Z.

 The sigmoid activation function effectively squashes the output values between 0 and 1 (cf. Figure <ref>). Moreover, in specific contexts such as domain classification tasks, the output of the sigmoid function assumes a significant interpretation. For instance, in classifying cells into Darcy or Forchheimer domains, the output of the sigmoid function directly corresponds to the likelihood that a given cell belongs to either category; for example, if the output is 0.7, it means that the probability that the cell belongs to the Darcy class is 70.





  §.§.§ Cost function



In the backward propagation, the parameters W and b are trained and updated. Backpropagation involves computing the gradient of the cost, or loss, function with respect to the network's weights. In our case, where the output is a binary classification vector, we use the cross-entropy cost function, namely,


    c(W, b) = - 1/sk∑_i=0^k∑_j=0^s [ y_i, jlog(a_i, j) + (1 - y_i, j)  log(1- a_i, j)].

Here, s is the size of the dataset, k is the size of the output vector (i.e., the size of the mesh), y_i, j is the element in the position (i,j) in the true label vector, a_i, j is the element (i,j) in the probability vector corresponding to the label predictions. Cross-entropy provides a measure of dissimilarity between the true labels and the predicted probabilities, lower values of cross-entropy indicating better model performance.

To optimize computational efficiency and expedite the training process, we implement an early-stopping criterion. Specifically, we halt the training procedure when 15 consecutive iterations yield no discernible improvement. This stopping criterion is compatible with the characteristics of our dataset, as we aim to strike a balance between computational resource use and achieve optimal model performance. Moreover, we introduce an additional criterion: we terminate the network training if the discrepancy in improvements falls below a certain threshold. This threshold varies for each test case and differs between cross-validation and training of the selected network. Specifically, considering that the cross-validation phase is more time-consuming, we opt for a higher threshold value, so that we stop before. Conversely, when computing on the entire training dataset, the computational time is more manageable, allowing for the adoption of a slightly smaller threshold value.




 §.§ Hyperparameters


Hyperparameters are a set of fixed parameters within a neural network architecture that remain unchanged during training. These parameters play a crucial role in shaping the network's performance and behavior. Unlike trainable parameters, like weights, hyperparameters are predetermined either manually or through optimization procedures prior to training. Examples of hyperparameters include the learning rate, the architecture's depth (number of hidden layers), the width (number of neurons) of each layer.

Tuning hyperparameters is fundamental for optimizing the performance of the network, as different configurations can lead to significantly different outcomes in terms of accuracy, metrics, convergence speed, and generalization ability <cit.>. 



  §.§.§ Cross-validation



To tune the hyperparameters, we employ the widely used method of cross-validation, specifically opting for the κ-fold cross-validation, where κ denotes the user-specified number of folds—in our case, 5. Cross-validation provides several advantages, including the avoidance of biased validation-set selection and more effective data use. It ensures that each example is included in the training set exactly once, promoting robust generalization to all dataset samples <cit.>. 

After initially partitioning our dataset into training and test sets according to a split of 95 and 5, the training set is further divided into five folds of approximately equal sizes. Subsequently, a sequence of models is trained, each time using a different fold as the validation set while the remaining folds are used for training. This process, illustrated in Figure <ref>, is repeated for all five splits, resulting in the collection of five vectors of metric values (cf. Appendix <ref> for a detailed description of the metrics). Then, we compute the mean of the chosen metric over the five folds to obtain an overall evaluation of the neural-network model. 




Care should also be taken when splitting the dataset to avoid potential issues related to data order. We shuffle the data to eliminate any inherent ordering in the samples based on labels.





































  §.§.§ Systematic methodology

As shown in Figure <ref>, our methodology involves a systematic approach to model development and evaluation. Initially, we partition the dataset into training and testing subsets, employing the former for cross-validation procedures detailed in Appendix <ref>. Leveraging this framework, we engage in hyperparameter tuning to extract optimal configurations, using them to train the model on the entirety of the training data, producing the final model. We then subject this model to evaluation using the test set and the training results.

In addition to the number of hidden layers and number of nodes, the hyperparameter we tune is the learning rate; we decide to try 4 different values, namely, 0.1, 0.01, 0.0075 and 0.001, thus spanning three different orders of magnitude and keeping a reasonable balance between precision and computation time. 

In the hyperparameter tuning phase, we adopt a random-search strategy. Recognizing that our input vectors span at least two orders of magnitude, we posit the effectiveness of using hidden layers with a high number of nodes. This architectural choice facilitates the diffusion of information across a multitude of nodes, leading to convergence towards a complete vector representation. Additionally, our empirical observations indicate that networks characterized by layers with an increasing number of nodes from input to output tend to show better performance, aligning with the inherent progression from inputs to desired outputs. Accordingly, we explore various random combinations of node and hidden-layer counts (cf. Tables <ref> and <ref>).



 §.§ Metrics



We define four fundamental quantities, referred to as metrics, integral to the evaluation of neural network models predicting mesh-cell classifications. In our context, a single example yields a vector representing mesh cells as output. Specifically, we classify GF cells as positive and Darcy cells as negative, the former being the prediction we wish to miss the least, as, indeed, predicting GF when the truth is Darcy is not as detrimental precison-wise as predicting Darcy when the truth is GF. Four distinct outcomes are possible:


    
  * True Positive (TP): the neural network correctly identifies a positive output cell;
    
  * True Negative (TN): the neural network correctly identifies a negative output cell;
    
  * False Positive (FP): the neural network incorrectly labels a negative output cell as positive;
    
  * False Negative (FN): the neural network incorrectly labels a positive output cell as negative.





  §.§.§ Precision and recall



One widely used metric in evaluating neural network performance is the error rate (Err), representing the frequency of classification errors over a given set of examples. It is defined as

    Err = FP+FN/FP+FN+TP+TN,

where FP, FN, TP and TN are the aggregated quantities for every cell across all examples in the set. An alternative metric, the classification accuracy (Acc), is the frequency of correct classifications and is related to the error rate by Acc = 1 - Err.

However, error rates may not provide a comprehensive view, especially in imbalanced datasets where one class significantly outnumbers the other, which is what we expect in our case where most domain should be classified as negative, that is, as Darcy. In our case, the abundance of Darcy cells necessitates additional metrics, namely, precision (Pr) and recall (Re):


    Pr = TP/TP+FP  and  Re = TP/TP+FN.

Precision quantifies the likelihood that the classifier is correct when labeling an example as positive, while recall measures the probability that a positive example is correctly recognized.

In our study, recall is more relevant than precision due to the detrimental impact of false negatives compared to false positives. Indeed, as already mentioned, misclassifying a cell as GF when it is actually Darcy merely entails the resolution of a nonlinear constitutive law (cf. (<ref>)), incurring an increase in computational time but no numerical inaccuracies. Conversely, misidentifying a cell as Darcy when it is truly GF leads to a substantial numerical error. Consequently, in selecting the most suitable network for our test cases, we prioritize those exhibiting higher recall since indeed a smaller FN yields a higher value. This parallels situations in medical diagnosis, where correctly identifying a patient with condition X constitutes a true positive, while failing to diagnose a patient with X constitutes a false negative—this latter outcome, medical practitioners endeavor to minimize and thereby advocate a low FN rate, i.e., a high recall.



  §.§.§ Performance plots


To evaluate further the performance of our trained neural network, we wish to study its behavior depending on the choice of the threshold in our binary classification. After the training process, the model outputs real values confined in the range [0, 1]; to get a binary value, 0 or 1, we need to fix a threshold separating the output values mapped to 0 from those mapped to 1. The commonly used threshold is 0.5, but it can be adjusted to achieve a desired trade-off between precision and recall. To get the desired trade-off, we examine all possible thresholds using the precision-recall curve. The proximity of the curve to the top-right corner indicates balance in the classifier performance, meaning that the points in this region represent high precision and high recall for the same threshold.

Another commonly used tool for analyzing classifier behavior at different thresholds is the Receiver Operating Characteristics (ROC) curve. Similar to the precision-recall curve, the ROC curve considers all possible thresholds for a given classifier but illustrates the trade-off between the False Positive Rate (FPR) and the True Positive Rate (TPR), where TPR is synonymous with recall and FPR, also called fall-out, is the fraction of false positives out of all negative samples:

    FPR= FP/FP+TN.

In the ROC curve, the ideal performance is reflected by a curve close to the top-left corner, indicating high recall while maintaining a low positive rate. Our ideal scenario aims for the top-left corner, where there is a zero error rate on the negatives and 100% accuracy on the positives. The point nearest to the top left is regarded as the most balanced operating point in the sense that it represents high TPR and high FPR. Moreover, the ROC curve is often summarized using a single metric: the Area Under the Curve (AUC). A higher AUC signifies better overall performance, and a perfect AUC of 1 implies that all positive points have a higher score than all negative points. Notably, AUC is particularly valuable for imbalanced classification problems, as it provides a more informative metric than accuracy.

A further method to compare our neural networks is the parity plot. For each example in the set, we compare the prevalence of the predicted output against the expected prevalence. The prevalence for example j∈{1,…,s} (Prv_j) is the ratio of the sum of positive cells in the jth example to the total number k of cells: 

    Prv_j = 1/k∑_i=1 
    cell i positive^k y_ij,

where y_ij is the entry in the ith row and jth column of the output matrix Y. In the parity plot, the reference line, represented by the diagonal equation y = x, denotes perfect model performance, where, for every example, the model predicts the exact number of positive (i.e., GF) cells as present in the mesh, that is, the true and predicted prevalences match. The distance of each point in the plot, i.e., each example, from the diagonal indicates the extent to which the model deviates in the number of positive cells predicted. While this metric is not flawless, as a model could predict the exact number of positive values but in the wrong cells, it is complemented by other metrics to provide a comprehensive evaluation of the model.

Finally, one of the most comprehensive tools for representing the results of binary classifications are confusion matrices. These are two-by-two matrices in which the rows correspond to the true classes and the columns to the predicted classes. Each entry indicates the frequency with which a cell belonging to the class in the row is classified in the class in the column. Entries on the main diagonal indicate correct classifications, while entries off the diagonal indicate incorrect ones, which provides a granular view of the performance of the model <cit.>.




















abbrv










 Paper Title: Gradient dynamics model for drops of volatile liquid on a porous substrate.






s.hartmann@uni-muenster.de
ORCID ID: 0000-0002-3127-136X
Institut für Theoretische Physik, Universität Münster, Wilhelm-Klemm-Str. 9, D-48149 Münster, Germany
Center for Nonlinear Science (CeNoS), Universität Münster, Corrensstr. 2, 48149 Münster, Germany


u.thiele@uni-muenster.de
http://www.uwethiele.de
ORCID ID: 0000-0001-7989-9271
Institut für Theoretische Physik, Universität Münster, Wilhelm-Klemm-Str. 9, D-48149 Münster, Germany
Center for Nonlinear Science (CeNoS), Universität Münster, Corrensstr. 2, 48149 Münster, Germany


§ ABSTRACT 
  We present a mesoscopic hydrodynamic model for a spreading drop of volatile partially wetting liquid on a solid porous layer of small thickness. Thereby, evaporation takes place under strong confinement, i.e., we consider a drop that spreads on one of two parallel plates that form a narrow gap. Our gradient dynamics model describes the coupled dynamics of the vertically averaged vapor density profile, the drop height profile, and the vertically averaged saturation profile in the porous layer. The underlying free energy incorporates saturation-dependent wettability, capillarity of the drop and within the porous layer, air and vapor entropy, and an entropic contribution relevant close to complete filling of the porous layer. After developing the model we discuss the resulting sorption isotherm and illustrate typical drop spreading and imbibition behavior including the formation of a saturation halo within the porous substrate.


Gradient dynamics model for drops of volatile liquid on a porous substrate
    Uwe Thiele
    August 5, 2024
==========================================================================





§ INTRODUCTION


The behavior of liquids in and on porous media has been studied for a long time <cit.> due to the wide spectrum of natural and technical systems where this is relevant. Examples include the motion of particles with a porous outer layer in a liquid <cit.>, mushy regions occurring during solidification of alloys <cit.>, droplets of simple or complex liquids spreading on and within porous solids <cit.>, reactive drops within a porous medium <cit.>, the dewetting of a liquid film on a porous medium <cit.>, thermocapillary instabilities in systems involving porous layers <cit.>, the infiltration of water into soil <cit.>, the coating of porous substrates <cit.>, drops on polymer layers and brushes <cit.>, and phase behavior in nanoporous materials without <cit.> and with light-switched properties <cit.>.

Particular interest has focused on liquid flow past a porous medium <cit.> as well as on the motion and stability of moving liquid-gas interfaces (imbibition fronts) within a porous medium <cit.> including cases where chemical reactions occur <cit.>. Also the absorption of vapors into porous materials is of interest <cit.>. Note finally that the creation of liquid-infused slippery substrates is often based on porous substrate layers <cit.>.


There are various approaches to modeling the dynamics of the imbibition process, i.e., the flow of liquid into and within porous material. In one approach the wet and dry regions within the porous medium are separated by a sharp imbibition front and the flow within the “wet” part is described via Darcy's law v⃗=-(κ/η)∇ p. It describes the velocity field v⃗, which is driven by the pressure gradient ∇ p between the liquid reservoir and the Laplace pressure inside the pores of the porous medium <cit.>. Here, κ denotes the permeability of the porous medium, and η the dynamic viscosity of the liquid.







A spreading droplet on a permeable membrane (porous medium with only vertical pores) is modeled by <cit.> using a thin-film equation with a sink term that is driven by the same pressure as the convective flux (see their section II). In particular, their  can be written as a gradient dynamics model <cit.> for the film height h:

    ∂_t h  = ∇·[Q∇δℱ/δ h]
        - M(δℱ/δ h - p_vap),

with the mobilities Q=h^3/3η, M=κ/η, the free energy ℱ=∫ [σ (∂_x h)^2/2+ϱ_liq g h^2/2] d x, and p_vap=0. Here, σ is the interface tension, ϱ_liq is the mass density of the liquid, and g is the gravitational acceleration. In this model, no lateral motion within the porous layer is accounted for.

<cit.> also describe in their section III a model for a droplet on a fully saturated porous substrate where lateral Darcy flows may occur. Furthermore, a model with an explicit wet-dry boundary within the porous layer is considered (their section IV). The process of imbibition into the vertical pores is included in Ref. <cit.>, resulting in the coupled dynamics of height profiles on top and inside the porous medium, modeled through two coupled thin-film equations.
Both height profiles of the form h_i(x, t) are measured from the porous-liquid interface, i.e.,
every pore consists of a filled and an empty section.
This approach does not allow for lateral transport within the porous layer and is also followed by <cit.>.

Partially filled pores can be accounted for in phase-field models of infiltration <cit.>. The governing equation describes the dynamics of a filling ratio within the porous medium and is mathematically closely related to a thin-film equation. Such models amend Richards equation <cit.>, which itself extends Darcy's law to unsaturated porous media.
In a related approach, <cit.> have recently presented a gradient dynamics model for a volatile droplet on a grafted polymer substrate, which describes the coupled dynamics of the droplet, the liquid within the substrate and the vapor, also cf. the review in Ref. <cit.>.

Here, we adapt the latter approach by replacing the polymer brush substrate with a porous medium of small finite thickness, and by furthermore incorporating the driving by capillary pressure as employed by <cit.>. The resulting model effectively extends Eq. (<ref>) with additional equations for the vapor dynamics and the dynamics of the liquid within the pores. It can be applied to studies of spreading drops of volatile liquids on porous substrates that are initially unsaturated.
The pursued gradient dynamics approach to mesoscopic hydrodynamics is based on the observation that in the Stokes limit, i.e., without inertia, the dynamics of a hydrodynamic system is simply driven by the minimization of a free energy functional. Such a thermodynamically consistent approach has for a long time been employed for the description of diffusive processes like the decomposition of binary mixtures via the Cahn-Hilliard equation <cit.>, before it was noted by <cit.> that the long-wave equation <cit.>, that describes the dewetting of a thin layer of simple nonvolatile liquid, belongs to the same class of models <cit.>. Subsequently, the finding was extensively employed in the analysis of such systems and, more importantly, extended to more complex thin-film systems <cit.>. Writing the hydrodynamic thin-film equations in gradient dynamics form automatically ensures that the derived equations respect the principles of thermodynamics, i.e., provides a convenient consistency check. This facilitates, in particular, the incorporation of additional effects via inclusion of further degrees of freedom and energy contributions <cit.>. 

The mesoscopic hydrodynamic model is presented in section <ref> as a three-field gradient dynamics. Particular attention rests on the involved transport processes and the underlying energy functional. The subsequent section <ref> discusses the resulting sorption isotherm of the porous medium, while section <ref> analyzes the coupled spreading, evaporation and imbibition dynamics of a sessile drop on a porous substrate. Finally, we conclude in section <ref> with a brief discussion.






§ MESOSCOPIC HYDRODYNAMIC MODEL


 
    



 §.§ Three-Field Gradient Dynamics



  
The gradient dynamics model that we develop in the following includes fields for the local vapor concentration above the substrate and the local amount of liquid within the porous medium. A sketch of the considered geometry is shown in Fig. <ref>.
Importantly, we assume that both, the gas layer and the porous substrate, are thin as compared to lateral extensions such that the local vapor concentration, as well as the local filling ratio of the pores, are approximately homogeneous in vertical direction. For the vapor layer this is ensured by considering a confined geometry, i.e., the gas phase is limited by a horizontal solid plate situated at a distance d from the substrate. Such a setup for the description of the diffuse vapor dynamics was introduced for a volatile drop on a smooth solid substrate by <cit.> and employed for such a drop on a brush-covered substrate in Ref. <cit.>.


Given these assumptions, we use three variables ψ_i(𝐱, t) to describe the state of the system formed by the drop, porous substrate, and vapor layer.
The general gradient dynamics framework reads <cit.>

    ∂_t ψ_i = ∇·[ ∑_b=1^3  Q_ij ∇δℱ/δψ_j]
        - ∑_j=1^3  M_ij δℱ/δψ_j

where the subscripts i, j = 1, 2, 3 refer to the three fields and the 3× 3 mobility matrices  with components M_ij and Q_ij represent the conserved and nonconserved dynamics, respectively.

The variable ψ_1 describes the number of liquid particles within the drop per substrate area and relates to its profile height h as

    ψ_1(x⃗, t)= ρ_liq h(x⃗, t).

where ρ_liq is the constant and uniform particle number density of the liquid.
Further, ψ_3 describes the number of vapor particles within the gas layer per substrate area  and relates to the height-averaged vapor density ρ_vap(𝐱, t) via

    ψ_3(x⃗, t)= ρ_vap(x⃗, t) [d-h(x⃗, t)].

For simplicity, we consider the system as isothermal and assume that the total particle density ρ_tot in the gas layer is constant and uniform. This assumption reflects that the velocity of sound is much larger than the velocity of all dynamics processes we are interested in.

Here, ρ_tot is the sum of the vapor particles and of the remaining particles of the atmosphere constituents, simply labeled as “air”. Therefore,

    ρ_tot = ρ_vap(x⃗, t) + ρ_air(x⃗, t).


Finally, the variable ψ_2(x⃗, t) is the number of liquid particles within the porous layer per substrate area. It is proportional to the local pore filling ratio Θ(x⃗, t), i.e.,

    ψ_2(x⃗, t)= ρ_liq z_0 b  Θ(x⃗, t) = ρ_liqζ(x⃗, t)

where b is the uniform and constant porosity (void fraction) and z_0 is the thickness of the porous layer. We have also introduced ζ(x⃗, t)=z_0 b  Θ(x⃗, t) as the effective height of the substrate-contained liquid.




 §.§ Transport Processes

The conserved mobilities 𝐐 account for three processes: (i) viscous motion within the drop, (ii) Darcy-type diffusive transport of liquid particles within the substrate, and (iii) diffusive transport of vapor particles within the gas phase. This automatically neglects any dynamic coupling between the phases.
In particular, there is no direct dynamic coupling between the liquid in the drop and the liquid in the porous layer, i.e., Q_12=Q_21=0. This should be a good approximation at small porosity, but could in principle be improved along the lines of <cit.>, i.e., by considering Darcy-Brinkman dynamics in the porous layer.
The resulting  mobility matrix is diagonal, namely,

    𝐐
        =
        [ 1/ρ_liq ψ_1^3/3 η                 0                 0;                 0         D̂_porψ_2                 0;                 0                 0  1/k_B T D_vapψ_3 ]

with the vapor diffusion coefficient D_vap. The mobility of the liquid within the porous substrate, D̂_por, needs to be determined, e.g., from Darcy's equation <cit.> and will depend on the permeability κ of and the effective viscosity η_eff=η/b in the porous layer.
Following the phase-field approach of <cit.>, we obtain

    D̂_por = z_0κ k_r(Θ)/η_eff,

where k_r(Θ) is a relative permeability, i.e., an increasing convex function of the liquid saturation Θ. For an example of such a function see Fig. 2 of Ref. <cit.>. Darcy's law is recovered for k_r(Θ) = const.


The nonconserved mobilities 𝐌 account for all three transfer processes, namely, (i) evaporation (or condensation) between the film and the gas phase, (ii) evaporation (or condensation) directly between the substrate and the gas phase, and (iii) liquid transfer between the porous substrate and the drop. We assume that these processes are solely driven by the partial pressures of the phases and assign corresponding Onsager coefficients. The resulting matrix reads

    𝐌 = [  M_im + M_ev        -M_im        -M_ev;        -M_im M_im + M_ev'       -M_ev';        -M_ev       -M_ev'   M_ev+M_ev' ].

where M_ev and M_ev' are the Onsager coefficient of phase change for the drop and the substrate, respectively, and M_im is an imbibition rate coefficient. The particular form of the matrix ensures that ψ_1+ψ_2+ψ_3 is conserved, i.e., ∂_t (ψ_1+ψ_2 +ψ_3) = -∇·j⃗_ψ_1+ψ_2 +ψ_3.

As the porous medium exposes only a fraction of the contained liquid to the surface via the pores, we ensure that the evaporation rate from the substrate surface scales with the porosity employing the simple assumption

    M_ev' = b M_ev.

Keeping the overall structure of the matrix, in principle, more complicated dependencies may be incorporated into the M's.




 §.§ Free Energy Functional


Next, we discuss the underlying free energy functional ℱ[ψ_1, ψ_2, ψ_3]. We employ the long-wave approximation <cit.> and, for convenience, use the fields h, ρ_vap, and ζ as abbreviations for the relations containing only ψ_i. We have

    ℱ =   ∫_Ω[ γ_lg/2 (∇ h)^2_liquid-gas interface energy + γ_sl(Θ)_liquid-substrate interface energy + f_wet(h, Θ)_wetting energy+(h+ζ) f_liq(ρ_liq)_liquid bulk energy
                + (d-h) f_vap(ρ_vap)_vapor energy + (d-h) f_air(ρ_air)_air energy + z_0Γ/2(∇Θ)^2  +  z_0Ψ(Θ)_substrate energy] d x dy,

where f_liq, f_vap, and f_air are bulk liquid, vapor, and air energies per volume, γ_lg is the constant liquid-gas interface energy, γ_sl is the (adaptive) filling ratio-dependent liquid-substrate interface energy, and f_wet(h, Θ) is the (per area) wetting energy.

The remaining terms (last two) represent the energetic description of the porous substrate layer that we base on the work of <cit.>.
Namely, the substrate energy contains the capillary potential Ψ(Θ) and a nonlocal term that encodes the apparent interface energy Γ of imbibition fronts. The factor z_0 transforms the per-volume contributions into a per-area contribution in our height-averaged setting. It also relates the filling ratio Θ to the effective height of liquid in the pores, ζ=z_0 b Θ, see Eq. (<ref>).
The capillary potential is related to the capillary pressure function J(Θ)∼ -∂_ΘΨ. Analytical expressions for J and the relative permeability k_r(Θ) are given, e.g., by the van Genuchten-Mualem model (vGM) <cit.>, where

    k_r    = √(Θ)[1 - (1-Θ^1/m)^m]^2,
    
        J    = p_c  (Θ^-1/m - 1)^1/n.

Here, p_c is the scale of the capillary pressure, n>1 is a measure for pore size distribution, and m=1-1/n.
The capillary pressure constant p_c results from the Laplace pressure inside the pores and relates to the capillary rise h_cap via Jurin's law <cit.>, i.e.,

    p_c = 2 γ_lgcosθ_eq/r_0 = ϱ_liq g h_cap

with the typical pore radius r_0 ≈√(κ/b), the liquid mass density ϱ_liq, and the equilibrium contact angle θ_eq of the liquid on the pure (non-porous) substrate material.



Note that the capillary pressure ∂_ΘΨ is always negative for 0<Θ<1, zero for Θ=1, and undefined for Θ>1.
In other words, the energetic minimum of the capillary potential is at full saturation Θ = 1. The case Θ>1 corresponds to the (unphysical) oversaturation of the porous medium with liquid, i.e., the volume of the contained liquid would be larger than the space available in the pores. To prevent such unphysical behavior in our dynamical model, we additionally incorporate and entropic contribution arising for the two-phase system formed by the liquid and the gas within the pores <cit.>. For the nearly saturated liquid of interest, this gives the entropic contribution to the local free energy

    f_entropic(Θ) = b  ρ_liq  k_BT   (1-Θ)log(1-Θ).

The complementary entropic contribution of the gas in the pores is negligible against the capillary potential that is part of the employed  vGM model (<ref>).
We combine the entropic contribution (<ref>) with the previously employed vGM expression (<ref>) into the capillary potential Ψ. Here, we use this to directly define the pressure

    ∂_ΘΨ = - J + ∂_Θ f_entropic.

At low porosity b, the entropic part only corresponds to a small correction to the capillary pressure (and resulting potential), as can be assessed in Fig. <ref>. There both the capillary pressure ∂_ΘΨ and potential Ψ are illustrated.
Nevertheless, including the entropy is crucial as it ensures that the capillary pressure diverges when approaching complete filling, Θ→ 1, i.e., the minimum of the capillary potential is always at Θ < 1.

To account for the adaptive wettability of the porous substrate we follow an approach similar to <cit.>. Namely, we assume a power law dependence of the Hamaker constant on the volume fraction c of the substrate material in the porous layer.

    f_wet(h, Θ) = A_0 c^a ( h_p^3/5h^5 - 1/2h^2),

where A_0  is the Hamaker constant of a poreless substrate, and a ≥ 0 is some constant exponent. The substrate volume fraction is c=1-bΘ, i.e., the complement to the liquid concentration.

Furthermore, we assume that the interface energy between the drop and the porous layer adapts with the same power law as used in the wetting potential. This ensures consistency with the adaption of a macroscopic substrate-gas interface energy, similar to the case of a polymer-brush substrate discussed in Ref. <cit.>. Hence, we write for the substrate-liquid interfacial energy

    γ_sl(Θ) = γ_sl, dry c^a

with some positive exponent a. This implies that the interface energy decreases when the substrate fills with liquid.

Vapor and air are considered to correspond to ideal gases, cf. Ref. <cit.>. In consequence, their respective free energy densities are purely entropic

    f_vap = k_B T ρ_vap[ log(Λ^3ρ_vap) - 1 ]
          and  
        f_air = k_B T ρ_air[ log(Λ^3ρ_air) - 1 ]

with the mean free path length Λ. This completes the discussion of all modeling elements.



 §.§ The kinetic model

The dynamic equations (<ref>) involve the variations of the free energy functional, which compute as

    δℱ/δ h = ρ_liqδℱ/δψ_1   = -γ_lgΔ h + ∂_h f_wet(h, Θ) + f_liq
    δℱ/δζ = ρ_liqδℱ/δψ_2   = ∂_ζ[ γ_sl(Θ) + f_wet(h, Θ) + z_0 Ψ (Θ) ] + f_liq - Γ/z_0 bΔΘ
    δℱ/δψ_3   = k_B T log( ρ_vap/ρ_tot-ρ_vap),

where in the final equation we have used that the vapor particle density is much smaller than the total gas particle density, which itself is much smaller than the liquid density ρ_vap≪ρ_tot≪ρ_liq, as discussed in more detail by <cit.>.

Inserting the energy variations as well as the mobility matrices (Eqs. (<ref>) and (<ref>)) into the three-field gradient dynamics Eq. (<ref>) gives the resulting dynamic equations 

    ∂_t ψ_1 / ρ_liq   = ∇·[ ψ_3^3/3η ρ_liq^2∇δℱ/δψ_1] - j_ev - j_im,
    ∂_t ψ_2  / ρ_liq   = ∇·[ D_por/ρ_liq ψ_2  ∇δℱ/δψ_2] - j_ev' + j_im,
    ∂_t ψ_3 / ρ_liq   = ∇·[ D_vap (d-h) ∇ρ_vap] / ρ_liq + j_ev + j_ev'.

The transfer fluxes are

    j_ev   = M_ev/ρ_liq(δℱ/δψ_1 - δℱ/δψ_3),
    
        j_ev'    = M_ev'/ρ_liq(δℱ/δψ_2 - δℱ/δψ_3),
    
        j_im   = M_im/ρ_liq(δℱ/δψ_1 - δℱ/δψ_2),

namely, the film evaporation/condensation flux, the evaporation/condensation flux of the liquid contained in the porous layer, and the imbibition flux, respectively.

Finally, for an efficient numerical treatment, we reformulate the governing equation in terms of the dimensionless relative vapor concentration (or relative humidity) ϕ(x⃗, t) = k_BT ρ_vap(x⃗, t)/p_sat and the dimensionless filling ratio of the substrate Θ(x⃗, t) = ζ / (z_0 b).

    ∂_t h    = ∇[ h^3/3η∇δℱ/δ h] - j_ev - j_im
    ∂_t Θ   = ∇[ κ k_r(Θ)/η∇δℱ/δζ] + 1/z_0 b (j_im - j_ev')
    ∂_t [(d-h)ϕ]    = ∇[ D_vap (d-h) ∇ϕ] + ρ_liq k_B T/p_sat (j_ev + j_ev').

This completes the model development. The model is now employed to discuss in the next section the resulting sorption isotherm of the porous medium. The subsequent section <ref> analyzes the coupled spreading, evaporation and imbibition dynamics of a sessile drop on a porous substrate. 


§ SORPTION ISOTHERM OF A POROUS MEDIUM


Before we employ numerical tools to study the behavior of the coupled three-phase model, we consider spatially homogeneous equilibrium states, where h(x⃗, t)=const, Θ(x⃗, t)=const, and ϕ(x⃗, t)=const.
These states can be assessed analytically, as the governing equations strongly simplify when all spatial and temporal derivatives are set to zero. In addition, we assume that the liquid film is either very thick or that it corresponds to a thin precursor layer, i.e., h≫ h_p or  h=h_p, respectively. In both cases the wetting potential is negligible, f_wet(h, Θ)≈ 0.

Evaluating the partial pressures (Eq. (<ref>)) for a homogeneous equilibrium state yields

    δℱ/δψ_1   = 1/ρ_liqf_liq
    δℱ/δψ_2   = 1/ρ_liq{∂_ζ[ γ_sl(Θ) + f_wet(h, Θ) + z_0 Ψ(Θ) ] + f_liq}
    δℱ/δψ_3   = k_B T log( ρ_vap/ρ_tot-ρ_vap).

In thermodynamic equilibrium, the system is at rest. This implies that the transfer fluxes j_ev, j_ev', and j_im are zero as the partial pressures balance. In contrast, a larger pressure in the film than in the substrate causes an absorption of liquid into the substrate until the pressures equilibrate.
Their balance results in coexistence conditions for the three phases.

When a thick homogeneous film is in equilibrium with an ambient vapor phase, the air is considered saturated with vapor.
Hence, comparing the film and vapor partial pressures defines the saturation vapor density, i.e.,

             δℱ/δψ_1     = δℱ/δψ_3|_ρ_vap=ρ_sat

    ⇔40mu          f_liq/ρ_liq     = k_B T log( ρ_sat/ρ_tot - ρ_sat ).

Below, this result is used to relate the liquid free energy f_liq to the vapor saturation pressure. Note that it is independent of the substrate state.

Similarly, equating the film and substrate pressures, δℱ / δψ_1 = δℱ / δψ_2, allows one to define the equilibrium filling ratio for a substrate that is covered with a thick film

    ∂_ζ[ γ_sl(Θ) + f_wet(h, Θ) + z_0 Ψ(Θ) ] = 0.

Next, equating the substrate and vapor pressures, δℱ / δψ_2 = δℱ / δψ_3, gives the equilibrium substrate filling ratio as a function of the ambient vapor density.

    ∂_ζ[ γ_sl(Θ) + f_wet(h_p, Θ) + z_0 Ψ(Θ) ] + f_liq = ρ_liq k_B T log( ρ_vap/ρ_tot - ρ_vap).

Using Eq. (<ref>) to express the liquid bulk free energy through the saturation pressure, the equation can be written as

    ϕ(Θ) = exp{∂_ζ[ γ_sl(Θ) + f_wet(h_p, Θ) + z_0 Ψ(Θ) ] /ρ_liqk_BT}.

The result implicitly defines the sorption isotherm Θ(ϕ), i.e., the substrate state as a function of the vapor concentration.
If the porosity is low (or the substrate is sufficiently thick), the term related to the (bulk) capillary potential dominates the interface terms. This is implied by their scaling behavior with respect to the substrate thickness z_0 or the porosity b.  Thus, we simplify the definition of the isotherm by neglecting the interface terms and write

    ϕ(Θ) = exp{∂_ΘΨ(Θ)/b ρ_liq k_B T}.


Fig. <ref> shows sorption isotherms Θ(ϕ) of a porous substrate at fixed dimensionless capillary pressure constant p_c / (ρ_liq k_BT) = 0.1 for various values of the porosity b and the exponent n.



§ SPREADING DYNAMICS: HALO FORMATION







Next we consider an example for the coupled spreading, evaporation and imbibition dynamics of a sessile drop on a porous substrate.  Fig. <ref> shows four corresponding snapshots of a simulation of the developed three-field porous substrate model, Eq. (<ref>), in a radial geometry. The boundary conditions are chosen such that the vapor particles are allowed to freely escape from the radial domain at the outer boundary (at r=20mm), where the lab humidity ϕ_lab = 10 is enforced. The full set of parameters is given in Tab. <ref>. The parameters represent realistic values, however, our choice is not geared towards a particular experiment. The simulation is performed employing the finite element method implemented in the package  <cit.>. Details regarding the model transformed into radial geometry can be found in Ref. . 

At each of the four chosen times, Fig. <ref> contains three panels: The top panel shows the three transfer fluxes between the phases, namely, liquid-to-vapor evaporation/condensation flux j_ev, substrate-to-vapor evaporation/condensation flux j_ev', and liquid-to-substrate imbibition flux j_im. The central panel visualizes the state of the system with the drop given in dark blue, the ambient vapor concentration presented as a light blue shading, and the porous substrate state as a red-to-yellow shade below the droplet. The shading represents the local filling ratio, where darker colors correspond to a larger liquid content of the substrate. The  bottom panel gives the humidity and filling ratio profiles of gas layer and porous substrate layer, respectively.

Inspection of Fig. <ref> indicates that in the initial second, fast imbibition of liquid occurs underneath the drop (green line in bottom panels of Fig. <ref>(a) and (b)). In parallel, evaporation results in saturation of the vapor phase directly above the droplet that here has the properties of water (red line in bottom panels of Fig. <ref>(a) and (b)). Interestingly, there is some absorption of liquid from the vapor into the porous layer outside the drop (green line in top panels of Fig. <ref>(a) and (b)).
Over the next minutes (Fig. <ref>(c) and (d)), lateral diffusion becomes the dominating process. On the one hand, the vapor diffuses what represent the limiting process for the evaporation of the drop as we are in the diffusion-limited regime, see Ref. <cit.> for an extensive discussion of the different regimes. On the other hand, also the substrate filling is extended far beyond the contact line of the drop. This occurs not only due to diffusion within the substrate but also because there is a significant condensation of liquid from the vapor phase into the substrate. We emphasize that the strength of this effect depends on the ratio between the various rates for the competing diffusive processes. For instance, if the substrate had a larger permeability, transport within the substrate could be faster than transport in the gas phase implying that one could in the first minutes observe evaporation of liquid from the substrate outside the drop into the gas phase.

Here, the condensation continues for about a minute (between Fig. <ref>(c) and (d)). Then, the process reverses and liquid evaporates from the porous layer into the gas phase. This results in the emergence of a quasi-stationary macroscopic “halo” in the substrate filling ratio. The system establishes a continuous flow of liquid from the drop radially outwards through the porous substrate. As it then evaporates thereby accelerating the transport through the gas phase the halo develops.
The resulting halo persists for the full time until the droplet has completely evaporated. Similar results have been obtained for the case of a liquid drop evaporating on a polymer-brush covered substrate <cit.>. However, there the brush does not only adapt its “filling ratio” but also its surface topography.



§ CONCLUSION





We have presented a mesoscopic hydrodynamic model in gradient dynamics formulation for a sessile drop of volatile partially wetting liquid on a solid porous layer that is able to describe situations where several dynamic processes compete that are all driven by the same underlying energy functional. The latter incorporates saturation-dependent wettability, capillarity of the drop and within the porous layer, air and vapor entropy and an entropic contribution relevant close to complete filling of the porous layer that avoids unphysical behavior. The modeled transport and transfer processes are the spreading of the droplet, the evaporation of liquid from the drop into the ambient gas or condensation of vapor from the ambient gas, vapor diffusion in the gas phase, the vertical absorption of liquid from the drop or the vapor into the porous substrate, and the lateral imbibition dynamics of the liquid within the porous substrate layer. These coupled processes are captured by spatially two-dimensional partial differential equations for the particle densities per substrate area in the three regions: liquid drop, porous layer and gas phase. These particle densities are closely related to the drop height profile, the vertically averaged saturation profile in the porous layer, and the vertically averaged vapor density profile, respectively. The approach assumes strong confinement, i.e., we have (i) considered a sessile drop on one of two parallel plates that form a narrow gap, and (ii) a thin porous substrate layer. This has generalized the approach of Ref. <cit.> for drops of volatile liquids within a gap between smooth solid plates, and partially parallels such models for drops on polymer brushes <cit.>. In contrast, here, the imbibition process is incorporated adapting and extending models for the dynamics of the saturation profile in a porous medium that represent diffuse-interface generalizations of Darcy's law and Richards' equation used, e.g., to study gravity-driven saturation fronts within porous media <cit.>.

The incorporation of drop and vapor dynamics now allows for the adaptation of the extended model to a number of practical problems. An example important in the context of rhizospheric soil in the presence of plant root-produced exopolysaccharides is the water transfer between a drop and an amphiphilic porous medium, studied for nonvolatile liquids in <cit.>. There, one could introduce the interplay between ambient humidity and saturation dynamics. Other examples that one could now investigate with consideration of both, imbibition and vapor dynamics, include inkjet printing on layers of paper <cit.> and on microporous membranes <cit.>, and sliding drops on porous substrates <cit.>. Furthermore, the approach could be expanded to cover various suspensions and solutions on porous substrates <cit.> by combining the gradient dynamics approach presented here with elements of Refs. <cit.>.

In the present work we have given the mesoscopic hydrodynamic model in long-wave approximation <cit.>. However, as discussed in Ref. <cit.> in the spirit of the Cahn-Hilliard equation for phase separation one may also use a hybrid model where the mobilities are kept in long-wave approximation but the energy functional is made more precise by using a full-curvature formulation. It has been shown that this often results in more exact models, see e.g., Refs. <cit.>. Here, the full-curvature formulation is obtained by replacing the long-wave metric factor 1/2(∇ h)^2 in Eq. (<ref>) by the full expression √(1+(∇ h)^2). Note that the basic gradient dynamics can, in principle, also be adapted to situations like substrates with chemical or topographic heterogeneities <cit.>. The employed gradient dynamics approach also provides a relatively simple, but by definition thermodynamically consistent approach to more complex situations <cit.>. In this way, our present approach could in the future be further expanded to cover situations where the underlying governing equations are not yet completely known. Examples include systems where imbibition results in the swelling of the porous medium as for thin deformable porous media <cit.>, liquid-infused slippery substrates that are often based on porous substrate layers <cit.>, imbibition into loose material as a powder bed <cit.>, spreading and imbibition of drops of liquid mixtures, and photocatalytically active lubricant-impregnated surfaces <cit.>.



§ DATA AVAILABILITY STATEMENT

The underlying data needed to reproduce shown results are publicly available at the data repository zenodo, see:

Hartmann, S. and Thiele, U. (2024). Data Supplement for: “Gradient dynamics model for drops of volatile liquid on a porous substrate” [Data set]. Zenodo. https://doi.org/10.5281/zenodo.1312782210.5281/zenodo.13127822.





The authors acknowledge support by the Deutsche Forschungsgemeinschaft (DFG) via grant No. TH781/12-1 and TH781/12-2 within Priority Program (SPP) 2171 Dynamic Wetting of Flexible, Adaptive, and Switchable Surfaces; UT would further like to thank the Isaac Newton Institute for Mathematical Sciences, Cambridge, for support and hospitality during the program Anti-Diffusion in Multiphase and Active Flows (ADIW04) where some work on this paper was undertaken. The program was supported by EPSRC grant No. EP/R014604/1. We acknowledge fruitful discussions with Florian Cajot, Philippe Beltrame, Laura Gallardo Dominguez, Patrick Huber as well as many discussions on adaptive substrates with Jan Diekmann, Daniel Greve, and Christopher Henkel at the University of Münster, and with participants of the events organized by SPP 2171.







 Paper Title: Pattern Formation of Freezing Infiltration in Porous Media.






ndjones@caltech.edu

amoure@caltech.edu

 rubyfu@caltech.edu

 Department of Mechanical and Civil Engineering, California Institute of Technology, 1200 E California Blvd, Pasadena CA, 91125



§ ABSTRACT 
Gravity-driven infiltration of liquid water into unsaturated porous media can be a spatially heterogeneous process due to the gravity fingering instability.
When such infiltration occurs in a subfreezing porous medium, liquid water can readily freeze, leading to both the removal of liquid water available for transport and a reduction in local permeability.
As a result of the coupling between gravity fingering and freezing, macroscopic frozen structures can form that record the shape and history of the wetting front.
These structures have been observed in the field in terrestrial snowpack and glacial firn layers and are believed to have profound impacts on how liquid water and its accompanying thermal content distribute during infiltration.
However, a more detailed physics-based understanding of freezing infiltration has been missing.
In this work, we use a thermodynamic nonequilibrium infiltration model to investigate the emergence of refrozen structures during water infiltration into an initially homogeneous and subfreezing porous medium.
From scaling analysis, we recover the relevant nondimensional groups that govern the physics of the freezing infiltration process.
We identify two key mechanisms caused by freezing that reduce the effective infiltration rate, calculated as the maximum depth of infiltration per elapsed time.
In the first mechanism, the effective infiltrate rate decreases because a portion of the liquid water is consumed due to freezing, and such effect can be well quantified by the freezing Damköhler number.
For the second mechanism, we report on a new phenomenon termed secondary fingering, where new flow paths are established in between the primary infiltration channels.
We find that secondary fingering reduces the degree of flow channelization and thus weakens the effective rate of infiltration via flow field homogenization.










Pattern Formation of Freezing Infiltration in Porous Media
    Xiaojing Fu
    August 5, 2024
==========================================================









§ INTRODUCTION

When a viscous fluid infiltrates into an initially dry porous medium under gravity, an instability can readily occur that results in the formation of preferential flow pathways.
In the absence of macro pores or other types of heterogeneities in the porous medium <cit.>, pore-scale competition amongst gravitational, viscous, and capillary forces at the wetting front gives rise to this instability.
This phenomenon, often referred to as the gravity fingering instability, is widely observed in the field <cit.> and studied in laboratory experiments <cit.> for various porous media such as sand, soil, or snow.
Preferential flow in unsaturated media will enhance the effective infiltration rate of the wetting-phase fluid compared to stable flow <cit.>, and has shown to accelerate the arrival of solutes or contaminants <cit.> into deeper regions of the domain.

Gravity fingering has recently gathered a renewed interest in the field of snow and firn hydrology.
In this context, water and its associated thermal content enter the snowpack from radiation-induced surface melting or rain-on-snow events <cit.> and actively modify the thermal budget and permeability structure of the porous ice.
Experiments of gravity-driven water percolation both in the field, as shown in Fig. <ref>(a), and in the laboratory in 3D snow columns have demonstrated that the process is intrinsically unstable, giving rise to preferential pathways <cit.>.
Meanwhile, as snow exists below the freezing point of water, heat transfer and phase change processes readily occur during the infiltration process.
In contrast to isothermal gravity fingering, here the effect of phase change will hinder the vertical infiltration rate of meltwater by converting the amount of liquid water available for transport into solid ice and consequently lowering the local permeability in the snow.
This refreezing of melt within the snowpack forms cylindrical frozen structures known as ice pipes on the order of centimeters in width at the site of preferential flow pathways <cit.>.
Additionally, during contact with discrete snow layers below the surface, vertical flow can be stopped and redirected laterally, forming horizontal frozen structures commonly referred to as ice lenses <cit.>.
Evidence of these refrozen structures has been widely observed in the field <cit.>, both from excavated snow pits [Fig. <ref>(b)] as well as radar detection <cit.> for larger-scale frozen structures such as ice lenses or slabs.
The formation of refrozen structures is known to affect the thermal budget of the snowpack due to latent heat release <cit.>, and the lasting structures affect the bulk hydraulic, thermodynamic, and structural properties of the snowpack. 




Motivated by these observations, there have been a series of experimental and modeling efforts to better understand the infiltration process of water through permeable ice such as snow or firn.
Experimental studies that investigate gravity-driven preferential flow in porous ice typically operate at isothermal conditions <cit.> in which no phase change is permitted to occur.
Additionally, due to the opacity of porous ice, directly imaging the macroscopic infiltration process is difficult, and requires the dissection of experiments a posteriori to investigate the flow patterns <cit.> or, more recently, the employment of non-intrusive techniques such as neutron radiography <cit.> or magnetic resonance imaging <cit.>.
Many numerical studies have been performed that investigate the formation of refrozen structures (in particular, ice lenses) using one-dimensional models <cit.> without considering the flow instability.
Preferential flow in porous ice has also been investigated through numerical simulations, but these studies mainly consider isothermal conditions <cit.> or are limited by the assumption of local thermodynamic equilibrium <cit.>.
Despite the efforts mentioned above, a systematic investigation of the mechanical details of infiltration into subfreezing porous media and the resulting mass transfer dynamics is still lacking.

In this work, we investigate gravity-driven unstable infiltration of liquid water into an initially dry porous medium composed of ice and air.
We use a recently developed model <cit.> that couples a two-phase flow model of unstable infiltration with nonequilibrium thermodynamics of ice-water heat transfer and phase change. 
In this model, water flow is prescribed with a Darcy velocity that takes contributions from gravity, capillary pressure, and macroscopic surface tension effects.
The model captures phase change between liquid water and ice by assigning independent temperature fields to the water and ice phases and thus allowing local thermodynamic non-equilibrium (LTNE).
The LTNE assumption allows us to impose phase change rates based on thermodynamic properties and local temperatures of the porous media instead of fixed rates of phase change.
We perform high-resolution numerical simulations of the model on a 2D rectilinear domain and focus our analysis on the nonlinear regime past the onset of the fingering instability.
Our simulations capture the formation of vertical refrozen structures during infiltration over a wide range of flow rates and thermal conditions.
Quantitative analysis of the simulations shows that the effective infiltration rate coupled with phase change can be well predicted by the freezing Damköhler number.
We also show that increased rates of both freezing and water flux can decrease the heterogeneity of the flow field by forming additional preferential pathways.
We report on a new phenomenon we term as “secondary fingering”, where new infiltration channels form between pre-established ones.
This phenomenon occurs only under the regime of high rates of both freezing and flow, and arises primarily due to the permeability reduction from freezing.
The secondary fingering process is in contrast to classical gravity-driven infiltration, in which the infiltrating fluid tends to travel through pre-existing pathways.






§ GOVERNING EQUATIONS

We consider a porous matrix of ice, water, and air, in which the volume-averaged quantities water saturation S(𝐱,t) and porosity ϕ(𝐱,t) determine the phase composition throughout the domain. The volumetric liquid water content (LWC) is defined as LWC=ϕ S and the solid fraction of ice is 1-ϕ. 
Flow of water is governed by a Darcy velocity u(𝐱,t). 
The temperature of the water and ice phases are  T_w(𝐱,t) and T_i(𝐱,t), respectively. 
The full details of this model can be found in Ref. <cit.>. 
Here, we provide a brief summary of the governing equations and their underlying assumptions.

The conservation equations for ice and water mass take the form:

    ∂(1-ϕ)/∂ t=-R_m W_SSA(T_int-T_melt), 
    ∂(ϕ S)/∂ t+∇·u=ρ_i/ρ_w R_m W_SSA(T_int-T_melt),

where ρ_i and ρ_w are the densities of ice and water, respectively, T_int is the temperature of the ice-water interface, and T_melt=0 ^∘C is the melting point of ice. 
The phase change rate coefficient R_m is defined as R_m=c_p,w/(L_solβ_sol), where c_p,w is the specific heat capacity of liquid water, L_sol is the latent heat of solidification of ice, and β_sol is the kinetic attachment coefficient for ice growth from liquid water <cit.>. 
The wet specific surface area W_SSA represents the ice-water interfacial area per unit volume:

    W_SSA=SSSA_0/ϕ_0ln(ϕ_0)ϕln(ϕ),

where SSA_0 is the initial specific surface area. 
The interfacial temperature T_int is an averaged pore-scale quantity that is defined from the volume integrated Stefan condition:

    T_int-T_melt/L_sol/c_p,w=-β_solv_n,

where v_n is the volume averaged velocity of the fluid-solid interface and is taken to be positive for freezing.
Following <cit.>, the interface temperature takes the form:

    T_int(T_i,T_w)=k_i/r_iT_i+k_w/r_wT_w/c_p,wρ_w/β_sol+k_i/r_i+k_w/r_w,

where k_i and k_w are the thermal conductivity of the ice and water phases, and r_i and r_w are thermal length scales of the ice and water phases, respectively.
The parameters r_i and r_w represent thermal diffusion lengths that are determined from pore-scale simulations of the generalized Stefan problem for solidification (further details in <cit.>).
Note that the right-hand side expressions in Eqs. (<ref>) and (<ref>) differ by a factor of ρ_i/ρ_w and have opposite signs, which ensures mass conservation during phase change.

The model assumes local thermodynamic nonequilibrium by tracking the temperatures of the ice and water phases independently:

    ∂ [(1-ϕ) T_i]/∂ t = ∇·(D_i (1-ϕ) ∇ T_i ) + W_SSAD_iT_int-T_i/r_i, 
    ∂ (ϕ S T_w)/∂ t + ∇·(u T_w) = ∇·(D_w ϕ S ∇ T_w ) + W_SSAD_wT_int-T_w/r_w,

where and D_i and D_w are the thermal diffusivities of ice and water, respectively. 

To capture unstable gravity-driven infiltration in porous media, here we adopt the unsaturated flow model proposed by Cueto-Felgueroso and Juanes <cit.>:

    u = -K_s(ϕ)k_r(S)∇Π(S),

where K_s(ϕ) is the saturated hydraulic conductivity, k_r(S) is the relative permeability, and Π(S) is the total flow potential.
We use an empirical expression for the snow hydraulic conductivity <cit.>:

    K_s(ϕ)=3(d_i/2)^2ρ_w g/μ_wexp[-0.013ρ_i(1-ϕ)],

where d_i is the average ice grain diameter, g is the gravitational acceleration, and μ_w is the dynamic viscosity of water, all of which we assume to be constant. The relative permeability takes the form of a convex function of saturation <cit.>:

    k_r(S)=(S-S_r/1-S_r)^a,

where S_r is the irreducible saturation and the exponent a>1 is dependent on the porous media. Here, we take S_r=1×10^-3 and a=5. The expression for the flow potential is <cit.>:

    Π(S)=y-ψ(S) - √(κ)∇·(√(κ)∇ S),

where y is the vertical coordinate (increasing with height), ψ(S) is capillary suction derived from Leverett scaling, and κ(S) is the capillary pinning function:

    ψ(S)  =  h_cap S^-1/α{ 1-exp[ β(S-ν_e) ] ( 1+βα/α-1 S ) },
    κ(S) = h_cap^2∫_0^S ψ(S)  dS = h_cap^3 α/α-1 S^α-1/α{ 1-exp[ β(S-ν_e) ] },

where h_cap, α, β, and ν_e are constants that are calibrated from water retention curves for a given porous material <cit.>. 
These functions along with the values for these parameters are shown in Fig. <ref>.
Note that in the case of κ=0, the classical Richards equation for unsaturated flow in porous media is recovered. 






§ PROBLEM SETUP AND SCALING ANALYSIS

We apply the above model to the problem of uniform infiltration in 2D, in which we introduce liquid water into a subfreezing 2D porous medium through the top boundary at a constant flow rate U, as illustrated in Fig. <ref>.
The incoming water temperature is prescribed as T_melt, and the initial temperature of the porous ice is T_i,0.
A no-flux boundary condition is imposed on the lateral boundaries and bottom boundary of the domain for both flow and thermal energy.
We identify the following characteristic scales in our system: |Δ T_0|=|T_i,0-T_melt| is the characteristic temperature, U is the characteristic velocity, L=SSA_0^-1 is the length scale, and hence the product t_c=(SSA_0U)^-1 is the characteristic time scale.
We use SSA_0^-1 as the length scale as it provides a more appropriate volume-averaged measurement of the average grain size.
Note that SSA_0^-1∼ d_i for spherical ice grains. 
The nondimensional equations for mass and thermal conservation of each phase then take the following form:

    ∂ϕ/∂ t   = Da W_SSA(T_int-T_melt), 
    ∂ (ϕ S)/∂ t+∇·u   =ρ_i/ρ_wDa W_SSA(T_int-T_melt), 
    ∂ [(1-ϕ) T_i]/∂ t   = Pe_i^-1[∇·( (1-ϕ) ∇ T_i ) + W_SSA(T_int-T_i)], 
    ∂ (ϕ S T_w)/∂ t + ∇·(u T_w)    = Pe_w^-1[∇·(ϕ S ∇ T_w ) + W_SSA(T_int-T_w)],





The Damköhler number Da sets the ratio between the rate of phase change and the rate of advection: 

    Da=c_p,w|Δ T_0|/L_solUβ_sol.

Applying Eq. (<ref>) to the above definition allows us to rewrite Da as the ratio between the freezing velocity at the ice-water interface, v_n, and the rate of advection: 

    Da=Ste/Uβ_sol=v_n/U.

Here, Ste is the Stefan number, which sets the ratio between the amount of sensible heat to the latent heat of solidification: 

    Ste=c_p,w|Δ T_0|/L_sol.

As Da increases, the rate of freezing becomes comparable to the rate of infiltration.
The thermal Péclet numbers associated with ice and water set the ratio between advection and thermal diffusion:

    Pe_i=UL/D_i,      Pe_w=UL/D_w.

The thermal Péclet numbers control how quickly heat is transferred by diffusion both ahead of the infiltration front as well as in the spanwise direction.
When Pe is sufficiently small, thermal diffusion dominates and any thermal nonequilibrium is due to the kinetic limitations of phase change.
As Pe_i and Pe_w differ only by the thermal diffusivity of each phase, we refer to Pe=Pe_i in this work.

The above scaling analysis introduces the nondimensional groups that control the heat transfer and phase change dynamics of the problem.
The analysis assumes the natural length scale of the porous media (SSA_0^-1), which correlates with the grain size.
However, the original isothermal problem of gravity-driven infiltration bears its own scaling analysis that yields additional nondimensional groups that control the length scale and onset of the fingering pattern.
Applying the same scaling analysis to Eqs. (<ref>) and (<ref>), we arrive at the nondimensional equations for velocity and flow potential:

    u = -K'_s(ϕ)/R_sk_r∇Π', 
    Π'=y'-N_gr^-1ψ'(S)-N_nl(√(κ')∇·(√(κ')∇ S)),

where ψ'(S)=ψ(S)/h_cap, κ'=κ/h_cap^3, and N_gr and N_nl are the gravity number and nonlocal number, respectively.
Note that in the absence of phase change, K_s is constant, and U/K_s is defined as the flux ratio R_s <cit.>.
The influence of these two additional nondimensional groups on the stability of the flow pattern is investigated in Ref. <cit.>, and they are defined:

    N_gr=L/h_cap,      N_nl=Γ/ρ g L^3,

where Γ is a coefficient in the nonlocal free energy potential that is associated with an effective surface tension of the wetting front interface <cit.>.
Following Ref. <cit.>, we take Γ=ρ g κ, and recognizing that κ∼ h^3_cap  from Eq. (<ref>), we arrive at the relation:

    N_nl=N_gr^-3.

Increasing N_gr has been shown to decrease the intrinsic length scale of the gravitational fingering instability, leading to thinner fingers.
In addition to the hydraulic properties of the porous media such as the average grain size diameter d_i or the capillary height h_cap, both the flux ratio R_s and the initial saturation S_0 of the porous media have been shown to influence the onset and nonlinear regimes of the pattern formation process <cit.>.


As we focus on the effects of the supplied flux U and the rate of freezing on the pattern formation process, in this work we do not vary N_gr nor N_nl.
Although the entry flux U has been shown to influence the size and spacing of fingers in an isothermal scenario, in this work, we explore a more narrow range of flux ratios between 1.2×10^-4 to 5.3×10^-4 (U between 6.3 to 27 mm/h), and thus the variability of the finger width and spacing is negligible.
Larger input fluxes may result in significantly wider fingers in these simulations, however here we chose the range of U to reflect appropriate values of surface melt rates for natural snowpack reported in the literature <cit.>.






§ SIMULATION SETUP AND NUMERICAL METHODS

We conduct high-resolution numerical simulations of the model to investigate the dynamics and patterns for flow and porosity structures during water infiltration into subfreezing porous ice.
The simulations are performed on a 2D rectilinear domain of 1 m × 2 m (L_x× L_y) as shown in Fig. <ref>(c) and Fig. <ref>.
Initially, the domain is composed of dry porous ice with a uniform initial ice temperature T_i,0≤ 0 ^∘C, uniform porosity ϕ(𝐱,0)= 0.456, and average ice grain diameter d_i=1.5 mm which mimics the hydraulic properties of coarse snow.
We assume the porous media is initially at the irreducible saturation S(𝐱,0)= S_r=1× 10^-3. 
The initial specific surface area is estimated from Ref. <cit.> to be SSA_0=3514 mm^-1.
We approximate the kinetic attachment coefficient for ice growth from Ref. <cit.> as β_sol=800 s/m.

To explore the effect of the relative strength of water freezing rate on the overall infiltration pattern, we systematically vary the surface meltwater flux U and the initial ice phase temperature T_i,0. 
We vary U between 6.3–27 mm/h to explore flow rates near values reported in Ref. <cit.> for natural systems and consider initial ice temperatures between 0 to -20 ^∘C.
The domain has 400×800 (N_x× N_y) mesh elements, with the exception of simulations presented in Sec. <ref>, in which we use a finer mesh resolution.
Equations (<ref>), (<ref>), (<ref>), (<ref>), and (<ref>) are solved using isogeometric analysis <cit.>—a finite element method that implements B-splines as basis functions.
The time advancement scheme is a generalized–α method with an adaptive time step based on the number of Newton-Raphson iterations.
To implement the above methods, we develop a code using the open source libraries PETSc <cit.> and PetIGA <cit.>.








§ RESULTS




 §.§ Impact of freezing on infiltration depth

Freezing can directly hinder gravity-driven infiltration through two mechanisms.
In the first mechanism, mass in the liquid phase is lost as it is converted into ice at the site of freezing and thus the amount of total water available for transport is reduced.
In the second mechanism, an increase in the ice volume fraction leads to a reduction in local permeability which lowers the Darcy velocity of infiltration.
Both of these effects will slow the effective rate of infiltration of the wetting phase into the porous medium compared to the isothermal problem without freezing.
In Fig. <ref>, we demonstrate this effect by plotting the liquid water content LWC and porosity ϕ at a fixed time (t_f=212 min) after constant infiltration of U=27 mm/h for varying T_i,0.
With decreasing T_i,0, the wetting front extends shallower into the domain; at the same time, the local porosity reduction due to freezing becomes more pronounced.




To quantify the decrease of the infiltration depth due to freezing, we define the depth of infiltration L_infil over a fixed time t_f by measuring the vertical distance from the top of the domain to the front of infiltration using the horizontally-averaged liquid water content LWC_x=(ϕ S)_x as shown in Fig. <ref>(a).
In Fig. <ref>(b), we plot L_infil at t_f=212 min as a function of both T_i,0 and U.
We find that, for a fixed U, the depth of infiltration decreases monotonically with decreasing T_i,0; meanwhile, at a fixed T_i,0, L_infil monotonically increases with increasing U.
Applying the appropriate scaling to our data and defining the effective infiltration velocity as u=L_infil/t_f, we find that the curves in Fig. <ref>(b) can be collapsed into a master curve prescribing the normalized effective infiltration velocity u/U as a function of Da, as shown in Fig. <ref>(c).
In particular, the master curve reveals two distinct regimes of infiltration behavior identified by the threshold Da=40.
When Da<40, the rate of freezing is comparable to the flow rate, and the nonlinear interplay between infiltration and freezing leads to complex behavior in u/U.
In this regime, increasing Da reduces u/U drastically, and the exact relationship between u/U and Da is highly nonlinear and weakly dependent on U. 
When Da>40, the rate of freezing dominates the rate of advection, and u/U becomes controlled solely by the rate of freezing and can be prescribed by a linear function of Da.






 §.§ Porosity structure of a freezing finger

Our simulation results thus far demonstrate that the initial fingering pattern creates heterogeneous regions of low porosity in an initially homogeneous porous medium, as demonstrated in Fig. <ref>(b) and Fig. <ref>. 
The reduction in porosity is directly proportional to the rate of freezing and is focused predominantly at the macroscopic ice-water interface where thermal gradients that drive freezing are the strongest (Eq. (<ref>)).
In this section, we study the detailed porosity structure of a freezing finger by considering the formation and infiltration of a single finger into unsaturated permeable ice with subfreezing temperature T_i,0.
We conduct finer resolution simulations than presented in Sec. <ref> and <ref>, using a domain size of 0.1 m × 0.5 m (L_x× L_y) with 256 × 1280 elements (N_x× N_y).
For computational efficiency, we impose a thin “pinning” layer at the top of the domain with a lower permeability to promote the fingering instability more quickly.
For the hydraulic properties of the porous medium, we use approximated parameter values from Ref. <cit.> for coarse snow.

We perform a suite of these simulations by systematically varying U and Ste using the same values in Sec. <ref>.
To analyze the results, we consider a cross-section of ϕ at y=0.375 m [Fig. <ref>(a)] at a sufficiently late time in which local thermal equilibrium is achieved.
The porosity change inside the finger is small due to the fast downward advection of thermal content which quickly brings the inner region of the finger into thermal equilibrium.
Meanwhile, the outer edges of the finger see sustained and significant freezing, as its nonequilibrium state is maintained by the diffusive heat exchange with the non-infiltrated region [Fig. <ref>(b)].
We note however, that the ice-water interface eventually approaches thermal equilibrium [Fig. <ref>(c)] as well, allowing the wall thickness to approach a plateau value as the freezing process halts.




Because we only explore a range of relatively low U that does not influence the finger width, we observe that U only affects the downward finger velocity and does not modify the finger geometry or spanwise heat transfer. 
As a result, the wall thickness and changes in porosity are only affected by Ste. 
The equilibrium configuration of the porosity field exhibits an annulus of a low-porosity region surrounding a still-porous region in which water is still able to percolate through [Fig. <ref>(a)].
We find that both the wall thickness [Fig. <ref>(d)] and the porosity of the ice walls [Fig. <ref>(c)] are only functions of Ste and are independent of the imposed meltwater velocity U.
The porosity inside the finger decreases only slightly with Ste, as shown in Fig. <ref>(c).



 §.§ Formation of secondary fingers

The reduction in the effective infiltration velocity u caused by freezing can be explained, to the first order, by the consumption of melt and the reduction in local permeability.
Here, we highlight an additional mechanism, which we term secondary fingering, that weakens the localized nature of the infiltration flow paths and decreases u.

During unstable infiltration, an initial set of fingers with roughly uniform spacing forms, which we denote as the primary fingers as shown in Fig. <ref>(a) at t=80 min.
The majority of freezing occurs at the ice-water interface of the primary fingers due to high thermal gradients across the interface (Fig. <ref>).
The latent heat release and accompanying diffusive heat transfer from this freezing warms the unsaturated region between each finger, leading to a region of uniform thermal equilibrium behind the fingering front [Fig. <ref>(c)].
As the regions between each finger have remained unsaturated, local permeability is higher than the wetted regions that have undergone freezing.
With flow still supplied at the top of the domain, new preferential pathways, which we term the secondary fingers, emerge in these regions of higher permeability and local thermodynamic equilibrium, as shown in Fig. <ref>(a) for t>80 min.
Since the conditions for flow are more favorable in these regions, the secondary fingers quickly propagate towards the front near the primary fingers until they reach the front and begin to undergo freezing, as shown in Fig. <ref>(b).
The formation and propagation of secondary fingers will slow down the overall propagation of the wetting front, as water available for transport is now diverted into the unsaturated regions in between the primary fingers.




The reduction in effective infiltration rate due to the formation of secondary fingers is also corroborated by a reduction in the degree of channelization of the flow structure.
To demonstrate this, we quantify the lateral flow path distribution by measuring the variance in saturation. 
We introduce S_y(x) as the y-averaged saturation along the x-axis.
We then define the degree of flow channelization as the horizontal variance of S_y: 

    σ^2=⟨S^2_y⟩-⟨S_y⟩^2.

For a uniform flow, regardless of saturation overshoot in the y-direction, S_y will be constant along x and thus σ^2 will be zero.
In Fig. <ref>, we demonstrate that, as the formation of secondary fingers redistributes flow into regions where the primary fingers do not visit, the area of the unsaturated region decreases as more channels occupy the domain, and thus σ^2 decreases.
We also note that before the onset of secondary fingers, σ^2 increases with increasing Ste and can even surpass σ^2 of flow configurations without secondary fingers.
In these scenarios, more freezing occurs at the ice-water interface of each finger for higher Ste (Sec. <ref>), and thus the effective finger width decreases, leading to sharper gradients in S_y(x) and therefore a higher variance.
Additionally, for Ste>0.1 (T_i,0<-7.5 ^∘C), some fingers have the potential to meander and begin to propagate diagonally.
Only the secondary fingers exhibit this meandering behavior, as the porosity structure at the site of their formation is heterogeneous due to freezing from the primary set of fingers.
This can cause slightly more freezing at one side of the meandering finger due to differing local thermodynamic conditions on either side. 
For an initially homogeneous medium, we postulate that this small amount of phase change causes a “runaway" effect, causing the meandering finger to continue propagating diagonally until it combines with another finger.
The coalescence of two fingers here will also raise σ^2, as shown in Fig. <ref>(a) insets.




In Sec. <ref>, we demonstrate that the competition between the rate of phase change and the rate of advection, or the Damköhler number, controls the normalized effective infiltration rate u/U. 
Here, we find that the compounding effects of phase change, as controlled by Ste, and thermal advection, as controlled by Pe, predict the formation of secondary fingers.
In Fig. <ref>, we present a Ste–Pe phase diagram that predicts the emergence of secondary fingers.
Both sufficiently high rates of freezing around the primary fingers (high Ste) as well as sufficient flow available for transport (high Pe) are required to form secondary fingers.
Thus, the product of these nondimensional numbers, StePe controls the emergence of secondary fingers.
As shown in Fig. <ref>, we find that secondary fingers only appear for StePe>10^-4 (dashed curve).









§ CONCLUSIONS

In this paper, we study the effects of freezing on the unstable infiltration of water into porous media.
We use a continuum model <cit.> to describe gravity-driven infiltration coupled with thermal transport and ice-water phase change in an initially unsaturated and subfreezing porous media.
We present high-resolution numerical simulations of the model applied to various thermal conditions and flow rates and identify key nondimensional parameters that govern the patterns of flow and the resulting frozen structures.
Our results demonstrate that the effective infiltration rate can be well characterized by the freezing Damköhler number (Sec. <ref>).
In particular, we identify a regime for Da<40 in which thermal advection and the freezing kinetics are in strong competition, leading to a nonlinear decrease in the effective infiltration rate with increasing Da.
We also identify a linear regime (Da>40) where the effects of freezing dominate the infiltration process and the effective infiltration rate decreases linearly with increasing Da.
In Sec. <ref>, we focus on the characteristics of vertical structures, or ice pipes, and their formation.
We find that the ice pipes are initially created by freezing at the macroscopic ice-water interface while the interior region of the finger remains porous and continues to facilitate downward transport of water.
Upon future cycles of environmental forcing, this saturated region has the possibility to fully freeze which could result in a solid column of ice, as observed in the field <cit.>.
In addition to the consumption of liquid water and the decrease in local permeability, we identify a third mechanism—the emergence of secondary fingers—that reduces the effective infiltration rate by diverting water into new flow channels behind the initial fingering front (Sec <ref>).
We find that both sufficient rates of freezing and adequate thermal transport, as described by the nondimensional product StePe, is required to produce secondary fingers.
By measuring the variance of S_y(x), we show that the creation and propagation of secondary fingers reduces the degree of channelization of the flow field.


An important assumption we make in this study is that local freezing does not alter the capillary pressure function ψ(S).
This function, also called the water retention curve, has been documented only for a handful of various snow types, but only under isothermal flow conditions <cit.>.
In freezing flows through porous media, both the volume-averaged ice grain diameter d_i and the ice fraction (1-ϕ) would increase at the site of phase change, and the capillary suction force may strengthen or weaken locally and modify the infiltration dynamics.
Future studies involving numerical simulations at the pore-scale and/or laboratory experiments would be valuable in determining how the freezing process affects ψ(S).
Further, as natural snowpack is often a stratified, heterogeneous porous medium, the interaction between gravity-driven flow and discretely layered porous media with varying hydraulic and capillary properties, and in particular the formation of ice lenses, will be a topic of future studies.















 Paper Title: Lack of self-similarity in transverse velocity increments and circulation statistics in two-dimensional turbulence.







Laboratoire de Physique de l'École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France


Université Côte d'Azur, Observatoire de la Côte d'Azur, CNRS, Laboratoire Lagrange, Boulevard de l'Observatoire CS 34229 - F 06304 NICE Cedex 4, France



Université Côte d'Azur, Observatoire de la Côte d'Azur, CNRS, Laboratoire Lagrange, Boulevard de l'Observatoire CS 34229 - F 06304 NICE Cedex 4, France



§ ABSTRACT 
  We study the statistics of longitudinal and transverse structure functions, as well as velocity circulation in the inverse energy cascade of two-dimensional turbulence. By means of direct numerical simulations of the incompressible Navier–Stokes equations, we show that transverse structure functions exhibit an anomalous scaling, in contrast to the self-similar behavior of longitudinal ones. We argue that the intermittency of transverse structure functions explains the one observed for velocity circulation statistics, although they are driven by different turbulent structures. 


Lack of self-similarity in transverse velocity increments and circulation statistics in two-dimensional turbulence
    Giorgio Krstulovic
    August 5, 2024
==================================================================================================================



Two-dimensional (2D) turbulence is a paradigmatic system in fluid dynamics as it strongly differs from three-dimensional (3D) turbulence. One of the most remarkable phenomena taking place in 2D turbulent flows is the inverse energy cascade, a process in which energy is transferred towards large scales leading to the formation of large-scale coherent structures <cit.>. This mechanism is relevant in geophysical flows such as atmospheres and oceans, which exhibit quasi-2D properties due to the suppression of motion in one direction induced by confinement, stratification, rotation, or other effects <cit.>. A prime example of this phenomenon is Jupiter's giant red spot <cit.>.

Unlike the direct energy cascade in three dimensions <cit.>, it has been widely accepted that the 2D inverse energy cascade is not intermittent. More precisely, most of the attention has been devoted to the longitudinal velocity increments  δ u^∥_r  = (( + ) - ()) ·r̂, which have been shown to be self-similar for a given length scale r within the inertial range <cit.>. It follows that the longitudinal structure functions (LSF) of order p 


    S_p^∥(r) = ⟨ | δ u^∥_r |^p ⟩∼ r^ζ_p^∥


exhibit scaling exponents ζ_p^∥ that follow a self-similar dependence p/3, consistent with Kraichnan-Leith-Batchelor (KLB) theory <cit.>. Note that this is in stark contrast to 3D homogeneous and isotropic turbulence, where strong fluctuations in the velocity field lead to deviations from the Kolmogorov self-similar theory (K41) predicting ζ_p^∥=p/3. Deviations from this scaling are typically described using multifractal theories <cit.>.  

In recent years, the study of the intermittent behavior in turbulent flows has taken a new direction through the analysis of velocity circulation, a conserved quantity in ideal fluids defined as the line integral of the velocity field around a closed loop _r of linear length scale r


    = ∮__r·.


High-resolution direct numerical simulations (DNS) of 3D turbulence showed an intermittent behavior in the statistics of velocity circulation, with scaling exponents λ_p of a different nature than velocity increments <cit.>. That is, the naive relation between the scaling exponents of structure functions and circulation moments, expressed as λ_p = ζ_p + p, is not satisfied. This behavior was also observed in quantum turbulence <cit.>, where the nature of vortices drastically differs from that of classical fluids, motivating the development of new intermittency theories based on circulation <cit.>. A recent experimental study in quasi-2D turbulence showed that the statistics of velocity circulation is intermittent <cit.>, and was later confirmed in numerical simulations of 2D classical and quantum turbulence <cit.>. This striking observation is in strong contrast with the established self-similar picture of 2D turbulent flows. An explanation for the surprising difference between fluctuations of circulation and velocity increments remains elusive.  

A priori, the relation between increments and circulation can be understood when writing explicitly the velocity circulation around a squared loop of opposite vertices (x_0,y_0) and (x_1,y_1) = (x_0+r,y_0+r)


    Γ_r    =   ∫_x_0^x_0+r[ u_x(x, y_0+r) - u_x(x, y_0) ]  x +  ∫_y_0^y_0+r[ u_y(x_0+r, y) - u_y(x_0,y) ]  y.


We note that the integrands correspond to the transverse velocity increments in different directions. 
The transverse scaling exponents ζ_p^⊥ are defined from the transverse structure functions (TSFs) as


    S_p^⊥(r) = ⟨ | δ^⊥_r |^p ⟩∼ r^ζ_p^⊥,


with δ^⊥_r = δ_r - δ u^∥_r r̂ the transverse velocity increments. It follows that a refined dimensional guess for the circulation scaling exponent would be λ_p=ζ_p^⊥ + p, but this relation does not hold in 3D turbulence.

Scaling exponents of LSFs and TSFs in 3D homogeneous and isotropic turbulence are known to differ, the latter being more intermittent <cit.>. Furthermore, high-resolution DNS showed that the transverse scaling exponents ζ_p^⊥ saturate for high-order moments <cit.>. In 2D, the TSFs have received much less attention.

In this Letter, we study the statistics of velocity increments and circulation in the inverse energy cascade of 2D turbulence by means of DNS. We look independently at the longitudinal, transverse and circulation structure functions and do a general comparison of their scaling properties. 
We also derive an analytical relation between the circulation and the transverse scaling exponents.




We describe the dynamics of a 2D flow using the incompressible Navier-Stokes equations for the scalar vorticity field ω = (×) ·ẑ 


    ∂_t ω + {ω, ψ} = ν∇^2 ω - αω + f,


where the vorticity and the stream function satisfy the relation ω = -ψ, with ν the kinematic viscosity and f an external forcing. The Poisson brackets are defined as {ω, ψ} = ∂_x ω∂_y ψ - ∂_y ω∂_x ψ, and the term -αω is a linear frictional damping that dissipates energy at large scales. 
We analyze data from two different DNS, one provided by Guido Boffetta 2D turbulence database (RUN-A) with N_x = N_y = N=2048 linear collocation points <cit.>, and a second one at a larger resolution performed by our group (RUN-B) with N=6144 <cit.>. 
In both cases, Eq. (<ref>) is solved in a 2D periodic square of side 2π using a pseudo-spectral method and evolved using a second-order Adams-Bashforth (RUN-A) or Runge-Kutta (RUN-B) scheme. 
For RUN-A, the forcing is a random white noise in time with a Gaussian correlation in space, which satisfies ⟨ f(,t) f(0, t') ⟩ = F_0 l_f^2 δ(t-t') exp[-x^2/(2ℓ_f^2)] with ℓ_f the forcing scale <cit.>. For RUN-B, we use a constant in time Gaussian forcing in Fourier space f̂() = exp[-(||-k_f)^2/ (2 Δ k^2)], with k_f = ℓ_f^-1 the forcing wave number and Δ k its width. We choose the parameters α and k_f in order to maximize the width of the inertial range.

Due to the implementation of different forcings in each simulation, the statistical properties of the 2D turbulent flow slightly differ at the forcing length scale, but become negligible in the inertial range. 
The turbulent properties of the flow are averaged in time once the system reaches a statistically steady state. We average 28 fields for RUN-A and 453 fields for RUN-B. The parameters of each DNS are shown in Table <ref>.

The energy spectra for both runs develop a clear k^-5/3 scaling law between the large-scale dissipation wave number k_α∼ℓ_α^-1 with ℓ_α≃ϵ^1/2α^-3/2, and the forcing wave number k_f (see Fig. <ref>). 



In this same range of scales, the energy flux 


    Π(k) = ⟨^<k·[ ·] ⟩


takes negative values, signature of the energy cascading towards large scales. The superscript ^<k indicates that the velocity field considers only wave numbers such that k < || and ϵ = k_f^-1∑_k=1^k_fΠ(k) is the mean energy flux. 
The formation of a condensate at small wave numbers is eluded due to the damping term.
Note that the energy flux is not exactly flat in the inertial range due to the use of a linear damping. It has been observed that employing an hypoviscosity term flattens the energy flux in the inertial range <cit.>.
It is expected that in the limit of an infinite scale separation / →∞, the flux becomes constant in a larger range of scales. Figure <ref> (c) shows the energy spectra compensated by the KLB prediction C ϵ^2/3 k^-5/3 with C = 6 <cit.>.

For the analysis of structure functions, we compute the velocity increments along lines in both the x and y directions.  
The top row of Fig. <ref> shows the LSF defined as in Eq. (<ref>) for even-order moments up to p=10 for both DNS. 



The insets display the local slopes, defined as the logarithmic derivative ζ_p^∥(r) = log S_p^∥ / log r. Note that for large scales r >, the flow is decorrelated and the structure functions become flat, while for small scales r < they follow the viscous scaling S_p ∼ r^2p. In the inertial range, the LSFs present logarithmic corrections which affect the scaling properties of the flow <cit.>. The deviations from a pure power-law scaling are stronger for RUN-A than for RUN-B, probably due to the different resolutions used in each dataset, as well as the peculiarities of the forcing. 
To reduce these systematic deviations, we employ the extended self-similarity (ESS) method with respect to the third-order moment S_3^∥(r) = ⟨ |δ u^∥_r|^3 ⟩ <cit.>. The shaded area indicates the inertial range ℓ_f < r < ℓ_α defined from the ESS method. 
Note that, due to the implementation of different forcings, the moments at the injection scale clearly differ from each other. Nevertheless, this effect in the inertial range is suppressed employing the ESS method, as shown in the bottom row of Fig. <ref>. The new local slopes defined as log S_p / log S_3 display clear scaling properties that follow the well-known self-similar behavior ζ_p^∥ = p/3 <cit.>. 



Following the same procedure, we compute the TSFs. Figure <ref> shows the TSFs for both datasets.
Again, the structure functions display logarithmic corrections in the inertial range that we reduce by employing the ESS method with respect to S_3^⊥. 
Strikingly, the local slopes deviate from the self-similar prediction, suggesting that fluctuations of transverse increments are stronger than longitudinal ones. This behavior is somehow similar to the case of 3D turbulence, where the ζ_p^⊥ saturate for p ≥ 10 <cit.>. We do not observe this saturation in 2D turbulence. 

We now focus on the circulation statistics to see if there is a relation with the intermittency of TSFs.
We compute the velocity circulation around squared planar loops of size r over the whole space. The calculations are performed in Fourier space to preserve the spectral precision from the simulations <cit.>. 
The circulation moments ⟨ |Γ_r|^p ⟩ are shown in Fig. <ref>. 
The local slopes also exhibit poor scaling properties, that are improved by employing the ESS method with respect to the second-order moment ⟨ |Γ_r|^2 ⟩. The insets of the bottom panels in Fig. <ref> show the deviations of the local slopes with respect to the self-similar prediction 4p/3. 





Figure <ref> summarizes the results of this work. 
The left panels show the longitudinal and transverse scaling exponents obtained from both databases using the ESS method, exhibiting a self-similar behavior for the former and an anomalous scaling for the latter. The error bars indicate the maximum and minimum values of the local slopes in the inertial range. 
The right panel shows the circulation scaling exponents λ_p obtained from the ESS method, and we compare them with the transverse exponents via ζ_p^⊥ + p. We also include experimental data extracted from <cit.>, obtained without employing the ESS method. 
The insets show the anomalous deviations of the scaling exponents with respect to the self-similar predictions ζ_p^ss=p/3 and λ_p^ss=4p/3, i.e. 


    ζ_p^∥ - ζ_p^ss/ζ_p^ss,  ζ_p^⊥ - ζ_p^ss/ζ_p^ss,   and  λ_p - λ_p^ss/λ_p^ss.


Despite the clear differences observed in the forcing scales between the two numerical simulations, the scaling exponents obtained from the ESS method in the inertial range are statistically equivalent. The different trend observed in the inset of Fig.<ref>.a might be due to statistical errors and it is less than 3%. Remarkably, both datasets lead to the same transverse anomalous scaling. Moreover, circulation scaling exponents measured in experiments coincide well with the ones measured in both simulations without using the ESS method.
These results show the robustness of the turbulent properties of the inverse cascade in 2D flows, at least when the ESS method is used. 


Starting from Eq. (<ref>) relating circulation and transverse velocity increments, it is possible to show using the Hölder inequality that (see Supplemental Material)


    λ_p ≤ζ_p^⊥ + p.


This inequality implies that, if the TSFs are intermittent, i.e. ζ_p^⊥ < p/3 for p>3, then the circulation moments also are, as it follows that λ_p < 4p/3 for p>3. This relation is obeyed in our simulations, as shown in Fig. <ref>.c.

We remark that in the 3D case, it has been shown that the TSFs saturate for p ≥ 10 to a constant value <cit.>. Equation (<ref>), which also holds in 3D, implies that the saturation of the ζ_p^⊥ leads to dλ_p/dp≤1 for large p. This fact strongly constraints existing models of velocity circulation exponents.
Assuming the bifractal fit proposed in <cit.>, the equality of Eq. (<ref>) would take place at p = 14.5, which is beyond the limit of statistical convergence for current numerical simulations. A similar issue arises with the multifractal model proposed in <cit.> at p ≈ 13. The log-normal model proposed in <cit.> satisfies inequality (<ref>) for all p. However, it fails for p>33 as the predicted exponents start to decrease. 

To conclude, we analyzed the statistical properties of two-dimensional turbulence from direct numerical simulations of the incompressible Navier-Stokes equations of two different databases. By forcing at small scales, a stationary inverse energy cascade is generated which exhibits well-known features, such as the k^-5/3 scaling law for the energy spectrum and a negative energy flux. The study of high-order moments structure functions confirmed the well-known self-similar behavior of longitudinal velocity increments but revealed anomalous deviations for the transverse ones. We argue that the intermittent behavior of this quantity explains the anomalous exponents of velocity circulation through the inequality (<ref>). 
For a future work we propose to investigate the inverse energy cascade in a dual cascade regime to study how non-local effects of the enstrophy cascade affect large-scale fluctuations. 
We also propose to study more realistic geophysical quasi-2D flows such as rotating or stratified turbulence. The characterization of structure functions and circulation moments in these systems could bring some insights into anisotropic effects and could be useful for atmospheric modeling. 



§ ACKNOWLEDGMENTS


  We are grateful to Guido Boffetta for fruitful scientific discussions and for sharing his database. 
  This work was supported by the Agence Nationale de la Recherche through the project GIANTE ANR-18-CE30-0020-01. GK was also funded by the Simons Foundation project "Collaboration in Wave Turbulence" (award ID 651471)
  Computations were carried out at the Mésocentre SIGAMM hosted at the Observatoire de la Côte d'Azur.









[pages=-]sm.pdf



 Paper Title: Transition reversal over a blunt plate at Mach 5.


	[
    Franz Rothlauf
    
==================



§ ABSTRACT  
In this work, the stability and transition to turbulence over an experimental blunt flat plate with different leading-edge radii are investigated. The freestream Mach number is 5, the unit Reynolds number is 6×10^7  m^-1, and the maximum nose-tip radius 3 mm exceeds the experimental reversal value. High-resolution numerical simulation and stability analysis are performed. Three-dimensional broadband perturbation is added on the farfield boundary to initiate the transition. The highlight of this work is that the complete physical process is considered, including the three-dimensional receptivity, linear and nonlinear instabilities, and transition. The experimental reversal phenomenon is favourably reproduced in the numerical simulation for the first time. Linear stability analysis shows that unstable first and second modes are absent in the blunt-plate flows owing to the presence of the entropy layer, although these modes are evident in the sharp-leading-edge case. Therefore, the transition on the blunt plate is due to nonmodal instabilities. Numerical results for all the blunt-plate cases reveal the formation of streamwise streaky structures downstream of the nose (stage I) and then the presence of intermittent turbulent spots in the transitional region (stage II). In stage I, a preferential spanwise wavelength of around 0.9 mm is selected for all the nose-tip radii, and low-frequency components are dominant. In stage II, high-frequency secondary instabilities appear to grow, which participate in the eventual breakdown. By contrast, leading-edge streaks are not remarkable in the sharp-leading-edge case, where transition is induced by oblique first and Mack second modes. The transition reversal beyond the critical nose-tip radius arises from an increasing magnitude of the streaky response in the early stage, while the transition mechanism keeps similar qualitatively.



Transition reversal, boundary layer transition, hypersonic flow, blunt body





§ INTRODUCTION


Hypersonic boundary layer stability and transition have drawn extensive attention due to fundamental and engineering importance. The drag force and the surface heat flux of hypersonic vehicles can be increased by several times after the transition to turbulence. Hence, it is of great interest to predict or control the hypersonic boundary layer transition. Possible transition processes include receptivity to external disturbances, transient growth, eigenmodal growth, parametric resonance and mode–mode interactions, breakdown to turbulence, and bypass mechanisms <cit.>. The detailed transition path depends on the level of environmental disturbances. 

During the design of hypersonic vehicles, the leading edge is usually blunted to mitigate local heating. In the meantime, approximate nose bluntness also delays the boundary layer transition, which favourably reduces aerothermal load. In terms of the physical mechanism, theoretical and computational studies have demonstrated that the appearance of the entropy layer near the bow shock has a stabilisation effect on the main instability modes of boundary layers  <cit.>. According to linear stability analysis, first and second modes are usually the dominant instability modes in supersonic and hypersonic boundary layers, respectively <cit.>. The two modes are of vortical and acoustic nature, respectively. With increased nose bluntness, these two modes tend to be stabilised near the nose region due to increasingly favourable pressure gradient, decreased local Reynolds number and lower Mach number. Furthermore, another inviscid-type instability can be supported inside the entropy layer, which is called the entropy-layer mode <cit.>. Some literatures have found that the entropy-layer normal mode is not remarkable enough to resist the overall stabilisation effect of increasing nose bluntness <cit.>. Unstable entropy-layer
mode exists in the nose region with low frequencies and small grow rates <cit.>. It should be cautious that the entropy-layer normal mode is not identical to the entropy-layer instability, where the latter may be of nonmodal nature.



With regard to experimental efforts, <cit.> performed a systematic study of the nose bluntness effect on the transition, which is a continuation of an earlier publication <cit.>. Three facilities, including two wind tunnels and one shock tunnel, produced the same data trend of the bluntness effect. The slender cone models were tested at various Mach numbers and unit Reynolds numbers with different nose-tip radii. The transition locations were obtained from measurements of surface heat transfer. The informative data from Stetson display two distinct regimes. One is the small-bluntness regime, where the transition location moves
downstream with increasing bluntness. The other is the large-bluntness regime, where
the transition location moves upstream rapidly. This phenomenon is called transition reversal. The reversal performance is clear in a _R–_t plot, where _R and _t are the horizontal and the vertical axis, respectively. Here, _t is the Reynolds number based on the freestream condition and the transition onset location x^∗_t, _R is based upon the nose-tip radius R_n^∗, and the asterisk denotes dimensional quantities. The critical reversal value for the horizontal axis is referred to as _R,c. Owing to the stabilisation effect of the entropy layer, the small-bluntness regime is conceivable. In the large-bluntness regime (_R>_R,c), the transition onset becomes more sensitive to the roughness effect. Stetson speculated that early frustum
transition for the large-bluntness regime was dominated by disturbances originating near the nose tip, and that these disturbances were closely related to the roughness effect. Stetson also reminded that model nose-tips were polished before each
run. However, this operation only removed surface material protrudion, while cavities remained after repetitive runs. Therefore, the roughness effect was not eliminated in a rich dataset.

Although the large-bluntness regime may be sensitive to roughness, there is no solid evidence that roughness is a necessary condition for transition reversal. In addition to Stetson's work, the transition measurements on blunt cones <cit.>, ogive-cylinders <cit.> and blunt flat plates <cit.> have reported transition reversal in different facilities. Part of them explicitly claimed that model surfaces had been polished in certain runs <cit.>. The root-mean-square (r.m.s.) roughness height was within/around a micrometer after polishing. Among these studies, <cit.> detected less frequent events of turbulent bursts after surface polishing. <cit.> found that polishing led to a laminar flow at large bluntness (R_n^∗=15.24 mm), whereas the existence of distributed roughness gave rise to early transition. However, their results did not exclude the possibility that _R,c might be shifted to a higher value at runs with polishing. With polished models, <cit.> reported continuous occurrence of transition reversal with different leading-edge shapes. In other words, transition reversal was not eliminated over a relatively smooth wall. Experimental scientists also pointed out that the transition reversal behaviour can be affected by multiple factors, such as freestream turbulent intensity, surface roughness, pressure gradient,
wall temperature, flow separation, etc. Nevertheless, the dominant factor is difficult to determine.

In terms of mechanism-related measurements, <cit.> examined the
boundary-layer instabilities over sharp and blunt cones by PCB sensors at Mach 10. Beyond the critical nose-tip radius, the transition occurred ahead of the appearance of second-mode instabilities, and the pressure signature was weak. At large bluntness, both <cit.> and <cit.> observed the onset of transition that was away from the entropy swallowing point and close to the nose region.  Schlieren images of <cit.> and <cit.> revealed that the second-mode rope-like structures for sharp cones were replaced by wisp-like structures above the boundary layer edge for largely blunted cones.  Laser-induced-fluorescence-based
schlieren measurements by <cit.> over blunt cones also reported flow structures that were dissimilar to second-mode ones over sharp cones. More recent work of <cit.> ascribed the appearance of elongated structures above the boundary layer at large bluntness to nonmodal instabilities. 


Regardless of the confirmed reversal phenomenon, it is difficult to uncover the physical mechanism by the wind-tunnel experiment only. Challenges include insufficient resolution of flow field, decreased reproducibility of large-bluntness early transition, and reliable theoretical tools that correlate the experimental data satisfactorily. Parallel and nonparallel stability analyses have shown that modal amplification of first-mode, second-mode,
and entropy-layer instabilities is not strong enough to account for transition reversal including Stetson's experiment <cit.>. Due to the non-orthogonality of the linearised Navier–Stokes (N–S) equation, nonmodal instabilities may exist even if the flow is modally stable. <cit.> used the optimal transient growth theory to explain wind-tunnel observations such as nose-tip transition. The transient growth analysis provides the upper bound of the propagating disturbance amplitude, which can be escalated by two to four orders of magnitude <cit.>. Following that, <cit.> attempted to explain transition reversal in this framework. Stationary disturbances, originating from the nose tip and propagating in the entropy layer, were reported to experience significant nonmodal amplification over largely blunted cones at Mach 6. Over various blunt bodies, transient growth analysis has also shown pronounced nonmodal instabilities that may be connected with experimental early transition <cit.>. 

Another popular framework is the resolvent analysis (input–output analysis), which seeks the optimal response of the linear system to external input forcings <cit.>. By computational fluid dynamics (CFD) and resolvent analysis, <cit.> identified 3-D low-frequency streamwise streaks near the nose region of the blunt cone. Furthermore, the streaky response becomes stronger with a larger nose-tip radius. In spite of the shed new insights, the transient growth analysis or resolvent analysis does not provide direct evidence for transition reversal. In real-life situations, the input forcing or the inflow optimal disturbance that leads to maximal energy amplification may be `not physically realizable' <cit.>. Recently, <cit.> have included the shock/disturbance interaction in the resolvent analysis. At Mach 5.8, the improved framework demonstrated that the receptivity of the blunt-cone flow to freestream disturbances is of highly three-dimensional (3-D) nature. In those cases, energetic first-mode and entropy-layer instabilities were identified with tens of kilohertz. 

In recent decades, CFD has been an effective tool to throw light on flow mechanisms. For the concerned high-speed blunt-body flows, numerous direct numerical simulations (DNS) have been conducted to reveal the receptivity process <cit.> and the breakdown scenario <cit.>. The nonlinear interaction induced by oblique waves, propagating inside the entropy layer, was found to be significant to the final transition.   <cit.> compared the numerical inclined structure in the entropy layer with the experimental schlieren image, and reached a qualitative agreement. The lift-up and Orr-like mechanisms were deduced to be responsible for the amplification of entropy-layer instabilities <cit.>. The mechanisms causing the initiation of entropy-layer instabilities were not studied. Among the DNS studies on nonlinear stages, the blunt-cone flow by <cit.>  and the ogive-cylinder forebody flow by <cit.> included the effect of a varying nose-tip radius. However, with perturbations added downstream of the shock, they did not capture the transition reversal. The authors suspect that the failure to reproduce transition reversal is attributed to the absence of upstream receptivity. Applying two-dimensional (2-D) DNS, <cit.> seeded random pressure noise in front of the detached shock of an experimental blunt flat plate at Mach 6. Beyond the critical nose-tip radius, the frequency of the most amplified disturbance deviates from the second-mode semi-empirical value. The resulting growth rate is increased compared to small-bluntness cases, and the temperature fluctuation turns to peak outside the boundary layer. By employing tunnel-like noise on the freestream boundary of 3-D DNS, <cit.> reported inclined structures that resembled experimental schlieren observations over a blunt cone at Mach 8.
Inspired by recent progress, the authors attach importance to the inclusion of upstream receptivity when studying the transition reversal.

In summary, current studies only provide indirect supports for the reason of transition reversal. A more direct connection between the informative CFD/theoretical results and the experimental transition measurement is expected to be established. To the best knowledge of the authors, no high-fidelity numerical simulation has successfully reproduced and explained the transition reversal in wind-tunnel experiments, which is the objective of this paper. To this end, the complete physical process from the shock/diturbance interaction to the transition to turbulence is explored. The detailed flow physics is envisaged to be revealed. The article is organised as follows. The investigated physical problem and flow conditions are described in <ref>. The numerical and theoretical tools are introduced in <ref>. The simulation strategy and numerical details are shown in <ref>. A comparison with experimental transition data is first depicted in <ref>. The flow mechanisms and the related discussions are displayed in <ref>. Concluding remarks are given in <ref>. A brief study of mesh convergence is provided in Appendix <ref>, and additional information is shown in Appendixes <ref> and <ref>.





§ PROBLEM DESCRIPTION

An experimental flat plate model with a cylindrically blunted or sharp leading edge is investigated. The tests were conducted in a Ludwig-type wind tunnel UT-1M in Russia by <cit.>. Figure <ref> gives a schematic drawing of the simulated problem. The difference from the blunt-cone flow is that, entropy swallowing does not seem to occur within a practical distance in the considered blunt-flat-plate flow. Therefore, the impact of the entropy layer is persistent, and the effect of entropy swallowing is excluded. Furthermore, no surface roughness is placed in the present numerical simulation. Thus, the nose bluntness effect tends to be isolated in the flat-plate configuration, which simplifies the problem. The angles of attack and sideslip are considered to be zero. A Cartesian coordinate system (x, y, z) is constructed with the origin at the centre of the cylindrical nose, corresponding to the streamwise, wall-normal and spanwise velocities (u, v, w). An orthogonal body-fitted coordinate system (ξ, η, z) is also defined, which is along the wall-tangent, wall-normal and spanwise directions, respectively. The freestream conditions of the tunnel are given as follows: Mach number M_∞ =5, total temperature T^∗_0 = 468 K, static temperature T^∗_∞ = 78 K and unit Reynolds number ^∗_∞ =6×10^7  m^-1. The subscript `∞' refers to the freestream quantity. The suggested surface temperature is the room temperature T^∗_w =293 K, which corresponds to T^∗_w/T^∗_0≈0.626. The subscript `w' represents the quantity at the wall.

In this paper, the primitive variables are nondimensionalised by the corresponding freestream quantities except that the pressure p is by the freestream ρ^∗_∞u^∗ 2_∞, where ρ represents density. The reference length scale for nondimensionalisation is L^∗_ref=1 mm, which is in the same order of magnitude as the downstream boundary layer thickness. Under the current setting of ^∗_∞ =6×10^7  m^-1, the critical nose-tip radius for transition reversal in the experiment is estimated to be about R_n,critical^∗=2.37 mm. Different nose-tip radii R_n^∗ are considered, namely 0 (sharp leading edge), 2 mm, 2.7 mm and 3 mm. Thus, nose-tip radius that is either smaller or larger than the critical value has been considered. 





§ METHODOLOGY OVERVIEW



 §.§ Numerical simulation

The 3-D compressible N–S equations in the Cartesian coordinate system can be written in a dimensionless conservation form:

    ∂Q/∂ t + ∂F/∂ x + ∂G/∂ y + ∂H/∂ z = 1/(∂F_v/∂ x + ∂G_v/∂ y + ∂H_v/∂ z),

where t denotes time, Q = (ρ, ρ u, ρ v, ρ w, ρ E)^T is the vector of conservative variables, F, G and H represent the vectors of inviscid fluxes, and F_v, G_v and H_v refer to the vectors of viscous fluxes. Detailed expressions of the fluxes can be found in <cit.>. The symbol E represents the total energy per unit mass, and the superscript `T' indicates matrix transpose. A calorically perfect gas (air) model is assumed with a constant specific heat ratio of γ = 1.4. Sutherland's law is adopted to calculate the dynamic viscosity μ, and then the thermal conductivity κ is computed with a constant Prandtl number Pr=0.72. Simulations of the 2-D laminar base flow, the 2-D instability and the full 3-D transitional flow are performed using an in-house finite-volume-based solver called PHAROS <cit.>. This solver has been well applied and validated in relevant physical problems including the 3-D instability of double-cone flows <cit.> and transitional flat-plate boundary layers <cit.>. 

In wind-tunnel experiments such as <cit.> and <cit.>, the transition onset Reynolds number at the critical nose-tip radius is generally in the order of 10^7. As a result, it is computationally expensive to perform DNS with several nose-tip radii. In this work, we conduct affordable direct numerical simulations, and follow the DNS setup of the finest resolution of <cit.> in a supersonic turbulent flat-plate boundary layer. In detail, a combination of dimensionless grid spacings Δ x^+=10∼20, Δ y_w^+<1 and Δ z^+<10 and a seventh-order upwind-biased numerical scheme for construction of inviscid fluxes will be applied. Here, during the evaluation of Δ x^+, Δ y_w^+ and Δ z^+, the friction velocity u_τ is calculated based on the turbulent skin friction coefficient C_f via van Driest II correlation. The correlation of C_f is a function of the momentum thickness Reynolds number _θ. The detailed evaluation procedures can been found in <cit.> and <cit.>. 

Note that the pre-transitional and transitional regions rather than the fully developed turbulent region are the main research concerns of this paper. A mesh convergence study is conducted, and the results are displayed in Appendix <ref>. In general, the adopted mesh resolution is sufficient to achieve the research objective. It should be remarked that the frequency spectra and the amplitude of incoming disturbances were not given in most experimental studies, including the considered one by <cit.>. The frequency-dependent spectra are usually difficult to measure accurately in the wind tunnel. Thus, with insufficient physical information, it is demanding and fortuitous to reproduce the accurate transition locations via numerical simulation. As mentioned by <cit.>, what is useful should be the relative change and trend of data rather than the transition Reynolds number itself. As a consequence, the present numerical simulation is expected to reproduce the reversal phenomenon (tendency) rather than the exact transition locations.    




 §.§ Linear stability analysis

Compressible linear stability theory (LST) is utilised to identify the normal-mode instability. The instantaneous flow field can be represented as the sum of mean and fluctuating quantities. The variable vector ϕ=(u,v,p,w,T)^T, can be decomposed into

    ϕ(ξ,η,z,t)=ϕ̅(ξ,η)+ϕ'(ξ,η,z,t),

where the overbar represents the time-averaged flow. In the present laminar flow analysis, the base flow ϕ̅ is two-dimensional, i.e., w̅=0. With the normal-mode ansatz, the small-amplitude disturbance can be expressed by 

    ϕ' = ϕ̂(η)exp[ i( αξ +β z - ω t)]+ c.c.

Here, ϕ̂ represents the eigenfunction, α is the complex streamwise wavenumber, β is the spanwise wavenumber, ω is the angular frequency, and c.c. denotes complex conjugate. The linearised N–S equation can be derived from (<ref>). Under the quasi-parallel flow assumption, the linear stability equation can be then derived and written as  <cit.>

    H_ηη∂^2ϕ̂/∂η^2+ H_η∂ϕ̂/∂η+ H_0ϕ̂  = 0,

where H_ηη,  H_η,  H_0 are 5×5 matrices. The boundary condition is given by

    {[ û = v̂ = ŵ = T̂ = 0,η = 0;  û = v̂ = ŵ = T̂ = 0,η→∞, ].

while p̂ is solvable from the wall-normal momentum equation on the boundary. The stability equation is finally transformed to a complex eigenvalue problem with respect to α. The operators of equation (<ref>) are related to α, β and the local base flow. The local growth rate σ=-α_i is positive if the mode is unstable. The linear stability analysis is performed by our in-house code, which has been well validated by benchmark cases <cit.>. A global spectral collocation method is utilised to obtain the global spectrum, and a local algorithm is employed to improve the eigenmodes <cit.>. 




§ SIMULATION STRATEGY



 §.§ Numerical method


Figure <ref> displays the simulation strategy. A shock-capturing method is adopted, following a recent practice in the cross-flow-dominated hypersonic transition subject to free-stream acoustic noise <cit.>. For cases with blunted leading edges, the structured mesh is iteratively designed to be entirely aligned with the shock shape near the detached shock. The wall-normal distribution of grid points is clustered near the shock and the wall using a hyperbolic tangent function. In the wall-normal direction, at least 140 points are located in all the fully developed turbulent boundary layers. The grid spacing is uniform in the spanwise direction and mostly in the streamwise direction. In the vicinity of the leading edge, the grid is clustered in the streamwise direction. 

As a  first step, the 2-D laminar base flow is obtained by the same numerical scheme as 3-D simulations, expect that the data parallel-line relaxation method is used for efficient convergence to a steady state <cit.>. For 3-D cases, mixed numerical schemes are employed. The inviscid flux is reconstructed by the seventh-order upwind scheme in the smooth region away from the shock and away from the nose-tip region in the range x>0. The inviscid flux is then calculated using the Harten-Lax-van Leer-Contact (HLLC) Riemann solver <cit.>. In the remaining spatial domain, i.e., near the discontinuity detected by a Ducros sensor <cit.> or near the nose tip (x≤0), the inviscid flux is reconstructed by the second-order monotonic upstream-centered scheme for conservation laws (MUSCL) scheme with limiters <cit.>, and then calculated by the modified Steger–Warming scheme <cit.>. The above setup ensures that the complete shock region is calculated by the second-order MUSCL scheme. The purpose is to obtain a high resolution in the downstream smooth region in conjunction with a robust solution near the strong discontinuity and the large-gradient flow near the nose tip. The viscous fluxes are discretised by the second-order central scheme, which has been applied in previous hypersonic transition simulations <cit.>. To perform time-accurate simulations, the three-stage third-order total variation diminishing Runge–Kutta
method is employed for time marching. The boundary conditions are given as follows:
free-stream conditions are imposed on the far-field boundary, extrapolation is used on the
outflow boundary, and isothermal, no-slip and no-penetration conditions are enforced on
the wall boundary. The periodic condition is utilised on the spanwise boundary. Next to the outflow boundary, sponge zones are placed to minimise the reflection
of disturbances <cit.>.



 §.§ Case description




Case information about the computational domain and mesh resolution is given in table <ref>. The symbols L_x and L_z represent the length and the width of the computational domain, and n_x, n_y and n_z refer to the mesh node numbers in the three directions. Cases R0, R2, R2.7 and R3 are simulated to reveal the bluntness effect on the flow transition. In the early pre-transitional stage of blunt-plate cases, about 22 points are used in the spanwise direction for each spacing of the most amplified streamwise streak. The spanwise width of the computational domain is able to contain around 13 most amplified streaks. We have also conducted a prior test for case R2 by either reducing the spanwise width to two-thirds of the baseline value or increasing n_y from 251 to 351. No visible change was found in the characterisation of steady and unsteady flow fields. The mesh convergence with respect to n_x and n_y is also confirmed in the 2-D receptivity study in Appendix <ref>. In the streamwise direction, around 15 points are employed for the pronounced high-frequency short-wavelength structure contained in the wave packet of blunt-plate cases. The streamwise length of the computational domain for case R2 was found to be insufficient to establish the fully developed turbulence. As a consequence, the domain of case R3 is further extended. 

To examine the mesh resolution effect, a new case R3F is simulated with an evidently shorter streamwise length and a finer resolution. The grid spacings (Δ x^+,Δ y^+,Δ z^+) of case R3F approach the DNS setup for hypersonic wall-bounded flows by <cit.> and <cit.>. The results in Appendix <ref> indicate that the current mesh resolution is sufficient for 3-D transitional studies. Another case R2C is simulated with a smaller total number of grid points and thus less computational cost. Case R2C presents the same flow phenomena as the baseline case R2. Data collected from a long-time simulation of case R2C will be used for spectral proper orthogonal decomposition (SPOD) analysis. 



 §.§ Broadband disturbance model

To trigger the transition and mimic a real-life environment, broadband disturbances are added on the farfield boundary in front of the shock. As a consequence, the receptivity to freestream disturbances is included in the simulation. The receptivity process has been extensively investigated for hypersonic flows over flat plates, wedges, cones, etc. at different flow conditions, e.g., by <cit.>. The instability waves in boundary layers are found to be about three to five times more receptive to slow
acoustic waves than fast acoustic, vorticity and entropy waves. Furthermore, the
intensity of acoustic disturbances increases rapidly with Mach
number. The more pronounced receptivity to slow acoustic waves has also been verified in response to broadband disturbances <cit.>. Acoustic disturbances, radiated mostly from the nozzle-wall turbulent boundary layer, tend to dominate the overall
disturbance environment of wind tunnels at Mach 2.5 or above <cit.>. A more recent combined experimental and numerical study confirmed that the slow acoustic wave is the dominant acoustic mode in noisy hypersonic wind tunnels <cit.>. Therefore, the slow acoustic wave is adopted as a representative disturbance model for the freestream boundary condition. 

In this paper, the three-dimensional broadband acoustic-wave model of <cit.> is employed. The merit of the broadband model is that the flow is able to select the preferential frequency and wavenumber naturally rather than adopt artificially imposed ones. The dimensionless pressure perturbation for a Fourier mode (m, n) with respect to the frequency and the spanwise wavenumber is given by 

    p'_m,n = A_m[ cos (β _nz + ψ _m,n) + cos ( - β _nz + ψ _m,n)]cos (α _mx - ω _mt + φ _m),

where
m = 1,2,...,M_f and n = 0,1,...,N_β. Here, M_f and N_β are the total numbers of frequencies and nonzero spanwise wavenumbers, respectively. The symbols ψ _m,n (n0) and φ _m represent random constant phase angles. In other words, once all the phase angles are randomly generated at t=0, the angle values will not change with time. The phase angles are also unchanged for cases with different nose-tip radii, which excludes the effect of the initial phase difference. For n = 0, it is enforced that β _0 = ψ _m,0 = 0. The symbols α _m and ω _m represent the streamwise wavenumber and the angular frequency for the m-th frequency component, and β _n is the spanwise wavenumber for the n-th wavenumber component. The Fourier modes are actually higher-order harmonics, which is indicated by
equation

    [ ω _m=mω _1,  β _n=nβ_1. ](𝑎,𝑏)

Moreover, ω _m = 2πf_m, where f_m is the frequency. 
In accord with <cit.>, the streamwise wavenumber and the angular frequency is linked through α _m = ω _m/(1 ± 1/M_∞). Here, 1 ± 1/M_∞ is the dimensionless phase speed of the fast/slow acoustic wave, and the slow-acoustic-wave one is finally chosen. In this paper, the incidence angle of the acoustic wave on the x–y plane is assumed to be zero, which was also adopted by <cit.>. The oblique wave angle on the x–z plane is given by θ _m,n = arctan(β _n/α _m).


In equation (<ref>), the dimensionless amplitude is A_m = A_m^ ∗ /( ρ _∞ ^ ∗ u_∞ ^∗ 2) is equally set for each spanwise-wavenumber component. In other words, no preferential spanwise wavenumber is imposed. The frequency-dependent dimensional amplitude A_m^ ∗ is determined by the following relation

    A_m^ ∗/p_∞ ^ ∗  = {[   √(C_Lf_m^ *  - 1Δf^ * /2),             f_m^ *  ≤ 40 kHz; √(C_Uf_m^ *  - 3.5Δf^ * /2),                   otherwise, ].

which is fitted from the measured frequency spectra of noise in the Arnold Engineering Development Complex (AEDC) Hypervelocity Wind Tunnel 9 <cit.>. The law of f_m^ *  - 3.5 at high frequencies has been reported by the measured noise data in various tunnel conditions with M_∞ ranging from 6 to 14, including the Hypersonic Ludwieg Tube Braunschweig, the Purdue Boeing/AFOSR Mach-6 Quiet Tunnel, the NASA 20-Inch Mach 6, the Sandia Hypersonic Wind Tunnel at Mach 8 and the AEDC Tunnel 9 <cit.>. The law of f_m^ *  - 1 at low frequencies was also verified by the DNS data of the tunnel noise at different Mach numbers <cit.>. The amplitude constants are C_L = 3.953 ×10^ - 4 and C_U = 126.5 ×10^6 in SI units. In the present model, the dimensional frequency ranges from 10 kHz to 1000 kHz with interval of Δf^ ∗ = 5 kHz. Meanwhile, the spanwise wavenumber β^∗ _n  ranges from 600 m^-1 and 20600 m^-1 with interval of Δβ ^ * = 800 m^-1, which is expected to achieve a broadband state. The mode numbers are finally M_f = 199 and N_β = 26. The corresponding spanwise wavelength varies from 0.3 mm to 10.5 mm, and the oblique wave angle θ _m,n on the x–z plane spans from 4^∘ to 89^∘. 


Following the dispersion relations of slow acoustic waves by <cit.> and <cit.>, we obtain the perturbations of other primitive variables by
equation

    [ u'_m,n =  - p'_m,nM_∞cosθ _m,n,;            v'_m,n = w'_m,n = 0,;  T'_m,n = (γ  - 1)M_∞ ^2p'_m,n. ](𝑎,𝑏,𝑐)

Finally, the total perturbation of the variable ϕ=(u,v,p,w,T)^T is given by

    ϕ '(x,z,t) = B_rescaled A_rescaled∑_n = 0^N_β∑_m = 1^M_fϕ '_m,n,

while the instantaneous density ρ is directly calculated from the equation of state for ideal gas. For 2-D receptivity studies (N_β=0), the parameters are set to A_rescaled=B_rescaled=1 in equation (<ref>). For 3-D transitional simulations, the amplitude rescaling parameter is set to A_rescaled=0.366, such that the spanwise averaged p^∗'_∞,rms of the 3-D wave is numerically equal to the 2-D counterpart determined by (<ref>). The detailed intensity of the pressure fluctuation is p^∗'_∞,rms/p̅^∗_∞=2.85 %. Note that the amplitude and spectrum of incoming disturbances were not given in the experimental study of <cit.>. A precursor simulation of case R2 reports that the designated amplitude leads to a significant advance of the transition onset compared to the experimental data. This observation might indicate that the intensity of real tunnel noise in <cit.> should be further lower. To approach the experimental data, another rescaling parameter is multiplied and given by B_rescaled=0.6. Finally, the resulting amplitude of the pressure fluctuation is p^∗'_∞,rms/p̅^∗_∞=1.71 %. At each time step of the simulation, the instantaneous quantity on the farfield boundary is forced to be the superimposition of the base-flow quantity and the perturbation.  




§ PHENOMENON OF TRANSITION REVERSAL


Since the research subject is the transition reversal, we first present the reversal phenomenon manifested by experimental and computational data. Figure <ref> shows the spanwise- and time-averaged Stanton number for the present cases R0, R2, R2.7 and R3. The Stanton number is defined by

    St=q_w^∗ / [ ρ _∞ ^ ∗ u_∞ ^ ∗ c_p∞^ ∗( T_aw^ ∗  - T_w^ ∗)],

where q_w^∗ is the wall heat flux, and T_aw^ ∗ is the adiabatic (recovery) wall temperature for turbulent flows. The legend `laminar' refers to the curve of the 2-D laminar flow, while the `turbulent' Stanton number is obtained from Reynolds analogy. To figure out the `turbulent' Stanton number, the skin friction coefficient C_f is calculated based on van Driest II correlation, as described in  <ref>. The Reynolds analogy factor 2St/C_f was incipiently recommended to be Pr^-2/3 by <cit.>, whereas a following high-speed experiment by <cit.> indicated a lower value. Consistent with <cit.> and <cit.>,  the Reynolds analogy factor is set to 1.0 in this paper. As shown by figure <ref>, the St-curves after transition collapse well onto the turbulent empirical formula as well as the experimental range of <cit.>. An exception is that the length of the computational domain for case R2 is insufficient to see the transition end. A further investigation on the mean velocity profile and the associated law of the wall is given in Appendix <ref>. A fully developed turbulent state is indicated for case R3 in an approximate range x>180. 

To directly compare with the experimental result, the determination approach of the transition onset location is kept unchanged to <cit.>. This approach has been commonly applied in hypersonic transition measurements. In detail, the mean Stanton number result is mapped onto a log_10(_x)–log_10(St) plot, where _x is the x-based Reynolds number. The transition onset is then defined by the intersection point of approximate laminar and transitional lines. A graphical example is provided in Appendix <ref>. The transition onset Reynolds number is then given in table <ref>. Obviously, the transition onset reverses when the nose-tip radius exceeds 2 mm. 

Despite that a rescaling parameter B_rescaled is employed in equation (<ref>), the transition onset is still earlier than the experimental one. As aforementioned, the data trend instead of the exact transition Reynolds number is of interest. We keep the current computational setup unchanged instead of adjusting B_rescaled repeatedly to match the experimental result. For a straightforward comparison, the transition onset Reynolds numbers _t for all the CFD cases are multiplied by a factor 1.45. This operation does not alter the tendency arising from the bluntness effect. Figure <ref> compares the CFD and experimental data. A fitting curve was given in conjunction with the raw data by <cit.>. Regardless of the more deviated case R0, the overall tendency of the nose bluntness effect is reproduced by the present numerical simulation. <cit.> mentioned that the data scattering was increased evidently with small nose bluntness, which might partly account for the deviation of the case-R0 state. In the following section, the detailed flow mechanism will be analysed by numerical and theoretical tools.









§ FLOW MECHANISM



 §.§ Two-dimensional response









The Mach number contour of the two-dimensional (2-D) steady base flow is shown in figure <ref> for the four cases with different nose-tip radii.  Following the criterion of <cit.>, the edge of the boundary layer is defined as the location where
the local total enthalpy is 0.995 times the freestream value. Furthermore, the edge of the entropy layer is defined as the location where the local entropy increment is 0.25 times the value at the wall. Here, the total enthalpy and the entropy increment are defined by H^∗=c_p^∗ T^∗+(u^∗ 2+v^∗ 2)/2 and Δ S^∗=c_p^∗ln(T^∗/T^∗_∞)-R^∗ln(p^∗/p^∗_∞), respectively. The symbols c_p^∗ and R^∗ represent the specific heat at constant pressure and the gas constant for air, respectively. The thicknesses of the boundary layer and the entropy layer are plotted against x in figure <ref>. In addition to the total enthalpy criterion, the nominal thickness δ_0.99 of the boundary layer is also shown in figure <ref>(a) for case R0. Clearly, the presence of a blunted leading edge thickens the boundary layer. Away from the nose tip, the value of the nose radius has no visible impact on the boundary layer thickness. Moreover, an increasing nose-tip radius makes the entropy layer thicker. Different from the cone geometry, entropy swallowing is not seen in a distance of interest. In other words, the boundary layer is entirely covered by the entropy layer. This feature may enable a long-distance stabilisation effect of the entropy layer on the normal-mode instability. Furthermore, the Mach number on the boundary layer edge evolves to be around M=2.7 downstream for cases R2, R2.7 and R3. Therefore, unstable Mack second mode, usually appears above Mach 4, is probably not supported in the considered blunt-plate flow.

Figure <ref> shows the instantaneous pressure fluctuation at t^∗= 0.5 ms when the 2-D unsteady flow is fully established. For case R0, the broadband planar acoustic waves interact with the weak leading-edge shock first and the boundary layer subsequently. Clear double-cell signature of the pressure fluctuation for Mack second mode is captured in the downstream boundary layer. For blunt-plate cases, the planar acoustic waves interact with first the detached shock and then the weak expansion wave forming in the vicinity of the junction point (x,y)=(0,R_n). No Mack-second-mode like structure is captured throughout the computational domain. With varying nose-tip radius, the structure of p' is not substantially altered. This might suggest some similarities in the instability evolution for the considered blunt-plate cases, which will be further examined in 3-D studies.

Figure <ref> shows the streamwise evolution of Chu's energy and the r.m.s. of the wall pressure fluctuation p'_rms. The Chu's energy E_Chu is defined as the wall-normal integral of Chu's energy density, which is given by <cit.>

    E_Chu = 1/2∫_0^∞[ ρ̅( u'^2 + v'^2 + w'^2) + T̅/γ M_∞ ^2ρ̅ρ '^2 + ρ̅/γ( γ  - 1)M_∞ ^2T̅T'^2] dη.

Different length of the statistical window is used, and no visible difference in the concerned quantity is found in figure <ref>. Thus, statistical convergence is reached. The sharp-leading-edge case supports the amplification of 2-D Mack second mode, while an approximately neutral state is reported on the downstream flat plate for all the blunt-plate cases. The downstream magnitude of p'_rms satisfies p'_rms,R2<p'_rms,R2.7<p'_rms,R3<p'_rms,R0. Meanwhile, the final Reynolds number based on the transition onset satisfies  Re_R2>Re_R2.7>Re_R3>Re_R0. Note that a higher-amplitude fluctuation usually corresponds to an earlier transition onset if the other factors are unchanged. Thus, the tendency of the response magnitude of the disturbance, dependent on the nose-tip radius, is coincidently consistent with the performance of the transition onset. With regard to Chu's energy, the base flow of case R0 does not possess a thick entropy layer compared to the others. As a result, disturbance energy outside the boundary layer is relatively small for case R0, and the integrated E_Chu is considerably lower than the other cases. However, for blunt-plate cases, E_Chu,R2<E_Chu,R2.7<E_Chu,R3 still holds true. The above observation may suggest that the bluntness effect has already been reflected, though incompletely, in the 2-D physical problem.



 §.§ Linear stability analysis




To clarify the mechanism of the linear instability, it is necessary to perform a normal-mode stability analysis. For the sharp-leading-edge case, the laminar base flow is extracted at a pre-transitional location x=50 for stability analysis. Subsequently, the one-dimensional (1-D) base flow along the wall-normal direction is transformed into another 1-D mesh, which is implemented by cubic spline interpolation via Intel MKL. Algebraic stretching is employed for the new 1-D mesh points, which are more robust for spectral difference in the stability analysis. The algebraic stretching function has been incorporated into the spectral collocation method by <cit.>. For the new 1-D mesh, n_y,new≥161 is found to be sufficient to achieve convergence of eigenvalues. The following `coarse mesh' and `fine mesh' in figure <ref> represent the mesh point number of n_y,new=201 and n_y,new=221, respectively. 



The local growth rate σ=-α_i is calculated in the space of (ω,β), where σ>0 suggests an unstable mode. Figure <ref> clearly shows the unstable range of the second mode and the oblique first mode.  The growth rates of the first and second modes peak at around (ω,β)=(0.9,2.75) and (ω,β)=(6.4,0), respectively. These two states correspond to dimensional frequencies of 127 kHz and 902 kHz, respectively. The wavelength of the most unstable first mode is λ_z^∗=2.28 mm. However, for all the blunt-plate cases, no discrete mode with σ>0 is identified in the range x∈[5, 200], ω∈[0, 8] and  β∈[0, 7]. It is not surprising that the unstable second mode is absent, since the boundary-layer-edge Mach number is only 2.7 downstream. With regard to the most unstable first mode, the eigenspectra of case R0 at x=50 as well as case R2 at x=50 and x=150 are shown in figure <ref>. The complex phase velocity is defined by c=ω/α, where c_i>0 indicates an unstable mode. Clearly, the discrete mode becomes stable even when the nose-tip radius is only R_n=2. These findings are consistent with existing knowledge of the stabilisation effect of the entropy layer. The non-swept plane stagnation flow near the nose tip is also known to be linearly stable to three-dimensional normal-mode instabilities <cit.>. The takeaway information in this case is that the transition to turbulence subject to freestream broadband disturbances, if possible, should be probably due to nonmodal instability mechanisms. The considered blunt-plate cases exclude the possibility of normal-mode instabilities.



 §.§ Three-dimensional instability and transition







The time history of the Stanton number at different streamwise stations of case R2 is recorded and shown in figure <ref>. For the probed laminar-flow location x=50, the perturbations exert no substantial influence on the mean heat flux compared to the undisturbed laminar value at t=0. In the transitional region (x=100, x=150, x=200 and x=250), clear signature of intermittency is observed. Windows with high-amplitude perturbations appear 
intermittently when turbulent spot (shown later) is present. During the intermittent turbulent window, the mean heat flux is escalated to around 4∼6 times the laminar value. Furthermore, the degree of intermittency, i.e. the ratio of the turbulent time to the total time, is visibly increased from the early stage x=100 to the late stage x=250. The presence of intermittency is observed for all the 3-D cases in this paper. Note that a flow-through time unit is t=260 for case R2. After a stationary state is reached, statistical calculations of mean and r.m.s. quantities are performed during an interval of t_2-t_1=1770 or t_2-t_1=1010, and no visible difference is found between the two choices.

Figure <ref> gives a Q-criterion visualisation of the flow structure for case R3. Figure <ref> further shows the streamwise velocity perturbation u_perturb. Here, the perturbation is the difference between the instantaneous field and the laminar flow, i.e., u_perturb(x,y,z,t)=u(x,y,z,t)-u_laminar(x,y). As shown by figure <ref>, streamwise-vortices like structures are formed immediately downstream of the blunt nose. Figure <ref> depicts that the development of streamwise vortices or
streamwise streaks is mostly beneath the boundary layer edge (dashed line) and far away from the entropy layer edge (dashed-dotted line). The outer entropy layer is only marginally disturbed. In the approximate range 60<x<120 of figure <ref>(a), turbulent spots are generated, which are surrounded by streamwise vortices and then convected away towards downstream. For a spatially fixed probe, intermittent signal of the physical quantity can be recorded, as shown in figure <ref>.  Similar to the numerical finding of <cit.>, a core region of the turbulent spot can be developed, where the skin friction or heat flux can be escalated. The increase in heat transfer is also recorded in the turbulent window in figure <ref> and later in figure <ref>. Figure <ref> shows the contours of the instantaneous Stanton number. For all blunt-plate cases, a notable observation is that streamwise streaks appear immediately downstream of the nose region. Following that, the turbulent spot surrounded by the streaky laminar flow emerges, which has a finite and gradually increasing spanwise width. In the core region of turbulent spots, high-frequency components become more evident (see figure <ref>), which are associated with secondary instabilities under the streaky laminar flow. Eventually, the turbulent spot spans in the whole spanwise domain and develops to fully established turbulence. By contrast, for case R0 under the same environment of freestream disturbances, no strong signature of the streamwise streak is observed in the vicinity of the leading edge. Nonetheless, three-dimensionality is still seen in the pre-transitional and transitional regions of case R0, which might be related to first-mode instabilities. To clarify the physical cause, spanwise-wavenumber spectral analysis will be performed later in conjunction with LST. The streamwise streaks are further visualised by the mean Stanton number in figure <ref>. The streaky spanwise spacings of cases R2, R2.7 and R3 seem to be close to each other.











Figure <ref> shows the distribution of spanwise-averaged r.m.s. of the fluctuating velocities at the laminar-flow location x=20. The statistical feature |u'|≫|v'|, |u'|≫|w'| is clear signature of streamwise streaks for blunt-plate flows. More importantly, the maximum of u'_rms gradually grows with increasing bluntness. The respective peak values are 0.045, 0.063 and 0.073 for cases R2, R2.7 and R3. The corresponding wall-normal heights η are 0.067, 0.045 and 0.039, respectively, which are inside the boundary layer. Therefore, as nose bluntness is increased, the streaks are enhanced, and simultaneously the peak fluctuation positions move towards the wall. The finding of the strengthened streaks with nose bluntness is consistent with the resolvent analysis of blunt-cone flows by <cit.>. The above observation also accounts for the preceding enlarged deviation from the 2-D laminar flow in wall heat transfer with increasing nose bluntness (see figure <ref>(b–d)). In comparison, the difference between 3-D and 2-D laminar flows for case R0 is ignorable in figure <ref>(a). This performance indicates that the streamwise streak does not play an important role in wall heat transfer of the sharp-leading-edge case.





Figure <ref> presents the contour of the instantaneous spanwise velocity w on the x–y plane, which is able to characterise the three-dimensionality. Different from the cone geometry that was frequently reported, e.g. by <cit.>, the blunt flat plate displays relatively weak signature of flow structures in the outer entropy layer. The discrepancy in the observed structure may be attributed to the different feature of the entropy layer. The mean streamline outside the boundary layer is oriented outward the wall in the blunt-flat-plate flow, whereas it is toward the wall before entropy swallowing in the blunt-cone flow. Thus, the perturbations outside the boundary layer appear to have a small impact on the boundary layer transition over the blunt flat plate. In the near-wall region of figure <ref>, the evolving streamwise streaks downstream of the nose tip (0<x<75) display staggered positive and negative regions of w' on the plate.  As aforementioned, the plane stagnation flow is linearly stable to normal-mode perturbations. Therefore, the formation of the strong streamwise streaks in the vicinity of the nose tip is likely to be due to the nonmodal instability. Farther downstream, the presence of packets containing small-scale structures possibly implies secondary instabilities and intermittent breakdown to turbulence.





 §.§ Spatial and temporal spectra


This subsection aims to quantify the spatial and temporal spectra, which point out the dominant scales in space and time. Figure <ref> gives the spanwise Fourier transform of the Stanton number, which is plotted against the streamwise coordinate and the spanwise wavelength λ_z. To reduce the randomness effect, ensemble average of the Fourier-transformed St is taken with thousands of snapshots during a long time period. With regard to the sharp-leading-edge case R0, two pronounced spanwise wavelengths λ_z=2.28 and λ_z=1.14 appear successively in the pre-transitional region x<x_t, where the transition onset is x_t=72. The first leading wavelength λ_z=2.28 agrees very well with the most amplified first-mode wavelength by LST in  <ref>. The second leading wavelength λ_z=1.14 emerges later, which has a weaker priority over other wavelengths compared to λ_z=2.28. It is conceivable that λ_z=1.14 arises from the nonlinear interaction of the dominant oblique first mode. Specifically, the mode–mode interaction in the Fourier space can be expressed by

    (f_1,β_1)-(f_1,-β_1)→ (0,2β_1),

where f_1 and β_1 denote the first-mode frequency and spanwise wavenumber, respectively. Mode (0,2β_1) is the generated streamwise vorticity mode. This stationary mode possesses a spanwise wavenumber twice the first-mode one, and thus a spanwise spacing half of the first-mode spanwise wavelength. Therefore, the three-dimensionality in the pre-transitional region of case R0 is partly induced by the oblique first mode and the resulting streamwise vorticity (or stationary streak) mode. For blunt-plate cases, it is interesting to observe a preferential wavelength λ_z=0.91 that does not vary with the nose-tip radius. The preferential wavelength is formed in and  through out the pre-transitional regions. The value of this wavelength is visibly consistent with the streak visualisation in figure <ref>. In this early stage, turbulent spots have not yet appeared, or they are insignificant. Note that the intensity of the added freestream disturbance is uniform with respect to the spanwise wavenumber/wavelength. It is thus inferred that the preferential wavelength is selected by the leading-edge receptivity process. Furthermore, different blunt-plate cases share similar spanwise spatial spectra. The main difference appears to be the transition location that is manifested by the appearance of the filled wavenumber spectra in figure <ref>. In that region with full spectra, the energy is concentrated on the large scales of λ_z rather than the dissipative small scales, which is consistent with the energy cascade property of turbulent flows.






In addition to the spatial spectra, temporal Fourier transform is also conducted and shown in figure <ref>. For case R0, the most amplified second-mode frequency is calculated by LST via the Newton-Raphson method, which maximises the growth rate σ. The resulting frequency is represented by the dashed line in figure <ref>(a). A good agreement in the most pronounced second-mode frequency is reached between CFD and LST, and the difference is less than 100 kHz. For the remaining blunt-plate cases, similar spectral characteristic is observed again for the frequency spectrum. In the pre-transitional region, low-frequency components below 20 kHz are dominant for cases R2, R2.7 and R3. Farther downstream, higher-frequency components grow rapidly and fill the frequency spectrum. The growth of the high-frequency band is likely to be associated with secondary instabilities. No preferential frequency of hundreds of kilohertz is found. The spectral characteristics in figures <ref>(a–d) also resemble the experimental results of <cit.>, where the signal was measured by PCB sensors over sharp and blunt cones.

The time history of wall pressure at several probes is further used to calculate power spectral density (PSD) for cases R0, R2 and R3. To this end, Welch's method is applied, and the time sequence is segmented with 50 % overlapping. A Hamming windowing function is used to reduce spectral leakage. The PSD results in the pre-transitional, transitional and nearly turbulent regions are shown in figure <ref>. The PSD of the Stanton number is also calculated, and the performance is similar to that of the wall pressure fluctuation. For case R0 in the laminar region, figure <ref>(a) illustrates that the PSD peak at around 1000 kHz falls within the unstable second-mode frequency range. For other blunt-plate cases, no Mack-mode-like high-frequency component emerges at x=50. In the transitional region at x=100, no prominent frequency is identified for all the cases. The PSD of the large-bluntness case R3 exceeds that of case R2 throughout the frequency range at x=100. This noteworthy observation indicates that the energy advantage of case R3 over R2 is overall and broadband, rather than characterised by a particular frequency. The disappearance of a single Mack second mode for case R0 is possibly related to the fact that x=100 is already in the late nonlinear stage, as shown by figure <ref>(a). At the nearly turbulent position x=250, the PSD curves of cases R2 and R3 become close to each other, which indicates proximity to a fully established turbulent state. 

To conclude, transition to turbulence in the sharp-leading-edge flat-plate flow is probably induced by Mack second mode and oblique first mode simultaneously. By contrast, the blunt-plate cases are dominated by low-frequency streamwise streaks with the same spanwise spacing and the subsequent high-frequency secondary instabilities. The characteristics of spatial and temporal spectra are found to be similar among cases R2, R2.7 and R3. The main quantitative difference is that the streaky response is increased by growing nose bluntness, as illustrated by figure <ref>. It is thus deduced that the transition reversal is caused by enhanced receptivity to freestream disturbances without changing the transition mechanism essentially.








 §.§ Spectral proper orthogonal decomposition

SPOD analysis is further conducted, which is a data-driven method to extract coherent structures or modes from flow fields. The data can be collected from experimental or high-fidelity computational studies. The conventional proper orthogonal decomposition (POD) seeks a set of deterministic modes, which are only spatially coherent. The SPOD modes are orthogonal in the context of a space–time inner product operation. As a result, the improved SPOD method captures the dominant modes that are coherent in space and time  <cit.>. Mathematically,
SPOD modes are the eigenvectors of a cross-spectral density (CSD)
tensor at each frequency. In practice, N_t snapshots of the fluctuating flow field are constructed into a snapshot matrix

    Q=[ q_1,  q_2, ...,  q_N_t].

To estimate the CSD, the data are segmented into N_blk
overlapping blocks with N_FFT snapshots in each, as

    Q^(k)=[ q_1^(k),  q_2^(k), ...,  q_N_FFT^(k)],

where the j-th column in the k-th block is

    q_j^(k)= q_j+(k-1)(N_FFT-N_ovlp)+1.

Here, N_ovlp is the number of overlapping snapshots for each block. The data segmentation serves to increase the number
of ensemble members. A Hamming windowing function is applied to each block. Subsequently, discrete Fourier transform is applied to each windowed block, and the CSD matrix is constructed for each discrete frequency. SPOD modes are then computed as the eigenvectors of the CSD matrix for each frequency, which are organised in a descending order of its energy. In this paper, Chu's energy norm is chosen to evaluate the SPOD mode energy. The SPOD analysis is performed based on the open-source code provided by <cit.>.

To compromise the need to converge the SPOD modes and the limitation of computational resources, we follow the parameteric setup by <cit.>, who carried out SPOD analysis of a transitional boundary layer dataset. To reach statistical convergence of the spectral density, the number of the flow realisation N_blk≥20 was considered to be enough. In this paper, we take N_blk=20, and choose a reasonable resolution of N_FFT=512  with 75% overlapping. Based on the relationship

    N_blk = ⌊N_t - N_ovlp/N_FFT - N_ovlp⌋,

where ⌊·⌋ represents the floor operator, the total snapshot number should be at least N_t=2944. The snapshots are collected after the flow field reaches statistical stationary. For a low frequency of interest, say 20 kHz, the corresponding period is t_1^∗=5×10^-5 s. To obtain a good temporal resolution, 8 periods are fitted into one block, which yields  N_FFT/8=64 snapshots for each period of interest. As a result, the physical snapshot sampling time Δ t^∗_snap=t_1^∗/64 and sampling frequency f^∗_s=1/Δ t^∗_snap=1280 kHz. The Nyquist frequency is thus 640 kHz, and the minimum resolved frequency is f_min^∗=1/(N_FFTΔ t^∗_snap)=2.5 kHz.  










Since the general flow characteristics are similar for all the blunt-plate cases, it is able to provide sufficient physical insights by examining SPOD modes of case R2C only. Figures <ref>–<ref> show the three leading SPOD modes on the x–y plane for three chosen frequencies, namely low frequency 20 kHz, medium frequency 120 kHz and high frequency 550 kHz, respectively. In general, the three frequencies display distinct disturbance patterns. The low-frequency SPOD mode displays signature of streamwise streaks immediately downstream of the nose region. The streamwise length of the streak is around λ_x=30 near the leading edge, which is evidently larger than the boundary layer thickness. For the medium frequency, the leading-edge SPOD mode structure looks more complicated. The disturbances are developed in both the entropy layer and the boundary layer. The pattern in the entropy layer somewhat resembles the wisp-like structure in the experimental measurement of <cit.>. In general, the SPOD mode energy is still more concentrated in the boundary layer for the medium frequency. In terms of the low and medium frequencies, there is visible signature in the relatively upstream region x<100 for the first leading SPOD mode. This observation possibly suggests that the receptivity to the upstream disturbance is not ignorable for the low- and medium-frequency components. In comparison, the high-frequency SPOD mode disappears in the upstream region and emerges suddenly in the range x>140. Therefore, the high-frequency component is more likely to originate from secondary instabilities in the developed distorted base flow rather than from the freestream receptivity. 


To quantify the instability evolution, figure <ref> displays the streamwise dependence of the maximal streamwise velocity fluctuation |u'_max| for the first leading SPOD mode. For relatively low-frequency modes, the order of magnitude of the velocity fluctuation is already in the range 10^-3∼10^-2, which is maintained in the transitional region (downstream of the arrow). For the highlighted high-frequency (545 kHz) mode, the order of magnitude of |u'_max| undergoes an escalation from around 10^-6 to 10^-3 during the transition. In other words, high-frequency SPOD modes are rather weak in the pre-transitional region and grow rapidly in the transitional region. Physical factors other than the secondary instabilities do not seem convincing to account for such a sudden growth. The described instability evolution further supports that the high-frequency SPOD mode emerges from secondary-instability-like mechanisms.





Figure <ref> shows the frequency spectra for the 20 leading SPOD modes. For the first leading SPOD mode, the mode energy peaks at discrete frequencies of a series of harmonics, which resembles the SPOD energy feature in the controlled transition by <cit.>. These peak frequencies are integer times the frequency interval Δ f^∗=f^∗_1/2=5 kHz. Here, f^∗_1 is the fundamental (minimal) frequency that is imposed in the 3-D broadband disturbance model. Since 10 kHz, 15 kHz, 20 kHz, etc. are added in the model, the lowest frequency peak 5 kHz in figure <ref> should originate from the nonlinear difference interaction between 15 kHz and 10 kHz. In terms of the second and higher SPOD modes, the signature of the tuned frequencies becomes weaker, and the impact of the initial harmonic behaviour in the freestream disturbance model fades away.

In addition to the symmetry plane, the snapshots of the transverse plane were also stored for SPOD analysis. Figure <ref> provides the first leading SPOD mode on the z–y plane in the streaky laminar flow region. Instantaneous, time-averaged and SPOD-modal flow fields are listed from top to bottom. The contours of the streamwise and spanwise velocities are shown on the left and right columns, respectively. Based on the time-averaged results, the 3-D laminar flow is visibly distorted and displays spanwise periodic distributions in u_mean and w_mean. The spanwise spacing manifested in the contour of w_mean is close to the preferential spanwise wavelength λ_z=0.91 in  <ref>. The SPOD mode in figure <ref> exhibits staggered high- and low-speed streaks mainly inside the boundary layer. Unlike the aligned pattern of w_mean, the SPOD modes look inclined and more irregular on the z–y plane. The two-stage transition scenario over the present blunt flat plates becomes apparent based on the above SPOD analysis.







§ CONCLUSIONS

In the present paper, the stability and transition to turbulence of high-speed boundary layers over sharp and blunt flat plates are investigated. To initiate the boundary layer transition, 3-D broadband slow-acoustic-wave disturbances are added on the freestream boundary. The frequency spectrum obeys the distribution obtained from tunnel measurements, while no preferential spanwise wavenumber is imposed in the combination of harmonics. As the nose-tip radius exceeds 2 mm, the delay trend of the transition onset location reverses. A good agreement is reached between the numerical and experimental results with respect to the reversal tendency, which is reported for the first time.

A prior 2-D receptivity simulation displays a plateau of energy growth downstream of the blunt nose region. By contrast, Mack-mode like structures appear in the sharp-plate flow, which undergo remarkable energy amplification. Linear stability analysis identifies unstable Mack second mode and oblique first mode simultaneously for the sharp-plate flow, while no significant unstable normal modes are found for the blunt-plate flow. As a result, the transition to turbulence for the considered blunt-plate flow is due to nonmodal instabilities. Following that, 3-D high-resolution simulations report a two-stage transition mechanism, including the formation of a 3-D streaky laminar flow and a subsequent secondary instability stage. In stage I, low-frequency streamwise streaks are formed immediately downstream of the nose for blunt-plate cases, whereas their signature is not evident in the sharp-plate flow. A unified preferential spanwise wavelength 0.91 mm is observed for blunt-plate cases with different nose-tip radii. In comparison, the wavelengths for the most amplified oblique first mode and the resulting streamwise-vorticity mode are pronounced in the sharp-plate flow. With regard to the frequency spectra, low-frequency components within 20 kHz are dominant for the blunt-plate cases, while Mack second mode with around 1 MHz emerges in the sharp-plate flow. In stage II, intermittent turbulent spots appear in the streaky flow, convect downstream and span the spanwise domain. High-frequency components grow rapidly during this process, which arise from secondary instabilities. In this stage, the spatial and temporal spectra as well as the intermittency feature of blunt-plate flows with various nose-tip radii are similar. The crucial difference is the growing streaky response in stage I as the nose-tip radius is increased. This initial factor gives rise to earlier birth of turbulent spots and thus an advance of the transition onset with large nose bluntness. However, the transition reversal does not suggest an essentially different transition mechanism beyond the critical nose-tip radius.

The present computational data do not either contradict or confirm the roughness explanation by experimental investigators. The possible nose-tip roughness is expected to increase the streaky response in stage I, which further moves the transition onset forward with large nose bluntness. The present work also demonstrates that the employed 3-D broadband disturbance model is a useful choice for similar transition studies in the future. 








§ MESH CONVERGENCE STUDY

Figure <ref>(a) shows the mesh resolution effect on the 2-D receptivity study of case R2. The convergence in the evolutions of the Chu's energy and the wall pressure fluctuation r.m.s. is confirmed. This observation also implies the convergence of the base flow. For 3-D studies, the spanwise wavelength of the dominant streak, the breakdown scenario, the transition location, and other statistical quantities are examined successively. The main results are not obviously affected by increasing the mesh resolution to that of case R3F, which is close to the previous hypersonic DNS resolution of <cit.> and <cit.>. Figure <ref>(b) shows the streamwise development of the Chu's energy and the maximal streamwise velocity fluctuation. The energy growth in the late transitional stage appears to be slightly affected. Nonetheless, the overall deviation is acceptable. Hence, the main conclusions keep unchanged with increasing mesh resolution.








§ LAW OF THE WALL FOR MEAN VELOCITY PROFILES

To further examine the fully developed turbulent region, the mean velocity profile is compared with the law of the wall. Case R3 is adopted for analysis, which possesses the longest fully developed turbulent region among the considered cases. In order to remove the influence of mean density variation due to the compressibility effect, the van Driest transformation for the mean velocity is utilised as

    U_VD^ +  = ∫_0^u̅^ + √(ρ̅/
    				.
    				-ρ̅_w) du̅^ + (y),

where u̅^ +=u̅/u̅_τ. Figure <ref> plots the transformed mean velocity profiles at different streamwise locations. The viscous sublayer law U_VD^+=y^+ and the log law U_VD^+=2.5 ln(y^+)+5.8 are also shown. Through the van Driest transformation, the intercept of the log law is higher than the incompressible counterpart 5.0, which has been encountered in previous DNS of hypersonic wall-bounded flows <cit.>. The slope appears to collapse onto that of the standard log law starting from around x=180. Combining with the Stanton number result in figure <ref>(d), we deduce that a fully developed turbulent state has been established since then for case R3.





§ DETERMINATION OF TRANSITION ONSET LOCATIONS

To determine the transition onset location as the experimental investigator did, figure <ref> plots log_10(St) versus log_10(_x) for different cases. The transition onset Reynolds numbers _t correspond to the displayed intersection points. Transition onset locations of cases R0 and R3 are given as an example.



[Funding]This research is supported by the Hong Kong Research Grants Council (no. 15216621, no. 15217622, no. 15204322 and no. 25203721) and the National Natural Science Foundation of China (no. 12102377).

[Declaration of interests]The authors report no conflict of interest.



jfm







