Paper Title: Absolute-value based preconditioner for complex-shifted Laplacian systems.







Xuelei Lin School of Science, Harbin Institute of Technology,	Shenzhen 518055, China

       linxuelei@hit.edu.cn
Corresponding author: Congcong Li Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong SAR 
             22482245@life.hkbu.edu.hk      
Sean Hon Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong SAR

        seanyshon@hkbu.edu.hk        






Absolute-value based preconditioner for complex-shifted Laplacian systems
    Xuelei LinCongcong Li Sean Hon 

    Received: date / Accepted: date
=========================================================================












































The complex-shifted Laplacian systems arising in a wide range of applications.

In this work, we propose an absolute-value based preconditioner for solving the complex-shifted Laplacian system. In our approach, the complex-shifted Laplacian system is equivalently rewritten as a 2× 2 block real linear system. With the Toeplitz structure of uniform-grid discretization of the constant-coefficient Laplacian operator, the absolute value of the block real matrix is fast invertible by means of fast sine transforms. For more general coefficient function, we then average the coefficient function and take the absolute value of the averaged matrix as our preconditioner. 

With assumptions on the complex shift, we theoretically prove that the eigenvalues of the preconditioned matrix in absolute value are upper and lower bounded by constants independent of matrix size, indicating a matrix-size independent linear convergence rate of MINRES solver. Interestingly, numerical results show that the proposed preconditioner is still efficient even if the assumptions on the complex shift are not met. The fast invertibility of the proposed preconditioner and the robust convergence rate of the preconditioned MINRES solver lead to a linearithmic (nearly optimal) complexity of the proposed solver. The proposed preconditioner is compared with several state-of-the-art preconditioners via several numerical examples to demonstrate the efficiency of the proposed preconditioner. 





65F08 65F10 65M06




Absolute-value based preconditioner for complex-shifted Laplacian systems
    Xuelei LinCongcong Li Sean Hon 

    Received: date / Accepted: date
=========================================================================





§ INTRODUCTION

In this work, we consider iteratively solving the following  complex-shifted Laplacian system:

    (K + λ I)𝐳=𝐟,

where K ∈𝐑^M × M is derived from the discretization of the Laplace operator -∇(a(𝐱) ∇) based on finite difference scheme on rectangular domains and 0 < ǎ≤ a(x) ≤â, λ=α+β i is a complex number with i denoting the imaginary unit and α,β∈ℝ, I is identity matrix of proper size, 𝐳 is the unknown vector to be solved, and 𝐟 is the given right-hand side vector. 

The complex-shifted Laplacian matrix was initially introduced as a “shifted Laplace" preconditioner for handling the Helmholtz equations as documented in <cit.>. As a preconditioner, its application follows with the computation of its inverse times some given vectors, from which complex shifted Laplacian system (<ref>) arises. Besides, the complex-shifted Laplacian system (<ref>) also arises other scenarios, such as preconditioned iterative solutions of local and non-local evolutionary equations with Laplacian operator as spatial term; see, e.g., <cit.>. In these scenarios, the triangular matrices arising from discretization of temporal derivatives are further approximated by α-circulant matrices which have complex eigenvalues. As a result, the approximated time-space matrices are block diagonalizable with each eigen-block being a complex-shifted Laplacian matrix. Solving linear systems with the approximated time-space coefficient matrices gives rise to complex-shifted Laplacian systems of form (<ref>). These applications motivate us to study fast solvers for (<ref>).


 
Although direct methods have the potential to offer benefits, solving complex-shifted linear systems (<ref>) using these methods can be computationally expensive. This serves as strong motivation for us to explore and develop more efficient solvers for such systems. The efficient computation of Helmholtz-like complex symmetric systems has been extensively explored in the literature. For the multigrid method, notable references include <cit.>.  In addition, for the preconditioned Krylov subspace
 methods like GMRES or BiCGStab methods, relevant contributions can be found in <cit.>. In <cit.>, the Preconditioned Modified Hermitian and Skew-Hermitian Splitting method (PMHSS) has been proposed for the complex linear system (<ref>), which is based on Hermitian and Skew-Hermitian splitting of the coefficient matrix. However, the corresponding theoretical analysis relies on assumptions that K+α I is symmetric positive definite (SPD) and that β I is symmetric positive semi-definite (SPSD). Recently, the authors generalized the additive element-wise Vanka smoother <cit.> for solving complex-shifted Laplacian systems in ParaDIAG algorithms <cit.>. Nevertheless, the convergence theory for these iterative solvers has considerable room to be developed.


Instead of directly working on (<ref>), we study a preconditioner for the following equivalent 2× 2 block real linear system (a saddle point system)

    [     β I K + α I; K + α I    -β I ]_=:𝒜[ 𝐳_1; 𝐳_2 ]=
    		[ 𝐚; 𝐛 ],
 
 where λ=α+β i, z=z_1+z_2 i, f=a+b i, with i=√(-1) denoting the imaginary unit. Such a “complex-to-real" equivalent transformation is not new and several preconditioners for such saddle point systems have been studied in the literature. In <cit.>, Schur complement based preconditioners have been proposed for the saddle point system. Recently, the preconditioned square block (PRESB) preconditioner has been proposed in <cit.> for the saddle point system, which is based on block triangular factorization formulas and thus easily invertible. Nevertheless, when applying the PRESB preconditioner to (<ref>), its theoretical analysis relies on the assumption that  K + α I+β I is SPD <cit.>. Moreover, inverting K + α I+β I to implement these preconditioners presents a challenge. When K + α I+β I is indefinite, its inversion is as difficult as to invert the original system (K+λ I)z=f. To summarize, iterative solver for (<ref>) with indefinite K + α I+β I is still a challenging problem.
 
 For any real symmetric matrix H, define its absolute-value as
 
    | H|:=Q^ T diag(|λ_1|,|λ_2|,...,|λ_M|)Q,

 where Q^ T diag(λ_1,λ_2,...,λ_M)Q is the unitary diagonalization of H; in particular, if λ_i's are nonnegative, we define the square root of H as 
 
    √( H):=Q^ T diag(√(λ_1),√(λ_2),...,√(λ_M))Q.

 
  Then, our absolute-value preconditioner for (<ref>) is given by
 	
    Ã:=|𝒜|=√(𝒜^T*𝒜)   =   [ [ √((K + α I)^2+ β^2 I);                       √((K + α I)^2+ β^2 I) ]].

		
	To demonstrate the effectiveness of the proposed preconditioner, we will show that the preconditioned Ã^-1𝒜 has only two distinct eigenvalues ± 1, leading to a two-step convergence of the MINRES solver (i.e., the MINRES solver for the preconditioned system will find the exact solution to (<ref>) within two-iterations). In each iteration of the preconditioned MINRES solver, it requires to compute  matrix vector multiplications Ã^-1 v for some given vector v. When a( x) is a constant, Ã^-1 is diagonalizable by discrete sine transforms and such computing such matrix-vector multiplication requires 𝒪(Mlog M) operations. When a( x) is non-constant, computing such matrix-vector multiplication would be expensive. To remedy this, we employ the following absolute-value based preconditioner in the case of non-constant a( x).
	
    𝒫 = [ [ √((K + α I)^2+ β^2 I);                       √((K + α I)^2+ β^2 I) ]],

where K = γ L for γ=√(ǎâ). It is clear that L is the discretization matrix of the constant Laplacian -∇^2, which can be diagonalized by the discrete sine transforms and its eigenvalues are larger than a positive constant c_0. It is important to note that when a(x) is equal to 1, the preconditioner 𝒫 is exactly the preconditioner 	Ã and K=L. Such a preconditioner is always fast invertible no matter K+α I is definite or not. Moreover, we prove that the preconditioned MINRES solver with the proposed preconditioner has a convergence rate independent of matrix size under assumptions on α and β[See Theorem <ref>], even if K + α I may be indefinite in such assumptions. 
Hence, the proposed preconditioning technique somewhat relaxes the restriction on definiteness of K + α I, compared with the results in the literature. However, as mentioned above, our preconditioner relies on the uniform grid discretization. The uniform grid discretization does not exist when the domain of interested is irregular one. In the futrue, we will resort to the technique of domain decomposition to partition  the major area of an irregular domain with uniform grid so that our preconditioner is applicable to the major area.

	
	The contribution of this work is twofold: first, we start with the real symmetric equivalent form of the complex-shifted Laplacian system (<ref>) and propose a novel absolute-value based preconditioner for the real symmetric system. Thanks to the real symmetric equivalent form, we can employ the short-recurrence Krylov subspace solver, MINRES, to solve the preconditioned system, which helps reducing the computational cost of iterative steps. Second, we develop a convergence theory for the preconditioned MINRES solver, showing that the  proposed solver has a matrix-size independent convergence rate.
	
	The paper is organized as follows. In Section <ref>, we provide the convergence analyses for our proposed preconditioned solver. Numerical examples are given in Section <ref> for showing the effectiveness of our proposed preconditioners. At last, the concluding remarks are given in Section <ref>.
	


§ THEORETICAL RESULTS

In this section, we investigative the convergence rate of MINRES solver for the preconditioned system. 
The following lemma will be exploited as framework for analyzing the convergence behavior of the proposed preconditioned MINRES method.
<cit.>
	Let P∈ℝ^N× N and A∈ℝ^N× N be a symmetric positive definite matrix and a symmetric nonsingular matrix, respectively. Suppose Σ( P^-1 A)∈[-a_1,-a_2]∪[a_3,a_4] with a_4≥ a_3>0, a_1≥ a_2>0 and a_1-a_2=a_4-a_3. Then, the MINRES solver with P as a preconditioner for the linear system A x= y∈ℝ^N× 1 has a linear convergence as follows
	
    || r_k||_2≤ 2(√(a_1a_4)-√(a_2a_3)/√(a_1a_4)+√(a_2a_3))^⌊ k/2⌋|| r_0||_2,

	where r_k= P^-1 y- P^-1 A x_k denotes the residual vector at the kth iteration with x_k (k≥ 1) being the kth iterative solution by MINRES; x_0 denotes an arbitrary real-valued initial guess; ⌊ k/2⌋ denotes the integer part of k/2.







 §.§ Spectral analysis of the ideal preconditioner Ã^-1𝒜




The following theorem reveals the eigendecomposition structure of a class of certain 2 × 2 block matrices, including the matrix 𝒜 as a special case. Note that such a block structure can be regarded as a special case in our early work on asymptotic eigenvalue distribution <cit.>.


	Let 
	
    ℳ = [  θ I_n  A_n^*;    A_n -θ I_n ]∈ℂ^2n × 2n
 with θ∈ℝ and A_n ∈ℂ^n × n, and let |ℳ|:=√(ℳ^2). Then, |ℳ|^-1ℳ is both Hermitian and unitary. In other words, |ℳ|^-1ℳ has only two distinct eigenvalues ± 1.


	Considering the singular value decomposition of A_n = U_n Σ_n V_n^*, we have the following eigendecomposition of ℳ by direct computations
	
    ℳ   =   [         θ I_n V_n Σ_n U_n^*; U_n Σ_n V_n^*        -θ I_n ]
       =   [ V_n    ;     U_n ][  θ I_n    Σ_n;    Σ_n -θ I_n ][ V_n    ;     U_n ]^*  
       =   [ V_n    ;     U_n ]𝒬_=:𝒬[  √(Σ_n^2+θ^2 I_n)                  ;                   -√(Σ_n^2+θ^2 I_n) ]𝒬^*
    			[ V_n    ;     U_n ]^*_=𝒬^*,
 
	where 𝒬 is real orthogonal given by 
	
    𝒬
    		=
    		[                         Σ_n D_1^-1                        -Σ_n D_2^-1; (√(Σ_n^2+θ^2 I_n)  - θ  I_n)D_1^-1  (√(Σ_n^2+θ^2 I_n)  + θ I_n)D_2^-1 ]

	with
	
    D_1 =  √((-√(Σ_n^2+θ^2 I_n)  + θ I_n)^2 + Σ_n^2)

	and
	
    D_2 = √((√(Σ_n^2+θ^2 I_n)  + θ I_n)^2 + Σ_n^2 ).

	It is obvious that both diagonal matrices D_1 and D_2 are invertible, since θ≠ 0 by assumption. 
	
	With (<ref>), it is easy to see that
	
    |ℳ|   =   √(ℳ^2)
       =   √(𝒬[[ Σ_n^2+θ^2 I_n;               Σ_n^2+θ^2 I_n ]]𝒬^*)
       =   𝒬[ [ √(Σ_n^2+θ^2 I_n);                  √(Σ_n^2+θ^2 I_n) ]]𝒬^*.

	Therefore, we have
	
    |ℳ|^-1ℳ=𝒬[ [  I_n;      -I_n ]]𝒬^*.

	It is easy to see from the equality above that |ℳ|^-1ℳ is both unitary and Hermitian with ± 1 as its eigenvalues.
	The proof is concluded.
















































































Let 𝒜 and 𝒜̃ be defined by (<ref>) and (<ref>), respectively. Since K + α I is symmetric, it is straight forward to verify that Ã^-1𝒜  has only two distinct eigenvalues ± 1 with using Theorem <ref>. Then, with Lemma <ref>, we immediately obtained the following theorem.


For any real initial guess, the MINRES solver with Ã as preconditioner will find the exact solution within 2 iterations.


Since the spectrum of Ã^-1𝒜 is equal to {-1,1}, the convergence factor given in Lemma <ref> is equal to 0 when k=2. That means || r_2||_2=0 with r_2 being the residual vector at 2nd iteration. The result follows.







 §.§ Spectral analysis of 𝒫^-1𝒜


In this subsection, we provide a theorem which supports the optimal preconditioning effect of 𝒫.

Denote by ρ( C) and σ( C) respectively the spectral radius and the spectrum of a square matrix C. Denote by λ_min( H) and λ_max( H) respectively the maximal eigenvalue and the minimal eigenvalue of a Hermitian matrix H. 

For any real symmetric matrices C_1, C_2∈ℝ^k× k, denote C_2 ≻ ( or≽)   C_1
if C_2- C_1 is positive definite (or semi-definite). Especially, we denote C_2 ≻ ( or≽)   O, if C_2 itself is positive definite (or semi-definite).
Also, C_1 ≺ ( or≼)   C_2 and O≺ ( or≼)   C_2  have the same meanings as those of C_2 ≻ ( or≽)   C_1 and C_2 ≻ ( or≽)   O, respectively.


Let two Hermitian positive semi-definite matrices H_1 and H_2  be simultaneously diagonalizable. Then, for any non-zero vector z of compatible size, it holds that

    √(2)/2≤z^*√(H_1^2+H_2^2)z/z^*(H_1+H_2)z≤ 1.



 Since H_1 and H_2 are both simultaneously diagonalizable and Hermitian, we can rewrite H_1 and H_2 as
 
    H_1=Q^*Λ_1Q,    H_2=Q^*Λ_2Q,

 for some unitary matrix Q and diagonal matrices Λ_1 and Λ_2. Then,
 
    z^*√(H_1^2+H_2^2)z/z^*(H_1+H_2)z   =z^*√(Q^*(Λ_1^2+Λ_2^2)Q)z/z^*Q^*(Λ_1+Λ_2)Qz
       =z^*Q^*√(Λ_1^2+Λ_2^2)Qz/z^*Q^*(Λ_1+Λ_2)Qz
       z̃=Qz====z̃^*√(Λ_1^2+Λ_2^2)z̃/z̃^*(Λ_1+Λ_2)z̃.

 Notice that Λ_1 and Λ_2 are both diagonal matrices with nonegative diagonal entries. For two non-negative  numbers, c_1 and c_2, it is well-known that
	
    √(2)/2(c_1+c_2)≤√(c_1^2+c_2^2)≤ c_1+c_2.

 Therefore, 
 
    √(2)/2≤z̃^*√(Λ_1^2+Λ_2^2)z̃/z̃^*(Λ_1+Λ_2)z̃≤ 1,

 which implies that 
 
    √(2)/2≤z^*√(H_1^2+H_2^2)z/z^*(H_1+H_2)z≤ 1.

 The proof is complete.


    For positive numbers θ_i, η_i(1 ≤ i ≤ m), it obviously holds that

    min _1 ≤ i ≤ mθ_i/η_i≤(∑_i=1^m η_i)^-1(∑_i=1^m θ_i) ≤max _1 ≤ i ≤ mθ_i/η_i .



(See, e.g., <cit.>)


  (i) ǎL≼ K≼âL.

  (ii) There exists a positive constant c_0>0 independent of M such that L≽ c_0I.
	





Assume ǎ c_0 + |β|- |α| > 0, and define γ=√(ǎâ).	Let 𝒜,𝒫∈ℝ^2M × 2M be defined by (<ref>) & (<ref>), respectively. Then, 


  (i) σ(𝒫^-1𝒜)⊂[-μ_0,-1/μ_0]∪[1/μ_0,μ_0] for α≥ 0;

  (ii)σ(𝒫^-1𝒜)⊂[-μ̃_0,-μ̃_1]∪[μ̃_1, μ̃_0] for α<0 and ǎ c_0 + |β|+α > 0,

where

    μ_0 = √(2â/ǎ),    μ̃_0=√(2)(c_0â+|β|-α)/c_0γ+|β|+α,  μ̃_1=√(2)(c_0ǎ+|β|+α)/2(c_0γ+|β|-α).
 


	
	By matrix similarity, we have
	
    σ(𝒫^-1𝒜)=σ(𝒫^-1/2𝒜𝒫^-1/2)=σ(𝒫^-1/2Ã^1/2Ã^-1/2𝒜Ã^-1/2Ã^1/2𝒫^-1/2).

	Suppose λ is an eigenvalue of 𝒫^-1/2Ã^1/2Ã^-1/2𝒜Ã^-1/2Ã^1/2𝒫^-1/2. Then,
	
    |λ|≤  ||𝒫^-1/2Ã^1/2Ã^-1/2𝒜Ã^-1/2Ã^1/2𝒫^-1/2||_2≤ ||𝒫^-1/2Ã^1/2||_2||Ã^-1/2𝒜Ã^-1/2||_2||Ã^1/2𝒫^-1/2||_2.

	Since σ(Ã^-1/2𝒜Ã^-1/2)=σ(Ã^-1𝒜)⊂{-1,1} and Ã^-1/2𝒜Ã^-1/2 is Hermitian, it is clear that
	
    ||Ã^-1/2𝒜Ã^-1/2||_2=1.

	Then, the upper bound estimation of |λ| reduces to
	
    |λ|≤   ||𝒫^-1/2Ã^1/2||_2||Ã^1/2𝒫^-1/2||_2=||𝒫^-1/2Ã^1/2||_2^2.

	Notice that
	
    ||𝒫^-1/2Ã^1/2||_2^2=λ_max(𝒫^-1/2Ã𝒫^-1/2).

  
  
  
  
		
		
		
  
		
		
		
  
		
		
		
  
	Let z be any eigenvector of 𝒫^-1/2Ã𝒫^-1/2 corresponding an eigenvalue λ_max(𝒫^-1/2Ã𝒫^-1/2). Then, it holds that
	
    λ_max(𝒫^-1/2Ã𝒫^-1/2)
       =    z^ T𝒫^-1/2Ã𝒫^-1/2 z/ z^ T z
       z̃=𝒫^-1/2 z≠ 0========   z̃^ TÃz̃/z̃^ T𝒫z̃
       =   [z̃_1^ T,z̃_2^ T]  blkdiag(√((K + α I)^2+ β^2 I) ,√((K + α I)^2+ β^2 I))[ z̃_1; z̃_2 ]/[z̃_1^ T,z̃_2^ T] blkdiag(√((K + α I)^2+ β^2 I) ,√((K + α I)^2+ β^2 I))[ z̃_1; z̃_2 ]
       =   z̃_1^ T√((K + α I)^2+ β^2 I)z̃_1 + z̃_2^ T√((K + α I)^2+ β^2 I)z̃_2/z̃_1^ T√((K + α I)^2+ β^2 I)z̃_1 + z̃_2^ T√((K + α I)^2+ β^2 I)z̃_2,

	which together with Proposition <ref> and Lemma <ref> implies that
	
    |λ|≤λ_max(𝒫^-1/2Ã𝒫^-1/2)   ≤max_z̃_1≠ 0,z̃_2≠ 0{z̃_1^ T√((K + α I)^2+ β^2 I)z̃_1 /z̃_1^ T√((K + α I)^2+ β^2 I)z̃_1 ,  z̃_2^ T√((K + α I)^2+ β^2 I)z̃_2/z̃_2^ T√((K + α I)^2+ β^2 I)z̃_2}
       =max_z̃_1≠ 0z̃_1^ T√((K + α I)^2+ β^2 I)z̃_1/z̃_1^ T√((K + α I)^2+ β^2 I)z̃_1
       ≤√(2)max_z̃_1≠ 0z̃_1^ T(|K + α I|+ |β| I)z̃_1/z̃_1^ T(|K + α I|+ |β| I)z̃_1.


 
By Proposition <ref> and Lemma <ref>(i), we have

    |λ|    ≤√(2)max{z̃_1^ T(K + α I)z̃_1/z̃_1^ T(K + α I)z̃_1, 1}
       ≤√(2)max{z̃_1^ TK z̃_1/z̃_1^ TKz̃_1, 1}
       ≤√(2)max{âz̃_1^ TL z̃_1/γz̃_1^ TLz̃_1, 1}= √(2â/ǎ)=μ_0,   if α≥ 0.

	That implies
	
    λ∈[-μ_0,μ_0],   if α≥ 0.


For α<0, Lemma <ref>(ii) implies that

    |K + α I|+ |β| I≽ |K|-|α|I+|β|I=K̅+|β|I-|α| I   =γ L+|β|I-|α| I
       ≽ (ǎc_0+|β|-|α|)I.

Hence, 

    |λ|   ≤√(2)max_z̃≠ 0z̃_1^ T(|K + α I|+ |β| I)z̃_1/z̃_1^ T(|K + α I|+ |β| I)z̃_1
       ≤√(2)max_z̃≠ 0z̃_1^ T(K +(|α|+|β|) I)z̃_1/z̃_1^ T(|K + α I|+ |β| I)z̃_1
       ≤√(2)max_z̃≠ 0z̃_1^ T(âL +(|α|+|β|) I)z̃_1/z̃_1^ T(|K + α I|+ |β| I)z̃_1
       ≤√(2)max_z̃≠ 0z̃_1^ T(âL +(|α|+|β|) I)z̃_1/z̃_1^ T(γ L+|β|I-|α| I)z̃_1
       =√(2â/ǎ)[1+max_z̃≠ 0[(â^-1-γ^-1)|β|+(â^-1+γ^-1)|α|]z̃_1^ Tz̃_1/z̃_1^ T[L+γ^-1(|β|-|α|) I]z̃_1]
       ≤√(2â/ǎ)[1+max_z̃≠ 0[(â^-1-γ^-1)|β|+(â^-1+γ^-1)|α|]z̃_1^ Tz̃_1/z̃_1^ T[c_0I+γ^-1(|β|-|α|) I]z̃_1]
       =√(2)(|β|+|α|+c_0â)/γ c_0+|β|-|α|
       =√(2)(|β|-α+c_0â)/γ c_0+|β|+α
    =μ̃_0,   if α<0 and γ c_0+|β|-|α|>0,

where the 3rd inequality is obtained from Lemma <ref>(i); the 4th inequality is obtained from (<ref>); the 5th inequality is obtained from Lemma <ref>(ii).
	That implies

    λ∈[-μ̃_0,μ̃_0],   if α<0 and γ c_0+|β|-|α|>0.


 It remains to estimate the lower bound of |λ|.
 Denote by σ_min( C), the minimal singular value of a matrix C. As 𝒫^-1/2Ã^1/2Ã^-1/2𝒜Ã^-1/2Ã^1/2𝒫^-1/2 is Hermitian, it is clear that
	
    |λ|   ≥σ_min(𝒫^-1/2Ã^1/2Ã^-1/2𝒜Ã^-1/2Ã^1/2𝒫^-1/2)
       =1/||(𝒫^-1/2Ã^1/2Ã^-1/2𝒜Ã^-1/2Ã^1/2𝒫^-1/2)^-1||_2
       =1/||𝒫^1/2Ã^-1/2Ã^1/2𝒜^-1Ã^1/2Ã^-1/2𝒫^1/2||_2
       ≥1/||𝒫^1/2Ã^-1/2||_2||Ã^1/2𝒜^-1Ã^1/2||_2||Ã^-1/2𝒫^1/2||_2.

	As Ã^1/2𝒜^-1Ã^1/2 is unitary, ||Ã^1/2𝒜^-1Ã^1/2||_2=1. Then,
	
    |λ|≥1/||𝒫^1/2Ã^-1/2||_2||Ã^-1/2𝒫^1/2||_2=1/||𝒫^1/2Ã^-1/2||_2^2=1/λ_max(Ã^-1/2𝒫Ã^-1/2).

	Moreover,
	
    λ_max(Ã^-1/2𝒫Ã^-1/2)
       =max_ w≠ 0 w^ TÃ^-1/2𝒫Ã^-1/2 w/ w^ T w
       w̃=Ã^-1/2 w≠ 0========max_w̃≠ 0w̃^ T𝒫w̃/w̃^ TÃw̃
       w̃^ T=[w̃_1^ T,w̃_2^ T], w̃_1,w̃_2∈ℝ^M× 1===============
       =max_w̃_1,w̃_2≠ 0[w̃_1^ T,w̃_2^ T] blkdiag(√((K + α I)^2+ β^2 I),√((K + α I)^2+ β^2 I))[ w̃_1; w̃_2 ]/[w̃_1^ T,w̃_2^ T] blkdiag(√((K + α I)^2+ β^2 I),√((K + α I)^2+ β^2 I))[ w̃_1; w̃_2 ]
       = max_w̃_1,w̃_2≠ 0w̃_1^ T√((K + α I)^2+ β^2 I)w̃_1 + w̃_2^ T√((K + α I)^2+ β^2 I)w̃_2/w̃_1^ T√((K + α I)^2+ β^2 I)w̃_1 + z̃_2^ T√((K + α I)^2+ β^2 I)w̃_2
       ≤max_w̃_1,w̃_2≠ 0max{w̃_1^ T√((K + α I)^2+ β^2 I)w̃_1 /w̃_1^ T√((K + α I)^2+ β^2 I)w̃_1 ,  w̃_2^ T√((K + α I)^2+ β^2 I)w̃_2/w̃_2^ T√((K + α I)^2+ β^2 I)w̃_2}
       =max_w̃_1≠ 0w̃_1^ T√((K + α I)^2+ β^2 I)w̃_1 /w̃_1^ T√((K + α I)^2+ β^2 I)w̃_1 ≤√(2)max_w̃_1≠ 0w̃_1^ T(|K + α I|+ |β| I)w̃_1 /w̃_1^ T(|K + α I|+ |β| I)w̃_1 ,

	where the last inequality comes from Lemma <ref>. That means
	
    |λ|≥1/√(2)max_w̃_1≠ 0w̃_1^ T(|K + α I|+ |β| I)w̃_1 /w̃_1^ T(|K + α I|+ |β| I)w̃_1 =√(2)/2max_w̃_1≠ 0w̃_1^ T(|K + α I|+ |β| I)w̃_1 /w̃_1^ T(|K + α I|+ |β| I)w̃_1 .

By Proposition <ref> and Lemma <ref>, we have

    |λ|   ≥√(2)/2max{max_w̃_1≠ 0w̃_1^ TKw̃_1 /w̃_1^ TK  w̃_1 ,1}
       ≥√(2)/2max{max_w̃_1≠ 0w̃_1^ TKw̃_1 /w̃_1^ TǎL  w̃_1 ,1}=1/2√(2ǎ/â)=1/μ_0,   if α≥ 0,

which together with (<ref>) implies that

    λ∈[-μ_0,-1/μ_0]∪[1/μ_0,μ_0],   if α≥ 0.



For α<0, Lemma <ref>(ii) implies that

    |K + α I|+ |β| I≽ |K|-|α|I+|β|I≽ǎL+|β|I-|α| I≽ (ǎc_0+|β|-|α|)I.

Hence,

    |λ|   ≥√(2)/2max_w̃_1≠ 0w̃_1^ T(|K + α I|+ |β| I)w̃_1 /w̃_1^ T(|K + α I|+ |β| I)w̃_1 
       ≥√(2)/2max_w̃_1≠ 0w̃_1^ T(|K + α I|+ |β| I)w̃_1 /w̃_1^ T(ǎL+|β|I-|α| I)w̃_1 
       ≥√(2)/2max_w̃_1≠ 0w̃_1^ T(K + |α |I+ |β| I)w̃_1 /w̃_1^ T[ǎL+|β|I-|α| I]w̃_1 
       =√(2ǎ)/2√(â)[1+max_w̃_1≠ 0[(γ^-1+ǎ^-1)|α|+(γ^-1-ǎ^-1)|β|]w̃_1^ Tw̃_1 /w̃_1^ T[L+ǎ^-1(|β|-|α|)I]w̃_1 ]
       ≥√(2ǎ)/2√(â)[1+max_w̃_1≠ 0[(γ^-1+ǎ^-1)|α|+(γ^-1-ǎ^-1)|β|]w̃_1^ Tw̃_1 /w̃_1^ T[c_0I+ǎ^-1(|β|-|α|)I]w̃_1 ]
       =√(2)(c_0ǎ+|β|-|α|)/2(c_0γ+|β|+|α|)
       =√(2)(c_0ǎ+|β|+α)/2(c_0γ+|β|-α)=μ̃_1,   if α<0 and c_0ǎ+|β|-|α|>0,

where the second inequality comes from (<ref>); the 4th inequality comes from Lemma <ref>(ii). The inequalities above together with (<ref>) indicates that

    λ∈[-μ̃_0,-μ̃_1]∪[μ̃_1,μ̃_0],   if α<0 and c_0ǎ+|β|+α>0.

The proof is complete.


With Lemma <ref> and Theorem <ref>, we immediately obtain the following theorem

Let r_k denote the residual vector at kth preconditioned MINRES iteration with 𝒫 as preconditioner for solving the system (<ref>) for k≥ 1. Let

    r_0=𝒫^-1[[ a; b ]]-𝒜 z_0,
 
denote the intial residual vector with arbitrary real initial guess z_0. Then,


  (i) 
    || r_k||_2≤ 2θ_1^⌊ k/2⌋|| r_0||_2,   if α≥ 0,

where the convergence factor θ_1:=μ_0^2-1/μ_0^2+1∈(0,1) with μ_0 defined in Theorem <ref> is independent of M.

  (ii)  
    || r_k||_2≤ 2θ_2^⌊ k/2⌋|| r_0||_2,   if α<0 and c_0ǎ+|β|+α>0,

where the convergence factor θ_2:=μ̃_0-μ̃_1/μ̃_0+μ̃_1∈(0,1) with μ̃_0 and μ̃_1 defined in Theorem <ref> is independent of M.






 §.§ Implementations


We first discuss the computation of the matrix-vector product 𝒜𝐯 for any given vector 𝐯. Since 𝒜 is a sparse matrix (due to K ∈𝐂^M × M being sparse), computing 𝒜𝐯 only requires linear complexity of 𝒪(M). 

For inversion of the preconditioner, we adopt different strategies in different cases.

Constant Laplacian -∇^2 on rectangular domain and uniform grid: In such case, we employ finite difference method or finite element method on uniform grid for discretization of Laplace operator. As a result, K is diagonalizable by the discrete sine transform, i.e., K = W Λ W^T, where W denotes a real symmetric orthogonal discrete sine transform;  Λ is diagonal matrix with positive diagonal elements. Since K is simultaneously diagonalizable by W, it is clear that 𝒜̃ is fast diagonalizable. Especially,

    𝒜̃   =
    
    
    
    
       [ [ W;   W ]]
          ×[ [ √((Λ+α I)^2+β^2 I);                    √((Λ+α I)^2+β^2 I) ]]
          ×[ [ W;   W ]].

Hence, computing 𝒜̃^-1 v for a given vector v requires only 𝒪(Mlog M) operations in such case. As 𝒜̃ is already fast invertible in such case. Therefore, in such case, we adopt the ideal preconditioner 𝒜̃ for solving the system (<ref>).


Variable Laplacian -∇(a( x)∇) on rectangular domain and uniform grid: 
Our preconditioning technique begins with replacing a(𝐱) by a constant γ (γ = √(ǎâ)) to obtain the constant-coefficient matrix K, where K = γ L. Note that L is the discretization matrix of the constant Laplacian -∇^2. In such case, we adopt 𝒫 as preconditioner. Similarly, computing 𝒫^-1 v for a given vector v requires only 𝒪(Mlog M) operations.






§ NUMERICAL EXPERIMENTS


To demonstrate the efficiency of the proposed preconditioning technique, we in this section test and compare its performance with several state-of-the-art solvers, including the multigrid methods with Jacobi smoother and Vanka-type smoother <cit.> , PRESB preconditioning technique <cit.> and PMHSS preconditioning technique <cit.>. As both the PRESB and PMHSS preconditioners are non-Hermitian preconditioner, we apply GMRES solver to solve the preconditioned systems by the two preconditioners. The implementation of  the PRESB preconditioner and PMHSS preconditioner involves solving linear systems with K + (α+β) I as coefficient matrix, which is challenging when  K + (α+β) I is indefinite. To address this issue, we apply the ILU (Incomplete LU Decomposition) method to invert this part.

The experiments were conducted using GNU Octave 8.2.0 on a Dell R640 server with dual Xeon Gold 6246R 16-Cores 3.4 GHz CPUs and 512GB RAM running Ubuntu 20.04 LTS. The CPU time in seconds was measured using the built-in tic/toc functions in Octave. The MINRES solver and GMRES solver were implemented using the built-in minres function and gmres function, respectively in Octave. For the PRESB and PMHSS methods, K + (α+β) I is of incomplete factorization type and is obtained
via Matlab’s function ilu. The exact solution was generated using the built-in randn function in Octave, where the exact solution is represented as randn(M) + randn(M)* i. Here, randn denotes a random number drawn from a standard normal distribution, and i represents the imaginary unit. Additionally, for the MINRES and GMRES solver, we employed a zero initial guess and set a stopping tolerance of 10^-8 based on the reduction in relative residual norms, unless otherwise specified. For the remaining parameter settings, we followed the same configuration as described in <cit.>. When employing the PRESB and PMHSS preconditioners, we utilized GMRES(50) method as the iterative scheme. The iteration was terminated when the residual norm was reduced by a factor of 10^-8.  


In the tables given in the following examples, “Iter" stands for the iteration number by required by MINRES for convergence and “CPU” indicates the CPU time estimated in seconds for convergence using the Octave built-in function tic/toc.


 Constant Laplacian case: a(x)=1.

In the first example, we tested the saddle point system (<ref>) using MINRES with the proposed preconditioner Ã in (<ref>) and the constant-coefficient complex-shifted Laplacian system (<ref>) by the MGM method with the Vanka-type smoother proposed in <cit.>, respectively. Table <ref> (a) shows the iteration and CPU time of MINRES when the preconditioner Ã is applied and Table <ref> (b) shows the numerical results of Multigrid with Vanka-type solver. As we know, λ=α+β i. From Table <ref>, it can be observed that regardless of whether the complex shift λ is positive or negative, there is no significant variation in the numerical results. 






Observing the data for Table <ref> for a wide range of degrees of freedom (DoF), it can be concluded that the optimal preconditioner in MINRES achieves the anticipated two-step convergence. In contrast, the Vanka-type solver in Multigrid demonstrates higher iteration numbers and CPU times compared to our proposed preconditioner. Specifically, the iteration number of the Vanka-type solver is approximately seven times that of our proposed preconditioner, while the CPU time is approximately three to five times longer than our method for the same degree of freedom (DoF). Regardless of the selection of (α, β), the stability of the iteration count in MINRES with the preconditioner 𝒜̃ implies that its convergence rate remains optimal even as the grids are refined. This observation provides support for the theoretical results presented in Theorem <ref> and demonstrates the robustness of the proposed solver.

In summary, the results from the provided data demonstrate that the MINRES method with the optimal preconditioner is superior to the Multigrid method with a Vanka-type solver in terms of both iteration number and CPU time for the Constant Laplacian case ((a(x)=1)).


 Variable Laplacian case: a(x)=(20+ x_1^2)(20+ x_2^2).

In the second example, as previously mentioned, we performed experiments to demonstrate the versatility of our approach. Specifically, we conducted tests on the variable-coefficient complex-shifted Laplacian system (<ref>) using the MGM method with the Jacobi smoother and the GMRES method with the PMHSS preconditioner. For the saddle point system (<ref>), we evaluated the performance of the MINRES method with the proposed preconditioner 𝒫 given in (<ref>), as well as the GMRES method with the PRESB preconditioner. The iteration count and CPU time of MINRES with the preconditioner 𝒫 are presented in Table <ref> (a), while Table <ref> (b) displays the numerical results obtained with the Multigrid method using the Jacobi solver. The numerical results shown in the remaining Tables <ref> (c) and <ref> (d) are from GMRES(50) with the PRESB preconditioner and GMRES(50) with the PMHSS preconditioner, respectively. Similarly, across a wide range of α and β values, we observed that the proposed preconditioner 𝒫 consistently outperformed the Jacobi solver and the remaining two GMRES solvers in terms of CPU time and iteration counts. 

From Table <ref> (a), it can be observed that the iteration count of our proposed method remains stable at 14, even when the matrix size is twice that of the Multigrid method. This demonstrates our advantage in terms of computational efficiency. In contrast, the Jacobi solver in Multigrid exhibits significantly higher iteration numbers and CPU times for the same spatial grid M. Specifically, the iteration count of the Jacobi solver is more than twice that of our proposed preconditioner, and the CPU time greatly exceeds that of our method. Constructing a sparse variable-coefficient Laplacian matrix also requires a considerable amount of time.  For Tables <ref> (c) and (d), although the numerical results of GMRES with the PMHSS preconditioner are significantly better than those of GMRES with the PRESB preconditioner, neither approach matches the efficacy of the preconditioner we have proposed.  
     

    It is worth noting that when M≥ 2^10-1, the runtime of program exceeds 3600 seconds, and we chose to terminate the program at that point.








§ CONCLUSIONS

In this study, we have devised optimal preconditioners for efficiently solving complex-shifted Laplacian systems, considering both constant and variable coefficient cases. For the constant coefficient case, our approach ensures optimal convergence when used in conjunction with the MINRES method, while maintaining computational efficiency. For the variable coefficient case, we have demonstrated the convergence of our algorithm under the relevant assumptions. To validate the efficacy of our proposed preconditioners, we conducted numerical experiments. The results demonstrate that our preconditioners perform as expected, further confirming their effectiveness.

Although our preconditioner relies on the uniform grid discretization only existing in regular domain,  we will consider extending our preconditioning technique to the problem on irregular domain via domain decomposition method in the future.




§ ACKNOWLEDGMENTS

The work of Sean Hon was supported in part by the Hong Kong RGC under grant 22300921, a start-up grant from the Croucher Foundation, and a Tier 2 Start-up Grant from Hong Kong Baptist University. 	The work of Xue-lei  Lin was partially supported by research grants: 2021M702281 from China Postdoctoral Science Foundation, 12301480 from NSFC,  HA45001143 from Harbin Institute of Technology, Shenzhen, HA11409084  from Shenzhen.



§ DATA AVAILABILITY STATEMENT

The data of natural images and codes involved in this paper are available from the corresponding author on reasonable request.



§ DECLARATIONS


	
  1.I confirm that I have read, understand and agreed to the submission guidelines, policies and submission 
	declaration of the journal.

	
  2.I confirm that all authors of the manuscript have no conflict of interests to declare.

	
  3. I confirm that the manuscript is the authors' original work and the manuscript has not received prior publication 
	and is not under consideration for publication elsewhere.

	
  4. On behalf of all Co-Authors, I shall bear full responsibility for the submission.

	
  5. I confirm that all authors listed on the title page have contributed significantly to the work, have read the 
	manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.

	
  6.  I confirm that the paper now submitted is not copied or plagiarized version of some other published work.

	
  7. I declare that I shall not submit the paper for publication in any other Journal or Magazine till the decision is 
	made by journal editors. 

	
  8. If the paper is finally accepted by the journal for publication, I confirm that I will either publish the paper
	immediately or withdraw it according to withdrawal policies.

	
  9. I understand that submission of false or incorrect information/undertaking would invite appropriate penal 
	actions as per norms/rules of the journal.

 *
 
plain      
 




	
	




	
	
	




	
	
	
	





	
	




	




	
	




	




	
	




	
	




	
	
	
	




	
	




	
	
	




	




	




	
	







	
	





 




	
	




	
	




	




	
	







 
	




	
	




	
	
	




	
	




	





	
	




	
	




	
	
	




	




	
	




	
	




	
	




	
	




	
	




	




	




	
	




	
	




	
	




	
	




	




	



	




















































 Paper Title: Convergence analysis of inner-iteration preconditioned GMRES.


Time-resolved Stochastic Dynamics of Quantum Thermal Machines
    Gabriel T. Landi
    Received ...; accepted ...
=============================================================




§ ABSTRACT 
The objective of this paper is to understand the superlinear convergence behavior of the inner-iteration preconditioned GMRES method. In order to understand the phenomenon, we analyze the convergence using the Vandermonde matrix which is defined using the eigenvalues of the coefficient matrix. Although eigenvalues alone cannot explain the convergence, they may provide an upper bound of the residual, together with the right hand side vector and the eigenvectors. For the diagonalizable case, 
if the eigenvalues of the coefficient matrix are clustered, the upper bound of the convergence curve shows superlinear convergence, when the norm of the matrix obtained by decomposing the right hand side vector into the eigenvector components is not so large. We especially analyze the effect of inner-iteration preconditioning for least squares problems, where the eigenvalues cluster towards 1.








§ INTRODUCTION

The generalized minimal residual method (GMRES) <cit.>, is a robust iterative method for the numerical solution of nonsymmetric square systems of linear equations. GMRES is a generalization of the MINRES method <cit.> which was developed for symmetric systems.GMRES generates a Krylov subspace, and finds the solution in the Krylov subspace by minimizing the residual. Consider the problem with square coefficient matrix

    Ax=b,    A∈ℝ^n× n,    b∈ℝ^n.

Let x_0 be the initial solution (in all our numerical experiments, we set x_0=0), the initial residual 
r_0=b-Ax_0. Generate the Krylov subspace with A and r_0.

    𝒦_k(A, r_0)= span{r_ 0, Ar_ 0, …, A^k- 1r_ 0}.

At each step, we seek z_k∈𝒦_k(A, r_0) and x_k=x_0+z_k, such that the residual
 r_k=b-Ax_k=b-A(x_0+z_k)=r_0-Az_k is minimized, i.e. min_z_k∈𝒦_k(A, r_0)r_0-Az_k_2.
GMRES minimizes the residual on an expanding Krylov subspace. Thus, the residual decreases monotonically.

Let q_m(x) be the minimal polynomial of the nonsingular matrix A, i.e., 

    0=q_m(A)=α_0I+α_1A+…+α_mA^m,

where I is the identity matrix and α_0≠ 0. It follows that

    A^-1=-1/α_0∑_j=0^m-1α_j+1A^j

This representation of A^-1 characterizes x=A^-1b as a member of a Krylov subspace <cit.>.One reason why the Krylov subspace method is efficient is that the solution of the linear system Ax=b may belong to a Krylov subspace of degree much less than the size of A.
The degree of the polynomial with respect to A and b, can be even smaller than the degree of the minimal polynomial of A.
Note that

   


    max_r_0=1min_p∈π_kp(A)r_0≤max_r_0=1min_p∈π_kp(A)r_0=min_p∈π_kp(A),

where π_k denotes the set of polynomials of degree at most k and with value 1 at the origin, and r_0=b-Ax_0 <cit.>.


This inequality shows that the norm of a polynomial of A bounds the residual if the r_0 is a constant. We can select the polynomial as the characteristic polynomial, which is determined by the eigenvalues. This implies that the eigenvalues can determine the upper bound of the residual.
On the other hand, there is a famous result that any non-increasing convergence curve (residual norm versus iterations) is possible with the matrix A which can be chosen to have any desired eigenvalues <cit.>. See also <cit.>. This implies that the eigenvalues alone cannot determine the convergence of the residual.Is there a conflict between the above two results? Can the eigenvalues decide the convergence behavior of GMRES? In fact, there is no conflict between them. The eigenvalues give an upper bound for the non-increasing residual curve together with the right hand side vector and the eigenvectors. The convergence is influenced by the distribution of the eigenvalues, the projection on each eigenvectors and the normality of the matrix.


The maximum steps to converge is governed by the degree of the minimal polynomial of A, which determines the inverse of A. Moreover, with a certain b, the degree is less. Within the degree, it can even converge faster to a lower level and stagnate after the degree, which is the phenomenon in GMRES with inner-iteration preconditioning <cit.>.

Although any non-increasing curve can occur, convergence can be improved by clustering the distribution of eigenvalues of A, when the right hand side vector is fixed. See Ipsen's work <cit.> for how the projection of b onto the eigen-space affects the residual bounds.  The Ritz values of GMRES are also related to our analysis <cit.>.


The normality of A tends to help the convergence. Large condition number κ(V) of V, the matrix consisting of the eigenvectors of A, can hinder the convergence <cit.>.


Our work mainly shows that the clustering of eigenvalues has an effect of lowering the degree of the minimal polynomial of A with respected to r_0, make the GMRES converge faster, when A is diagonalizable, which may be the essence of the inner-iteration preconditioning.The organization of the paper is as follows.  In section 2, we present a brief analysis of the upper bound of the residual. In section 3, we apply the reasoning process from section 2 to the clustered eigenvalues scenario. In section 4, we offer a concise review of the inner-iteration preconditioned GMRES method for least squares problems, and provide a thorough analysis of this method through the framework established,  culminating in the presentation of a theorem. In section 5, we present an illustrative example constructed based on the work of Greenbaum et al.  <cit.>. This example is then interpreted using the theorem established in the preceding section. Finally, in section 6, we conclude the paper.



§ UPPER BOUND OF THE RESIDUAL

Assume  A∈ℝ^n× n is diagonalizable, and d is the grade of 𝒦_k(A,b), which means d is the smallest integer such that
K_d(A,b)=K_d+1(A,b). Consider the system of linear equations

    Ax=b,    b∈ℝ^n.
Let b=c_1v_1+c_2v_2+…+c_dv_d, where v_i∈ℂ^n  (v_i_2=1) are the eigenvectors of A corresponding to eigenvalues of λ_i∈ℂ (i=1, 2, …, d) and c_i are the corresponding weights, 1≤ i≤ d. Thus, b has the following representation.

    b=[c_1v_1,c_2v_2,…,c_dv_d][1,⋯,1]^𝖳.

Then

    Ab   =A(c_1v_1+c_2v_2+…+c_dv_d)=c_1Av_1+c_2Av_2+…+c_dAv_d
    
       =c_1λ_1v_1+c_2λ_2v_2+…+c_dλ_dv_d
       =[c_1v_1,c_2v_2,…,c_dv_d][λ_1, λ_2, …,λ_d]^𝖳.

Similarly, 

    A^kb=[c_1v_1,c_2v_2,…,c_dv_d][λ_1^k, λ_2, …,λ^k_d]^𝖳.

    Hence, for the kth iterative solution x_k=[b, Ab,…, A^k-1b]y_k of Ax=b,


    min_x_k=x_0+z_k, z_k∈𝒦_k(A,r_0)b-Ax_k_2   =min_y_k∈ℂ^kb-A[b, Ab,…, A^k-1b]y_k_2
       =min_y_k∈ℂ^k[c_1v_1,c_2v_2,…,c_dv_d][1, 1, …, 1]^𝖳-[c_1v_1, c_2v_2, …,c_dv_d]Λ_d^k y_k_2
       =min_y_k∈ℂ^k[c_1v_1, c_2v_2, …,c_dv_d]([1, 1, …, 1]^𝖳-Λ_d^k y_k)_2

    where

    Λ_d^k=(
      [   λ_1 λ_1^2     ⋯ λ_1^k;   λ_2 λ_2^2     ⋯ λ_2^k;     …     …     …     …;   λ_d λ_d^2     ⋯ λ_d^k;       ])∈ℂ^d× k

is a Vandermonde matrix.Thus, we have

    min_x_k=x_0+z_k, z_k∈𝒦_k(A,r_0)b-Ax_k_2   =min_y_k∈ℂ^k[c_1v_1,c_2v_2,…,c_dv_d](Λ_d^k y_k-[1, 1, …, 1]^𝖳)_2
    
             ≤min_y_k∈ℂ^k[c_1v_1,c_2v_2,…,c_dv_d]_2Λ_d^k y_k-[1, 1, …, 1]^𝖳_2 
       =min_y_k∈ℂ^kVdiag[c_1, c_2, …, c_d]_2Λ_d^k y_k-[1, 1, …, 1]^𝖳_2.




 
  
   
    

 
  
   
    
    
    
    
    

 
If v_1, v_2, …, v_d are orthonormal, then b_2=√(c_1^2+…+c_d^2)=c_2, where  c=[c_1, c_2, …, c_d]^𝖳. Let x=[x_1, x_2, ⋯, x_d]^𝖳.

Then,

    [c_1v_1,c_2v_2,…,c_dv_d]_2=   max_x_2=1[c_1v_1,c_2v_2,…,c_dv_d][x_1, x_2, …, x_d]^𝖳|_2 
    
        =   max_x_2=1c_1x_1v_1+c_2x_2v_2+…+c_dx_dv_d_2 
    
        =   max_1≤ i≤ d|c_i|
    
       =   c_∞,

and we have

    min_x_k=x_0+z_k, z_k∈𝒦_k(A,r_0)b-Ax_k_2/b_2≤c_∞/c_2min_y_k∈ℂ^kΛ_d^k y_k-[1, 1, …, 1]^𝖳_2.

Saad <cit.> gives the bound

    r_k≤κ(V)r_0_2max_i=1,2,…, n|p(λ_i)|,

where p is any polynomial of degree ≤ n which satisfies the constraint p(0)=1, and the vector x_k∈𝒦_d(A, r_0) associated with r_k=b-Ax_k=p(A)r_0,
for the residual using an eigenvalue decomposition A=VDV^-1 where D is a diagonal matrix consisting of the eigenvalues of A. The bound contains κ(V)=V_2/V^-1_2. However, (<ref>) does not contain κ(V). It contains 
[c_1v_1,c_2v_2,…,c_dv_d]_2,
and min_y_k∈ℂ^kΛ_d^k y_k-[1,⋯,1]^𝖳_2, which 

depends on Λ_d^k instead of V. Moreover, our bound involves c_i, which is a different from (<ref>). 



§ CLUSTERED CASE

Consider the case when the eigenvalues of A have a clustered structure, where there are s clusters, and each eigenvalue λ_i∈ℂ, 1≤ i≤ d belongs to a cluster around a center γ_j∈ℂ with a small radius ϵ, i.e. λ_i=γ_j+ϵ_i, 1≤ j≤ s,0≤ϵ_i≤ϵ≪ 1, where ϵ≡max(ϵ_1, ϵ_2, …,ϵ_d), and γ_1, γ_2, …, γ_s are distinct, i.e. γ_j_1≠γ_j_2 if j_1≠j_2. 

At step k, replace λ_i in (<ref>) by γ_j+ϵ_i to obtain

    Λ_ϵ≡Λ_d^k=(
      [     γ_1+ϵ_1 (γ_1+ϵ_1)^2           ⋯ (γ_1+ϵ_1)^k;     γ_1+ϵ_2 (γ_1+ϵ_2)^2           ⋯ (γ_1+ϵ_2)^k;           …           …           ⋯           …;     γ_2+ϵ_i (γ_2+ϵ_i)^2           ⋯ (γ_2+ϵ_i)^k;           …           …           …           …;     γ_s+ϵ_d (γ_s+ϵ_d)^2           ⋯ (γ_s+ϵ_d)^k;             ])∈ℂ^d× k.


Then,

    Λ_ϵ≈Λ_ϵ
    = Λ_s+P.



    Λ_ϵ=(
      [               γ_1+ϵ_1         γ_1^2+2γ_1ϵ_1                     ⋯     γ_1^k+kγ_1^k-1ϵ_1;               γ_1+ϵ_2         γ_1^2+2γ_1ϵ_2                     ⋯ γ_1^k
       +kγ_1^k-1ϵ_2;                     …                     …                     ⋯                     …;               γ_2+ϵ_i         γ_2^2+2γ_2ϵ_i                     ⋯     γ_2^k+kγ_2^k-1ϵ_i;                     …                     …                     …                     …;               γ_s+ϵ_d         γ_s^2+2γ_sϵ_d                     ⋯     γ_s^k+kγ_s^k-1ϵ_d;                       ]) ∈ℂ^d× k.



    Λ_s=(
      [   γ_1 γ_1^2     ⋯ γ_1^k;   γ_1 γ_1^2     ⋯ γ_1^k;     …     …     ⋯     …;   γ_2 γ_2^2     ⋯ γ_2^k;     …     …     …     …;   γ_s γ_s^2     ⋯ γ_s^k;       ]  )∈ℂ^d× k,     P=(
      [         ϵ_1     2γ_1ϵ_1           ⋯ kγ_1^k-1ϵ_1;         ϵ_2     2γ_1ϵ_2           ⋯ kγ_1^k-1ϵ_2;           …           …           …           …;         ϵ_d     2γ_sϵ_d           ⋯ kγ_s^k-1ϵ_d;             ])∈ℂ^d× k.

Deleting identical rows of Λ_s, we obtain Λ_s,

    Λ_s=(
      [   γ_1 γ_1^2     ⋯ γ_1^k;   γ_2 γ_2^2     ⋯ γ_2^k;     …     …     …     …;   γ_s γ_s^2     ⋯ γ_s^k;       ]) ∈ℂ^s× k.

Hence, denote the ith row of P as 
[ϵ_i   2γ_j(i)ϵ_i  …   kγ^k-1_j(i)ϵ_i] to indicate that the elements of the ith row are related to the center γ_j(i).

Let y_1=min_y∈ℂ^kΛ_s y-[1,⋯,1]^𝖳_2. Then, 


 
    min_y∈ℂ^kΛ_ϵ y-[1, 1, …, 1]^𝖳_2   ≤Λ_ϵ y_1-[1, 1, …, 1]^𝖳_2
       ≈Λ_s y_1-[1, 1, …, 1]^𝖳+Py_1_2 
       ≤Λ_s y_1-[1, 1, …, 1]^𝖳_2+Py_1_2
       = Λ_s y_1-[1, 1, …, 1]^𝖳_2+Py_1_2=Py_1_2    (k=s),
 
 since Λ_k is nonsingular, because γ_1, γ_2, …, γ_k are distinct.

 


 
  
   
    
   
  


Let y_1=(y_1^1, y_1^2, ⋯, y_1^k)^𝖳, then

    (Py_1)_i=(ky_1^kγ_j(i)^k-1+(k-1)y_1^k-1γ_j(i)^k-2+⋯+y_1^1)ϵ_i,     1≤ i≤ d.

  
   
    
     
     Define the polynomial f(γ) using y_1 as
    
    f(γ)=y_1^kγ^k+y_1^k-1γ^k-1+⋯+y_1^1γ-1.

  Since f(γ_j)=0 is the jth equation of Λ_k y-[1,⋯,1]^𝖳=0, and y_1 is the solution to this system, it follows thatγ_j (j= 1, 2, …, k) are the roots of f(γ)=0.
 Then, we have  
   
    f(γ)=(-1)^k-11/∏_j=1^k γ_j∏_j=1^k(γ-γ_j),

    
    f'(γ_j(i))=(-1)^k-11/∏_j=1^k γ_j∏_j=1,⋯,k,  j≠ j(i)(γ_j(i)-γ_j),

    
    f'(γ_j(i))=ky_1^kγ_j(i)^k-1+(k-1)y_1^k-1γ_j(i)^k-2+⋯+y_1^1,

    and 
    
    (Py_1)_i=f'(γ_j(i))ϵ_i.

    Thus, at step k, we have

    min_x_k=x_0+z_k, z_k∈𝒦_k(A,r_0)b-Ax_k_2   ≤Vdiag[c_1, c_2, …, c_d]_2min_y∈ℂ^kΛ_d^k y-[1,…,1]^𝖳_2 
       ≈Vdiag[c_1, c_2, …, c_d]_2(f'(γ_j(1))ϵ_1, f'(γ_j(2))ϵ_2, …,f'(γ_j(d))ϵ_d)^𝖳_2
       ≤ϵVdiag[c_1, c_2, …, c_d]_2(f'(γ_j(1)), f'(γ_j(2)), …,f'(γ_j(d)))^𝖳_2.

Consider some ϵ_i=0, the estimation (<ref>) only contains centers γ_j(i) which at least have two eigenvalues. 


§ INNER-ITERATION PRECONDITIONING



 §.§ Inner-iteration by NR-SOR for BA-GMRES

Hayami et al. <cit.> proposed preconditioning the m× n rectangular matrix A of the least squares problem 

    min_x∈ℝ^nb-Ax_2,     A∈ℝ^m× n,     b∈ℝ^m

 by an n× m rectangular matrix B from the right or the left, and using the generalized minimal residual (GMRES) method <cit.> for solving the preconditioned least squares problems (AB-GMRES and BA-GMRES methods, respectively).
For ill-conditioned problems, AB-GMRES and BA-GMRES were shown to be more robust compared to the preconditioned CGNE and CGLS, respectively. Note here that BA-GMRES works with Krylov subspaces in n-dimensional space, whereas AB-GMRES works with Krylov subspaces in m-dimensional space. Algorithm <ref> is the algorithm of BA-GMRES. 

The BA-GMRES method <cit.>, applies GMRES to

    BAx=Bb,      A∈ℝ^m× n,     B∈ℝ^n× m,     b∈ℝ^m,

and is equivalent to the original least squares problem (<ref>)
if and only if ℛ(B^𝖳BA)=ℛ(A). If we let B=A^𝖳, we have the normal equations


    A^𝖳Ax=A^𝖳b.
.

One can precondition this system by an explicit matrix P∈ℝ^n× n, which is given by

    PA^𝖳Ax=PA^𝖳b.

Forming an explicit matrix P requires computation time and storage space, especially when there is a requirement to form the normal equation matrix A^𝖳A explicitly.Applying NR-SOR to the normal equations for l steps, which avoids forming the normal equation matrix A^𝖳A of (<ref>) explicitly, is mathematically equivalent to providing a preconditioning matrix P^(l) such that

    P^(l)A^𝖳Ax=P^(l)A^𝖳b.
Introducing a stationary iteration method inside the GMRES iteration instead of forming an explicit preconditioning matrix to precondition GMRES, gives the inner-iteration preconditioned GMRES <cit.>. Morikuni <cit.> presents different stationary iterative methods combined with AB-GMRES and BA-GMRES and compares with other methods.As other earlier work, we mention FGMRES <cit.>, which is more related to AB-GMRES but applies different preconditioners at each step. SOR was used as inner preconditioners with GCR <cit.>, and SOR as inner preconditioners with GMRES <cit.>.Using NR-SOR as inner-iteration preconditioners is a way of implicit preconditioning, but has an explicit form for theoretical analysis. In the NR-SOR, let A^𝖳A=M-N, when M is nonsingular. Then, 

    P^(l)A^𝖳A   =(∑_i=1^l-1(M^-1N)^i+ I)M^-1A^𝖳A
       =(∑_i=1^l-1(M^-1N)^i+ I)M^-1(M-N)
       =(∑_i=1^l-1(M^-1N)^i+ I)( I-M^-1N)
       =∑_i=1^l-1(M^-1N)^i+ I-∑_i=1^l(M^-1N)^i
       = I-(M^-1N)^l
         
       = I-H^l

where H=M^-1N. Hence, the eigenvectors of P^(l)A^𝖳A remain the same for l=1, 2, ….The algorithm for using NR-SOR as inner-iteration preconditioner in BA-GMRES is as follows.



 §.§ Eigenvalue distribution for inner-iteration preconditioning

We use a test matrix <cit.> to explain our analysis. Let

    A= U(
      [   1   1                   0;         0.9 0.9            ;                   ⋱   ⋱    ;                     0.1 0.1;   0                        ;     ]) 
    V^𝖳∈ℝ^100× 20.

where U∈ℝ^100× 100 and V∈ℝ^20× 20 are orthogonal matrices computed with the QR factorization of random
matrices. Thus, A is rank-deficient, with rank 10. The residual bound given in <cit.>, represented by the exponent of the spectral radius after taking the logarithm, closely resembles a straight line, providing a pessimistic estimate of the actual residual and failing to adequately explain the observed superlinear convergence .
In order to analyze the convergence of BA-GMRES, in (<ref>), let A=B^(l)A=P^(l)A^𝖳A, b=B^(l)b=P^(l)A^𝖳b. 
Figure <ref> shows the nonzero singular values of A. Figure <ref> shows the nonzero eigenvalue of A^𝖳A. Figure <ref> shows the nonzero eigenvalues of H=M^-1N. Figure <ref> shows the eigenvalues of B^(l)A=P^(l)A^𝖳A= I-H^l,l=4. Figure <ref> shows the eigenvalues of B^(l)A=P^(l)A^𝖳A= I-H^l,l=8.  Table <ref> gives the values for the above figures.










 §.§ Estimation for the test matrix

As for the test matrix of (<ref>), A∈ℝ^100× 20, A=B^(l)A=P^lA^𝖳A= I-H^l has only one cluster of eigenvalues around the center 1, and the others are separate eigenvalues as shown in Figure <ref> and <ref> for l= 4, 8. 


Thus, according to Table <ref> where d=10 in (<ref>), we have as in (<ref>) with k=d=10

    Λ_ϵ≡Λ_10^10=(
      [      1+ϵ_1  (1+ϵ_1)^2          ⋯ (1+ϵ_1)^10;      1+ϵ_2  (1+ϵ_2)^2          ⋯ (1+ϵ_2)^10;      1+ϵ_3  (1+ϵ_3)^2          ⋯ (1+ϵ_3)^10;      1+ϵ_4  (1+ϵ_4)^2          ⋯ (1+ϵ_4)^10;          1          1          ⋯          1;        λ_6      λ_6^2          ⋯     λ_6^10;        λ_7      λ_7^2          ⋯     λ_7^10;        λ_8      λ_8^2          ⋯     λ_8^10;        λ_9      λ_9^2          ⋯     λ_9^10;       λ_10     λ_10^2          ⋯    λ_10^10;            ])∈ℂ^10× 10.

For step k<d,


    Λ_ϵ=(
      [     1+ϵ_1 (1+ϵ_1)^2         ⋯ (1+ϵ_1)^k;     1+ϵ_2 (1+ϵ_2)^2         ⋯ (1+ϵ_2)^k;     1+ϵ_3 (1+ϵ_3)^2         ⋯ (1+ϵ_3)^k;     1+ϵ_4 (1+ϵ_4)^2         ⋯ (1+ϵ_4)^k;         1         1         ⋯         1;       λ_6     λ_6^2         ⋯     λ_6^k;       λ_7     λ_7^2         ⋯     λ_7^k;       λ_8     λ_8^2         ⋯     λ_8^k;       λ_9     λ_9^2         ⋯     λ_9^k;      λ_10    λ_10^2         ⋯    λ_10^k;           ])∈ℂ^10× k.

Since, ϵ=max_k|ϵ_k|<10^-10 (k=1,2,3,4), which is tiny,


    Λ_ϵ≈Λ_ϵ=(
      [  1+ϵ_1 1+2ϵ_1      ⋯ 1+kϵ_1;  1+ϵ_2 1+2ϵ_2      ⋯ 1+kϵ_2;  1+ϵ_3 1+2ϵ_3      ⋯ 1+kϵ_3;  1+ϵ_4 1+2ϵ_4      ⋯ 1+kϵ_4;      1      1      ⋯      1;    λ_6  λ_6^2      ⋯  λ_6^k;    λ_7  λ_7^2      ⋯  λ_7^k;    λ_8  λ_8^2      ⋯  λ_8^k;    λ_9  λ_9^2      ⋯  λ_9^k;   λ_10 λ_10^2      ⋯ λ_10^k;        ])∈ℂ^10× k.

Seperating Λ_ϵ into two matrices, Λ_ϵ=Λ_s+P, where


    Λ_s=(
      [      1      1      ⋯      1;      1      1      ⋯      1;      1      1      ⋯      1;      1      1      ⋯      1;      1      1      ⋯      1;    λ_6  λ_6^2      ⋯  λ_6^k;    λ_7  λ_7^2      ⋯  λ_7^k;    λ_8  λ_8^2      ⋯  λ_8^k;    λ_9  λ_9^2      ⋯  λ_9^k;   λ_10 λ_10^2      ⋯ λ_10^k;        ])∈ℂ^10× k,     P=(
      [  ϵ_1 2ϵ_1    ⋯ kϵ_1;  ϵ_2 2ϵ_2    ⋯ kϵ_2;  ϵ_3 2ϵ_3    ⋯ kϵ_3;  ϵ_4 2ϵ_4    ⋯ kϵ_4;    0    0    ⋯    0;    0    0    ⋯    0;    0    0    ⋯    0;    0    0    ⋯    0;    0    0    ⋯    0;    0    0    ⋯    0;      ])∈ℂ^10× k.



    Λ_s=(
      [      1      1      ⋯      1;    λ_6  λ_6^2      ⋯  λ_6^k;    λ_7  λ_7^2      ⋯  λ_7^k;    λ_8  λ_8^2      ⋯  λ_8^k;    λ_9  λ_9^2      ⋯  λ_9^k;   λ_10 λ_10^2      ⋯  λ_6^k;        ])∈ℂ^6× k.
For k=6, we have

    detΛ_ s=∏_6≤ i<j≤ 10(λ_i-λ_j)∏_i=6^10(λ_i-1).
Since λ_i≠λ_j≠ 1≠ 0, (6≤ i<j≤ 10),

    detΛ_ s≠ 0 ⇒rankΛ_s=6   (k=6).
Note 

    Vdiag[c_1, c_2, ⋯, c_d](Λ_sy-[1,1,⋯,1]^𝖳)_2≤Vdiag[c_1, c_2, ⋯, c_d]_2Λ_s y-[1,1,⋯,1]^𝖳_2

holds in general.Λ_sy-[1,1,⋯,1]^𝖳_2=0 and Λ_s y-[1,1,⋯,1]^𝖳_2=0 share the same solution y if k=6. Note, rankΛ_s≤ 6 for k≤ 6. rankΛ_s=k (1≤ k≤ 6), rankΛ_s=6   (k>6), if λ_i≠λ_j≠ 1≠ 0, (6≤ i<j≤ 10).Let y_1 =min_y_∈ℛ^kΛ_s y_k-[1, 1, …, 1]^𝖳_2Note
 
    min_y Λ_ϵ y-[1, 1, …, 1]^𝖳_2   ≤Λ_ϵ y_1-[1, 1, …, 1]^𝖳_2
       ≈Λ_s y_1-[1, 1, …, 1]^𝖳+Py_1_2 
       ≤Λ_s y_1-[1, 1, …, 1]^𝖳_2+Py_1_2
       = Λ_s y_1-[1, 1, …, 1]^𝖳_2+Py_1_2    (k=6)
        
       = Py_1_2    (k=6)
 
Notice that when k=6, Λ_s y_1-[1, 1, …, 1]^𝖳_2=0.

    Λ_ϵ y-[1, 1, …, 1]^𝖳_2≤Py_1_2,

where y_1=(y_1^1, y_1^2, …, y_1^6)^𝖳, and

    Py_1=(
      [ ϵ_1y_1^1+2ϵ_1y_1^2+…+ 6ϵ_1y_1^6; ϵ_2y_1^1+2ϵ_2y_1^2+…+ 6ϵ_2y_1^6; ϵ_3y_1^1+2ϵ_3y_1^2+…+ 6ϵ_3y_1^6; ϵ_4y_1^1+2ϵ_1y_4^2+…+ 6ϵ_4y_1^6;                               0;                               0;                               0;                               0;                               0;                               0;                                 ]).

Since



    Λ_s y_1-[1, 1, …, 1]^𝖳_2=0.


    (
      [      1      1      ⋯      1;    λ_6  λ_6^2      ⋯  λ_6^6;    λ_7  λ_7^2      ⋯  λ_7^6;    λ_8  λ_8^2      ⋯  λ_8^6;    λ_9  λ_9^2      ⋯  λ_9^6;   λ_10 λ_10^2      ⋯ λ_10^6;        ])(
      [ y_1^1; y_1^2; y_1^3; y_1^4; y_1^5; y_1^6;       ])-(
      [ 1; 1; 1; 1; 1; 1;   ])=(
      [ 0; 0; 0; 0; 0; 0;   ]),

    which means 1, λ_6, λ_7, λ_8, λ_9 and λ_10 are roots of
    
    f(γ)=y_1^6γ^6+y_1^5γ^5+y_1^4γ^4+y_1^3γ^3+y_1^2γ^2+y_1^1γ-1=0.

     
      
    
    Thus,
    
    f(γ)=-1/λ_6λ_7λ_8λ_9λ_10(γ-1)(γ-λ_6)(γ-λ_7)(γ-λ_8)(γ-λ_9)(γ-λ_10)

    and
      
    f'(1)=-1/λ_6λ_7λ_8λ_9λ_10(1-λ_6)(1-λ_7)(1-λ_8)(1-λ_9)(1-λ_10).

    Also, we have
    
    f'(γ)=6y_1^6γ^5+5y_1^5γ^4+4y_1^4γ^4+3y_1^3γ^2+2y_1^2γ+y_1^1

    and
     
    f'(1)=6y_1^6+5y_1^5+4y_1^4+3y_1^3+2y_1+y_1^1.

     Let ϵ= max{ϵ_1,ϵ_2,ϵ_3,ϵ_4}<10^-10. Note that
    
    Py_1_2   =(f'(1)ϵ_1, f'(1)ϵ_2 , f'(1)ϵ_3, f'(1)ϵ_4)^𝖳_2
       ≤ϵ(f'(1), f'(1), f'(1), f'(1))^𝖳_2
       =2ϵ|f'(1)|.

Since Vdiag[c_1, c_2, …, c_d]_2=2.5068   (V is computed by eigenvalue decomposition of A. Then, decompose b on V to get c_i), (<ref>) gives the final estimate at the 6th iteration, and

    B^(l)r_s_2   =   B^(l)(b-Ax_s)_2
       ≤Vdiag[c_1, c_2, …, c_d]_2 Λ_ϵ y-[1, 1, …, 1]^𝖳_2
       ≃Vdiag[c_1, c_2, …, c_d]_2 Py_1_2
       <Vdiag[c_1, c_2, …, c_d]_2 2ϵ|f'(1)|
       =2Vdiag[c_1, c_2, …, c_d]_2ϵ|-1/λ_6λ_7λ_8λ_9λ_10(1-λ_6)(1-λ_7)(1-λ_8)(1-λ_9)(1-λ_10)|
       ≤ 5.136× 10^-10×1/0.4754×0.4901×0.0675×0.0001× (1.8999× 10^-7)^2
       =3.49× 10^-29.

If we choose λ_6 and λ_7 to be in the cluster around 1 , then ϵ<10^-6, and at the 4th iteration we obtain B^(l)r_s_2<3.49× 10^-12.
 
 
 Figure <ref> shows B^(l)r_s_2 versus the number of iterations in quadruple precision arithmetic (double precision arithmetic limits the observation). At the 4th iteration B^(l)r_s_2 is approximately 10^-12, and at the 6th iteration B^(l)r_s_2 is approximately 10^-29, which is close to the estimation. Thus, although A has 10 different singular values, the eigenvalue of the preconditioned matrix B^lA is contained in a cluster around 1. Within several steps,
B^(l)r_s_2 converges to a tiny level. In other words, the residual norm converges to near zero before the grade d.Ipsen's upper bounds for the non-normal matrix B^(l)A in <cit.> gives B^(l)r_6_2 < cϵr_0_2, where c is a constant that reflects the distance from separate eigenvalues to the cluster center 1 which is smaller than 0.5 and also reflects the non-normality of B^(l)A which is related to V_2, and r_0_2=4.55. Thus, the value of this bound is about 10^-1. Ipsen's estimation for normalmatrices in <cit.> is B^(l)r_6_2≈ (1/3)× 0.7^5 r_0_2≈0.0560r_0_2, which is larger than our estimate. However, B^(l)A is non-normal. Our work can be regarded as extending this estimate to the diagonalizable case. Bounds involvingexponents of the spectral radius after taking log gives a straight line. Our paper is devoted to illustrating superlinear convergence.Then, we have the following theorem.

     The matrix obtained by left preconditioning A by inner-iteration is denoted by A=B^(l)A. Let the radius of the cluster of eigenvalues around 1 be ϵ (ϵ≪ 1). The outlier eigenvalues are λ_i, 1≤ i<k-1. Then, at the kth iteration, the residual B^(l)r_k can be bounded to first order (𝒪(ϵ)) by V diag [c_1, c_2, ⋯, c_d]ϵ√(n-k+1) Π_i=1^k-11-λ_i/λ_i, where the same definitions as in (<ref>) and (<ref>) are used.


      
    B^(l)r_k_2   =   B^(l)(b-Ax_k)_2
       ≤Vdiag[c_1, c_2, …, c_d]_2 Λ_ϵ y-[1, 1, …, 1]^𝖳_2
       ≃Vdiag[c_1, c_2, …, c_d]_2 Py_1_2
       <Vdiag[c_1, c_2, …, c_d]_2 √(n-k+1)ϵ|f'(1)|
       =Vdiag[c_1, c_2, …, c_d]_2ϵ√(n-k+1) Π_i=1^k-11-λ_i/λ_i.
 
   



§ INFLUENCE OF NON-NORMALITY

To illustrate how a large condition number of the eigenvectors of A slows down the convergence, especially when the eigenvalues are well
 clustered,
we construct the example in <cit.> by choosing the eigenvalues as 1, 1.01, and 1.001. Then, the characteristic polynomial is given by

    (λ-1)(λ-1.01)(λ-1.001)=λ^3-3.011λ^2+3.02201λ-1.01101,
 and the companion matrix of A is given by

    A^B= (
      [        0        0   1.0110;        1        0 -3.02201;        0        1    3.011;          ]),

giving non-increasing residual series r_0_2=1, r_1_2=0.99, r_2_2=0.98. Then, we have 
    g=(√(r_0_2^2-r_1_2^2), …, √(r_2_2^2-r_3_2^2))^𝖳=(0.1411,0.1404,0.98)^𝖳.
Choosing U to be the identity matrix,  U*b=g. Thus, B=(b, u_1,…,u_n-1) is given as follows.

    B= (
      [ 0.1411      1      0; 0.1404      0      1;   0.98      0      0;        ])

Then, we finally have

    A=BA^BB^-1=(
      [       0 -2.8794  1.4329;       1  3.1529 -0.5957;       0  0.9908 -0.1419;         ]).
If we solve Ax=b by GMRES, the norm of b=f(0)=1, the residual series is 0.99, 0.98.

    V= (
      [ -0.4103  -4.105  0.4085;  0.8165  0.8165 -0.8165; -0.4062 -0.4060  0.4080 ]),    κ(V)=1.1079× 10^6.


    c=(3.4563× 10^6,-3.1070× 10^6,0.3493× 10^6)^𝖳,    Vdiag[c_1, c_2, …, c_d]_2=4.6607× 10^6.
The Vandermonde part Λ_d^1 y_1-[1, 1, …, 1]^𝖳_2 in (<ref>) for the first iteration is

    (-0.0037, 0.0063,-0.0027)^𝖳_2=0.0078.

Hence, the bound for the first iteration is Vdiag[c_1, c_2, …, c_d]Λ_d^1 y_1-[1, 1, …, 1]^𝖳_2=3.6353× 10^4, which is very large compared to r_1_2=0.99.The Vandermonde part Λ_d^2 y_2-[1, 1, …, 1]^𝖳_2 in (<ref>) for the second iteration is

    (-0.4407× 10^-5,  -0.0485× 10^-5, 0.4892× 10^-5)^𝖳_2=6.6026× 10^-6.

Hence, the bound for the second iteration isVdiag[c_1, c_2, …, c_d]Λ_d^2 y_2-[1, 1, …, 1]^𝖳_2=3.0772, which is larger than r_2_2=0.98, but the bound is tighter compared to the first iteration. This is because the Vandermonde part converges fast. This implies that the clustering of eigenvalues gives a superlinear bound, although how close the bound and the true convergence curve are, is influenced by Vdiag[c_1, c_2, ⋯, c_d]_2.Applying BA-GMRES to Ax=b with l steps NR-SOR inner-iteration preconditioning with the relaxation parameter ω=1.1, we obtain κ(V)=25.69, Vdiag[c_1, c_2, …, c_d]_2=4.28. As shown in Table <ref>, with the increase in l, the bound of the residual decreases monotonically and approaches the actual residual. Then, the Vandermonde part of P^(5)A for the second iteration is 3.9036× 10^-3. Thus, the bound is 1.7079×10^-2, which is slightly larger than the actual residual 4.1286×10^-3. This shows that the inner-iteration preconditioning works for this extreme example. 



§ CONCLUSIONS


In the inner-iteration preconditioning of least squares problems, one starts with the normal equations so that the original matrix is a normal matrix. It seems that the NR-SOR inner-iterations do not harm the normality (well conditioning of the eigenvectors) a lot according to numerically experiments. The eigenvectors remain  unchanged, while the eigenvalues cluster as the inner-iteration steps increases. Thus, the method maintains the condition number of eigenvectors, at the same time improving the distribution of the eigenvalues. That is why the method shows superlinear convergence. Our way of analyzing gives hints on how the preconditioning works. We will extend the work from the diagonalizable case to the non-diagonalizable case using the Jordan canonical form in the future.









§ ACKNOWLEDGMENTS

We would like to thank Dr. Keiichi Morikuni for useful discussions.







§ CONFLICT OF INTEREST

The authors have no conflict of interest to declare.

unsrt  
  




 Paper Title: A Waveguide Port Boundary Condition based on approximation space restriction for Finite Element Analysis.



wpbc]A Waveguide Port Boundary Condition Based on Approximation Space Restriction for Finite Element Analysis










[1]Francisco T. Orlandinif102348@dac.unicamp.br

2]Philippe R. B. Devloophil@unicamp.br


1]Hugo E. Hernández-Figueroahugo@unicamp.br


*[1]DECOM, Faculdade de Engenharia Elétrica e Computação, Universidade Estadual de Campinas, Brazil

[2]Departamento de Estruturas, Faculdade de Engenharia Civil, Arquitetura e Unicamp, Universidade Estadual de Campinas, Brazil





A Waveguide Port Boundary Condition (WPBC)
    based on the restriction of the approximation space is presented
    in the context of Finite Element Analysis.
    As well as reducing the computational domain in the same manner as the traditional
    WPBC,
    the proposed scheme further reduces the degrees of freedom at the waveguide ports,
    simplifies the implementation and seamlessly provides post-processing results such
    as the reflected power in each waveguide mode.
    The boundary condition is thoroughly derived, and numerical examples are used
    as a support for the discussion on topics such as the needed number of modes
    to be employed at a waveguide port. Finally, a nanograting-based plasmonic sensor
    is analysed to illustrate further possibilities of the scheme.










[
    [
    August 5, 2024
==================





§ INTRODUCTION

From submarine communications cables to Radio Frequency Identification systems,
electromagnetics waveguides are of great importance in several fields of electrical engineering.
Moreover, advancements in Photonic Crystal Fibres (PCFs) have allowed the guiding and confining light in
dimensions previously unattainable until recent decades.
Therefore, the class of waveguide-fed optical devices is an ever growing class, fuelling thus the
need for an accurate numerical analysis of complex waveguide structures.

One class of numerical method that has been employed for this end is the Finite Element Method (FEM),
which, among its desirable features, present a versatility to deal with complex geometries.
The frequency domain analysis of these devices is a task often needed in the design process, as one could
then compute reflectivity/transmission spectra, field distributions and signals coupling.

However, FEM schemes require a closed computational domain, and the truncation of open domains has been a subject
of study for several years, as show in <cit.>, in the context of Finite Difference schemes.
In the context of waveguide-fed structures, one special case of open domains of particular interest are the
waveguide ports, in which one can inject a given incident wave and absorb the reflected
waveguide modes from the structure, and it its thus important to truncate the domain appropriately
at the ports.

The Perfectly Matching Layers (PMLs), introduced in <cit.>, are a popular choice
among the Computational Electromagnetics community. The PMLs can be interpreted as a fictitious region
around the computational domain of interest that are able to attenuate incident waves, and are often placed
behind a waveguide port. However, not only they result in an expansion of the domain (and, naturally,
in an increase of degrees of freedom in a region away from the domain of interest) but adjusting the PMLs
for absorbing evanescent waves as well as propagating waves can be a challenging task<cit.>,
specially over a range of frequencies.

Among other approaches<cit.>,
of special significance for the present work is the
Waveguide Port Boundary Condition (WPBC) <cit.>, which
is based on an orthogonality relationship between waveguide modes and can accurately absorb the reflected
modes. In <cit.> the WPBC is described for rectangular waveguides, and in <cit.> a more
general form is presented.

However, there are a couple of issues, the first being the orthogonality relationship
between the waveguide modes. Taking a step-index optical fibre as an example of
a waveguide with an open cross section, one can verify that only the guided modes
present an exponential decay away from the core. With radiation modes, which are
also needed to represent a reflection, this does not happen. How could one
derive an orthogonality relationship for a mode that is unbounded in the spatial domain
and thus is not even square-integrable?

This property of some of the physical solutions translate to the computational domain
as well, given that a lossless open waveguide cross section truncated with PMLs
result in a non-self adjoint problem, and the orthogonality must be carefully studied,
as recently discussed in <cit.>.

At last, even if the orthogonality has been dealt with,
the traditional form of the WPBC involves double integrals of the boundary
term, which results in a more complicated implementation.

A novel form of the WPBC is presented in this paper, wherein the dofs
at the boundary are associated with the waveguide modes due to an approximation space restriction.
Its expressions are derived and the orthogonality relationship of the eigenmodes
is discussed for lossy, transverse-anisotropic waveguides,
allowing for its application in a general waveguide. The paper is organised as follows:
Section <ref> presents and discusses the weak formulations and the necessary Hilbert spaces.
Section <ref> dwells on the orthogonality relationship between the FEM solutions
of the waveguide analysis and from thereon develops the
new WPBC formulation with approximation space restriction, along with a discussion on
aspects related to its implementation. Finally, in Section <ref>, numerical
results are presented and strategies for the application of the WPBC are discussed,
including a discussion on a strategy to determine the needed number of modes
for a reflectionless boundary condition in a more complex scenario.



§ PRELIMINARIES



 §.§ Hilbert spaces

The relevant function spaces for the FEM formulations are the following:

    Ω ={ ϕ: Ω→ℂ |  ∫_Ωϕ·ϕ^* dΩ < ∞ },
       Ω ={ ϕ∈Ω |  ϕ∈ [Ω]^d },
       Ω = 
    { ϕ∈ [Ω]^d |  ϕ∈ [Ω]^d' }.



The Ω space refers to the scalar functions which are square-integrable in the domain Ω,
Ω is a subspace of Ω in which the gradient is also square-integrable and, finally,
Ω denotes the subspace of [Ω]^d, where d is the dimension of Ω,
for which the curl is well defined and it lies in [Ω]^d', with d'=2d-3 for d=2,3.

For Ω and Ω we also have the associated trace spaces.
Restricting the discussion to quasi-polynomial spaces for clarity,
the trace of a function with support on Ω is associated with its restriction on the boundary Γ.
For Ω, the trace is merely the restriction of the function on Γ,
while for Ω the trace is the tangential component of the function. Formally, one can write

    tr: ϕ∈Ω→tr ϕ =ϕ|_Γ∈Γ
       tr: ϕ∈Ω→
    tr ϕ =
        n×ϕ|_Γ∈Γ^d',

and, having the traces defined
[Actually, the traces reside in subspaces of the corresponding Ω spaces,
  but we avoid defining fractional Hilbert spaces given the scope of the paper.],
present the subspaces

    Ω =
        { ϕ∈Ω |  tr ϕ|_Γ_d=0},
       Ω =
    { ϕ∈Ω^d |  tr ϕ|_Γ_d = 0},

which are used throughout this work for imposing homogeneous Dirichlet boundary conditions on the
Γ_d⊆Γ boundaries.




 §.§ Weak formulations

All the weak formulations presented in this paper are derived from the inhomogeneous Helmholtz
wave equation, shown in Equation <ref>.
The derivations are presented here for the complex electric field, however, they
can also be formulated using the magnetic field due to the duality of the Maxwell's equations.


    [μ^-1_r(E)]-k_0^2ϵ_ rE
      = F.


The relative electric permittivity and magnetic permeability are shown here in the tensor form
as to contemplate anisotropy, and are denoted, respectively, by ϵ_r and μ_r.
F is a general source term and k_0 represents the wavenumber in vacuum.

The framework we are interested in is to perform the modal analysis of a waveguide which is embedded in a
domain in which we are interested to analyse the wave propagation, or scattering, resulting from the
incidence of one or more waveguide modes. This section is organised in the same fashion, discussing first the modal
analysis weak formulation followed by the scattering formulation.



  §.§.§ Modal analysis formulation


In the context of modal analysis, a waveguide is a structure invariant in the z-direction
[The scope of this work does not contemplate waveguides with periodicity in the propagation direction.],
which is said to be the propagation direction.
Moreover, the constitutive parameters ϵ_r and μ_r are now restricted to transverse anisotropy
and the source term F is zero.

Assuming that the waveguide has a (possibly unbounded) cross-section Ω contained in
the xy-plane, the modal analysis problem is posed as finding electric fields of the form


    E(x,y,z) = E(x,y)e^-jβ z

satisfying Equation <ref> with F=0. β is the propagation constant of the
waveguide mode.

Given the invariance in the propagation direction and the known z dependency of the electric field, the
modal analysis problem can be solved only in the two-dimensional cross section Ω. The weak form is
essentially the one presented in <cit.>, but supporting also transverse anisotropy.

The weak formulation is obtained by applying Equation <ref> in <ref>
and splitting the electric field as E=E_t+E_zẑ and the differential operator as
∇ = ∇_t+ ∂/∂ zẑ.
After some algebra, one can use the transformation of variables <cit.>:


    e_t=βE_t     e_z=-jE_z,

and obtain the following weak formulation:


Find non-trivial
(β^2,e_t, e_z) ∈ (ℂ×Ω×Ω)
such that:

    ∫_Ω[μ_r^-1( e_t)]·( ϕ)^*
      - k_0^2 ϵ_re_t·ϕ^*dΩ
    
      +β^2∫_Ω{μ_r^-1[( ∇_t e_z +e_t )×ẑ]}·[( ∇_t ϕ +ϕ)×ẑ]^*
    
      -k_0^2[ϵ_r(e_zẑ)]·(ϕ ẑ)^*
      dΩ = 0,
    ∀ϕ∈Ω , ϕ∈Ω.


The PML regions are incorporated naturally in the formulation by means of Equation <ref>,
and for simplification we have chosen to surround the domain with a Perfect Electric Conductor (PEC), resulting in
only Dirichlet boundary conditions in the electric field formulation.

Equation <ref> yields in the following algebraic problem:

Find non-trivial (β^2,e_t, e_z) ∈ (ℂ× [ℂ]^N × [ℂ]^M) such that:

    [ A_tt    0;    0    0 ]
        e_t 
    
        e_z
      
      = - β ^2
      [ B_tt B_tz; B_zt B_zz ]
        e_t 
    
        e_z
      ,

where:
equation-1


    [A_tt]_ij = ∫_Ω[
      μ_r^-1( φ_j ) ·( φ_i )^*
    .
    .
      - k_0^2 ϵ_r φ_j·φ_i^*] dΩ,
       [B_tt]_ij = ∫_Ω[μ_r^-1(φ_j×ẑ)]
        ·(φ_i×ẑ)^* 
        dΩ,
       [B_tz]_ij = ∫_Ω[μ_r^-1(φ_j×ẑ)]  ·(ϕ_i×ẑ)^*  dΩ,
       [B_tz]_ij = ∫_Ω[μ_r^-1(ϕ_j×ẑ)]  ·(φ_i×ẑ)^*  dΩ,
       
        [B_zz]_ij = ∫_Ω[μ_r^-1(ϕ_j×ẑ)]  ·(ϕ_i)^*
    
      - k_0^2 ϵ_r(ϕ_jẑ)( ϕ_iẑ)^* dΩ.



The generalised eigenvalue problem in Equation <ref> is Hermitian
and thus self-adjoint if the materials involved are lossless, for the tensors
ϵ_r and μ_r are real symmetric.
By inspection of Equation <ref> we see that it is not the case when
PMLs are involved, even if the materials were originally lossless, due to the losses
introduced by the PML region.

It is noteworthy that the De Rham compatibility between
Ω and Ω 
is a necessary condition<cit.>
to avoid spurious modes and ensure stability.
For all the examples in this paper, the chosen spaces form an exact sequence<cit.>,
meaning that the span of the gradient of the Ω space coincides
with the kernel of the curl of the Ω.




  §.§.§ Scattering formulation

The scattering weak form can be derived from Equation <ref> by means of the Galerkin method.
Now, we denote by Ω the computational for the scattering analysis,
which is one dimension larger than the modal analysis domain.
The electric field E is no longer restricted to the form of Equation <ref>, as most scenarios of interest involve reflections and discontinuities, and not
simply the undisturbed propagation of waveguide modes, so we simply look for E∈Ω.
The boundary is considered to be Γ = Γ_d ∪Γ_m, with Γ_m denoting the boundary
in which the WPBC mixed boundary condition is imposed, and PEC boundaries elsewhere.

The first step is to compute the inner product with a proper test function on both sides of the equation:

    ∫_Ω{[μ^-1_r(E)]}·ϕ^*
      -k_0^2ϵ_ rE·ϕ^*dΩ
    
      = ∫_ΩF·ϕ^*dΩ  ∀ϕ∈Ω.


Using the vector identity (A×B)=B·(A)-A·(B),

    ∫_Ω[μ_r^-1( E)]·( ϕ)^*+
    {[μ_r^-1( E)]×ϕ^*}
      -k_0^2ϵ_ rE·ϕ^*dΩ
    
      = ∫_ΩF·ϕ^*dΩ  ∀ϕ∈Ω.


Gauss's theorem can be applied on the divergence term, element by element:


    ∫_Ω{[μ_r^-1( E)]×ϕ^*}dΩ=
    ∑_K ∫_∂ K{[μ_r^-1( E)]×ϕ^*}·ndΓ=
    ∑_K ∫_∂ K[μ_r^-1E]·(ϕ^*×n) dΓ


Assuming now a tangential continuity of [μ_r^-1( E)]
between elements, all terms in the right-hand side of Equation <ref> vanish, and
the boundary terms on Γ_d vanish due to ϕ∈Ω. Finally, the problem can be posed as:

For F∈Ω given, find E∈Ω such that:

    ∫_Ω[μ_r^-1( E)]·( ϕ)^*
      - k_0^2 ϵ_rE·ϕ^*dΩ
    
      +∫_Γ_m[μ_r^-1E]·(ϕ^*×n) dΓ
    
      = ∫_ΩF·ϕ^*dΩ  ∀ϕ∈Ω.


It is worth noting that there are also no restrictions on the anisotropy of the involved media,
as long as they are still transverse anisotropic in the waveguide cross section.



 §.§ Source term modelling


The source term F is associated with a given physical source, and in this work we discuss
two different approaches for modelling a field injection in the domain.
One alternative is to model the source term as a plane current source<cit.>, propagating
the incident modes away from the plane in both sides.
In Figure <ref>, a general waveguide discontinuity is shown. The fictitious planes
Γ_in and Γ_out are the waveguide ports. In this approach, the waveguide ports need be PML backed, thus the extension of the domain in the z direction.




To do so, the field at the waveguide source is split into the incident field and the scattered field
as E=E_in+E_scatt. Assuming that only the scattered field has a continuous
tangential continuity of [μ_r^-1( E_scatt)],
the right-hand side term of Equation <ref> vanish everywhere
except at the waveguide cross section Γ.

At Γ, we have:


    ∫_Γn×[μ_r^-1( E_in)]·ϕ^* dΓ.


For computing the jump of the incident field, we write it as



    E_in,1 = ∑_iE_ie^-jβ_i z
    
      =∑_i (E_i,t+ẑE_i,z)e^-jβ_i z,
    E_in,2 = ∑_iE'_ie^jβ_i z
    
      =∑_i (E_i,t-ẑE_i,z)e^jβ_i z,


where E'_i is the backward propagating i-th mode.

For  waveguide modes, the curl can be computed as:

    (E(x,y)e^-jβ z) =
    [(jβE_t+ E_z)×ẑ+
      (∂ E_y/∂ x-∂ Ex/∂ y)ẑ]e^-jβ z,



    (E(x,y)e^ jβ z) =
    [(-jβE_t- E_z)×ẑ+
      (∂ E_y/∂ x-∂ Ex/∂ y)ẑ]e^jβ z,

and therefore, the tangential part of the curl[The z-component of Equations <ref> and <ref> are ignored due to the cross product with ẑ] reads as:


    n_1×μ_r^-1E_in,1 = 
    z×[∑_i(∇_tE_i,z×ẑ)
        +jβ_i (E_i,t×ẑ)]e^-jβ_i z,
    n_2×μ_r^-1E_in,2 = 
    
        -z×[-∑_i(∇_tE_i,z×ẑ)
        -jβ_i (E_i,t×ẑ)]e^jβ_i z,


which allows for:

    n×[μ_r^-1( E_in)]=
    ∑_i[(∇_tE_i,z+jβ_iE_i,t)×ẑ]2 cos(β_i z),

which is the sought source term, matching the term in <cit.> for TE/TM modes evaluated at z=0.

While this approach is effective, it involves backing the port with a PML region for absorbing the
counter propagating wave, which results in an undesirable enlargement of the computational domain.
The proposed alternative is to use the WPBC, which is the subject of the following section and it allows
for the domain to be truncated exactly at the source plane.



§ WAVEGUIDE PORT BOUNDARY CONDITION (WPBC)


In Figure <ref>, an (incident) waveguide port is shown, for which we aim to
develop a boundary condition.




At Γ_in, the total field can be written as a sum of incident and reflected fields.
Assuming that the leftmost domain of Γ_in, which is not in the computational domain, consists of a waveguide,
it is reasonable to assume that the reflected fields can be decomposed as a sum of backward propagating modes
[The prime sign for backward propagating modes is omitted from now onwards, and should the
  distinction be important, it will be explicitly discussed.],
that is

    E=E_in+E_ref = E_in+∑_i α_iE_ie^+jβ_i z.



This idea is closely related to the concept of eigenfunctions<cit.>:
the (possibly infinite) set of homogeneous solutions to a PDE with given boundary conditions form
a basis for the solution space of the same PDE subjected to the same boundary conditions with a given source term.



The modes of the waveguide at Γ are a natural candidate set
for expanding the solution of the scattering problem at Γ, and that is the basis for the theory of the WPBC,
introduced in <cit.> and detailed in <cit.>. The WPBC weakly enforces that, at Γ,
the solution will be a sum of waveguide modes weighted by the appropriate coefficients α_i so as to
represent the reflection. In this way, the domain before Γ is no longer discretised, as it was with
the PML backed current plane approach.

Upon inspection of Equation <ref>, it is clear that with an orthogonality condition of the modes
E_i at hand, an expression for α_i can be found.



 §.§ On the orthogonality condition of eigenmodes


The slab waveguide is the simplest example to illustrate the issues involving open waveguides. It consists of a plane core with higher refractive index deposited over a substrate and covered by cladding.
Due to its invariance in one of the transverse directions, the solutions for the slab waveguide can be split
into TE and TM modes. Moreover, either of the solution sets can be split into guided modes and radiation modes.

The guided modes form a discrete spectrum and are confined to the core and attenuate at infinity.
For the radiation modes, however, the situation is quite different: not only they do not attenuate at infinity,
but also they form a continuous spectrum<cit.>.
The former implies that the solutions do not have an associated norm, much less present any sort of orthogonality.

While this behaviour might seem unusual, it is aligned with the fact that the system under analysis is a
non-conservative open system, and thus the properties associated with Hermitian systems no longer hold<cit.>.

This naturally translates to the computational modelling of the slab waveguide.
As previously discussed, the computational domain must be truncated as to have finite dimensions, and this
truncation is often performed with PMLs.
As a consequence, the computational problem has materials with complex coefficients
and the resulting algebraic system is no longer Hermitian.
The continuous spectrum of radiation modes is now discretised into a discrete set of complex modes,
as thoroughly discussed in <cit.>, should the reader be interested in a deeper discussion on the topic.

Therefore, the orthogonality shown in <cit.> for rectangular waveguides do not hold.
While the discussion in <cit.>, allows for inhomogeneous waveguides, none of the examples present a PML or
a lossy region, and μ_r is assumed to be constant and isotropic.

For the reasons above, we state now a general orthogonality relationship for eigenmodes, supporting losses and
anisotropy, based purely on the algebraic eigensystem of the modal analysis.

Let us denote by (β_m^2,e_m) the solution (β_m^2,{e^m_t,e^m_z}) from problem <ref>
associated with the m-th mode. From <cit.>, we have then


    (β_n^2-β_m^2)⟨ Be_n, e_m^*⟩, = (β_n^2-β_m^2)e_m^TBe_n,

where ⟨·,·⟩ denotes the conjugate inner product, and we have used the fact that
the solution of the adjoint problem is the conjugate of the solution of the original problem.
Equation <ref> implies in

    (β_n^2-β_m^2)
      ∫_Γ{μ_r^-1[( ∇_t e_n,z +e_n,t)×ẑ]}·
    [( ∇_t e_m,z +e_m,t)×ẑ]
    
      -k_0^2[ϵ_r(e_n,zẑ)]·(e_m,z ẑ)
      dΓ = 0.


The term

    ∫_Γ{μ_r^-1[( ∇_t e_n,z +e_n,t)×ẑ]}·[( ∇_t e_m,z)×ẑ]
    
      -k_0^2[ϵ_r(e_n,zẑ)]·(e_m,z ẑ)
      dΓ,

corresponds to the right-hand side of the weak formulation evaluated at the solution {e^n_t,e^n_z}
tested against the test function associated with the dofs {0,e^m_z}.
Given that the left-hand side of the weak formulation is zero for such a test function (there are only terms
associated with e_t), this term must also be zero and it can be eliminated, simplifying thus
Equation <ref> to 

    (β_n^2-β_m^2)
    ∫_Γ{μ_r^-1[( ∇_t e_n,z +e_n,t)×ẑ]}·(e_m,t×ẑ)
    dΓ = 0.


Applying the inverse transformation of Equation <ref>, one obtains

    (β_n^2-β_m^2)
    ∫_Γ{μ_r^-1[( jβ_nE_n,t+ ∇_t E_n,z)×ẑ]}·(E_m,t×ẑ)
    dΓ = 0.


Now, we can make use of the expression for the
curl of a waveguide mode in Equation <ref>, repeated here for clarity

    <ref>(E(x,y)e^-jβ z) =
    [(jβE_t+ E_z)×ẑ+
      (∂ E_y/∂ x-∂ Ex/∂ y)ẑ]e^-jβ z,

and, observing that the z-component of Equation <ref> does not affect the inner product,
obtain:

    (β_n^2-β_m^2)e^jβ_nz
    ∫_Γ[μ_r^-1∇×(E_ne^-jβ_n z)]
      ·(E_m,t×ẑ)
    dΓ = 0.


Since


    H_ne^-jβ_nz = -1/jωμ_0μ_r^-1[∇×(E_ne^-jβ_nz)],

we have

    jωμ_0(β_n^2-β_m^2)∫_Γ(H_n×E_m)·ẑdΓ.


Thus, if the modes associated with degenerate eigenvalues are orthogonalised appropriately, we can then write


    ∫_Γ(H_n×E_m)·ẑdΓ, =
      κ_m, m=n
    
        0, m≠ n
        ,

  with
  
    κ_m = ∫_Γ(H_m×E_m)·ẑdΓ =
    -1/jωμ_0∫_Γ{μ_r^-1[( jβ_mE_m,t+ ∇_t E_m,z)×ẑ]}·
    (E_m,t×ẑ)
      dΓ



The results of this section are summarised in Figure <ref>,
where the magnitude of the tangential component of the cross product between the magnetic and electric fields
is plotted for the first two hundred computed modes of a step-index optical fibre, illustrating that for non-Hermitian
systems, the orthogonality holds only for the non-conjugated cross product.



 §.§ Waveguide port boundary condition


With the orthogonality condition at hand, we can finally derive the WPBC.
The derivation will be performed assuming that the port is aligned with the xy plane
and the outward normal vector is ẑ in the negative direction, as illustrated
in Figure <ref>.

Repeating for clarity Equation <ref>:

    <ref>E=E_in+E_ref = E_in+∑_i α_iE_ie^+jβ_i z,

it should be clear that Equations <ref> and <ref> allows for a closed expression for α_i, which is obtained by taking the cross product of <ref> with H_m followed by the dot product with
the outward normal vector -z and integrating over the boundary Γ, resulting in


    α_m = e^-jβ_m z/κ_m∫_ΓH_m×(E-E_in)·ẑ dΓ.


Since our formulation involves only the E field, we can eliminate H_m through Faraday's Law
applied to a backward propagating waveguide mode, that is


    H_m e^jβ_m z = -1/jωμ_0μ_r^-1(E_m e^jβ_m z),

which, if applied in <ref>, results in 

    α_m = 
      -e^-jβ_m z/jωμ_0κ_m
    ∫_Γ[μ_r^-1(E_m e^jβ_m z)e^-jβ_m z]×(E-E_in)·ẑ dΓ.


Upon the expansion of the expression for the curl of a backward propagating waveguide mode,

    α_m = -e^-jβ_m z/jωμ_0κ_m
    ∫_Γμ_r^-1[(-jβ_mE_m,t+ E_m,z)×z]
    ·[(E-E_in)×ẑ] dΓ.


Equation <ref> is a closed expression for α_m in terms of the
source field E_in and the total field E,
therefore suitable for usage in a mixed boundary condition term
for the scattering formulation. Since the boundary term of Equation <ref> is of the form

    ∫_Γ[μ_r^-1E]·(ϕ^*×n) dΓ,

we manipulate Equation <ref> as to obtain an expression for μ_r^-1E.
Taking its curl and applying the inverse relative permeability tensor:

    μ_r^-1E = 
    μ_r^-1E_in+∑_iα_i μ_r^-1[(E_i e^jβ_i z)]=
    μ_r^-1E_in+
    ∑_iα_i μ_r^-1[(-jβ_iE_i,t+ E_i,z)×z]e^jβ_i z,

and by application of the expression for α_i given in Equation <ref>:

    μ_r^-1E = μ_r^-1E_in
    
      -∑_i1/jωμ_0κ_i[(-jβ_i E_t,i+ E_z,i)×z]
    ∫_Γμ_r^-1[(-jβ_iE_,t+ E_i,z)×z]·[(E-E_in)×ẑ] dΓ.


The curl at the boundary can thus be expressed as

    μ_r^-1E+P(E) = U_in,

where

    P(E) = 
    ∑_i1/jωμ_0κ_iμ_r^-1[(-jβ_i E_t,i+ E_z,i)×z]
    ∫_Γμ_r^-1[(-jβ_i E_t,i+  E_z,i)×z]·(E×z) dΓ

and

    U^in = μ_r^-1E^in+P(E^in).

resulting in

    ∫_ΓP(E)·(ϕ^*×z) dΓ =
        ∫_ΓU_in·(ϕ^*×z) dΓ,

since at this boundary n=-z. Equations <ref> to <ref>
are similar to what has been developed in the literature
for the isotropic (and with homogeneous permeability) case.
Upon inspection of Equation <ref>, one realises that the double integrals mean that P(E) will
need to be evaluated for each trial function at Γ, which most likely translates to cumbersome implementation issues.

At this point, it might be useful to step back and reflect on the main aspects of the WPBC.
First, it enforces that the solution at Γ is a linear combination of waveguide modes.
Moreover, it computes the coefficients α_m such that the resulting combination represents the reflection
at the boundary. Both aspects are enforced weakly, since they are imposed by the weak formulation and
not at the approximation space level. It turns out that the WPBC can be greatly simplified if one of these
aspects is strongly enforced through restriction of the approximation space, which is the subject of the next
section.


 §.§ Approximation space restriction


The inspection of Equations <ref> and <ref> provides
an insight on how the WPBC could be simplified should the trial functions at the boundary
coincide with the tangential field components E_j,t.
Assuming that all the modes at the boundary were computed, this operation is equivalent
to a change of basis or, when just a few modes are applied, to apply a reduced basis
for the degrees of freedom at the port.
 
A comprehensive description of the implementation and the intricacies of the procedure is the subject of ongoing work; however, an illustrative example is provided here.

Approximation space restriction has been present in some - but not all - FEM codes for a couple of decades now,
specially in the context of hanging nodes support, as shown in Figure <ref>.



In this example, an Ω space is used for simplicity, and one can see
that the mesh is not Ω-conforming as shown by the relevant traces.
One approach to make this mesh conforming without further h-refinement is to restrict
the functions ϕ_3, ϕ_4 and ϕ_5, so they are no long free dofs.

In this case, we can see that defining basis functions ϕ_3' and ϕ_4' such that

    ϕ_3' = ϕ_3+1/2ϕ_4
    ϕ_4' = ϕ_5+1/2ϕ_4

will result in a conforming mesh. Therefore, we say that the functions {ϕ_3,ϕ_4,ϕ_4} are restricted
to ϕ_3' and ϕ_4', as Equation <ref> must always
hold for the space to be Ω-conforming.

In terms of implementation, the dependency coefficients can be computed through an ∂ K projection
of the traces and, naturally, this dependency must be stored in the computational mesh.
Then, the element matrix is computed as if no dependencies were present,
and subsequently the dependencies are computed before contributing to the global matrix.

Assuming that the functions ϕ_3, ϕ_4 and ϕ_6 have local support in the
element K_2, this element will contribute to the equations of ϕ_3', ϕ_4' and ϕ_6 and its dependency matrix reads as


    D_2 = [   1   0   0;   0 1/2   0;   0   0   1 ],

and the element matrix is transformed as

    K_2' = D_2^†K_2 D_2

as to incorporate the restrictions, where ^† stands for conjugate transpose.

Similarly, for the K_3 element,
assuming that its local-supported functions are ordered as ϕ_4, ϕ_5 and ϕ_7 results in the following dependency matrix


    D_3 = [ 1/2   0   0;   0   1   0;   0   0   1 ].





 §.§ Waveguide port boundary condition with approximation space restriction

The procedure described in Subsection <ref> can be applied to the WPBC: if
one restricts, at Γ, the approximation space to the tangential components of the waveguide modes
{E_t,i}, then the weak formulation can be simplified, as it is now only responsible
for computing the coefficients α_i. The simplification becomes clear when we evaluate P(E)
for a given transverse modal component, that is:



    P(E_t,j) = ∑_iμ_r^-1[(-jβ_i E_t,i+ E_z,i)×z]1/κ_i
    1/jωμ_0
    ∫_Γμ_r^-1[(-jβ_i E_t,i+  E_z,i)×z]·(E_t,j×z) dΓ,_-κ_i δ_ij

resulting in
  
    P(E_t,j) = -μ_r^-1[(-jβ_j E_t,j+ E_z,j)×z].


The biorthogonality was responsible for eliminating the double integral of the boundary term. We can apply the
same principle to the source term U^in: let us assume now that the incident wave can be projected
over forward propagating waveguide modes, E^in = ∑_kα_k E_k'e^-jβ_k z, where
E_k' = E_k,t-ẑE_z,t. This results in


    U^in = ∑_k{μ_r^-1[(jβ_kE_k,t- E_k,z)×ẑ]-.
    .
        μ_r^-1[(-jβ_kE_k,t+ E_k,z)×ẑ]
      }α_ke^-jβ_k z,

and

    U^in = -2∑_k
      μ_r^-1[(-jβ_kE_k,t+ E_k,z)×ẑ]
    α_ke^-jβ_k z.


Finally, we can now write the final expressions for the boundary term at the weak formulation:

+ẑ boundary:


    ∫_Γ{μ_r^-1[(jβ_jE_j,t+ E_j,z)×ẑ]}·(E_i,t^*×ẑ) dΓ=
    
        2∑_kα_k e^ jβ_k z
    ∫_Γ{μ_r^-1[(jβ_kE_k,t+ E_k,z)×ẑ]}·(E_i,t^*×ẑ) dΓ


-ẑ boundary:


    ∫_Γ{μ_r^-1[(jβ_jE_j,t- E_j,z)×ẑ]}·(E_i,t^*×ẑ) dΓ=
    
        2∑_kα_k e^-jβ_k z
    ∫_Γ{μ_r^-1[(jβ_kE_k,t- E_k,z)×ẑ]}·(E_i,t^*×ẑ) dΓ


Inspecting the final expressions should suffice to illustrate how the WPBC is simplified when restricting
the approximation space. As a final remark, it is important to assure that the waveguide modes used in
the expressions correspond to the boundary orientation. However, this is quite simple to deal with,
since changing the sign of the ẑ-component suffices
for alternating between forward/backward propagating modes.



 §.§ Implementation aspects


The contributions of the WPBC can be computed in a post-processing manner, using the solutions of the modal analysis.
Thus, the FEM scheme can simply assemble the matrix associated with Equation <ref>
and subsequently add the WPBC contributions.
Inspecting Equations <ref> and <ref> one can observe that, in the Hermitian problems,
the BC terms result in a diagonal matrix. For the more general case, this doesn't hold: the WPBC contributions
is actually a structurally symmetric complex matrix. Therefore, the resulting algebraic system is now no longer
complex symmetric.

One interesting remark is that, due to the approximation space restriction, at the waveguide port, the scheme will have
one dof per waveguide mode. Therefore, if the modes are normalised according to a quantity of interest,
such as power, then the dof itself provides automatically a result that normally one would need to
post-process to obtain, e.g., the power carried by each mode.

Before discussing the results, one important remark is needed on an efficient implementation of the WPBC: each element adjacent to the WPBC will contribute to all the equations
associated with the modes in this boundary. Therefore, the element matrix obtained after
the approximation space restriction can become quite large when using hundreds of modes
in the corresponding port.
Should the computational domain present reduced dimensions in the propagation direction,
which is often desirable in the analysis of a waveguide discontinuity, one can expect
to an impact in the assembly time due to the elements adjacent to the waveguide ports
when employing a large number of modes at the WPBC.

In order to mitigate this impact, these elements were grouped in patches, with one
element matrix by patch. The effect on the sparsity pattern of the resulting matrix
was not found to be significant, but it should be straightforward to recover the matrix
with the original sparsity pattern, without the element groups.




§ NUMERICAL RESULTS


In this section, numerical results are presented as to proceed with the discussion on the WPBC.
All the results were obtained using the 
library <cit.>, by means of the  library<cit.>, which acts as a wrapper
for electromagnetic problems. All the presented examples are present in the  repository,
and the corresponding scripts will be referenced accordingly.
All meshes were generated using the <cit.> python interface.
The Intel MKL library was used both for the sparse direct solver as for sparse BLAS-2 level operations,
and the  library<cit.> was used for its Krylov-based eigensolver.
Finally, graphic output was exported in  format and visualised in <cit.>.



 §.§ Initial validation: effect of unstructured meshes

In order to validate the scheme, a simple slab waveguide is analysed.
Given the planar nature of the waveguide, the modal and scattering analyses from <cit.> are performed,
using Ω-conforming elements in one and two dimensions, respectively.

The first experiment consists of a short section of a symmetric slab with width w=1, truncated by WPBC
boundaries denoted by Γ_in and Γ_out.
At Γ_in, a combination of the guided modes of the slab, obtained through the modal analysis,
is injected into the domain. As there is no discontinuity, the incident field should undisturbingly propagate along the
waveguide, presenting no reflections.
Denoting the incident field by E_inc = ∑_k α_k E_k'e^-jβ_k z, at a distance d from
Γ_in, the solution is expected to be


    E_ref = ∑_k α_k E_k'e^-jβ_k d.


As to verify the expected solution, the meshing of the slab was performed such that there is a plane (line) Γ_in^e at a distance d from Γ_in with nodes matching the nodes in Γ_in,
as shown in Figure <ref>.



Then, one can compare the difference between E_WPBC,
the field obtained through the scattering analysis, and E_ref. Moreover, E_ref
can also be compared with E_pml,
the field obtained in the scattering analysis with PML-backed plane sources.

The simulation was performed at the operational wavelength λ=1.55
and the PML parameters were computed according to Subsection <ref>
resulting in s_x=39.76 and s_z = 59.64. The used polynomial order for the Ω elements was 4,
and the remaining parameters are available at the script .

For E_inc set as α_1 = 0.5, α_2=2 and α_3 = 2.5, the computed field is shown
in Figure <ref> and the obtained relative errors are


    ‖E_ref-E_PML‖/‖E_ref‖ = 2.235e-06
    ‖E_ref-E_WPBC‖/‖E_ref‖ = 6.203e-08.




The WPBC approach resulted in an error almost 10^2 smaller than the one in the PML-backed propagation,
which is known to introduce numerical reflections<cit.>. Moreover, Equation <ref> provides an insight on how to determine the number of modes
to be used in the WPBC, which is the subject of the next section.

As a concrete measure of the impact of the element size and polynomial order on the
value of ‖E_ref-E_PML‖/‖E_ref‖,  Figure <ref> presents the computed values for a range of
element sizes and varying polynomial order. It is clear that the obtained
value is depends heavily on the mesh and must be computed for each new analysis.





 §.§ On the needed number of modes


The scenarios in which a WPBC is needed often involve reflections for which there is no analytical expression,
and for large problems, computing all the algebraic modes for the waveguide quickly becomes unpractical.
The last result of the previous subsection can be used as a tool for estimating if enough modes are being
employed in the WPBC: it is well known that Equation <ref>
is not expected to be satisfied at machine-level precision for unstructured meshes,
thus Equation <ref> can be seen as a tolerance level for analysing the efficiency of the WPBC.

In order to demonstrate it, a slab waveguide discontinuity is analysed, with core widths w_1=0.4
and w_2=1.5. On the slab with width w_1, the dominant mode is injected, and WPBCs are placed
on both sides 1 away from the junction.
For this experiment, the number of modes used in each WPBC, n_m^in and n_m^out, was varied from 3 to 300.

For each combination of (n_m^in,n_m^out), the following procedure was executed after
the scattering analysis was performed: at Γ_in^e (resp. Γ_in^e, now simply denoted by Γ^e),
a Finite Element space 𝒰 was created and,
since the mesh nodes are periodic to Γ_in (resp. Γ_out), the computed
waveguide modes at the WPBC were transferred to 𝒰.
Then, the 1D approximation space at Γ^e was restricted such that its only free dofs are the waveguide modes.
At the restricted 𝒰, one can compute u_p, a L^2(Γ^e)-projection of the scattering solution u,
and the norm r = ‖ u_p-u‖/‖ u ‖ provides a measure on how well the employed modes are able to represent
the solution at a close distance to the WPBC. A big r would indicate that there are higher-order modes being reflected
from the WPBC, and therefore more modes need be used at the boundary condition.

Figure <ref> shows the obtained r for the slab discontinuity scattering shown in Figure <ref>,
ran using the script .
The dashed line represents the numerical tolerance obtained from Equation <ref>: for this
analysis, 200 modes was considered sufficient on both domain truncations.







 §.§ Step fibre discontinuity


In order to evaluate the competitive of the scheme in terms of computational time,
propagation along a step-index optical fibre discontinuity was used.
Due to the size of the linear system resulting from the 3D scattering analysis, a iterative solver
was used, namely GMRES, with a preconditioning scheme to be discuss in this section.

The geometry of the problem is shown in Figure <ref>.
Element size was chosen as four elements per wavelength in the cladding and the
polynomial order as k=3.
This choice is due to limits on available computational resources, as the
memory requirements are larger when analysing 3D problems.
The fibres have n_core=1.4457 and n_clad=1.4378
and were analysed at λ_0=1.55,
with the radial PML region placed at 3.5 cladding wavelengths from the core.
At the chosen operational wavelength, even the dominant modes are not too confined
to the core, therefore the PML parameters were set at α_r = 0.4 and, for the
PML-backed ports, α_z=1, as to improve the conditioning of the matrix.

The coefficient
‖E_ref-E_WPBC‖/‖E_ref‖
was computed for a fibre with r=6 and resulted in 0.00277218,
which, while considerably larger than the one in Equation <ref>,
seems reasonable in the light of Figure <ref>.
This computation was executed with the script .




The step-index discontinuity was setup with two fibres of radii
r_1=6 and r_2=8, and the waveguide ports
were placed at a distance of 2 cladding wavelengths from the discontinuity.
Figure <ref> shows that 900 modes on each port were sufficient to
absorb the reflections due to the discontinuity,
therefore that was the chosen number of modes used in the comparison with
the PML-backed ports. 

For solving the resulting algebraic system of the scattering problem, a preconditioned
GMRES solver was chosen.
The preconditioning scheme was a block Jacobi preconditioner, with the following choice
of blocks, based on <cit.>:


    {𝒱}={ℰ_l.o.}⊕{ℬ𝒞}_sparse⊕{ℰ_h.o.}⊕{ℱ}⊕{ℐ}_c.


The block {ℰ_l.o.} is the block of the lowest order edge functions,
which is combined with {ℬ𝒞}, the equations from the WPBC boundary,
as a sparse block to be decomposed by a direct solver. Then, the remaining blocks
are the small dense blocks {ℰ_h.o.}, associated with the higher order
edge functions, and {ℱ}, with the face functions.
The dofs associated with the internal functions of each element are
eliminated through static condensation, therefore they do not appear explicitly in the
algebraic system.
In Table <ref>, the results of the script  are shown, comparing between the WPBC and PML-backed ports.
These were obtained in a computer with a Intel Xeon Gold 6130 CPU,
with 12 physical cores at 2.1 and 376 of RAM memory.
The measured times are split into assembly time t_a, solving time t_s and total time t_t.

One can see that, with reduced dimensions in the propagation direction and when a large number of modes is needed,
the assembly time of the WPBC setup is considerable.
However, the iterative solver converged in 40% less iterations,
and the memory consumption was significantly reduced,
as the resulting computational domain was able to be shortened.







 §.§ Plasmonic sensor


As a final demonstration of the capabilities of the WPBC,
the nanograting-based plasmonic sensor presented in <cit.> is analysed.
The device is introduced in the context of biologic sensors, and presents a periodicity
that allows for the numeric analysis to be performed in a unit cell employing periodic
boundary conditions.



The geometry of the device is shown in Figure <ref>. The numerical
setup consists in illuminating the device from the top boundary. The left and right
boundaries are set to be Floquet periodic, in which a zero-valued phase coefficient
is associated with a normal incidence angle. In <cit.>, the so-called DGC-SPR
sensor is compared with a simple copper grating.
Both devices consist of a copper layer with ridges. In the former case, the ridges
are made of the photoresist AZ5314E, in the latter, copper. The dielectric data is
presented in the supplementary material of the paper and the raw data was kindly
given to us upon request.

Even though the devices are not waveguides, the WPBC application is straightforward:
a one-dimensional modal analysis performed on the top boundary with periodic BCs
should provide the eigenfunctions for representing the reflected field produced by
illuminating the device, by the same concepts discussed in the introduction of Section
<ref>.

In Figure <ref>, the obtained field distributions corresponding
to Figures 2(e,f) of <cit.> are shown, illustrating then the application
of the WPBC in such a scenario.





§ CONCLUSIONS


An implementation of the Waveguide Port Boundary Condition based on the restriction
of the approximation space is presented, avoiding the double integrals involved in the
original form of such boundary condition.
The implementation is validated in two and three dimensions along with the presentation
of a strategy for determining the needed number of modes for scenarios with strong
reflections, which are often the case in the numerical analysis of waveguide discontinuities.
A computationally intensive three dimensional example is used for a comparative
performance analysis with PML-backed waveguide ports, showing that despite the
considerably larger assembly time, it still presents advantage in terms of processing
time due to the reduction of the computational domain in the absence of PMLs.
The computation of the fields distribution of a nanograting-based plasmonic sensor is used
as an illustration of possible applications of the developed technique apart from
waveguide analysis. Future applications of the technique are under active development, with particular emphasis on its utilisation in the analysis of multiple sequential waveguide discontinuities.



§ ACKNOWLEDGEMENTS


Francisco T. Orlandini is grateful to the support during the development of this work
provided by the Brazilian Research Council (CNPq) (Grant 445074/2020-5).

The author Philippe R. B. Devloo is also grateful to CNPq (grants 305823/2017-5 and 306167/2017-4) and extends his gratitude to EPIC – Energy Production Innovation Center, hosted by the University of Campinas(UNICAMP) and sponsored by Equinor Brazil and FAPESP – São Paulo Research Foundation (2017/15736-3).

Hugo E. Hernández-Figueroa is thankful to the São Paulo Research Foundation (FAPESP) under the projects 2021/06506–0 (StReAM), 2021/11380–5 (CPTEn), 2021/00199–8 (SMARTNESS), and (EMU) 2022/11596-0; and the Brazilian Agency CNPq, under the project 314539/2023-9 (HEHF’s research productivity grant).

Finally, the authors of <cit.> have kindly provided the dielectric data
for their plasmonic sensor allowing for us to analyse it as a final example in the
present work, and therefore deserve to be included here.





§ PML AS AN ANISOTROPIC ABSORBER


The main idea of the Berenger PML is to introduce a complex coordinate stretching on a region
of the domain such that prescribed losses are slowly increased in a way that, for the
exact wave equation, incident waves are
absorbed in a reflectionless manner. The PML can also be seen as an anisotropic absorber<cit.>, and
through the ideas in <cit.>, its equivalency can be shown.

Let us assume for now that the transformed coordinates inside the PML region are given by


    x' → s_x x,     y' → s_y y,     z' → s_z z,


where s_i, i=x,y,z is the complex scaling coefficient causing the attenuation in the i direction.
The coordinate transformation can be described by the Jacobian matrix


    J_PML =
      [ s_x   0   0;   0 s_y   0;   0   0 s_z ],

and the relative electric permittivity (magnetic permeability) ϵ_r (μ_r)
can be transformed by <cit.>:


    γ_PML = 1/J_PMLJ_PML^-1γJ^-T_PML,  γ = ϵ_r, μ_r,


which, for isotropic materials, result in


    γ_PML = γT,  T =
      [ s_y s_z/s_x           0           0;           0 s_x s_y/s_y           0;           0           0 s_x s_y/s_z ].


Finally, for a cylindrical PML, the procedure is carried out similarly by composing the Jacobians
of the transformations to and from the cylindrical coordinate system.


 §.§ Determining the PML coefficients


In <cit.>, the stretching coefficient s_i, i=x,y,z is computed as
s_i = α_max^i (ρ_i/d_i)^m, where ρ_i is the distance
from the beginning of the PML region, d_i is the width of the PML region and


    α_max^i = -(m+1)λ/4π n d_ilnR,

with n being the refractive index of the adjacent region and R the desired
reflection coefficient. In the present work, m=2 and R=10^-70 were used
as initial parameters for setting up the PMLs.







 Paper Title: A τ Matrix Based Approximate Inverse Preconditioning for Tempered Fractional Diffusion Equations.


	
	
		

a. School of Mathematics and Statistics, Beijing Technology and Business University, Beijing 100048, China
b. School of Applied Science, Beijing Information Science and Technology University, Beijing 100192, China
		§ ABSTRACT 
			Tempered fractional diffusion equations are a crucial class of equations widely applied in many physical fields. In this paper, the Crank-Nicolson method and the tempered weighted and shifts Grünwald formula are firstly applied to discretize the tempered fractional diffusion equations. We then obtain that the coefficient matrix of the discretized system has the structure of the sum of the identity matrix and a diagonal matrix multiplied by a symmetric positive definite(SPD) Toeplitz matrix. Based on the properties of SPD Toeplitz matrices, we use τ matrix approximate it and then propose a novel approximate inverse preconditioner to approximate the coefficient matrix. The τ matrix based approximate inverse preconditioner can be efficiently computed using the discrete sine transform(DST). In spectral analysis, the eigenvalues of the preconditioned coefficient matrix are clustered around 1, ensuring fast convergence of Krylov subspace methods with the new preconditioner. Finally, numerical experiments demonstrate the effectiveness of the proposed preconditioner.
		
		
			tempered fractional diffusion equationsapproximate inverse preconditioningτ matrix Krylov subspace method 
		
	
	
	
	

§ INTRODUCTION

	This paper discusses the numerical solution of variable coefficient tempered fractional diffusion equations (Tempered-FDEs) with initial-boundary value problems:
	
    {[ ∂ u( x,t )/∂ t=d( x ) ( _aD_x^β ,λ+_xD_b^β ,λ) u( x,t ) +f( x,t ) ,;                               u( a,t ) =0, u( b,t ) =0, t∈[ 0,T ] ,;                                    u( x,0 ) =u_0( x ) , x∈[ a,b ] ,;                                                                     ].

	where f(x,t) is the source term, d( x ) ≥ 0 is the diffusion coefficient function, and λ is a non-negative parameter. In Equation (1.1), _aD_x^β ,λ and _xD_b^β ,λ denotes the left and right Riemann-Liouville tempered fractional derivatives of the function u(x,t) with order β( 1<β <2 ), respectively defined by (see Baeumer and Meerschaert <cit.>):
	
    _aD_x^β ,λu( x ) =_aD_x^β ,λu( x ) -βλ ^β -1∂ _xu( x ) -λ ^βu( x )

	and
	
    _xD_b^β ,λu( x ) =_xD_b^β ,λu( x ) +βλ ^β -1∂ _xu( x ) -λ ^βu( x )

	where
	
    _aD_x^β ,λu( x ) =e_a^-λ xD_x^β( e^λ xu( x ) ) =e^-λ x/Γ( 2-β)∂ ^2/∂ x^2∫_a^xe^λ su( s )/( x-s ) ^β -1ds

	and
	
    _xD_b^β ,λu( x ) =e_x^λ xD_b^β( e^-λ xu( x ) ) =e^λ x/Γ( 2-β)∂ ^2/∂ x^2∫_x^be^-λ su( s )/( s-x ) ^β -1ds.

	
	Tempered fractional diffusion equations (Tempered-FDEs) are a crucial class of equations widely applied in fields such as biology, geophysics, and finance <cit.>-<cit.>. These equations are characterized by incorporating tempered fractional derivatives, modifying standard fractional diffusion equations by introducing an exponential tempering factor. This modification allows Tempered-FDEs to better simulate processes with finite propagation speed, accurately describing actual physical phenomena. However, analytical solutions to Tempered-FDEs and FDEs are generally difficult to obtain, leading to extensive research on the numerical solutions of FDEs in recent years <cit.>-<cit.>.
	
	For tempered fractional derivatives, stability of the resulting finite difference schemes can only be ensured if the spatial step size is sufficiently small when using standard fractional derivative approximations directly <cit.>. A novel shifted approximation method for tempered fractional derivatives was proposed in <cit.>, successfully developing an unconditionally stable numerical scheme for one-dimensional (1D) tempered fractional diffusion equations with constant coefficients. Additionally, in <cit.> proposed a Crank-Nicolson scheme for solving initial-boundary value problems of a class of variable coefficient tempered fractional diffusion equations, providing the discretization method used in this paper.
	
	Similar to fractional derivatives, tempered fractional derivatives also exhibit non-locality, resulting in dense or even full discrete coefficient matrices in the linear systems generated by discretization. Traditional direct solution methods, such as Gaussian elimination, require computational cost of O(N^3) and storage space of O(N^2), where N is the matrix size <cit.>. Given the symmetric positive definite Toeplitz structure in the coefficient matrix, matrix-vector multiplications of the discrete coefficient matrix and the preconditioner can be efficiently implemented using fast Fourier transforms <cit.> and discrete sine transforms <cit.>, respectively. For one-dimensional tempered space fractional diffusion equations, <cit.> proposed a scaled diagonal and Toeplitz approximate splitting (SDTAS) preconditioning, theoretically proving that the spectra of the resulting preconditioned matrix cluster around 1. For one-dimensional space fractional diffusion equations, <cit.> introduced an approximate inverse preconditioning matrix based on the τ matrix using the ideas of row-by-row inversion and interpolation, leveraging the properties of the symmetric positive definite Toeplitz matrix in the coefficient matrix. Results showed that preconditioned Krylov subspace methods converged at a superlinear rate with a total complexity of O(N log N).
	
	In this paper, we use the second-order finite difference scheme proposed in <cit.> to discretize the Tempered-FDEs. Specifically, we employ the Crank-Nicolson method for time discretization and the tempered weighted and shifts Grünwald formula for spatial discretization <cit.>. We keenly observe that the coefficient matrix of the resulting linear system is of the form I + DT, where I is the identity matrix, D is a diagonal matrix, and T is a symmetric positive definite (SPD) Toeplitz matrix. Given the SPD property of the coefficient matrix, we approximate it using τ matrix and construct a novel preconditioner using the row-by-row approximate inverse idea proposed by Pan and NG <cit.>. Due to the SPD nature of the Toeplitz matrix in the coefficient matrix, the preconditioner can be efficiently computed using discrete sine transforms <cit.>, with each step of the preconditioned Krylov subspace method having a computational complexity of O(N log N). This construction method was first proposed in <cit.> for coefficient matrices of the form D + T. Hence, we construct a row-by-row inverse preconditioner based on τ matrix approximation and apply it for the first time to the coefficient matrix form I + DT generated by Tempered-FDEs. Theoretically, we prove that the spectra of the resulting preconditioned matrix cluster around 1, ensuring rapid convergence of Krylov subspace methods with the new preconditioner. Numerical experiments demonstrate the effectiveness of the proposed preconditioner, outperforming other preconditioners.
	
	The rest of this paper is organized as follows. In section 2, we present the discretized system from the Tempered-FDEs (1.1). In section 3, we construct an approximate inverse preconditioner. In section 4, We conduct spectral analysis of proposed preconditioned matrices. In section 5, numerical experiments are implemented to demonstrate the performance of the proposed preconditioner. Finally, we give a brief conclusion of the entire article in section 6.
	
	

§ DISCRETIZATION OF TEMPERED-FDES


	Let N and M be positive integers, representing the number of spatial and temporal partitions, respectively. We define the spatial step size h = a/N+1 and the time step size Δ t = T/M. The spatial and temporal grids are given by x_i = ih for i = 0, 1, …, N+1 and t_j = jΔ t for j = 0, 1, …, M.
	
	Li and Deng <cit.> showed that the tempered fractional derivatives in (1.1) can be approximated using the tempered weighted and shifted Grünwald formula (Tempered-WSGD):
	
    _aD_x^β ,λu( x_i,t_j ) -λ ^βu( x_i,t_j ) =1/h^β∑_k=0^i+1g_k^( β)u( x_i-k+1,t_j ) -1/h^βρ _βu( x_i,t_j ) +O( h^2 ) ,

	and
	
    _xD_b^β ,λu( x_i,t_j ) -λ ^βu( x_i,t_j ) =1/h^β∑_k=0^N-i+2g_k^( β)u( x_i+k-1,t_j ) -1/h^βρ _βu( x_i,t_j ) +O( h^2 ) ,

	where ρ _β=( γ _1e^hλ+γ _2+γ _3e^-hλ) ( 1-e^-hλ) ^β, the weights g_k^( β) are given by
	
    g_k^( β)=γ _1w_0^( β)e^hλ,   		k=0,
    γ _1w_1^( β)+γ _2w_0^( β),   		k=1,
    ( γ _1w_k^( β)+γ _2w_k-1^( β)+γ 	_3w_k-2^( β)) e^-( k-1 ) hλ,   		k≥ 2.

	with w_0^( β)=1,w_k^( β)=( 1-1+β/k) w_k-1^( β),  for k≥ 1 and γ _1,γ _2,γ _3 satisfy the linear system
	
    {[ γ _1+γ _2+γ _3=1,;   γ _1-γ _3=β /2.;                   ].

	
	Let u_i^j≈ u( x_i,t_j ) ,f_i^j+1/2=f( x_i,t_j+1/2) ,t_j+1/2=1/2( t_j+t_j+1) and u_i^j+1/2=1/2( u_i^j+u_i^j+1) for i=1,...,N,j=0,1,...,M and 1<β <2. At the mesh point ( x_i,t_j+1/2), We consider the Crank-Nicolson technique for time discretization and the tempered-WSGD approximation for the tempered fractional derivatives to discretize the tempered-FDEs in (1.1). Ignoring the truncation error, we obtain the following second-order finite difference scheme:
	
    u_i^j+1-u_i^j/ t=   d_i/2( 1/h^β∑_k=0^i+1g_k^( β)u_i-k+1^j+1-1/h^βρ _βu_i^j+1+1/h^β∑_k=0^N-i+2g_k^( β)u_i+k-1^j+1-1/h^βρ _βu_i^j+1)
       +d_i/2( 1/h^β∑_k=0^i+1g_k^( β)u_i-k+1^j-1/h^βρ _βu_i^j+1/h^β∑_k=0^N-i+2g_k^( β)u_i+k-1^j-1/h^βρ _βu_i^j) 
       +f_i^j+1/2.


	We define the diagonal matrix as
	
    D=diag( d_1,d_2,...,d_n )

	and the vectors as
	
    u^j=[ u_1^j,u_2^j,...,u_n^j] ^T,f^j+1/2=[ f_1^j+1/2,f_2^j+1/2,...,f_n^j+1/2] ^T.

	
	For j=0,1,...,m and a given u^0, The differential format (2.3) can be rewritten in the following matrix form:
	
    ( I+DG ) u^j+1=( I-DG ) u^j+Δ tf^j+1/2,

	where I is the identity matrix, and G is a symmetric positive definite Toeplitz matrix, defined as
	G=G_β+G_β^T, where G_β^T represents the transpose of G_β. G_β can be represented as
	
    G_β=- t/2h^β( 
    			g_1^( β)   		g_0^( β)   		0   		...   		0   		0
    
    			g_2^( β)   		g_1^( β)   		g_0^( β)   		0   		...   		0
    ⋮   ⋱   ⋱   ⋱      ⋮
    
    			g_N-1^( β)   ⋱   ⋱   ⋱      		g_0^( β)
    
    			g_N^( β)   		g_N-1^( β)   		...   		...   		g_2^( β)   		g_1^( β)
    )
 
	
	 Hence, the coefficient matrix A of Tempered-FDEs (2.3) is the sum of the identity matrix plus the diagonal matrix multiplied by a SPD Toeplitz matrix, which we denote as
	
    A=I+DG

	
	
	By the following lemma 2.1, it is proven that the finite difference scheme (2.3) is unconditionally stable and second-order accurate in both space and time.
	
	
		
		Let S be the solution set of the linear system (2.2), if 1<β<2
		λ≥ 0,( γ _1,γ _2,γ _3 ) ∈𝒮,and
		
			    1.max{ 2( β^2+3β-4 )/β^2+3β+2,β^2+3β/β^2+3β+4 } <γ_1<3( β^2+3β-2 )/2( β^2+3β+2 );or   

			    2.( β-4 ) ( β^2+3β+2 ) +24/2( β^2+3β+2 )<γ_2<min{ ( β-2 ) ( β^2+3β+4 ) +16/2( β^2+3β+4 ),( β-6 ) ( β^2+3β+2 ) +48/2( β^2+3β+2 ) } ;or   

			    3.max{ ( 2-β) ( β^2+β-8 )/β^2+3β+2,( 1-β) ( β^2+2β)/2( β^2+3β+4 ) } <γ_3<( 2-β) ( β^2+2β-3 )/2( β^2+3β+2 )   
		
		we have
		
    g_1^( β)<0,   g_2^( β)+g_0^( β)>0,   g_k^( β)>0,  ∀ k≥ 3,  ∑_k=0^∞g_k^( β)=ρ _β≥ 0.

		
	


	

§ APPROXIMATE INVERSE PRECONDITIONER BASED ON Τ MATRIX

	
	From the previous section, we know that at each time step, we need to solve a large linear system of equations (2.4), where the coefficient matrix is defined as in Equation (2.5). Since the coefficient matrix A is asymmetric, we can use Krylov subspace methods for solving, such as GMRES. To improve the performance and stability of Krylov subspace methods, preconditioning of the linear system is necessary. Pan and Ng<cit.> proposed an approximate inverse preconditioner, and this preconditioning method has been widely used <cit.>.
	
	Based on the fact e_i^TA=e_i^TK_i and approximation e_i^⊺A^-1≈ e_i^⊺K_i^-1, where e_i represents the i-th column of the identity matrix, K_i=I+d_iG,i=1,2,…,N. we obtain the approximate inverse preconditioner P_1:
	
	
    P_1^-1=∑_i=1^Ne_ie_i^TK_i^-1,

	
	Since K_i is a Toeplitz matrix, one method is to approximate it with a circulant matrix, yielding a circulant matrix based preconditioner:
	
	
    P_C^-1=∑_i=1^Ne_ie_i^TC_i^-1,

	
	However, we note that G is an SPD Toeplitz matrix, we can use the τ matrix to approximate G<cit.>. For convenience, the first column of the G matrix can be expressed as - t/2h^β[ 2( g_1^( β)-ρ _α) ,g_0^( β)+g_2^( β),g_3^( β),...,g_N^( β)] ^T. According to <cit.>, we can compute the τ matrix approximation using Hankel correction:
	
	
    τ( G ) =G-HC( G ) ,

	where HC( G ) is a symmetric Hankel matrix, and its anti-diagonal elements can be computed from the corresponding elements in the following first column and last column:
	
	
    - t/2h^β( g_3^( β),g_4^( β),...,g_N^( β),0,0 ) ^T,   - t/2h^β( 0,0,g_N^( β),...,g_4^( β),g_3^( β)) ^T,

	
	It is well-known that τ( G ) defined in (3.1) can be diagonalized by the discrete sine transform<cit.>:
	
	
    τ( G ) =SΛ S,   S=( √(2/N+1)·sinπ ij/N+1) ,   i,j=1,2,...,N,

	where Λ =diag( λ _1,λ _2,...,λ _N ), S is the discrete sine transform matrix. 
	
	Clearly, G_j=I+d_jτ( G ) remains a τ matrix. Hence, using the τ matrix G_j to approximate the SPD Toeplitz matrix K_j, we obtain the τ matrix-based approximate preconditioner:
	
	
    P_2^-1=∑_i=1^Ne_ie_i^TG_i^-1.

	
	Using the preconditioner P_2^-1, we need to compute O( N ) discrete Sine transforms for each iteration, which is still computationally intensive. Pan and Ng <cit.> propose using interpolation methods can significantly reduce the computational complexity.
	
	Define {x_i}_i=1^N as the set of all discrete points in the interval [a, b], and the function q_k( x ) =1/1+λ _kd( x ), where λ _k∈ sp( τ( G ) ) ={λ _1,λ _2,...,λ _N},k=1,2,...,N. Selects l( l≪ N ) interpolation points {( x̃_j,q_k( x̃_j ) ) }_j=1^l in ( x_i,q_k( x_i ) ) }_i=1^N and then uses piecewise linear interpolation to obtain the interpolation function: 
	
	
    p_k( x )    =ϕ _1( x ) q_k( x̃_1 ) +ϕ _2( x ) q_k( x̃_2 ) +⋯ +ϕ _l( x ) q_k( x̃_l ) 
       =∑_s=1^lϕ _s( x ) q_k( x̃_s )

	
	We use formula (3.3) to diagonalize G_j as G_j=SΛ _jS, j=1,2,...,N, where S is the discrete sine transform matrix, and _j is the diagonal matrix composed of the eigenvalues of G_j. Then, applying interpolation (3.5) to approximate G_j^-1, we have:
	
	
    G_j^-1≈ S( ∑_s=1^lϕ _s( x_j ) Λ _s^-1) S,

	
	Replacing G_j^-1 in (3.4) with approximation values (3.6), we obtain the following preconditioner:
	
	
    P_3^-1   =∑_i=1^Ne_ie_i^TS( ∑_s=1^lϕ _s( x_i ) Λ _s^-1) S
       =∑_s=1^l( ∑_i=1^Ne_ie_i^Tϕ _s( x_i ))S( Λ _s^-1) S
       =∑_s=1^lΦ _sSΛ _s^-1S,

	where Φ _s=diag( φ _s( x_1 ) ,φ _s( x_2 ) ,...,φ _s( x_N ) ), Λ _s^-1=diag( q_1( x̃_s ) ,q_2( x̃_s ) ,...,q_N( x̃_s ) ), s=1,2,...,l are diagonal matrices. Finally, for a suitable number of interpolation points l, the computation of the preconditioner (3.7) requires O( lNlog N ) operations, which is acceptable.
	
	

§ SPECTRAL ANALYSIS

	In this section, we discuss the spectral properties of the preconditioned matrix P_3^-1A. We first present some off-diagonal decay property reasoning.
	 
	(<cit.>)
		Let A = (a_i,j)_i,j ∈ I be a matrix, where I = ℤ, ℕ or {1, 2, …, N}, then we say A belongs to the class ℒ_s, if 
		
    |a_i,j| ≤c/(1 + |i - j|)^s

		for s > 1, and some constant c > 0.
	
	
	(<cit.>)
		Let g_k^(β) be as defined in Section 2 with 1 < β < 2. Then it holds:
		
    g_k^(β) = (1 - β + 1/k)g_k-1^(β),   k = 1, 2, …,

		
    g_0^(β) = 1,   g_1^(β) = -β < 0,   1 > g_2^(β) > g_3^(β) > ⋯ > 0  and

		
    ∑_k=0^∞ g_k^(β) = 0,  ∑_k=0^m g_k^(β) < 0,   1 ≤ m < ∞.

		
	
		
	(<cit.>)
		Suppose g_k^(β) is defined by Section 2 with 1 < β < 2. Then
		
    g_k^(β) = 1/Γ(-β)k^β+1 (1 + O(1/k)),

		where Γ(x) is the Gamma function.
	
	
	(<cit.>)
		Let G, A, K_i and T_β be as defined in Sections 2 and 3, then G^-1, A^-1, K_i^-1 and G_β^-1∈ℒ_β+1, i.e.,
		
    |L_m,n| ≤c_0/(1 + |m - n|)^β+1,

		where c_0 is a positive constant and L = (L_m,n)_m,n ∈ I with I = {1, 2, …, N} can be G, A, K_i,G_β, G^-1, A^-1, K_i^-1 and G_β^-1.
	

	
		Let β∈ (1, 2) and L ∈ℒ_β+1. Then ∃ a constant ϖ s.t. L_∞≤ϖ.
	
	Proof. By making use of
	
    ∑_k=q+1^∞1/k^β+1≤∫_q+1^∞1/x^β+1  dx = 1/β q^β,

	we have 
	
    ∑_k=q+1^∞1/k^β+1≤1/β.

	Hence, 
	
    L_∞≤max_1 ≤ i ≤ n∑_j=1^n |L_i,j| ≤max_1 ≤ i ≤ n (|L_i,i| + ∑_j ≠ i |L_i,j|) = max_1 ≤ i ≤ n (c_0 + 2 ∑_k=2^∞c_0/k^β+1) ≤(2+β)c_0/β,

	
	then letting ϖ = (2+β)c_0/β, we obtain the result. □

	Next, we discuss the spectral properties of P_3^-1A by analyzing the approximation P_3^-1 - A^-1. Since
	
    P_3^-1 - A^-1 = P_3^-1 - P_2^-1 + P_2^-1 - P_1^-1 + P_1^-1 - A^-1,

	we will separately discuss the approximations P_3^-1 - P_2^-1, P_2^-1 - P_1^-1, and P_1^-1 - A^-1.
	
	First, we analyze the property of P_1^-1 - A. We present Lemma 4.5 with detailed proofs available in <cit.>
	
	
		for a given ε > 0,  there exists a constant c_1 and an integer N_1 such that for l≥ N_1 we have
		
		
    P_1^-1-A^-1_∞≤ c_1max_1≤ i≤ NΔ(x_i,l)+ε.

	where
		
    Δ(x_i,l)=max_i-l<k<i+l|x_k-x_i|=(l-1)h,

	
	
	
	Furthermore, we discuss the property of P_2^-1-P_1^-1. By the definition of P_1 and P_2 in section 3, we have
	
	
    P_2^-1-P_1^-1   =∑_i=1^Ne_ie_i^T( G_i^-1-K_i^-1)
       =∑_i=1^Ne_ie_i^TK_i^-1( K_i-G_i ) G_i^-1
       =∑_i=1^Nd_ie_ie_i^TK_i^-1( G-τ( G ) ) G_i^-1.

	
	We first prove through Lemma 4.6 that G_i^-1 is bounded.
	
	
		Let G_i be defined in section 3. Then we have G_i^-1_∞<η^-1.
	
	Proof. The proof method can refer to Lemma 4.7 in <cit.>, here η=1.
	
	As is shown in <cit.> that G-τ(G) can be split as E_1+F_1 with E_1 being in accordance with G-τ(G) in the upper left and lower right (N-1)×(N-1) sub-matrices and vanishing in the other entries, and rank(E_1)≤2(N-1). Further, if we let ε>0 be fixed, then we have F_1_2<ε.
	
	
	Because K_i^-1 has the off-diagonal decay property, then K_i^-1 can be rewritten as K̃_i+K̂_i, where
	
    K̃_i=[ 
    		*   ⋯   		*   		0   ⋯   		0
    ⋮   ⋱      ⋱   ⋱   ⋮
          ⋱      ⋱   		0
    
    		0   ⋱      ⋱      		*
    ⋮   ⋱   ⋱      ⋱   ⋮
    
    		0   ⋯   		0   		*   ⋯   		*
    ]   and  K̂_i=[ 
    		0   ⋯   		0   		*   ⋯   		*
    ⋮   ⋱      ⋱   ⋱   ⋮
    
    		0      ⋱      ⋱   		*
       ⋱      ⋱      		0
    ⋮   ⋱   ⋱      ⋱   ⋮
       ⋯   		*   		0   ⋯   		0
    ] .

	
	here '*' denotes the nonzero entries.
		
	Considering
	
    K_i^-1(G - τ(G))G_i^-1   = K_i^-1(E_1 + F_1)G_i^-1
       = K_i^-1E_1G_i^-1 + K_i^-1F_1G_i^-1
       = K_i^-1E_1G_i^-1 + (K̂_i + F̂_i)F_1G_i^-1
       = (K_i^-1E_1 + K̂_iF_1)G_i^-1 + F̂_iF_1G_i^-1,

	we derive
	
    (K_i^-1E_1+K̂_iF_1)G_i^-1_∞   ≤(K_i^-1_∞·E_1_∞+K̂_i_∞·F_1_∞)·G_i^-1_∞
       ≤ε(K_i^-1_∞+F_1_∞)·G_i^-1_∞

	and
	
    ∥∑_i=1^Ne_ie_i^G(K_i^-1E_1+K̂_iF_1)G_i^-1∥_∞    =max_1≤ i≤ Ne_i^G(K_i^-1E_1+K̂_iF_1)G_i^-1_1
       =max_1≤ i≤ N(K_i^-1E_1+K̂_iF_1)G_i^-1_∞
       ≤ε·max_1≤ i≤ N(K_i^-1_∞+F_1_∞)·G_i^-1_∞.

	Since K_i^-1_∞ and F_1_∞ are limited due to the off-diagonal decay characteristics of K_i^-1 and G, it follows that G_i^-1_∞ is also bounded. In other words, there is a constant c_2 > 0 such that
	
    ∑_i=1^N e_i^T (K_i^-1E_1 + K̂_iF_1)G_i^-1_1 ≤ c_2 ·ϵ.

	
	Next, we focus on the matrix product K̂_iF_1G_i^-1. Suppose the dimension of the blocks K̂_i matches the dimension of the block F_1. Otherwise, the smaller one can be extended. By examining the structure of K̂_i and F_1, we can compute the following results:
	
    K̃_iF_1G_i^-1=[ 0 0 … … 0 +; 0 0 … … 0 +; 0 0 … … 0 0; ⋮ ⋮ ⋮ ⋮; 0 0 … … 0 0; + 0 … … 0 0; + 0 … … 0 0 ], G_i^-1=[ + + … … + +; + + … … + +; 0 0 … … 0 0; ⋮ ⋮ ⋮ ⋮; 0 0 … … 0 0; + + … … + +; + + … … + + ],

	where ‘+’ denotes nonnegative entries. Therefore, we have
	
    rank( ∑_i=1^N e_i^T K̂_iF_1G_i^-1) ≤ 4ς,

	where ς is the dimension of the blocks in K̂_iF_1G_i^-1. Consequently,
	
    ∑_i=1^N e_i^T ( K_i^-1 - K_i^-1) = ∑_i=1^N e_i^T ( K_i^-1E_1 + K̂_iF_1 )G_i^-1 + ∑_i=1^N e_i^T F̂_iF_1G_i^-1,

	is a sum of a small norm matrix and a low rank matrix. To summarize, we obtain the following results.
	
	
		Let P_1 and P_2 be defined in Section 4. The approximation P_2^-1 to P_1^-1 satisfies
		
    P_2^-1 - P_1^-1 = E_2 + F_2,

		where E_2 and F_2 are matrices of small norm and low rank, respectively, i.e., E_2 < c_2 ·ϵ and rank(F_2) ≤ 4ς.
	

	We now consider the approximation of the preconditioner P_2 and the coefficient matrix A.
	
	
		Let P_1, P_2, and A be defined as previously stated. Then there exists N_2 such that for N > N_2, we have
		
    P_2^-1-A^-1=E_P_2+F_P_2,

		where E_P_2 and F_P_2 are of small norm and of low rank, respectively.
	
	
	Proof. Due to Lemma 4.5, we know
	
    P_2^-1-A^-1_∞≤max_1≤ t≤ NΔ(x_i,N_1)+ε=c_1(N_1-1)h+ε.

	Let  N_2  be an integer such that  (N_1-1)h<ε, then
	
    P_2^-1-A^-1    =P_2^-1-P_1^-1+P_1^-1-A^-1
       =E_2+F_2+P_1^-1-A^-1
       =E_2+(P_1^-1-A^-1)+F_2 
       ≜ E_P_2+F_P_2,

	where E_P_2=E_2+(P_1^-1-A^-1) and F_P_2=F_2. As
	
    E_P_2_∞=E_2_∞+P_1^-1-A^-1_∞<(c_1+c_2+1)·ε,

	then the conclusion follows.□
	
	Finally, we discuss the properties of P_3^-1-P_2^-1. We present the following conclusion, with detailed proof provided in <cit.>.
	
		Suppose ϖ is sufficiently small and l ≪ N. Denote by ϖ = max_1 ≤ i ≤ Nmax_1 ≤ j ≤ N{ | p_λ_j(d_i) - q_λ_j(d_i) | }. Then for any given ϵ > 0, ∃ N_3 > 0 (independent of N) such that
		
    P_3^-1 - P_2^-1 = E_3 + F_3,

		where E_3 and F_3 satisfy E_3_∞≤ϖ (2N_3 + 1) + ϵ and F_3 is of a low rank matrix, respectively.
	
	By combining the above theorems, we arrive at the following conclusion.
	
	
		Let P_3 and A be defined previously, then ∃ an integer N_3,such that for N >N_3 it holds
		
    P_3^-1-A^-1=E_P_3+F_P_3,

		where E_P_3 and F_P_3 are of small norm and of low rank, respectively.
	

	In the following section, we provide numerical experiments to illustrate the effectiveness of the proposed preconditioner.
	
	
	

§ NUMERICAL EXPERIMENTS

	In this section, we conduct numerical experiments on the studied Tempered-FDEs (1.1) to verify the efficiency of the proposed preconditioning matrices P_3^-1 (3.7) denoted as P_TAI^-1( l ) , where l represents the number of interpolation points. For comparison, we also use the Strang circulant approximate inverse matrix, the diagonal Toeplitz splitting preconditioning matrix, and the skew-diagonal Toeplitz splitting preconditioning matrix based on τ matrices. These three preconditioning matrices are denoted as P_CAI^-1( l ), P_DCS^-1 and P_SDTAS_τ^-1, respectively. All experiments are  performed via Matlab (version R2022b) on a personal computer with a 4.00GHz AMD Ryzen 9 7940H CPU, 16.00 GB of memory, and the Windows 11 operating system.
	
	In all experiments, we searched for the optimal number of interpolation points l in the proposed preconditioning matrices P_TAI^-1( l ). Therefore, for two experiments, we choose the optimal number of interpolation points l to be 8 and 12, respectively. We specify the same grid density in both space and time N= t=h, and for a certain initial point u^( 0 ), we define the relative error of the k-th iteration as follows:
	
    RES=‖ b-Au^( k )‖ _2/‖ b-Au^( 0 )‖ _2

	Let the initial point be a zero vector, the relative error tolerance be RES<10^-7, and the maximum number of iterations be 1000. We solve the problem using GMRES with the above preconditioners.
	
	For the following variable-coefficient tempered fractional diffusion equations (Tempered-FDEs):
	
    ∂ u(x,t)/∂ t=d(x)(_0𝐃_x^β,λ+_x𝐃_1^β,λ)u(x,t)+f(x,t),
    u(0,t)=0, u(1,t)=0, t∈[0,1],
    u(x,0)=0, x∈[0,1].

	The exact solution of the equation (5.1) is u( x,t ) ≃ te^-λ xx^3( 1-x ) ^3. The source term is given as follows:
	
    [ f( x,t ) = e^-λ xx^3( 1-x ) ^3-td( x ) [ e^-λ x∑_m = 0^3( ( -1 ) ^m( [ 3; m;   ]) Γ( 4+m )/Γ( 4+m-β)x^3+m-β).;  . +e^λ( x-2 )∑_j = 0^303^j/j!( ∑_m = 0^3( ( -1 ) ^m( [ 3; m;   ]) Γ( 4+m+j )/Γ( 4+m+j-β)( 1-x ) ^j+3+m-β)) ];                                                                         +2td( x ) ( λ ^β) e^-λ xx^3( 1-x ) ^3;                                                                                                               ]

	
	In this example, we conduct numerical experiments using the following two coefficients d( x ) of different complexity:
	
    d_1( x ) =e^5x/1+x

	
	and 
	
    d_2( x ) =e^3x+0.2/x( 1-x )

	
	The coefficient d_1( x ) and d_2( x ) is continuous over [ 0,1 ] but d_2( x ) has a singularity at point x=0 and x=1. Below, we present the numerical experiments for different coefficients of the equation.
	
	In the first experiment, We choose the coefficients d_1( x ) =e^5x/1+x, λ =1.5, β =1.2 and the remaining parameters γ _2, γ _3 are computed from the linear system (2.2). To illustrate the effectiveness of our proposed preconditioning matrices, we compared P_TAI^-1( l ), P_CAI^-1( l ), P_DCS^-1 and P_SDTAS_τ^-1. The parameters for the first two preconditioning matrices are ł=8, and the parameters for the latter two are β =0.10.
	
	
	we plot the two-dimensional images of the analytical solution and the numerical solution obtained using the preconditioners, as shown in Fig. 5.1:
	
	
	
	
	We use GMRES with P_TAI^-1( l ), P_CAI^-1( l ), P_DCS^-1 and P_SDTAS_τ^-1 preconditioners as well as GMRES without preconditioning to test. The results of the computations at discretization levels N=10^8,10^9,10^10,10^11,10^12 are shown in Table 1. Here, "IT" denotes the average number of iterations per step, and "CPU" denotes the total computation time (s) of the algorithm. To facilitate the comparison of the performance of different preconditioning matrices, we visualized the results from Table 1, as shown in Fig. 5.2. From Table 1 and Fig. 5.2, we can see that the performance of P_TAI^-1( l ) is significantly better than that of P_CAI^-1( l ), P_DCS^-1 and P_SDTAS_τ^-1. The number of iterations for each preconditioning method remains relatively stable as the discretization level increases, but the number of iterations for P_TAI^-1( l ) is notably lower than that for the other three. Similarly, in terms of computation time, the computation time for P_TAI^-1( l ) is much lower than that for the other three, and the growth in computation time for P_TAI^-1( l ) is slower as the discretization level increases.
	
	

		
	
	
	 To further illustrate the effectiveness of the proposed preconditioning matrices, we plot the eigenvalue distributions of the preconditioned coefficient matrices P_TAI^-1( l ), P_CAI^-1( l ), P_DCS^-1, P_SDTAS_τ^-1 and the original coefficient matrix A when N=10^8, as shown in Fig. 5.3. It can be observed that the eigenvalue distribution of the coefficient matrix A without preconditioning is highly dispersed, while the eigenvalue distributions of the preconditioned matrices AP_TAI^-1( l ) are concentrated around 1. Notably, the eigenvalue distribution of AP_TAI^-1( l ) is more concentrated than those of the other preconditioners.
	
	
	
	
	In the second experiment, we chose the coefficient d=d_2( x ), which has a singularity at point x=0 and x=1, and keep other parameters unchanged. Similarly, to illustrate the effectiveness of our proposed preconditioning matrices, we compared P_TAI^-1( l ), P_CAI^-1( l ), P_DCS^-1 and P_SDTAS_τ^-1. The parameters for the first two preconditioning matrices were l=12 and the parameters for the latter two were β =0.15.
	
	We plot the two-dimensional images of the excat solution and the numerical solution P_TAI^-1( l ) obtained using the preconditioners, as shown in Fig. 5.4. We use GMRES with P_TAI^-1( l ), P_CAI^-1( l ), P_DCS^-1 and P_SDTAS_τ^-1 preconditioners as well as GMRES without preconditioning to test. The results of the computations at discretization levels N=10^8,10^9,10^10,10^11,10^12 are shown in Table 2 and visualized in Fig. 5.5. In terms of the number of iterations, all preconditioning methods show relatively stable iteration numbers as the discretization level increases, but the number of iterations for P_TAI^-1( l ) is notably lower than that for the other three. In terms of computation time, the computation time for P_TAI^-1( l ) is slightly higher than that for P_SDTAS_τ^-1 at discretization level  N=10^8,10^9,10^10, but the difference is not significant. As the discretization level increases, the computation time for P_TAI^-1( l ) grows more slowly and is significantly lower than that for the other three preconditioners.
	
	
	
	
	
	
		
	
	
		
	
	
	We also plot the eigenvalue distributions of the preconditioned coefficient matrices P_TAI^-1( l ), P_CAI^-1( l ), P_DCS^-1, P_SDTAS_τ^-1 and A when N=10^8, as shown in Fig. 5.6. It can be observed that the eigenvalue distribution of the coefficient matrix without preconditioning is highly dispersed, while the eigenvalue distributions of the preconditioned matrices are concentrated around 1. Notably, the eigenvalue distribution of P_TAI^-1( l ) is more concentrated than those of the other preconditioners.
	
	
	

§ CONCLUDING REMARKS

		In this paper, the Crank-Nicolson method and the tempered weighted and shifts Grünwald formula are applied to discretize the tempered fractional diffusion equations. We then get the coefficient matrix of the discretized system having the structure of I + DT, where I is the identity matrix, D is a diagonal matrix, and T is a symmetric positive definite (SPD) Toeplitz matrix. Given the SPD property of the coefficient matrix, we approximate it using τ matrix and construct a novel preconditioner using the row-by-row approximate inverse idea proposed by Pan and NG <cit.>. The τ matrix based approximate inverse preconditioner can be efficiently computed using the discrete sine transform(DST). In spectral analysis, the eigenvalues of the preconditioned coefficient matrix are clustered around 1, ensuring fast convergence of Krylov subspace methods with the new preconditioner. Finally, numerical experiments demonstrate the effectiveness of the proposed preconditioner.
	
	

§ ACKNOWLEDGEMENT

		The authors would like to thank the anonymous referees for their constructive comments and suggestions for improving the manuscript.
	
	99
		
		Baeumer2010 B. Baeumer, M.M. Meerschaert, Tempered stable Levy motion and transient super-diffusion, J. Comput. Appl. Math., 233 (2010), pp. 2438–2448.
		
		Magin2004 R.L. Magin, Fractional calculus in bioengineering, Crit. Rev. Biomed. Eng., 32 (2004), pp. 1–104.
		
		Meerschaert2008 M.M. Meerschaert, Y. Zhang, B. Baeumer, Tempered anomalous diffusion in heterogeneous systems, Geophys. Res. Lett., 35 (2008), L17403.
		
		Zhang2016H. Zhang, F. Liu, I. Turner, S. Chen, The numerical simulation of the tempered fractional Black-Scholes equation for European double barrier option, Appl. Math. Model. 40 (2016) 5819–5834.
		
		Qu2017 W. Qu, Y. Liang, Stability and convergence of the Crank-Nicolson scheme for a class of variable-coefficient tempered fractional diffusion equations, Adv. Differ. Equ., 2017 (2017), 108.
		
		Tang2023 S.-P. Tang, Y.-M. Huang, A matrix splitting preconditioning method for solving the discretized tempered fractional diffusion equations, Numer. Algorithms, 92 (2023), pp. 1311–1333.
		 
		 Pan2014 J. Pan, R. Ke, M.K. Ng, H.-W. Sun, Preconditioning Techniques for Diagonal-times-Toeplitz Matrices in Fractional Diffusion Equations, SIAM J. Sci. Comput., 36 (2014), A2698–A2719.
		 
		 Huang2021X. Huang, H.-W. Sun, A preconditioner based on sine transform for two-dimensional semi-linear Riesz space fractional diffusion equations in convex domains, Appl. Numer. Math. 169 (2021) 289–302.
		 
		 Qin2023H. Qin, H. Pang, H. Sun, Sine transform based preconditioning techniques for space fractional diffusion equations, Numer. Linear Algebra Appl. 30 (2023) e2474.
		 
		 Sun2023L.-Y. Sun, S.-L. Lei, H.-W. Sun, J.-L. Zhang, An α-robust fast algorithm for distributed-order time–space fractional diffusion equation with weakly singular solution, Math. Comput. Simul. 207 (2023) 437–452.
		 
		 Zeng2022 M.-L. Zeng, J.-F. Yang, G.-F. Zhang, On τ matrix-based approximate inverse preconditioning technique for diagonal-plus-Toeplitz linear systems from spatial fractional diffusion equations, J. Comput. Appl. Math., 407 (2022), 114088.
		 
		 
		 
		 Zhang2023 C.-H. Zhang, J.-W. Yu, X. Wang, A fast second-order scheme for nonlinear Riesz space-fractional diffusion equations, Numer. Algorithms, 92 (2023), pp. 1813–1836.
		 
		 Liu2019J. Liu, H. Fu, H. Wang, X. Chai, A preconditioned fast quadratic spline collocation method for two-sided space-fractional partial differential equations, J. Comput. Appl. Math. 360 (2019) 138–156.
		 
		 Lu2021X. Lu, Z.-W. Fang, H.-W. Sun, Splitting preconditioning based on sine transform for time-dependent Riesz space fractional diffusion equations, J. Appl. Math. Comput. 66 (2021) 673–700.
		 
		 Qin2023H. Qin, H. Pang, H. Sun, Sine transform based preconditioning techniques for space fractional diffusion equations, Numer. Linear Algebra Appl. 30 (2023) e2474.
		 
		 Shao2022X.-H. Shao, C.-B. Kang, A preconditioner based on sine transform for space fractional diffusion equations, Appl. Numer. Math. 178 (2022) 248–261.
		 
		 Wang2015 W. Wang, X. Chen, D. Ding, S.-L. Lei, Circulant preconditioning technique for barrier options pricing under fractional diffusion models, Int. J. Comput. Math., 92 (2015), pp. 2596–2614.
		 
		 Li2016 C. Li, W. Deng, High order schemes for the tempered fractional diffusion equations, Adv. Comput. Math., 42 (2016), pp. 543–572.
		 
		 Wang2010 H. Wang, K. Wang, T. Sircar, A direct O(N log^2 N) finite difference method for fractional diffusion equations, J. Comput. Phys., 229 (2010), pp. 8095–8104.
		 
		Bini1990 D. Bini, F. Benedetto, A new preconditioner for the parallel solution of positive definite Toeplitz systems, in: Proc. Second Annu. ACM Symp. Parallel Algorithms Archit., Association for Computing Machinery, New York, NY, USA, 1990: pp. 220–223.
		
		Lin2023 X. Lin, X. Huang, M.K. Ng, H.-W. Sun, A τ-preconditioner for a non-symmetric linear system arising from multi-dimensional Riemann-Liouville fractional diffusion equation, Numer. Algorithms 92 (2023) 795–813.
	
		
		Chan1996 R.H. Chan, M.K. Ng, Conjugate Gradient Methods for Toeplitz Systems, SIAM Rev., 38 (1996), pp. 427–482.
		
		
		
		Huckle1994 T. Huckle, Iterative Methods for Toeplitz-like Matrices, 1994.
		
		Liu2016 H. Zhang, F. Liu, I. Turner, S. Chen, The numerical simulation of the tempered fractional Black-Scholes equation for European double barrier option, Appl. Math. Model., 40 (2016), pp. 5819–5835.
		
		Meerschaert2012 Y. Zhang, M.M. Meerschaert, A.I. Packman, Linking fluvial bed sediment transport across scales, Geophys. Res. Lett., 39 (2012), L20404.
		
		Chou2017 L.-K. Chou, S.-L. Lei, Fast ADI method for high dimensional fractional diffusion equations in conservative form with preconditioned strategy, Comput. Math. Appl. 73 (2017) 385–403.
		
		Du2019 N. Du, H.-W. Sun, H. Wang, A preconditioned fast finite difference scheme for space-fractional diffusion equations in convex domains, Comput. Appl. Math. 38 (2019) 14.
		
		Gan2023 D. Gan, G.-F. Zhang, Efficient ADI schemes and preconditioning for a class of high-dimensional spatial fractional diffusion equations with variable diffusion coefficients, J. Comput. Appl. Math. 423 (2023) 114938.
		
		Zhang2023D. Gan, G.-F. Zhang, Z.-Z. Liang, Preconditioning techniques of all-at-once systems for multi-term time-fractional diffusion equations, Numer. Algorithms (2023).
		
		Serra1999 S. Serra, Superlinear PCG methods for symmetric Toeplitz systems, Math Comp (1999).
		
		Stro2002 T. Strohmer, Four short stories about Toeplitz matrix calculations, Linear Algebra Its Appl. 343–344 (2002) 321–344.
	


 Paper Title: Predicting nonlinear-flow regions in highly heterogeneous porous media using adaptive constitutive laws and neural networks.





















 



Dodowhile







./fig/








margin=1.5in,includefoot,footskip=30pt


5pt 










§ 
0.5em#1[]


 §.§ 
[runin]0.5em#1[.]


  §.§.§ 
[runin]0.5em#1[.]


  
[runin]0.5em#1[.—]
*

§ 
0pt0.80.6
*

 §.§ 
0pt0.60.4
*

  §.§.§ 
0pt0.40.4
*

  
0pt0.20.2





equationsection


itemsep=-0.2em,topsep=0.3em

itemsep=-0.2em,topsep=0.3em





mystyle 
    0.3em 
    0.3em 
     
     
     
    . 
    .5em 
    #1 #2 (#3) 

plain
thmTheorem[section]
res[thm]Result
rem[thm]Remark
lem[thm]Lemma
cor[thm]Corollary
prop[thm]Proposition
hyp[thm]Hypothesis
assu[thm]Assumption
pb[thm]Problem
defn[thm]Definition
notn[thm]Notation
que[thm]Question
conv[thm]Convention
ex[thm]Example

namedthmstyle 
         
	 
	 
	 
	 
	 
	  
        
namedthmstyle

*genericthm
namedthm[1]

  
  
#1#2currentlabel#2
   






showonlyrefs
inter-unit-product=





















 
 
  |.   
  
⇀ 



 



 


∇
∇
÷∇·
∇·
Δ
 









 
 
 






R 
N 
𝒫 
M 
C 
ℬ 
S 


ε


ḍ
∫_^d
∫_^d×^d


d
D
∂




0
x
y
z
u
v
w
m
a
b
e
d
X
Y
Z
D
H
J
g
f
n
ν
τ
F
K
G
I
φ
Λ
Θ
λ
ϕ
L
V
B
U
D
X
∂


ℱ 
ℰ 
𝕀id 
ℋ
ℌ
𝒟 
G 
𝕂
𝔻
𝕂^-1




C[1]>m#1
L[1]>m#1





∇
  |.   













compat=1.18
matrix




shapes.geometric, arrows

rounded rect = [rectangle, 
rounded corners, 
minimum width=1cm, 
minimum height=1cm,
text centered, 
text width=2.5cm, 
draw=black, 
fill=red!30]

trap = [trapezium, 
trapezium stretches=false,
trapezium left angle=70, 
trapezium right angle=110, 
minimum width=1cm, 
minimum height=1cm, 
text width=2.5cm, 
text centered, 
draw=black, 
fill=blue!30]

rect = [rectangle, 
minimum width=1cm, 
minimum height=1cm, 
text centered, 
text width=2.5cm, 
draw=black, 
fill=orange!30]

arrow = [thick, ->, >=stealth]













Predicting nonlinear-flow regions in highly heterogeneous porous media using adaptive constitutive laws
 and neural networks
    Chiara Giovannini^1 ^2 ^3 Alessio Fumagalli^1 Francesco Saverio Patacchini^2
 August 5, 2024
    ^1 Department of Mathematics, Politecnico di Milano, p.za Leonardo da Vinci 32, 20133 Milano, Italy

^2 IFP Energies nouvelles, 1 et 4 avenue de Bois-Préau, 92852 Rueil-Malmaison, France

^3 Department of Engineering Mechanics, KTH Royal Institute of Technology, Brinellvägen 8, 114 28 Stockholm, Sweden

====================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
In a porous medium featuring heterogeneous permeabilities, a wide range of fluid velocities may be recorded, so that significant inertial and frictional effects may arise in high-speed regions. In such parts, the link between pressure gradient and velocity is typically made via Darcy's law, which may fail to account for these effects; instead, the Darcy–Forchheimer law, which introduces a nonlinear term, may be more adequate. Applying the Darcy–Forchheimer law globally in the domain is very costly numerically and, rather, should only be done where strictly necessary. The question of finding a prori the subdomain where to restrict the use of the Darcy–Forchheimer law was recently answered in <cit.> by using an adaptive model: given a threshold on the flow’s velocity, the model locally selects the more appropriate law as it is being solved. At the end of the resolution, each mesh cell is flagged as being in the Darcy or Darcy–Forchheimer subdomain. Still, this model is nonlinear itself and thus relatively expensive to run. In this paper, to accelerate the subdivision of the domain into low- and high-speed regions, we instead exploit the adaptive model from <cit.> to generate partitioning data given an array of different input parameters, such as boundary conditions and inertial coefficients, and then train neural networks on these data classifying each mesh cell as Darcy or not. Two test cases are studied to illustrate the results, where cost functions, parity plots, precision-recall plots and receiver operating characteristic curves are analyzed.


Keywords: porous medium, heterogeneous medium, adaptive constitutive law, neural network













§ INTRODUCTION



When modeling fluid flow in highly heterogeneous porous media, characterized by irregularly distributed permeability profiles, such as they may arise  when studying underground storage and management of CO2, geothermal energy, nuclear waste or freshwater <cit.>, Darcy's law, which is widespread in this context, may be challenged as noted in many studies <cit.>. Indeed, Darcy's law being a linear relation between fluid velocity (or, more precisely, seepage flux) and pressure gradient, it ignores inertial and frictional effects possibly arising in highly permeable areas, such as macropores, fractures or conduits, where high speeds may develop. Theoretical studies <cit.> provide admissible corrections in this particular setting, which consist in adding a nonlinear term to Darcy's law. The choice we make here is the so-called Darcy–Forchheimer law (cf. later for a generalization of it), which has the form

    - p = ( 1 + c_F√(k)/μ) ν/k ,

where  is the seepage flux, μ and ν are, respectively, the fluid's dynamic and kinematic viscosities, k is the medium's permeability and c_F is a dimensionless empirical inertial term which we refer to as the Forchheimer coefficient. The notation · stands for the Euclidean norm. Note that, for simplicity, we decide here to focus on scalar permeabilities, although tensor generalizations of this law are available <cit.>.

The Darcy–Forchheimer law is more accurate than Darcy's law thanks to its nonlinear term, but it is also more costly to solve numerically because of this same nonlinearity. Therefore, it may not be reasonable resource-wise to solve it everywhere in the domain, but, rather, it may be better to select appropriately a partition of the domain into two subdomains, where in one subdomain Darcy's law is still solved thanks to a slow flow and in the other the Darcy–Forchheimer law is solved because of a fast regime that does not allow to neglect inertial and frictional terms. To implement this, we choose to determine the partition a priori and then, once it is known, to use domain-decomposition techniques <cit.> and thus obtain a numerical strategy which is faster than a global Darcy–Forchheimer approach but still more precise than the global Darcy counterpart. A-posteriori strategies are also envisageable <cit.> but not investigated here. In this paper, we focus on the first step of this methodology, that is, the a-priori calculation of the domain partition, the domain-decomposition step being left for future examination.

One possible way of achieving a-priori knowledge of the Darcy and Darcy–Forchheimer regions may be found in the adaptive model presented in <cit.> and further tested numerically in <cit.>. There, the authors study a constitutive law that, given a fixed threshold on the magnitude of the seepage flux, automatically and iteratively selects Darcy's law or the Darcy–Forchheimer law: if the local flux has smaller magnitude than the threshold, Darcy's law is selected, otherwise the Darcy–Forchheimer law is. As a result of the simulation, one obtains a classification of each mesh cell as belonging to either the Darcy or the Darcy–Forchheimer subdomain. However, although the authors show that such a model is well posed, it is discontinuous at the transitions between the laws, making it rather unsuitable for numerical resolution via a classical, explicit fixed-point algorithm. Instead, in <cit.>, they propose a smoothed version of this adaptive model, where, thanks to a convolution, the law is regularized around the transitions. As a result, this regularized model yields approximations of the sought-after partition but is better suited for numerical purposes. The regularized model is, nevertheless, nonlinear and thus still expensive to run.

To circumvent this, we propose here to combine the regularized model, already available from <cit.>, with machine-learning techniques to provide a fast way of obtaining the partition. The regularized model is used to generate data with varying parameters, such as the Forchheimer coefficient and the boundary conditions and other physical parameters, and then neural networks are trained on these data. The networks output classifications of the mesh cells, by deciding for each cell whether it is Darcy or not. The validation of the networks is carried out by analyzing cost functions, parity plots, precision-recall plots and receiver operating characteristic curves. The codes for the regularized model and the neural networks are written in Python and available at <cit.>, while the underlying discretization and equation resolution are performed using the open source Python simulation tool PorePy <cit.>. 

We start by giving the details of the physical framework and equations at stake and by deriving the threshold on the magnitude of the seepage flux (Section <ref>); we then describe our overall machine-learning methodology and the two test cases we apply it to, the first coming from <cit.> and applicable to landfill management and the second extracted from the SPE10 benchmark reservoir scenario <cit.> (Section <ref>); and, finally, we discuss the results on these test cases (Section <ref>). In Appendix <ref>, we recall and discuss the basics of deep learning which we use in our methodology.










§ PHYSICAL FRAMEWORK



We position ourselves within the modeling context established by <cit.>; to ensure comprehensive understanding, we provide a summary here. To begin, let us elucidate some notations. We denote by Ω⊂^d, d∈, the porous medium and assume it is open, bounded, and possesses a Lipschitz boundary ∂Ω. The outward normal unit vector of ∂Ω is written n. Additionally, we use R_+ = [0, ∞) and N_0 = N∪{0}.

The unknowns in the problems discussed throughout this paper are the fluid’s pressure p: Ω→R and the seepage flux : Ω→R^d. The seepage flux is defined by = ρϕV, where ϕ is the medium’s porosity, and ρ and V are the fluid’s density and velocity, respectively. Furthermore, we use μ, ν and k to represent the fluid’s dynamic viscosity and kinematic viscosity and the medium’s scalar permeability, respectively. We assume that ρ, μ, ν, k Ω→ (0, ∞) and ϕΩ→ (0,1) are space-dependent knowns of the problem.



 §.§ Underlying model equations



In the porous material under consideration, we wish to solve the time-independent problem given by the conservations of mass and momentum. The former conservation is expressed as

    = q   in Ω;

here q Ω→R is a known fluid mass source. For the latter conservation, we choose what is often called the generalized Forchheimer (GF) law <cit.>:

    -  p + f = ( 1 + c_F(√(k)/μ)^m ^m ) ν/k ,

where fΩ→^d is a known vector of external body forces, such as gravity, c_FΩ→ (0, ∞) is the Forchheimer coefficient (also referred to as the Ergün number in the literature <cit.>, containing information on the porosity and tortuosity of the medium <cit.>) and m ∈[1,∞) may be called the Forchheimer exponent. When m=1, we recover the Darcy–Forchheimer (DF) law, as already discussed; qualitatively, Darcy's linear law is retrieved by imposing m=0 or c_F≡ 0. Here, · stands for the Euclidean norm in ^d. The conservation of momentum is also referred to as the constitutive law or seepage law.

With this pair of equations, we associate the following boundary conditions:

    ·n = u_0    on Σ_v,
    
            p = p_0    on Σ_p,

where Σ_v,Σ_p⊂∂Ω are relatively open in ∂Ω (i.e., Σ_v and Σ_p are each the intersection of an open subset of ^d with ∂Ω) and satisfy ∂Ω = Σ_v∪Σ_p and Σ_v∩Σ_p = ∅. Here, u_0Σ_v→ and p_0Σ_p→ are functions setting the conditions on the boundary for the flux and pressure.

Equations (<ref>), (<ref>) and (<ref>) form our model of interest. The underlying theoretical framework, including the appropriate functional spaces and weak formulations, are described in detail in <cit.>.



 §.§ Adaptive model


We can discern two distinct components within the GF law in (<ref>): a linear component given by (ν/k) and a nonlinear one represented by c_F (k/μ)^m u^m (ν/k). When the velocity, or seepage flux, is relatively low, addressing the nonlinear part becomes unnecessary and may be safely disregarded; this amounts to the ubiquitous Darcy law. Conversely, when the velocity is relatively high, the nonlinear part should be taken into account if one wants to study the inertial and frictional aspects of the flow; in this case, m≥ 1 and, as already mentioned, the particularly relevant case m=1 yields the well known DF law. 

In practice, one could always treat the nonlinear component since it adds precision to Darcy's law in any range of velocities. However, because its benefits are extremely limited in low-speed regions, i.e., in most part, of the medium, the gain in precision in these parts of the domain does not outweigh the numerical cost brought by the nonlinearity. It is thus reasonable only to solve the nonlinear model where strictly necessary, that is, where the flux magnitude is above a certain threshold and then keep Darcy's model where the magnitude is below this same threshold. The derivation of this a-priori threshold is given in Section <ref> below.

Denoting by u̅ the threshold, the dichotomy arising from this approach can be formulated as

    - p + f = 
        ν/k    if < u,
    ( 1 + c_F(√(k)/μ)^m ^m ) ν/k    if > u.

We refer to this law as the adaptive law, which, combined with (<ref>) and (<ref>) constitutes the adaptive model. The terminology stems from the fact that this model adapts naturally the constitutive law considered depending on the local Euclidean norm of the flux magnitude. What happens at the transition set { = u̅} is a delicate matter which we do not delve into here. Indeed, it requires the introduction of a multivalued reformulation of the adaptive law which does not enter the qualitative scope of the present discussion; we refer the reader to <cit.> for the complete details of the theory. The well-posedness of such an adaptive model is obtained by a straightforward generalization of <cit.> to space-dependent physical parameters μ, ν, k and c_F, which we further assume to be bounded below and above by strictly positive constants in Ω.

Note that the adaptive law (<ref>) can be rewritten as

    - p + f = ( 1_{ < u̅} + c_F(√(k)/μ)^m ^m 1_{ > u̅}) ν/k ,

where 1_A stands for the indicator function of set A and { < u̅} (respectively, { > u̅}) is a short-hand notation for {∈Ω() < u̅} (respectively, {∈Ω() > u̅}).



 §.§ Flux threshold



We now explain how to choose a priori the flux threshold, and then use this opportunity to define formally the Darcy and GF subdomains and introduce the Reynolds and Forchheimer numbers, which we shall use later. These notions are discussed in details for even more general seepage laws in <cit.>.



  §.§.§ Threshold derivation


Given a flux , the forces - p_D + f and - p_GF + f resulting from the Darcy and GF laws are, respectively, deducted from the following constitutive laws:

    - p_D + f = ν/k ,
    
        - p_GF + f = ( 1 + c_F(√(k)/μ)^m ^m ) ν/k .

Subtracting the equations in (<ref>) and using the second one as reference, we arrive at the following definition of local error d[] Ω→ [0,1), which can be found in <cit.> for the case m=1:

    d[] = c_F(√(k)/μ)^m ^m/1 + c_F(√(k)/μ)^m ^m.


Note that there exists a function ΔΩ×_+ → [0,1) such that, for all ∈Ω, there holds

    d[]() = Δ(, ()),

and Δ(, ·) is an increasing bijection. Then, let δ∈[0,1) and deduce that there is a unique wΩ→_+ so that Δ(, w()) = δ for all ∈Ω. We can then define the flux threshold u̅:

    u̅ = inf_Ω w ∈_+.

Then, using (<ref>) and (<ref>) and inverting Δ, one finds

    w = (δ/1 - δ)^1/mμ/c_F^1/m√(k),

which yields

    u̅ = (δ/1 - δ)^1/minf_Ωμ/c_F^1/m√(k).

In particular, when m = 1, i.e., the reference law is DF, we get

    u̅ = δ/1 - δinf_Ωμ/c_F√(k).




  §.§.§ Subdomains

Given again a flux , an error tolerance δ∈[0, 1) and the corresponding flux threshold u̅, we define the Darcy and GF subdomains as

    Ω_D[] = {∈Ω() < u̅}  and  Ω_GF[] = {∈Ω() > u̅},

which we may respectively refer to as the slow and fast subdomains.

If ∈Ω_D[], we know that solving at  Darcy's law (cf. first equation in (<ref>)) does not generate a local error greater than δ relative to solving at  the GF law (cf. second equation). Indeed, suppose ∈Ω_D[]; then, recalling (<ref>), (<ref>) and (<ref>), we get

    () < u̅≤ w(),

so that 

    d[]() = Δ(, ()) < Δ(, w()) = δ.

Hence, by solving Darcy's law in Ω_δ[] and the GF law in Ω_GF[], we make sure that the local error stays everywhere below δ.



  §.§.§ Reynolds and Forchheimer numbers

Given a flux , we define the Reynolds number Re[]Ω→_+ by

    Re[] = √(k)/μ.

Thanks to this notation, together with the notions of subdomains given in the previous section, given an error tolerance, we may rewrite the adaptive law (<ref>) as

    - p + f = ( 1_Ω_D[] + c_FRe[]^m 1_Ω_GF[]) ν/k .


We also define the mth Forchheimer number Fo_m[]Ω→_+ by

    Fo_m[] = c_F^1/m√(k)/μ = c_F^1/mRe[].

The dimensionless number Fo[]^m quantifies the weight of the nonlinear term in the GF law relative to 1, i.e., relative to the linear term. The first Forchheimer number, with m=1, resulting from considering the DF law as reference, is simply referred to as the Forchheimer number in the literature <cit.> and denoted by Fo[]. Given an error tolerance δ∈[0,1), one can further rewrite (<ref>) as

    - p + f = ( 1_Ω_D[] + Fo_m[]^m 1_Ω_GF[]) ν/k .

Moreover, the local error in (<ref>) can be reformulated as

    d[] = Fo_m[]^m/1 + Fo_m[]^m.

Also note that if at ∈Ω we have Fo_m[]()>(δ/(1-δ))^1/m, then ∈Ω_DF[]; thus, the value (δ/(1-δ))^1/m may be seen as a critical Forchheimer number above which nonlinear effects are surely nonnegligeable.




 §.§ Regularized model


The adaptive law, found in its different forms in (<ref>), (<ref>), (<ref>) and (<ref>), is discontinuous in the flux variable. Indeed, across the transition zone {∈Ω() = u̅}, the law jumps from Darcy's to the GF law. This discontinuity is not well suited for numerical resolution, especially when using a fixed-point algorithm. To circumvent this, the authors in <cit.> propose to use the variational, or energetic, formulation of the adaptive model to regularize the law via a convolution of the underlying energy functional. For completeness, we describe the main idea below, although we refer the reader to <cit.> for the details and the rigorous framework.

As just mentioned, the regularization of the adaptive law is based on an energetic consideration. Indeed, it is shown there that a pair (, p) is solution to the adaptive model (<ref>)-(<ref>)-(<ref>) if and only if it is a saddle point of the energy functional ℰ defined by

    ℰ(φ, ψ) = ∫_Ωf·φ - ∫_Ω q ψ + ∫_Σ_v u_0 ψ - ∫_Ωψ·φ - 𝒟(φ),

where

    𝒟(φ) = 1/2∫_Ω( 1_Ω_D[φ] + 2/m+2Fo_m[φ]^m 1_Ω_GF[φ]) ν/k φ^2.

The term 𝒟 is called the dissipation and is minimized on a certain restricted function space by a certain component of the solution  (cf. <cit.>). The discontinuity of the adaptive law is reflected in the discontinuity of the integrand of 𝒟 with respect to the variable φ. In fact, 𝒟 is not differentiable and merely has a subdifferential, which is equal to the right-hand side of the adaptive law in (<ref>). To make the dissipation smooth, its integrand is convolved with a Gaussian kernel G_ε of mean 0 and standard deviation some fixed ε>0. The convolution is applied on the space of flux magnitudes and yields the regularized dissipation 𝒟_ε, defined as

    𝒟_ε(φ) = 1/2∫_Ω G_ε * [ ( 1_Ω_D[φ] + 2/m+2Fo_m[φ]^m 1_Ω_GF[φ]) ν/k φ^2 ],

and the corresponding energy ℰ_ε, given by

    ℰ_ε(φ, ψ) = ∫_Ωf·φ - ∫_Ω q ψ + ∫_Σ_v u_0 ψ - ∫_Ωψ·φ - 𝒟_ε(φ).

The regularized model then consists in finding a pair (_ε, p_ε) which is a saddle point of ℰ_ε. Equivalently, after differentiation of the regularized dissipation, it consists in solving the model given by (<ref>) and (<ref>) together with the following law:

    - p_ε + f = G_ε' * [ ( 1_Ω_D[_ε] + 2/m+2Fo_m[_ε]^m 1_Ω_GF[_ε]) ν/k _ε^2 ] _ε,

where G_ε' is the derivative of the Gaussian kernel G_ε. The smooth model (<ref>)-(<ref>)-(<ref>) is well posed thanks to <cit.> and we decide to use Raviart–Thomas finite elements and an explicit fixed-point algorithm to generate data on which to train our neural networks (cf. Sections <ref> and <ref>).

Of course, when solving the regularized model as opposed to the adaptive one, we obtain a different partition into slow and fast subdomains, so that Ω_D[] differs from Ω_D[_ε] and so does Ω_GF[] from Ω_GF[_ε]. Nevertheless, thanks to the weak convergence result <cit.>, we expect that, although this has not yet been proved, the regularized subdomains are not far from those stemming from the adaptive model for a reasonably small choice of ε, such as ε = 0.1, which is the value we take for the numerical experiments below.









§ METHODOLOGY AND TEST CASES


In our pursuit of an a-priori partitioning of the domain into Darcy and Generalized Forchheimer (GF) zones, we leverage neural networks to predict the categorization of cells within a given mesh. Specifically, we employ a binary classification scheme, where 0 (or positive) denotes a GF cell and 1 (or negative) a Darcy cell. 



 §.§ Overview of methodology

For details on the fundamental notions of deep learning used to set up our neural networks, tune hyperparameters and validate performance, we refer the reader to Appendix  <ref>. To summarize, for each test case (cf. Section <ref>), having chosen a mesh and a set of input features, such as boundary conditions, our methodology is based on generating a dataset using the codebase <cit.>, which uses the regularized model described in Section <ref> to obtain a domain partition into Darcy and GF regions, on training a dense feed-forward neural network with two or three hidden layers and cross-entropy as cost function, on using cross-validation and recall to tune the underlying hyperparameters, and on evaluating the train and test results with recall, precision and various performance plots as metrics. Figure <ref> provides a flowchart of this methodology, where the ordering of the above steps is specified, as well as the train-test split ratio, the number of cross-validation folds, and the value of the  parameter from the regularized model.





 §.§ Test cases


We apply the above methodology to two test cases, whose results are given in Section <ref> below. Both are two-dimensional, the former coming from an application in landfill management <cit.>, the second taken from Layer 35 of the SPE10 benchmark reservoir scenario <cit.>. We present here the way the data are generated for these test cases as well as the set of hyperparameters optimized.



  §.§.§ Case 1: landfill management 


Landfills serve as critical repositories for waste disposal, necessitating efficient management strategies to mitigate environmental impact. Within this context, understanding the dynamics of rainwater infiltration is pivotal. In this study, we focus on the case where the influx is at the upper boundary acting like rainwater permeating soil. We impose zero-flux conditions on the left and right boundaries, and hydrostatic pressure at the bottom. In Figure <ref>, the vertical axis is aligned with the gravitational force, and we notice the presence of highly permeable channels, or macropores, of which we study two possible configurations: one has two channels traversing the whole domain vertically, exactly as in <cit.>, and the other features a network of seven intersecting channels.





  
Data generation
In this configuration, our dataset comprises s s_train + s_test = 2352 examples, each characterized by a 12-element input vector and a corresponding 2500-element output vector, n_0=12 being the number of input features and k=2500 the mesh size (cf. Figure <ref> for the notation). The input vector includes the influx at the top boundary (u_0, in □; cf. Section <ref>), the Forchheimer coefficient (c_F, dimensionless; cf. Section <ref>), the Forchheimer exponent (m, dimensionless; cf. Section <ref>), the error tolerance (δ, dimensionless; cf. Section <ref>), and the number of channels in the domain (2 or 7) and their associated porosity values. To translate porosities into permeabilities, we use the Kozeny–Carman formula, given by

    K = K_refϕ^3(1-ϕ_ref)^2/ϕ_ref^3(1-ϕ)^2,

where ϕ_ref and K_ref are respectively the reference porosity and reference permeability, which are calculated based on the grain-size distribution WT1 from the nuclear power plant Würgassen <cit.>. The calculation of these parameters is detailed in <cit.> and yields ϕ_ref = 0.35 and K_ref = 1.01e-9□.

To generate diverse values for the input features, we employ specific probability distributions. For the influx, we use a normal distribution with mean, standard deviation and cut-off interval given by the following values:

    μ_u_0 = 0.0105,   σ_u_0 = 0.0035,   I_u_0 = [0.001, 0.1].

Similarly, the Forchheimer coefficient follows a normal distribution with

    μ_c_F = 0.55,   σ_c_F = 0.4,   I_c_F = [0.1, 0.9].

These ranges are inspired by values in <cit.>. The Forchheimer exponent is too generated from a normal distribution with

    μ_m = 1,   σ_m = 1.5,   I_m = [1, 4].

We restrict the interval to m ≥ 1 which corresponds to the Darcy–Forchheimer seepage law (cf. Section <ref>). The error tolerance δ is selected from a uniform distribution within the interval [0.01, 0.25].

Seven values are randomly selected for each parameter. These values are then clipped, resulting in six distinct c_F, seven δ, seven u_0, and four m. Subsequently, we compute the product with every possible combination of these values. The dataset is further diversified by using either 2 or 7 channels, effectively doubling its size, yielding the 2532 examples. For the porosity values in each channel, we generate a uniformly distributed vector of 7× 1176 values ranging from 0.9 to 0.99; from this vector, we select either the first 2× 1176 values for the two-channel case or the entire vector for the seven-channel case.

By adopting these distributions, our aim is to align the dataset with the patterns and insights highlighted in <cit.> and ensure a meaningful and contextually relevant dataset for experimentation. Using these values, we input data into the code from <cit.>, generating a 2500-cell output vector for each input vector, i.e., an output of size k× s, representing the Darcy and GF binary mesh values.



  
Hyperparameters
Using cross-validation with 5 folds based on recall as metric, we tune the learning rate in the range [0.001, 0.1] and the number of nodes per hidden layer across the two networks given in Table <ref>.





  §.§.§ Case 2: Layer 35 of SPE10


Layer 35 of the SPE10 model plays a key role in reservoir simulation due to the intricate interplay between injection and production dynamics, and due to the highly permeable paths it features. In the context of our investigation, this horizontal layer is characterized by a single injection well through which water is pumped at the center of the domain, along with four production wells at the corners (cf. Figure <ref>). From the known depth of Layer 35, namely, 21.336 meters, we make the assumption of hydrostatic pressure on the entire boundary for this particular scenario, although no-flux boundary conditions could also be considered.





  
Data generation
Here, our dataset encompasses s=3888 examples, each characterized by a 4-element input vector and a corresponding 13200-element output vector, where n_0=4 is the number of input features and k=13200 is the mesh size (cf. Figure <ref>). Similar to the preceding case, the input vector incorporates the Forchheimer coefficient, the Forchheimer exponent and the error tolerance. Additionally, the parameter Q (measured in ) is introduced, corresponding to an external source (cf. Section <ref>) and used to assign values to distinct injection and production wells within the mesh.
Common values are retained for the shared parameters across cases; for Q, a normal distribution is employed, characterized by

    μ_Q = 100,   σ_Q = 30,    I_Q = [10, 300].


Nine values are randomly selected for each parameter. These values are then clipped, resulting in eight distinct c_F, nine δ, nine Q, and six m. Subsequently, we compute every possible combination of these values, reaching the 3888 examples.



  
Hyperparameters
Using again cross-validation with 5 folds and recall as performance metric, we tune the learning rate in the range [0.001, 0.1] and the number of hidden layers and nodes across the two networks given in Table <ref>.



§ RESULTS



We discuss now the results for the two test cases described in Section <ref>. 



 §.§ Case 1: landfill management 





  §.§.§ Hyperparameter tuning


Among the networks listed in Table <ref>, we highlight in yellow the one with highest recall, which is considered our main metric of interest (cf. Appendix <ref>). In particular, the table illustrates the argument from Appendix <ref>, namely, that the error rate has little discriminative power in our context: with the exception of networks employing a learning rate of 0.1, the variations in error rate between different networks are almost negligible. 



The percentage difference between the training cost and the validation cost (%cost) is defined as

    %cost = | c_val - c_train|/c_train,

where c_val and c_train are the costs of the validation and training, respectively. This metric measures the potential for overfitting and underfitting, and, despite already using cross-validation to mitigate overfitting, it provides an additional layer of control. Elevated values of %cost suggest overfitting or underfitting tendencies. For instance, in the first row of the second part of Table <ref>, a notable disparity in %cost coincides with notably low precision and recall values, indicative of suboptimal model performance maybe attributable to overfitting or underfitting.

Another metric in Table  <ref> is the variance on the recall. It is calculated by measuring the variance among the five recall values obtained from each fold. This analysis ensures that a high average recall value is not driven by an outlier, but reflects consistent performance across all folds. By examining the variance, we test whether the recall values are consistently high, indicating robust model performance, or whether they vary significantly, suggesting sensitivity to specific subsets of data. 
To compare these variances to the average recall values, we actually look at the standard deviation, which in the highlighted case is 0.08609. This value, compared to the average 0.8444, seems reasonable and further validates the reliability and consistency of the performance of our highlighted neural network.


From Table <ref>, it is evident that when the learning rate is set to 0.1, we encounter notably low precision and recall values. Examination of the cost function in this case yields Figure <ref> across all five folds for the two combinations of hidden layers and nodes. The cost's behavior appears to be nonconvergent, oscillating. 



During cross-validation, we use as threshold 10^-6 for the early-stopping criterion mentioned in Appendix <ref>.





































    
    

    







  §.§.§ Exploring selected network architectures




  
Training
Using the highlighted network in Table <ref>, the training is conducted on the entire training set. Figure <ref> displays the trend of the cost function during training and testing with a different test set (generated using the feature distributions in (<ref>)). Notably, the expected behavior is observed, where the function steadily decreases as the number of iterations increases. Eventually, the function stabilizes, indicating convergence. The similar behavior of the cost function in both training and testing suggests that overfitting is not occurring. In this instance, a threshold of 10^-8 is used for the early-stopping criterion.





  
Testing
In Table <ref>, the network's performance on the test set is presented. We specifically opt for a test set comprising 5% of the total dataset, for a total of 117 samples. Notably, the recall value demonstrates a valuable performance, reaching 92%. Conversely, the precision metric appears lower at 80%. However, given our emphasis on recall over precision, this performance seems satisfactory.



Figure <ref> illustrates that by adjusting the classification threshold, such as to 0.75, a higher recall is obtained. Nevertheless, this adjustment inevitably entails a trade-off with precision; the decision regarding the threshold is thus left to the user. The ROC curve in Figure <ref> showcases an exemplary characteristic, summarized by the notably high AUC value of 0.99662. We recall that an AUC of 1 signifies perfect classification, wherein all positive instances supersede negative instances in score ranking across all thresholds. Additionally, Figure <ref> presents the confusion matrix at a classification threshold of 0.5. The majority of cells are correctly classified. Notably, the number of false negatives (1224) is lower than that of false positives (3720), which aligns with the desired outcome. The parity plot shown in Figure <ref> shows that the test examples closely align with the bisector within the first quadrant. This indicates a near equivalence between the predicted GF cells and the actual ones. While this does not explicitly indicate the accuracy of the predictions in terms of position in the mesh, it nonetheless provides a valuable insight.



  
Testing on different test set
To conduct further verification, we test the network using a dataset stemming from different parameter distributions. 
We use normal distributions for the influx, the Forchheimer coefficient and the Forchheimer exponent with means, standard deviations and cut-off intervals given by:

    μ_u_0 = 0.02,   σ_u_0 = 0.0035,   I_u_0 = [0.00007, 0.05];
    μ_c_F = 0.35,   σ_c_F = 0.45,   I_c_F = [0.05, 0.95];
    μ_m = 2,   σ_m = 1.4,   I_m = [1, 4].

The error tolerance δ is selected from a uniform distribution within the interval [0.008, 0.28]. We then test the network by changing one distribution at a time and, finally, by changing all distributions simultaneously; see results in Table <ref>. There, we note that the metrics exhibit similar values to the test conducted previously (cf. Table <ref>), indicating that performance is consistent over slightly perturbed test sets, although we note a slight degradation in performance when the distribution for m is changed, which may be due to the lower variability of this feature in the training set, which in turn leads the network to have more difficulty generalizing.







  
Visualization
The visualization of the results is done via two test examples in Figure <ref>, accompanied by the corresponding input values in Table <ref>. There, n_chan corresponds to the number of channels and ϕ_i, i∈{1,…,}, to the porosities in each channel. Analysis of Figures <ref>–<ref> reveals instances where the network misclassifies, mostly predicting 'GF' instead of 'Darcy', but this is in line with our objectives since it avoids numerical errors, despite an increased computation time.







 §.§ Case 2: Layer 35 of SPE10





  §.§.§ Hyperparameter tuning


Among the listed networks in Table <ref>, the one highlighted in yellow has highest recall which, as detailed in Appendix <ref>, serves as our primary metric. This network demonstrates remarkable qualities, displaying high recall and precision, and low error rate with respect to the others. However, it is essential to keep in mind the context: in this scenario, we are predict 13200 cells for each example, with the majority being Darcy, hence we expect lower error rate compared to the first case study of Section <ref>. Another noteworthy aspect is the computational time, which is significantly higher than that of the first case (cf. Table <ref>), although this is understandable given the absence of GPU or parallel processing and the large volume of data to be predicted because of the mesh size.



As for the previous case study, we look at the behavior of the cost function in the cases with learning rate equal to 0.1, even if from Table <ref> the results for precision and recall seem quite similar to the other cases. From Figure <ref>, we can observe that the values of the cost functions also fluctuate, but the fluctuations tend to be smaller than in Figure <ref>, remaining consistently below 1 in amplitude in most cases. It is possible that delaying our early stopping may lead to convergence, despite the current appearance suggesting otherwise. In fact, to mitigate the computational cost without significantly compromising performance, we implement a 10^-5 threshold for the early-stopping criterion. This adjustment aims to speed up the calculation process without overly impacting the metrics.





  §.§.§ Exploring selected network architectures




  
Training
Using the network configuration shown in Table <ref>, training is performed on the entire training set. The trends of the cost function during training and testing are illustrated in Figure <ref>. The expected behaviour is observed with a constant decrease of the function as the number of iterations increases. Eventually, the function stabilizes, indicating convergence. The cost function in both training and testing behaves similarly indicating that there is no overfitting. In this scenario, an early stopping criterion with a threshold of 10^-8 is used.





  
Testing
Table <ref> shows the performance of the network on the test set, specially chosen to include 5% of the entire dataset, with a total of 194 samples. The recall value obtained stands at 99%, underlining the robustness of the network.



Both the precision-recall curve (Figure <ref>) and the ROC curve (Figure <ref>) present great characteristics, further confirming the results obtained. In particular, the AUC value is extremely close to 1, estimated at 0.99995. Furthermore, the confusion matrix (Figure <ref>) shows that the false negatives amount to only 579 instances out of the total number of cells, indicating an very efficient classification process. Also, in the parity plot illustrated in Figure <ref>, the plotted sample points closely adhere to the bisector within the first quadrant. Like in Case 1, this indicates a near equivalence between the predicted GF cells and the actual ones. However, it does not provide us with insights into their spatial distribution within the mesh.





  
Visualization
Figure <ref> gives the results for two test examples with the corresponding input values given in Table <ref>. The predictions demonstrate a high level of accuracy, beating that for the previous test case (cf. Figure <ref>), which is probably attributable to a larger dataset and reduced variance; with fewer inputs resulting in less disparate outputs, the performance of the network is significantly improved.








§ CONCLUSION


Given the obtained results, which help us locate the Darcy and generalized Forchheimer areas in a heterogeneous porous medium, the natural next step would be, as already mentioned in the introduction, to implement domain-decomposition methods. Overall, we would then have a complete approach partitioning a priori the domain into fast- and slow-flow regions and then solving the resulting heterogeneous model on a decomposed domain. The outcome is a method which is more precise than a global Darcy model and computationally faster than a global Forchheimer model; this, however, still remains to be studied. This comparative study would further quantify the advancements achieved through our partitioning strategy.

Turning now to the results presented in the present paper, we see a significant disparity in behavior between the SPE10 case (cf. Section <ref>) and the landfill management scenario (cf. Section <ref>), prompting further investigation of potential improvements. The datasets themselves have significant disparities. The landfill dataset, characterized by its smaller size and greater variance, comprises 12 inputs, each demonstrating considerable diversity. In fact, scenarios with only 2 channels feature 5 inputs at zero, while in scenarios with 7 channels, all porosity inputs approximate unity. As a result, these variations yield substantial disparities in the output images, particularly in the distribution of Darcy and generalized Forchheimer areas. Consequently, the networks struggle to generalize the behavior of the inputs effectively. In contrast, in the case of Layer 35 of the SPE10 scenario, with only 4 more homogeneous inputs and outputs, coupled with a larger sample size, the network adeptly generalizes the behaviors.

To resolve these disparities and improve performance, several strategies can be pursued. First, one may expect that enriching and diversifying the dataset for the landfill model should prove useful. We thus conducted cross-validation using various datasets and sample sizes. The findings, given in Table <ref>, reveal a persistent trend indicating that variations in dataset size exert a negligible influence on network performance. This observation underscores the suitability of the selected dataset size for our analytical purposes, suggesting that further expansion of the dataset would not yield benefits. Notably, despite the modest dataset size, the network demonstrates robust performance and effectively captures the underlying output patterns, which shows its efficacy in learning and generalizing from limited data. 




Second, training separate networks for the cases with 2 or 7 channels could be beneficial to Case 1, given the manageable computational overhead due to the smaller output size compared to Layer 35 of the SPE10 case given the coarser mesh (2500 cells vs. 13200 cells). Additionally, integrating dropout regularization during training may strengthen the robustness of the model. While incorporating L^2 regularization into the cost function typically helps mitigate overfitting and improve generalization, Figures <ref> and <ref> show that overfitting does not appear to be an issue in our case. Overall, these approaches offer promising avenues for enhancing our network performance and bridging the observed gap between the two scenarios.

Third, a more extensive hyperparametric tuning than that given here would be necessary to further validate our approach. Indeed, the presented search for optimal hyperparameters is rather narrow and searching for more hidden layers, nodes and optimizing algorithms would improve the validity of our results. 

Fourth, a notable enhancement could be obtained by switching from NumPy <cit.> to PyTorch <cit.> or Tensorflow <cit.>. NumPy is a powerful library for numerical computations, but PyTorch and Tensorflow are prevalent libraries in the deep-learning community which offer additional benefits specifically tailored for building and training neural networks, such as GPU acceleration and automatic differentiation.

Finally, a possible direction to take to generalize our model would be to introduce a permeability tensor into the model, instead of using a scalar. This is not trivial as there is no general consensus on the exact form the generalized Forchheimer law should take in this case; possible formulations are given in <cit.>. This would introduce anisotropies, particularly relevant to three-dimensional scenarios, and testing our neural-network approach in this case would be a significant improvement.

All in all, although there is still work to arrive at a complete understanding and validation of our approach, the results obtained so far are promising. They provide a solid foundation on which to build further refinements and investigations.



§ AKNOWLEDGEMENTS

This paper was made possible thanks to the support of IFP Energies nouvelles, the research laboratory where this work was conducted. The present research is part of the activities of “Dipartimento di Eccellenza 2023-2027”, Italian Minister of University and Research (MUR), grant Dipartimento di Eccellenza 2023-2027.


§ FUNDAMENTALS OF OUR NEURAL NETWORKS



Neural networks are particularly suited for recognizing intricate patterns and extracting essential features. Moreover, their generalizing power to new data makes them a particularly attractive tool for our purpose, which is to predict a vector representing mesh cells that exhibit common patterns <cit.>. We use feed-forward neural networks with dense layers implemented using the Numpy library <cit.> following a methodology outlined in the course <cit.> and using a class-based architecture to improve modularity and maintainability.

We delve below into the various components of our neural networks.




 §.§ Datasets


The datasets employed in this investigation are constructed using the codebase provided by <cit.>, after suitable modification so as to align them with the specific requirements of our study. 

Each analytical scenario we test in this paper involves generating an input vector for each instance, or example, within the dataset and generating a respective binary output vector. Each slot in the output vector represents a cell within our mesh. Upon obtaining each dataset, we divide it into the following sets:

    
  * a training set consisting of examples for tuning hyperparameters, such as number of hidden units and learning rate, and for training the network;
    
  * a test set for measuring generalization performance.

This division is crucial for assessing how well our model generalizes to new, previously unseen data. In particular, in the cross-validation step performed to optimize the hyperparameters, the training set is further subdivided into folds, each in turn playing the role of a validation set (cf. Appendix <ref>). The cross-validation is essential, as tuning hyperparameters on the entire training set is not viable due to prior exposure and may lead to overfitting, and the test set is reserved for reporting the final performance and is usable only once <cit.>.

A fundamental pre-processing step common to both test cases is the normalization of input data with respect to the training set. This enhances the model's ability to learn patterns and generalize across diverse instances within the dataset.



 §.§ Architecture




  §.§.§ Dense layers

We decide to exploit classical dense layers, also known as fully connected layers. These layers exhibit simplicity and flexibility, each neuron establishing deep connections with and receiving input from every neuron in its preceding layer, facilitated through matrix-vector multiplication. 
Broadly speaking, a multilayer perceptron, that is, a dense feed-forward neural network, comprises L+1 layers (with the input considered as Layer 0) and includes L-1 hidden layers, with the output layer being the Lth layer. Each layer can have varying numbers of neurons, namely, n_l where l ranges from 0 to L.



In our specific implementation,  input data is represented by a matrix with dimensions (n_0, s), where n_0 is the number of inputs and s is the size of the dataset. The neural network's operation can be dissected into two phases: the forward propagation and the backward propagation (also known as backpropagation).

During the forward propagation, matrix-vector multiplication is applied to calculate Z, the input to the activation function, which is a mathematical operation that may introduce nonlinearity to the network to learn complex patterns in the data. The dimensions of Z are (n_l, s), where l denotes the current layer. The computation is expressed as

    Z^(l) = W^(l) A^(l-1) + b^(l).

Here, W^(l) is the weight matrix of dimensions (n_l, n_l-1), b^(l) the bias vector with dimensions (n_l, 1), and A^(l-1) the activations from the previous layer or input data, with dimensions (n_l-1, s). The resulting matrix Z^(l) serves as the input to the activation function a, shaping the network's output according to A^(l) = a(Z^(l)). 



  §.§.§ Activation functions


Our neural network employs commonly used activation functions, specifically the Rectified Linear Unit (ReLU) and the sigmoid activation function. The ReLU activation function ReLu→ is defined as

    ReLU(Z) = max(0,Z),

and it is used in all hidden layers. One of the key advantages of ReLU is its computational efficiency compared to other activation functions like the sigmoid or the hyperbolic tangent, as it involves simple element-wise operations. Additionally, ReLU helps mitigate the vanishing gradient problem, occurring when the gradients calculated during backpropagation become extremely small as they propagate backwards through the network layers, eventually causing the gradients effectively to vanish. By producing zero outputs for negative inputs and transmitting positive inputs unaltered, ReLU ensures that it avoids saturation for positive inputs, thus enabling continuous gradient flow during backpropagation. This characteristic distinguishes ReLU from functions such as sigmoid or tanh, which are prone to gradient vanishing. Consequently, ReLU facilitates the information propagation across network layers during backpropagation, thereby promoting expedited convergence and safeguarding against neuron saturation <cit.>.
The output layer, responsible for binary output, employs the sigmoid activation function σ→(0,1):

    σ(Z) = 1/1+ e^-Z.

 The sigmoid activation function effectively squashes the output values between 0 and 1 (cf. Figure <ref>). Moreover, in specific contexts such as domain classification tasks, the output of the sigmoid function assumes a significant interpretation. For instance, in classifying cells into Darcy or Forchheimer domains, the output of the sigmoid function directly corresponds to the likelihood that a given cell belongs to either category; for example, if the output is 0.7, it means that the probability that the cell belongs to the Darcy class is 70.





  §.§.§ Cost function



In the backward propagation, the parameters W and b are trained and updated. Backpropagation involves computing the gradient of the cost, or loss, function with respect to the network's weights. In our case, where the output is a binary classification vector, we use the cross-entropy cost function, namely,


    c(W, b) = - 1/sk∑_i=0^k∑_j=0^s [ y_i, jlog(a_i, j) + (1 - y_i, j)  log(1- a_i, j)].

Here, s is the size of the dataset, k is the size of the output vector (i.e., the size of the mesh), y_i, j is the element in the position (i,j) in the true label vector, a_i, j is the element (i,j) in the probability vector corresponding to the label predictions. Cross-entropy provides a measure of dissimilarity between the true labels and the predicted probabilities, lower values of cross-entropy indicating better model performance.

To optimize computational efficiency and expedite the training process, we implement an early-stopping criterion. Specifically, we halt the training procedure when 15 consecutive iterations yield no discernible improvement. This stopping criterion is compatible with the characteristics of our dataset, as we aim to strike a balance between computational resource use and achieve optimal model performance. Moreover, we introduce an additional criterion: we terminate the network training if the discrepancy in improvements falls below a certain threshold. This threshold varies for each test case and differs between cross-validation and training of the selected network. Specifically, considering that the cross-validation phase is more time-consuming, we opt for a higher threshold value, so that we stop before. Conversely, when computing on the entire training dataset, the computational time is more manageable, allowing for the adoption of a slightly smaller threshold value.




 §.§ Hyperparameters


Hyperparameters are a set of fixed parameters within a neural network architecture that remain unchanged during training. These parameters play a crucial role in shaping the network's performance and behavior. Unlike trainable parameters, like weights, hyperparameters are predetermined either manually or through optimization procedures prior to training. Examples of hyperparameters include the learning rate, the architecture's depth (number of hidden layers), the width (number of neurons) of each layer.

Tuning hyperparameters is fundamental for optimizing the performance of the network, as different configurations can lead to significantly different outcomes in terms of accuracy, metrics, convergence speed, and generalization ability <cit.>. 



  §.§.§ Cross-validation



To tune the hyperparameters, we employ the widely used method of cross-validation, specifically opting for the κ-fold cross-validation, where κ denotes the user-specified number of folds—in our case, 5. Cross-validation provides several advantages, including the avoidance of biased validation-set selection and more effective data use. It ensures that each example is included in the training set exactly once, promoting robust generalization to all dataset samples <cit.>. 

After initially partitioning our dataset into training and test sets according to a split of 95 and 5, the training set is further divided into five folds of approximately equal sizes. Subsequently, a sequence of models is trained, each time using a different fold as the validation set while the remaining folds are used for training. This process, illustrated in Figure <ref>, is repeated for all five splits, resulting in the collection of five vectors of metric values (cf. Appendix <ref> for a detailed description of the metrics). Then, we compute the mean of the chosen metric over the five folds to obtain an overall evaluation of the neural-network model. 




Care should also be taken when splitting the dataset to avoid potential issues related to data order. We shuffle the data to eliminate any inherent ordering in the samples based on labels.





































  §.§.§ Systematic methodology

As shown in Figure <ref>, our methodology involves a systematic approach to model development and evaluation. Initially, we partition the dataset into training and testing subsets, employing the former for cross-validation procedures detailed in Appendix <ref>. Leveraging this framework, we engage in hyperparameter tuning to extract optimal configurations, using them to train the model on the entirety of the training data, producing the final model. We then subject this model to evaluation using the test set and the training results.

In addition to the number of hidden layers and number of nodes, the hyperparameter we tune is the learning rate; we decide to try 4 different values, namely, 0.1, 0.01, 0.0075 and 0.001, thus spanning three different orders of magnitude and keeping a reasonable balance between precision and computation time. 

In the hyperparameter tuning phase, we adopt a random-search strategy. Recognizing that our input vectors span at least two orders of magnitude, we posit the effectiveness of using hidden layers with a high number of nodes. This architectural choice facilitates the diffusion of information across a multitude of nodes, leading to convergence towards a complete vector representation. Additionally, our empirical observations indicate that networks characterized by layers with an increasing number of nodes from input to output tend to show better performance, aligning with the inherent progression from inputs to desired outputs. Accordingly, we explore various random combinations of node and hidden-layer counts (cf. Tables <ref> and <ref>).



 §.§ Metrics



We define four fundamental quantities, referred to as metrics, integral to the evaluation of neural network models predicting mesh-cell classifications. In our context, a single example yields a vector representing mesh cells as output. Specifically, we classify GF cells as positive and Darcy cells as negative, the former being the prediction we wish to miss the least, as, indeed, predicting GF when the truth is Darcy is not as detrimental precison-wise as predicting Darcy when the truth is GF. Four distinct outcomes are possible:


    
  * True Positive (TP): the neural network correctly identifies a positive output cell;
    
  * True Negative (TN): the neural network correctly identifies a negative output cell;
    
  * False Positive (FP): the neural network incorrectly labels a negative output cell as positive;
    
  * False Negative (FN): the neural network incorrectly labels a positive output cell as negative.





  §.§.§ Precision and recall



One widely used metric in evaluating neural network performance is the error rate (Err), representing the frequency of classification errors over a given set of examples. It is defined as

    Err = FP+FN/FP+FN+TP+TN,

where FP, FN, TP and TN are the aggregated quantities for every cell across all examples in the set. An alternative metric, the classification accuracy (Acc), is the frequency of correct classifications and is related to the error rate by Acc = 1 - Err.

However, error rates may not provide a comprehensive view, especially in imbalanced datasets where one class significantly outnumbers the other, which is what we expect in our case where most domain should be classified as negative, that is, as Darcy. In our case, the abundance of Darcy cells necessitates additional metrics, namely, precision (Pr) and recall (Re):


    Pr = TP/TP+FP  and  Re = TP/TP+FN.

Precision quantifies the likelihood that the classifier is correct when labeling an example as positive, while recall measures the probability that a positive example is correctly recognized.

In our study, recall is more relevant than precision due to the detrimental impact of false negatives compared to false positives. Indeed, as already mentioned, misclassifying a cell as GF when it is actually Darcy merely entails the resolution of a nonlinear constitutive law (cf. (<ref>)), incurring an increase in computational time but no numerical inaccuracies. Conversely, misidentifying a cell as Darcy when it is truly GF leads to a substantial numerical error. Consequently, in selecting the most suitable network for our test cases, we prioritize those exhibiting higher recall since indeed a smaller FN yields a higher value. This parallels situations in medical diagnosis, where correctly identifying a patient with condition X constitutes a true positive, while failing to diagnose a patient with X constitutes a false negative—this latter outcome, medical practitioners endeavor to minimize and thereby advocate a low FN rate, i.e., a high recall.



  §.§.§ Performance plots


To evaluate further the performance of our trained neural network, we wish to study its behavior depending on the choice of the threshold in our binary classification. After the training process, the model outputs real values confined in the range [0, 1]; to get a binary value, 0 or 1, we need to fix a threshold separating the output values mapped to 0 from those mapped to 1. The commonly used threshold is 0.5, but it can be adjusted to achieve a desired trade-off between precision and recall. To get the desired trade-off, we examine all possible thresholds using the precision-recall curve. The proximity of the curve to the top-right corner indicates balance in the classifier performance, meaning that the points in this region represent high precision and high recall for the same threshold.

Another commonly used tool for analyzing classifier behavior at different thresholds is the Receiver Operating Characteristics (ROC) curve. Similar to the precision-recall curve, the ROC curve considers all possible thresholds for a given classifier but illustrates the trade-off between the False Positive Rate (FPR) and the True Positive Rate (TPR), where TPR is synonymous with recall and FPR, also called fall-out, is the fraction of false positives out of all negative samples:

    FPR= FP/FP+TN.

In the ROC curve, the ideal performance is reflected by a curve close to the top-left corner, indicating high recall while maintaining a low positive rate. Our ideal scenario aims for the top-left corner, where there is a zero error rate on the negatives and 100% accuracy on the positives. The point nearest to the top left is regarded as the most balanced operating point in the sense that it represents high TPR and high FPR. Moreover, the ROC curve is often summarized using a single metric: the Area Under the Curve (AUC). A higher AUC signifies better overall performance, and a perfect AUC of 1 implies that all positive points have a higher score than all negative points. Notably, AUC is particularly valuable for imbalanced classification problems, as it provides a more informative metric than accuracy.

A further method to compare our neural networks is the parity plot. For each example in the set, we compare the prevalence of the predicted output against the expected prevalence. The prevalence for example j∈{1,…,s} (Prv_j) is the ratio of the sum of positive cells in the jth example to the total number k of cells: 

    Prv_j = 1/k∑_i=1 
    cell i positive^k y_ij,

where y_ij is the entry in the ith row and jth column of the output matrix Y. In the parity plot, the reference line, represented by the diagonal equation y = x, denotes perfect model performance, where, for every example, the model predicts the exact number of positive (i.e., GF) cells as present in the mesh, that is, the true and predicted prevalences match. The distance of each point in the plot, i.e., each example, from the diagonal indicates the extent to which the model deviates in the number of positive cells predicted. While this metric is not flawless, as a model could predict the exact number of positive values but in the wrong cells, it is complemented by other metrics to provide a comprehensive evaluation of the model.

Finally, one of the most comprehensive tools for representing the results of binary classifications are confusion matrices. These are two-by-two matrices in which the rows correspond to the true classes and the columns to the predicted classes. Each entry indicates the frequency with which a cell belonging to the class in the row is classified in the class in the column. Entries on the main diagonal indicate correct classifications, while entries off the diagonal indicate incorrect ones, which provides a granular view of the performance of the model <cit.>.




















abbrv










 Paper Title: Lyapunov weights to convey the meaning of time in physics-informed neural networks.









Université Paris Dauphine - PSL

CEREMADE, 
 Place du Marechal de Lattre de Tassigny, Paris 75016, FRANCE



[url]https://turinici.com

Gabriel.Turinici@dauphine.fr



§ ABSTRACT 
Time is not a dimension as the others. In Physics-Informed Neural Networks (PINN) several proposals attempted to adapt the time sampling or time weighting to take into account the specifics of this special dimension. But these proposals are not principled and need guidance to be used. We explain here theoretically why the Lyapunov exponents give actionable insights and propose a weighting scheme to automatically adapt to chaotic, periodic or stable dynamics. We characterize theoretically the best weighting scheme under computational constraints as a cumulative exponential integral of the local Lyapunov exponent estimators and show that it performs well in practice under the regimes mentioned above.




 physics-informed neural networks; causal weighting in PINNs; Lyapunox exponent








§ TIME IN PINNS : INTRODUCTION AND PREVIOUS WORKS


Physics-informed Neural Networks introduced in <cit.> proved to be a very successful paradigm invoked to solve complex mathematical equations such as time evolutions 
<cit.> <cit.> possibly in high dimensions <cit.>, 
 partial differential equations <cit.>
 or control systems <cit.>; this framework exploits the power of neural networks (including automatic differentiation) to represent the main unknown function which is the solution of some equation involving its derivatives. The PINNs can also incorporate different other elements such as (possibly noisy) measurements on the system or further data such as controls.

To improve PINN performance several leads have been followed; 
Sharma and Shankar
 <cit.> studied meshless discretization, Cho et al. <cit.> consider separable network architecture while Wang et al. <cit.> analyze 
best norm to express the loss. We borrow here from all these perspectives but we focus on time-dependent problems and more specifically on how the sampling of time points (or their weighting in the loss) impacts the overall performance. The time dimension has been long recognized to possess particular characteristics that deserve special attention. In a highly influential work on 
Allen-Cahn and Cahn-Hilliard  
phase field equations <cit.> Wight and Zhao introduce various sampling strategies in both space and time while McClenny and Braga-Neto
<cit.> make weights trainable which requires additional computational time; in the latter work the influence of the weights on the training loss is mediated by a function that becomes a hyper-parameter to be specified by the user.
In a related approach, 
<cit.> Wang et al. remarked that, especially for chaotic or near chaotic systems, the causality of the time dimension has to be respected and enforced to obtain good results. In particular they proposed that earlier times be given more weight in the loss functional, proportional to the exponential of the cumulative sum of the accumulated errors (up to that point in time, see formula (<ref>) below); a hyper-parameter denoted ϵ remains to be chosen and in practice it is iterated in a prescribed list. The procedure also requires some special termination criterion.  
On the other hand 
<cit.> Penwarden et al. also studied the time dimension and propose a scalable framework for causal sweeping strategies in PINNs. This and many other contribution recognized the necessity to combine PINNs sequentially in order to solve a problem set on a large time interval. While we agree with this perspective, we search here only for the best time weighting scheme on each of these individual intervals ; as such, our procedure can be combined with any other time sweeping protocol. Moreover, as the previous protocols have been rather prescriptive and proposed ad-hoc choices of time weighting, we focus here on the design of principled approaches to orient the choice of  weights.  Our procedure is on one hand flexible enough to recognize the main regime the evolution takes place into (stable, chaotic, periodic) and on the other hand it does not require additional hyper-parameters to adjust, which was a more laborious part of the previous approaches.

The balance of the paper is the following: in section <ref> we introduce the general framework and in section <ref> we give the main theoretical insights that motivate our procedure. The numerical experiments are the object of section <ref>, followed in sections <ref> and <ref> by concluding remarks. 




§ PRESENTATION OF THE FRAMEWORK
 
To set notations let us recall very briefly how PINNs operate. Suppose that the evolution equation is to be solved is written in the form :

    ∂_t u    = _t u
    
    
    u(0,x)     =u_0(x),  ∀ x ∈Ω
    
    u(t,x)    = u_bc(t,x),  ∀ t∈[0,T], x ∈∂Ω.

where u(·,·) is the main unknown, t is the time variable and x the spatial variable, 
_t is some operator possibly involving the 
spatial derivatives of u (e.g., for the heat equation _t = ∂_xx), u_0(·) and u_bc(·,·) are the initial and boundary conditions respectively. When the equation is set in finite dimension such as the Lorenz system in section <ref> then Ω will be discrete and ∂Ω = ∅. The solution u will be searched in the form of a neural network (NN) taking two inputs t and x and outputting an approximation _θ(t,x) of u(t,x) ; θ are the
NN parameters, see figure <ref>.
In order to simplify the presentation we will moreover assume as in <cit.> and related works, that we can construct the network _θ such that _θ satisfies exactly the initial and boundary conditions. For instance for the initial conditions one can simply shift the output of the NN :
_θ(t,x) ↦_θ(t,x)-_θ(0,x)+u_0(x)· t
as in <cit.> or  

    _θ(t,x) ↦_θ(t,x)-_θ(0,x)+u_0(x).

 as in <cit.>.
In order to train the NN a loss is formulated that includes as main ingredient the equation error w(t,x) defined as w(t,x) = ∂_t _θ(t,x) - _t _θ(t,x) or equivalently :

    ∂_t _θ(t,x) = _t _θ(t,x) + w(t,x),  _θ(0,x) = u_0(x).

Our use of this last equation is slightly different from the literature, see also figure <ref> : since the derivatives are here exactly computed using the automatic differentiation capabilities of the NN, the equation 
(<ref>) is also satisfied exactly. So we can consider the error w as being the main output and _θ as the solution of 
(<ref>) that is available at no additional cost.


The NN is trained to minimize a loss functional; the original proposal in <cit.> is 
∫_0^T ∫_Ω | w(t,x)|^2 dx dt but, later on, particular weighting schemes appeared that proposed a loss of the form :

    (θ) = ∫_0^T ∫_Ωρ(t) | w(t,x)|^2 dx dt,

with the weight ρ(t) ≥ 0 to be chosen in order to speed up the convergence and improve the quality of the output. For instance <cit.> use 
a discrete form of 

    ρ(t) = e^-ϵ∫_0^t w(t,x)^2_L^2_x

for some ϵ >0 ; such a choice will give more weight to low values of t. Another proposal is 
<cit.> who sets  
 ρ(t) = e^-ϵ t for some ϵ∈; the intuitive reason is that errors at earlier times will accumulate into larger errors in the final output so  the solution at initial times should be computed more precisely. Although these proposals are inducing better numerical properties, they remain somehow ad-hoc. We investigate below in a principled way 
 how the weights ρ contribute to the quality of the result and set it accordingly.





§ THEORETICAL RESULTS
 

To design time weighting schemes we need to know how the equation error w(t,·) at time t will impact the accuracy of the final result _θ(T,·). There is no general answer to this question but we will use a close concept, namely the Lyapunov exponent which describes  quantitatively 
the rate of separation of infinitesimally close trajectories. The Lyapunov exponent originates from the dynamic system analysis and, for some 1D ordinary differential equation z'(t) = f(t,z(t)) describes how two solutions, starting from z^1(0) and z^2(0) diverge in the limit of large times t; more precisely the Lyapunov exponent is the coefficient λ∈ such that | z^1(t)-z^2(t)| ∼ e^λ t | z^1(0)-z^2(0)| (for large t). The determination of the exact value of the Lyapunov exponent is a difficult task in dynamical systems and ever more so in evolution PDE, but we will retain this idea. This allows to state :

Let u, _θ as in  (<ref>) and (<ref>) respectively. Let λ(t) be such that 

    ⟨_t(u(t,·))- _t(_θ(t,·)), u(t,·)- _θ(t,·) ⟩/u- _θ^2≤λ(t).

Then

    u(T,·)- _θ(T,·)≤∫_0^T e^∫_t^Tλ(s)dsw(t,·) dt.

Here the norm is euclidean when Ω is finite and L^2 norm otherwise.

 Denote e(t) = u(t,·)- _θ(t,·). Using (<ref>) and (<ref>)  we obtain:

    1/2d/dte(t)^2 = ⟨d/dt e(t), e(t)⟩
    =  ⟨_t(u)- _t(_θ)- w(t,·), e(t)⟩
       ≤λ (t) ⟨  e(t), e(t)⟩ -
    ⟨ w(t,·), e(t)⟩≤λ (t)  e(t)^2 + w(t,·)· e(t)

This means that d/dte(t)≤λ (t)e(t) + w(t,·), and, using the notation Λ(t) = e^-∫_0^t λ(s) ds
we obtain d/dt(Λ(t)  e(t) ) ≤Λ(t) w(t,·) ; hence, since e(0) = 0, by integration from 0 to T we get 
Λ(T)  e(T) ≤∫_0^T Λ(t)  w(t,·) dt which gives the conclusion.


The result already gives qualitative insight into how the error at the final time T depends on errors at intermediary times t≤ T, which is the main idea of Lyapunov exponents. Let us take the simple case _t(u) = λ u with λ∈ constant; in this case 
(<ref>) is in fact an equality.
As in <cit.> we note that for a chaotic or divergent system, which is characterized by a strictly positive Lyapunov exponent i.e., Re(λ)>0 the trajectories diverge exponentially with respect to error in the equation (<ref>) or in the data. In this case the precision has to be large for t near zero because the factor e^λ (T-t) will multiply it resulting in large error in the final state. On the contrary, for values of t close to T the factor e^λ (T-t) is relatively small and a larger error in the equation can be tolerated because it will be multiplied by a smaller factor. What is interesting to note is that this also works the other way around for Re(λ)<0 which characterize  systems converging to stable equilibria. In this case initial errors in the equation will be "washed away" by the stable dynamics and are less important and estimation (<ref>) says that we can manage errors increasing exponentially with T-t as long as the exponent is smaller than |Re(λ)| !
Finally, for the (quasi-) periodic dynamics |λ|=1, all times t≤ T contribute equally to the final error and no special weighting should be enforced. 

To add more actionable quantitative details to the discussion above we 
will see now how this translates into weighting schemes for ρ(t); changing the weight ρ(t) indicates to the training algorithm that some values of t should be given priority when minimizing the error loss w(t,·). Of course, more computational resources are available better the algorithm will converge, so we will make our reasoning assuming a given computational level of error i.e., will assume that we can employ numerical algorithms that  make ∫_0^T ρ(t)  w(t,·)^2 dt small enough (but not zero), say equal to C_p and inquire which is the best weighting scheme for this class of errors. The idea is to see how well the choice of ρ guarantees a good behavior of the final error at time T. Note that the training will only minimize the loss without regards as to which of the possible functions w are selected, as long as 
 ∫_0^T ρ(t)  w(t,·)^2 dt reaches the desired level C_p. So here we will assume worse case scenario and see what is the ρ
 whose worse case scenario is the best. This will be formalized as looking for the ρ such that 
 {max_∫_0^T ρ(t)  w(t,·)^2 dt ≤ C_p∫_0^T e^∫_t^T λ(s)ds  w(t,·) dt }
is smaller, see proposition below.
To gain in generality we will accept weighting schemes with irregular densities and write for a law η on [0,T] with density ρ (maybe not a proper function) _τ∼η w(τ,·)^2 dt instead of 
 ∫_0^T ρ(t)  w(t,·)^2 dt.
 For technical reasons we will need the law η to have finite second order moment which we write η∈^2([0,T]) and we assume w is bounded.
The following result gives precise information on the best weighting to be used :

Let λ(s):[0,T] → with 
∫_0^T e^∫_0^t λ(s)ds  dt < ∞ and w such that 
t↦ w(t,·) is bounded. 


Then the  problem

    min_η∈^2([0,T]){max_w; _τ∼η[ w(τ,·)^2 ] ≤ C_p∫_0^T e^∫_t^T λ(s)ds  w(t,·) dt }

admits an optimum which is attained in some η^opt. Moreover the minimum η^opt has a density ρ^opt(t) which is proportional to 
e^- ∫_0^t λ(s) ds that is :

    ρ^opt(t) = e^- ∫_0^t λ(s) ds/∫_0^T e^- ∫_0^t λ(s) ds dt.



We first suppose that ρ is regular enough and look for the solution.
Denote Λ(t) = e^- ∫_0^t λ(s) ds and write 
∫_0^T e^∫_t^T λ(s)ds  w(t,·) dt  = 
1/Λ(T)∫_0^T Λ(t)  w(t,·) dt. Dismissing the constant 
1/Λ(T) we are thus maximizing ∫_0^T Λ(t)  w(t,·) dt under the constraint _τ∼η[ w(τ,·)^2 ] ≤ C_p.
The Cauchy's inequality informs that 

    ( ∫_0^T Λ(t)  w(t,·) dt )^2 ≤_τ∼η[ w(τ,·)^2 ] 
     _τ∼η[(Λ^2(τ)/ ρ^2(τ) ) ]

with equality only when Λ(τ)/ √(ρ(τ)) and √(ρ(t)) w(τ,·) are proportional i.e. 
w(τ,·) is proportional to  Λ(t)/ρ(t). In this case the maximal value is ∫_0^T Λ^2(t) / ρ(t) dt. 
So for a given ρ we cannot guarantee a final error better than
∫_0^T Λ^2(t) / ρ(t) dt (up to a multiplicative constant depending on C_p and Λ(T) but not on ρ).
Now the choice of ρ should set this integral to the smallest value possible under the constraint ∫ρ(t) dt = 1  and ρ≥ 0. Use again the Cauchy inequality to see that 

    ∫_0^T Λ^2(t) / ρ(t) dt  = 
    ∫_0^T ρ(t) dt ∫_0^T Λ^2(t) / ρ(t) dt ≥(∫_0^T Λ(t) dt)^2,

which shows that we cannot do better than (∫_0^T Λ(t) dt)^2, 
with equality when Λ^2(t) / ρ(t) and ρ(t) are proportional i.e.
ρ(t) is proportional to Λ(t), which is the conclusion. 

For any other, possibly non smooth, choice of ρ one can follow the same reasoning by regularization : take smooth approximations of ρ which, by the arguments above turn out to not be better than ρ^opt so passing to the limit one obtains that no non-smooth ρ can do better. 

So the best weights ρ(t) are proportional to 
e^- ∫_0^t λ(s) ds; this quantity is calculated from the values of λ(t). Ideally, one would like to have :

    λ(t) = ⟨_t(_θ(t,·)) - _t(u(t,·)), _θ(t,·)- 
        u(t,·)⟩/_θ(t,·)- u(t,·) ^2,

i.e., the λ that is realizing equality in (<ref>). But the right hand side depends on the exact solution which is the unknown and has to be approximated.  
We will estimate the λ(s) by taking simply :







    λ_n(t) ∼⟨_t(_θ_n(t,·))-_t(0), _θ_n(t,·)
        ⟩/_θ_n(t,·)  ^2

where θ_n is the value of the NN parameters after the n-th training step (_θ_n is the solution candidate at this iteration). 
This amounts to approximating the exact solution (which is unknown) to zero. Although this approximation seems rough, it the most universal to be made and in practice it gives good results because we do not want exact values for λ_n(t) but some general information e.g., on its sign and monotonicity. Of course, if additional information on the exact solution u(t,x) is available this could be incorporated into  (<ref>). 
Furthermore, note  that for the simple situation when 

_t(u) = _t(u) + ξ_t + α_t u,

with _t a anti-symmetric operator, ξ_t independent of u and α_t ∈ the choice (<ref>) results in λ_n(t) = α_t (for all n) which is exact and optimal.



§ NUMERICAL EXPERIMENTS
 

The code for the numerical experiments will be provided on Github https://github.com/gabriel-turinici. 



 §.§ Lorenz system
 

The Lorenz system is a set of ordinary differential equations that describes a simplified model of atmospheric convection. It exhibits chaotic behavior and has been widely studied in the field of dynamical systems; it is described by the following set of ordinary differential equations:

    x'(t) = σ (y - x), 	 
    y'(t) = x (ρ - z) - y, 	 
    z'(t) = x y - β z,







where x, y, and z are the state variables, and σ, ρ, and β are parameters.
Coherent with the literature <cit.> we will take final time T=0.5[In the literature time goes up to T=20 but in practice it is implemented as sequential solves on 40 time intervals of size 0.5.] and use the classical parameter set (initially studied by Lorenz) 
(σ, ρ, β) = (10, 28, 8/3) and the initial state 
(x, y, z)(0)=(1,1,1). This test case is notoriously difficult to solve with PINN  (the default scheme does not even converge, see below) because of its very high sensitivity with respect to errors in the data and equation.










The reference solution, considered exact, is the one obtained with  routine with default settings. 
For the initial condition we used the shift  in (<ref>). 
The NN has 5 hidden fully connected layers of 20 neurons each ('tanh' activation) and a final FC layer with output dimension 3 (to match the unknown dimension) with no activation. All layer initializations are Glorot Uniform. The NN has 1'783 trainable parameters (which compares very favorably with 1'054'723 training parameters used by <cit.> who employ 5 hidden FC layers with 512 neurons). We used a uniform time grid of 256 points (as in <cit.>) and 2000 iterations of the Adam optimizer; initially we tested with the default learning rate (0.001) but to accelerate convergence we then chose everywhere a learning rate of 0.01 (only exception being  figure <ref>  where we compare these two learning rates). All other optimizer parameters are the TensorFlow version 2.15 defaults. A run of the procedure takes 80 seconds on a T4 GPU.

We first show convergence results in figure <ref>. It is seen that the loss reaches low values (and will improve if more iterations are allowed); at the same time we plot (right figure) also the error at final time T between the reference solution and the PINN solution, that also behave well and shows that convergence occurred. This is to be compared with figure <ref> where the numerical solution shows high quality match with the baseline solution. We also checked (not shown here) that given more iteration the quality improves further (but then we cannot distinguish graphically the two).

We now pass to the inner workings of our procedure. First in figure <ref> (right) we note that the equation error is indeed of intuitive form: since the system is very sensitive to errors, it is best to set error to be lower for small t and larger at final times. 
This is coherent with the weights selected (automatically) by our algorithm, depicted in figure <ref> left, that are indeed decreasing. The weight assignment is the result of the estimators (<ref>) plotted in figure <ref> right. We plot both our estimator and the ground truth (<ref>) (that requires knowledge of the exact solution). The agreement is remarkable and it is seen that our estimator (<ref>) captures the good order of magnitude and sign while retaining a more smooth behavior.

As discussed above, we also investigated the influence of the learning rates; we plot in figure <ref> left 
a comparison of the losses during the optimization with two different learning rates, 0.001 (TensorFlow Adam optimized default) and 0.01. As expected from the general theory of stochastic optimization it is seen that the larger learning rate gives faster convergence but also larger final variance (oscillations) at the solution. 

Finally, in figure <ref> right we plot a confidence interval for the loss (learning rate set to 0.01 for all) after 1000 independent runs of the procedure. The variability is due to the initialization of the NN layers and is normal to have runs taking longer time. For all quantiles the decrease in loss is systematic. 








We now turn to comparing these results with equivalent results for two other time weighting algorithms, <cit.> (cumulative exponential error loss (<ref>)) and <cit.> (exponential weighting ρ(t) ∼ e^-λ t). 
We do not even compare with uniform time-weighting (which is the default PINN protocol) because this is known to not work well on Lorenz (and most chaotic) systems. The interested reader can nevertheless find in the supplementary material section <ref> this result.

For both we used exactly the same NN and optimizer settings.
We seeded the random number generation of numpy and TensorFlow with the same seed as in the previous case so all procedures start from the same initial network weights.
The 'causal' procedure seem to not have converged yet cf. figure <ref> even if we gave it 5 times more iterations. We did so because this procedure needs to change the value of ϵ in a schedule involving 5 preset levels in the list 0.01,0.1, 1, 10, 100. To each level we gave 2000 iterations (which is the total number of iterations used by our procedure). We checked that if given even more time the protocol does indeed converges. Obviously it attempts to exploit the same idea of using larger weights for t small resulting in lower equations errors at t small and larger errors for t large (cf. figure <ref> rightmost plot).
In figure <ref> left we plot the convergence of the procedure (the 5 ϵ levels are visible on left plot). Note that, since the Lorenz system is close to chaotic, significant equation errors of the early stages of the evolution equation destroy completely any confidence in the solution for final time T as is shown in figure <ref> right, where the final error between the exact and numerical solution does not appear to improve much. The loss is indeed decreasing but it does not convey enough relevant information to orient the convergence. See supplementary material for further explanation of the weights behavior in this procedure, which appear sensitive to the precise number of iterations run for each  value of ϵ.




We compare now with the exponential time weighting scheme in  <cit.>. The procedure has a hyper-parameter λ  that give the exponential decay of the weights. We tested several choices of parameters 
but will only show results for two of them,
λ=10 (which has the good order of magnitude) and 
λ=13.8; this latter value was chosen to have a posteriori the same average decay as our procedure (and thus should give comparable results). The results in figure  <ref> for λ=10 show that the procedure is sensitive to the choice of the λ parameter; when this parameter is given by some 'oracle' then the solution is much improved. When λ is farther away from the optimal value the results degrade.




 §.§ The Burgers' equation
 
The Burgers' equation models the one-dimensional flow of viscous  fluid and reads :

    ∂ u/∂ t + u ∂ u/∂ x = ν∂^2 u/∂ x^2,  t∈ ]0,T],  x∈]-1,1[ 
        u(0,x) = -sin(π x) 
        u(t,± 1)=0,  ∀ t ∈ [0,T].

where u(x, t) is the fluid velocity, ν=0.01/π is a real positive constant called the viscosity coefficient, x and t are spatial and temporal variables respectively. Total time is T=1.0.
These are supplemented by the initial 
(<ref>) and boundary (<ref>) conditions. For the initial condition we used the shift described in  (<ref>); on the other hand we did nothing special for the boundary conditions which were implemented by adding a corresponding term in the loss as it is classic in these cases <cit.>.

For the 'exact' baseline, as this problem is known to be difficult to solve, a first proposal was obtained with a  
finite difference scheme combining two half-step  Crank-Nicholson propagators for the viscosity term  with a Lax-Friedrichs  propagator for the transport part. A different computation can be proposed through the exact formula (2.5) used by <cit.> combined with Hermite polynomial quadrature ; see the supplementary material  <ref> for all scheme details and analytic formula. We checked that both agree up to 1.0e-4 and used quadrature analytic formula if precision beyond this was required.

As in <cit.> and <cit.> 
the NN architecture uses 8 fully connected hidden layers of 20 neurons each (3021 parameters to train) and learning rate of 0.01; we define a uniform grid of 50 points in time and 25 in space (which gives a total number of interior collocation points of 1250, almost an order of magnitude less than in the references cited). Procedure is run for 3000 iterations in 387 seconds on a T4 GPU.

We plot in figure <ref> the results for our procedure, the causal procedure <cit.> and the standard PINN <cit.> approach. Compared with the Lorenz system this is a non-chaotic case and all algorithms will eventually converge and give good results. The plot confirms that our procedure has some merits in using insights from the system during the initial iterations and improve efficacy. Data in supplementary material shows that this is realized by some counter-intuitive behavior of over-weighting final instants when t is close to T because at this point that the discontinuity is created (and detected by our procedure).







§ LIMITATIONS
 
Although the theoretical and empirical results appear encouraging, further work could improve some aspects of the procedure. The most apparent is the estimation of the Lyapunov exponents, which is now done in a very crude way; this could be refined or, if not possible, at least construct some trust regions to inform when these estimations are reliable or not.

Another aspect is  the interaction between the weights dynamic and the solution. This is common to all adaptive weights procedures but it may concern some cases where the weights given by (<ref>) induce instabilities in the solution preventing convergence (in the same vein as the well-known GAN "mode collapse" problem). An analysis of whether this can happen or how to prevent it could be required.

Finally, for the problems where the canonical, simple implementation of the PINN procedure already works well this algorithm may cause slight numerical computational overhead.



§ CONCLUSION
 

In this work, we presented a new approach to treat the time dimension within the framework of Physics-Informed Neural Network. We started from the observation that time instants are  not permutable and depending on the regime (chaotic, periodic, or stably convergent) errors in earlier instants affect differently the quality of the outcome. We proposed a theoretical explanation based on Lyapunov exponents and then a new algorithm built from this insight.

The procedure was then tested on two different cases, one chaotic (Lorenz system) and one stable but that induces singularities (Burgers' equation), both known to require careful numerical treatment. The proposed algorithm not only showcases robustness and practical efficiency but also addresses some limitations of earlier works that lacked a principled and theoretically grounded foundation. Of course, while our procedure is robust across various benchmarks, it is designed as a complementary addition to the existing methods rather than a one-size-fits-all solution. We believe this work contributes meaningfully to the field, offering a solid foundation for future research and application development.












§ SUPPLEMENTARY MATERIAL




 §.§ Standard PINN (no time weighting) results for the Lorenz system
 

As announced in section <ref> we plot in figure <ref> the solution of the procedure for the situation when the weights are not optimized at all but left all uniform. This case was allowed 30'000 iterations (10 times more than our standard setting) but did not even start to converge.





 §.§ Exact solution for Burgers' equation

To compare the PINN solution with a reference ground truth we solved Burger's equation with a finite difference scheme and compared with an analytic formula with Hermite-Gauss integration.



  §.§.§ Finite differences scheme for Burgers' equation

The finite difference scheme works on a spatial domain
discretized with nx = 400+1 spatial grid points (counting both segment extremities ± 1), 
i.e. Δ x =2/400 
and nt = 700 time steps (Δ t = 1/700);  each time step is implementing the following split-operator technique :

- first a Crank-Nicholson (CN) scheme over a Δ t/2 time step (taking into account the boundary conditions) for the diffusion part i.e. the solution by CN of the equation ∂_t u = ν∂_xxu 

- then a Δ t step of a Lax-Friedrichs scheme for the viscosity term, i.e. for the equation
∂_t u + u ∂_x u = 0.

- then a final Crank-Nicholson (CN) scheme over a Δ t/2 time step (taking into account the boundary conditions) for the diffusion part

The splitting technique is known to be of high order in time <cit.> 
which leaves the Crank-Nicholson part (second order in time) and Lax-Friedrichs scheme (first order in time and  space) as limiting factors. As for stability it is known that CN is unconditionally stable for the heat diffusion and since the matrix for the Lax-Friedrichs scheme is tridiagonal by Gersgorin theorem it is stable when max(|u|) ·Δ t/ Δ x<1.

In practice, if V∈^nx+1  is the current solution with V_j the component  representing the solution at point x=-1 + j·Δ x), the CN propagation for a Δ t/2 time step means solving the linear system in V^next : 

    V^next_j- V^next_j/Δ t/2 = ν/2(
         V_j+1+ V_j-1-2 V_j/Δ x^2  +
        V^next_j+1+ V^next_j-1-2 V^next_j/Δ x^2), 
     V^next_0=0=V^next_nx.


The  Lax-Friedrichs scheme means replacing V with :

    V^next_j- V_j+1+ V_j-1/2/Δ t +  (
         V_j+1^2/2- V_j-1^2/2/2 Δ x)=0, 
     V^next_0=0=V^next_nx.

Once this is solved the resulting 2D solution matrix is used as input to a 2D interpolation routine to obtain a linear interpolation function that is able to output values for any other point not necessarily on the space-time grid used by the finite difference resolution.



  §.§.§ Analytic formula for the solution of the Burgers' equation


Another way to find a solution is to used the analytic formula (2.5) from <cit.>. It reads

    u(t,x) = ∫_ u_0(x-η) ζ(x-η) e^-η^2/(4ν t) d η/∫_ζ(x-η) e^-η^2/(4ν t) d η,  where ζ(y) = e^-cos(π y) / (2 πν).

The integrals can be computed using Hermite-Gauss quadrature and in practice we used 50-th order formulas after a change of variables η = η' √(4 ν t ). For t < 10^-10 we used a first order Taylor development with respect to t using the exact formula for the time derivative from the Burgers' equation.




 §.§ Behavior of weights of the causal procedure (Lorenz system)

 
To explore the convergence properties of the causal protocol <cit.> we looked at the history of the weights used by the procedure during the resolution of the Lorenz system, see comments in section <ref>; these are plotted in figure <ref>.
The algorithm's protocol iterates the ϵ parameter ; at each change the weights associated with latter times are smaller which causes deterioration of the quality there. So the algorithm can only advance to the next value of ϵ only after the equation is satisfactory solved at initial times.





 §.§ Weights for Burgers' equation

We compare in figure <ref> the final weights of the three procedures considered for the Burgers' equation, we note that our proposal over-weights final times which is when the discontinuity appears. This is not the case of the other procedures ; this behavior explains the results in last columns of figure <ref> (rows 2 and 3) that clearly lack precision at that point in space.





 Paper Title: Precomputing approach for a two-scale phase transition model.





























































definitionDefinition
theoremTheorem
propositionProposition
prbProblem
corollaryCorollary
lemmaLemma
remarkRemark
assumptionAssumption


lemmaLemmalemmas
lemmaLemmaLemmas
thmtheoremtheorems
thmTheoremTheorems
algocfAlgo.Algorithm

acknowledgements § ABSTRACT  

[figure]font=normalsize,labelfont=bf,singlelinecheck=true


equationsection






†,1]Michael Eden
⋆]Tom Freudenberg
†]Adrian Muntean

[†]Department of Mathematics and Computer Science, Karlstad University, Sweden
[⋆]Center for Industrial Mathematics, University of Bremen, Germany



Precomputing approach for a two-scale phase transition model
    [
    August 5, 2024
============================================================

§ ABSTRACT 
In this study, we employ analytical and numerical techniques to examine a phase transition model with moving boundaries.
The model displays two relevant spatial scales pointing out to a macroscopic phase and a microscopic phase, interacting on disjoint  inclusions.
The shrinkage or the growth of the inclusions is governed by a modified Gibbs-Thomson law depending on the macroscopic temperature, but without accessing curvature information.
We use the Hanzawa transformation to transform the problem onto a fixed reference domain.
Then a fixed-point argument is employed to demonstrate the well-posedness of the system for a finite time interval. 
Due to the model's nonlinearities and the macroscopic parameters, which are given by differential equations that depend on the size of the inclusions, the problem is computationally expensive to solve numerically.
We introduce a precomputing approach that solves multiple cell problems in an offline phase and uses an interpolation scheme afterward to determine the needed parameters.
Additionally, we propose a semi-implicit time-stepping method to resolve the nonlinearity of the problem.
We investigate the errors of both the precomputing and time-stepping procedures and verify the theoretical results via numerical simulations.


Keywords:  Phase transitions, two-scale model, moving boundary problem, numerical analysis, simulation

MSC2020: 35R37, 65M60, 80A22, 35K55


[1]<michael.eden@kau.se>
footnote







§ INTRODUCTION

Phase transition models describe the heat dynamics in a system where phase transitions can occur, such as water-ice or austenite-pearlite microstructures in steel.
One of the main difficulties is the need to account for the changing geometries inherent to the transition process.
In some cases, there is an additional scale separation: the phase transition processes might happen at the microscale while our interest might be in the effective macroscopic behaviors.
Examples include microstructural changes in steel <cit.>, liquid–solid phase transitions in metals <cit.>, or thawing processes in permafrost <cit.>. In these cases, the focus is not on the geometric changes at the microscale but on their impact on macroscopic properties.
However, via homogenization techniques (e.g., two-scale convergence or asymptotic expansion) it is easy to see that the microstructural changes can not be neglected: In the resulting models, the phase transitions are still present on the microscale and directly influence the effective material parameters, see, e.g., <cit.>.
Such two-scale models (sometimes also called micro-macro model or distributed microstructures model) are computationally extremely costly, mainly due to the following two factors: 

    
  (a) Strong non-linear coupling between the scales: The macroscopic temperature field is driving the microstructural changes and the microstructural changes are in turn non-linearly influencing the macroscopic material parameters.
    
  (b) Family of moving boundary problems: Every macroscopic point is associated with corresponding moving boundary problems at the microscale both via the interaction with the microscopic temperature but also via the geometric changes influencing the heat conductivity via the associated cell problems.
 
Since one of the main goals of homogenization is the derivation of simpler models that are computationally less heavy, these issues might at first glance call into question the usefulness of such limit models.
There is, however, one important advantage these two-scale models have, namely the explicit separation of the scales.
This can be exploited in many different ways: via iteration procedures <cit.>, parallel computing <cit.>, machine learning <cit.>, and/or interpolation techniques <cit.>.

In this work, we propose, analyze, and implement a precomputing strategy in which the calculations of the macroscopic coefficients are pushed into an offline phase.
Instead of having to calculate the effective conductivity via geometry-dependent cell problems at every macroscopic point and at each time step, we precompute the conductivity instead for a range of potential parameter values and interpolate in between.
Please note, that this precomputing is perfectly parallizable.
We show that this interpolation is stable and that the introduced error in the two-scale system can be controlled via the step size in the parameter space and the order of the interpolation. 

This idea is not completely new and similar approaches have been outlined before, see <ref> for a short literature overview.
The non-linearities in the model are resolved with a semi-implicit time-stepping method which we show to converge linearly.
We confirm these convergence results via simulations and the corresponding FEniCS code is made freely available <cit.>.



 §.§ Two-scale phase transition models

We want to elaborate a bit on the origin and structure of the two-scale phase transition problem that we are looking at in this work.
The model in question is the homogenization limit of a highly heterogeneous two-phase system consisting of a connected phase Ω_^a(t) with disconnected inclusions of a different phase Ω_^b(t), see <ref>. 
Both domains depend on time due to the potential of phase transitions. 
Here, >0 is a small parameter denoting the heterogeneity of the system and the homogenization limit can be achieved via a (formal or rigorous) limit process →0.




A two-phase problem for such a phase transition problem could look like

    ∂_tθ_^a-k^aΔθ_^a   =f_^a        in Ω_^a(t), t>0,

    ∂_tθ_^b-^2k^bΔθ_^b   =f_^b        in Ω_^b(t), t>0,

    θ_^a   =θ_^b        on Γ_(t), t>0,

    -(k^a∇θ_^a+^2k^b∇θ_^b)·n_   =Lv_        on Γ_(t), t>0,

where L∈ denotes the Latent heat and v_ the normal velocity of the moving phase interface Γ_(t).
The particular -scalings are natural for this particular setup with inclusions, cf. <cit.>.
The normal velocity has to be implicitly or explicitly described via an additional equation.
The usual Stefan condition is θ_^a=θ_ref, where θ_ref is the reference temperature of the phase transition in question.[In a water/ice system under normal conditions, we have θ_ref=0^∘ C.]
Another option is Gibbs-Thomson law without curvature which postulates


    v_= (θ_^a-θ_ref)  on Γ_(t), t>0.



The model in question in this work is the limit of System (<ref>) for →0.
Such a limit procedure was conducted via two-scale convergence techniques for a slightly simplified one-phase equivalent of this system in <cit.>, and the results can easily, at least formally, be extended to this two-phase system via asymptotic expansion.
Via such a limit process, we can identify limits such that, in some sense, θ_^a→Θ and θ_^b→θ.
Here, Θ is the temperature field of the macroscopic phase which is defined on a non-changing macroscopic domain Ω and θ is the temperature field of the microscopic phase given on a changing microscopic domain Y(t).
Of course, Θ and θ are intricately coupled via energy conservation and the continuity condition <ref>.
Moreover, all material coefficients for the macroscopic heat problem are now dependent on the microstructural changes as well, rendering the problem highly nonlinear.
The exact setup of the two-scale problem is given in <ref>.

 



 §.§ Discussion on related literature

An early work on two-scale models in phase transitions is due to A. Visintin <cit.>, where the transition between the two scales is achieved via convolution with a Gaussian kernel but without explicitly looking at evolving microstructures.
More closely related to our work, in <cit.>, two-scale models for dendritic growth with prescribed geometry changes are formally derived via asymptotic expansion.
In <cit.> two-phase models for phase transitions are derived, albeit with a priori known geometry evolution.
Very recently, in <cit.>, an effective one-phase model for phase transitions was rigorously derived via two-scale convergence.

Very similar models and challenges can be found in the context of reactive transport or coagulation/fragmentation dynamics in evolving porous media which has garnered quite a bit of interest in recent years.
We point to the upscaling works in <cit.>, which are concerned with the derivation of two-scale models in that context, and note that the resulting models are structurally quite similar to the phase transition problem considered in this work.
Naturally, there are quite a few works looking more closely at the resulting two-scale models: In <cit.>, the well-posedness for such a two-scale model was established for a simplified setup where the geometry changes do not impact the diffusivity except via the porosity.
In <cit.>, strong local-in-time solutions for a two-scale model, even allowing for nonuniform geometry changes, are established via maximal parabolic regularity and a level set approach.

Such models often come with huge computational challenges due to their nonlinear nature and complex coupling.
Our main approach of dealing with this is a precomputing strategy in which the calculations of the effective macroscopic coefficients are pushed into an offline phase; introducing an interpolation error but significantly speeding up the calculations.
Similar ideas are often explored in the context of shape or topology optimization, see e.g.,  <cit.> and references therein.
In <cit.>, the convergence of a precomputing scheme for a two-scale dispersion model for porous media is shown, however, there are no geometry changes present.

Naturally, there are other approaches to tackle the computational complexity of two-scale models:
Instead of solving the entire coupled system in a classical FE approach, an alternative idea is to explicitly decouple the macro and micro temperatures by solving between them iteratively.
This is closely related to the FE^2 approach <cit.> and the Heterogeneous Multiscale Method <cit.>.
This approach has two main advantages: the calculations on the microscale can be run in parallel and an adaptive interpolation strategy can be used to only solve the microscale problems for a small number of macroscopic points.
However, for each time step, there may be a need for several iteration steps, which again can be costly, especially when the coupling is complex.
In addition, the iteration process introduces a new error.
The approach has been studied and investigated for many different two-scale problems, see <cit.>.
As a direct extension of the iterative scheme, an adaptive clustering approach has been developed in <cit.>.
Here, micro problems with comparable macroscopic data are paired together and solved via one representative system.
This allows for a speed-up and even enables fast simulations in higher-dimensional setups.
Of course, this approach produces further error sources, for which, to our knowledge, no estimates are available.
But the numerical results in <cit.> show good approximation accuracy. 

Another approach is the hierarchical FEM as they are used for multiscale problems in <cit.>.
Here, special FE spaces are constructed to reduce the DOFs of those functions which depend on multiple scales.
This method is comparable to a wavelet approach and allows for analytical error estimates, but is more complicated to set up than the previous methods.
Finally, we want to mention a model reduction approach which is often encountered in mechanics, see <cit.> and the references therein.
A model reduction for microscopic crystal growth was considered in <cit.>. 
This approach can lead to significant speed-ups, but requires, especially for time-dependent problems, a large amount of data to obtain reasonable models.



 §.§ Outline of the paper

The paper is structured as follows: In <ref>, we present the two-scale system, collect the mathematical assumptions, and introduce our notion of weak solutions.
In <ref>, we establish the existence of local-in-time weak solutions via a fixed-point argument by utilizing Hanzawa transformations (<ref>).
Moreover, we show that uniqueness and long-time existence are given when some additional estimates are met.
This is followed up by <ref>, where we introduce and discuss our precomputing strategy.
Here, we first show the stability of the resulting scheme with respect to the interpolation error (<ref>) before then analyzing our semi-implicit time-stepping method with which we resolve the non-linearities (<ref>).
We present some simulation scenarios in <ref>: in particular, we are showing convergence curves for our precomputing strategy.
Finally, we close with a short conclusion and discussion (<ref>).








§ SETTING OF THE GEOMETRY AND MODEL EQUATIONS

In this section, we introduce the geometric setup and the two-scale system modeling a two-phase composite medium with phase transformations taking place at the microscale.
We also collect the mathematical assumptions and present our notion of weak convergence.

To this end, let S = (0, T ), for T∈∪{∞}, denote the time interval of interest.
Let a Lipschitz domain Ω⊂^d (d=2,3, outside normal vector ν) represent the macroscopic domain, set Y = (0, 1)^d and let a C^3-domain Y_0⊂⊂ Y (outer normal vector n) represent the initial micro-domain.
We set Γ_0=∂ Y_0.
The micro-domain Y_0 is allowed to grow or shrink based on the dynamics of the problem and this change is characterized via a height parameter h inside a tubular neighborhood U_Γ⊂⊂ Y of Γ_0 with width 2a for some a>0.
Here, h=0 indicates no changes, h<0 shrinkage, and h>0 growth.
We denote the dynamic domain by Y(h) and interface by Γ(h),[Of course, Y(0)=Y_0 and Γ(0)=Γ_0.] respectively, via


    Γ(h):={γ+h n(γ) :  γ∈Γ_0},   Y(h):=(Γ(h)).

Please note that the growth of the micro-domain is cell uniform, i.e., the height h=h(t,x) only depends on time t∈ S and the macro variable x∈Ω.
As a consequence, shape changes are not possible (e.g., if Y_0 is a ball of radius r, Y(h) will be a ball of radius r+h).
For the exact setup of the transformation, we refer to <ref>, and, for a sketch of this setup, we refer to <ref>.




Now, let Θ=Θ(t,x) and θ=θ(t,x,y) denote the macroscopic and microscopic temperature, respectively, and h=h(t,x) the height function characterizing the micro-domain changes.

On the macroscale, we have a standard heat equation where all parameters non-linearly depend on h:


∂_t(C(h)Θ)-(K(h)∇Θ)   =F- L(h)∂_th-q(h)[θ]        in S×Ω,

-K(h)∇Θ·n   =0        on S×∂Ω,

Θ(0)   =Θ_0        in Ω.



Here, C denotes the heat capacity, K the effective heat conductivity, and L is the latent heat.
Moreover, q(h)[θ] S×Ω→ accounts for the energy flow from the micro into the macro region and F is a generic volume source.
The h-dependent coefficients are given via



    C   (-a,a)→(0,∞),     C(h)   =1-|Y(h)|,

    K   (-a,a)→^d×d,     K(h)_i,j   =∫_Y∖Y(h)𝒦(∇_yξ_j(h)+e_j)·(∇_yξ_i(h)+e_i)y,

     L   (-a,a)→,     L(h)   =|Γ(h)|,

    q(·)[θ]   (-a,a)→,     q(h)[θ]   =∫_Γ(h)κ∇_yθ·nσ_y.

Here, 𝒦, κ>0 are the heat conductivity of the macro- and micro-phase and ξ_j(h)∈ H^1_#(Y∖ Y(h)), j=1,...,d, solutions to the cell problems


∫_Y∖Y(h)𝒦(∇_yξ_j(h)+e_j)·∇_yϕy=0


for all ϕ∈ H^1_#(Y∖ Y(h)).


Please note that the coefficients <ref> represent what we typically expect in a two-scale limit system; see <cit.> for similar setups.
For example, C(h) represents the effective volumetric heat capacity of the macroscopic phase which is given by C(h)=|Y∖ Y(h)|=1-|Y(h)|.
This accounts for the competition with the micro-phase.


The heat problem for the macroscopic phase (<ref>) is strongly coupled with a Stefan-type problem for the microscopic phase in the form of



∂_tθ-_y(κ∇_yθ)   =f        in Y(h),  in  S×Ω,

θ   =Θ        on Γ(h),  in  S×Ω,

∂_th   =θ-Θ^ref        on Γ(h),  in S×Ω,

θ(0)   =θ_0        in Ω×Y(0).



The moving boundary condition (<ref>) is a modified Gibbs-Thomson law without curvature, where the normal velocity of the moving boundary, i.e., ∂_th, is determined by the difference of the temperature to a reference temperature.
As an example, for the water-ice system under normal conditions, we have Θ^ref=0^∘ C.
Here, we just assume Θ^ref∈.
The overall problem is both highly nonlinear due to the changing domains and is fully coupled via the following mechanism:


    
  * The flux term q in <ref> tracks the heat exchange between the phases. As a consequence, the microphase can act as a heat storage or heat sink.
    
  * <Ref> enforces continuity of the micro and macro temperatures at the micro-interface Γ(h). This also implies cell-uniform growth: Since Θ is y-independent, h must be as well.
    
  * The growing or shrinking domains directly influence the macroscopic coefficients (<ref>).

We point out that the Neumann flux appearing via q(h)[θ] in the strong form <ref> disappears in the weak form as it balances with the corresponding term in the micro part after integration by parts.
The main difficulties in the analysis of this system are the changing micro-geometries and the inherent non-linearity associated with those changes.
We do this by transforming the geometries into the fixed, initial geometries via the so-called Hanzawa transformation.
Details are given in <ref>.


    In our setup, it holds h(0)=0 which means that the initial micro-geometry is uniform in x∈Ω.
    However, we could easily consider an initial condition h∈ L^∞(Ω) as long as it is small enough (see also <ref>).
    As this would introduce some additional technical estimates in our analysis, we stick to a uniform initial setup. 




 §.§ Assumptions and weak forms

A few words, regarding our notation: In general, we use lowercase and capitalized symbols for the microscopic and macroscopic counterparts, e.g., f and F for the volume source densities.
For functions depending on both the macroscopic variable x∈Ω and y∈ Y, we take _y,∇_y to denote the spatial differential operators with respect to y.
Moreover, we take c,C>0 as generic constants (whose value can change even from line to line) and where we explicitly point out when they are independent of a specific parameter.
We start by formulating the assumptions we place on our geometry and data:

    
  (A1) The heat conductivities κ,𝒦∈^d× d are symmetric and positive definite.
    
  (A2) The volume source densities satisfy f∈ L^∞(S×Ω;C^0,1(Y_0)) and F∈ L^∞(S×Ω).
    
  (A3) The initial geometry Y(0)=Y_0⊂⊂(0,1)^d is a C^3 domain and Ω⊂^d is a Lipschitz domain. 
    The tubular neighborhood U_Γ⊂(0,1)^d of the interface Γ=∂ Y_0 has tubular radius a>0.
    
  (A4) The initial conditions satisfy θ_0∈ L^∞(Ω× Y_0) and Θ_0∈ L^∞(Ω).


Please note that the regularity presented in Assumptions (A2) and (A4) is higher than the standard parabolic regularity for two reasons: To be able to work with coordinate transforms, the micro heat production source needs to be continuous in the y-Variable.
We chose Lipschitz continuous to guarantee the uniqueness of solutions.
The boundedness of the data is needed to ensure boundedness of the solutions; without this, it is impossible to control the height functions (see <ref>).

Now, for any 0<T^*≤ T and h∈ L^∞((0,T^*)×Ω) satisfying h_∞≤ a^*=a/8, we introduce the corresponding non-cylindrical micro-domain


    Q[h](T^*)=⋃_(t,x)∈ (0,T^*)×Ω[{(t,x)}× Y(h(t,x)) ]⊂ (0,T')×Ω× Y.

and the Banach space


    𝒳[h](T^*)={(Θ,θ)∈ L^2(0,T^*;H^1(Ω))× L^2(Q[h](T^*)) : θ(t,x)∈ H^1(Y(h)),
    θ(t,x,y)=Θ(t,x) on Γ(h) for a.a. (τ,x)∈ (0,T')×Ω, y∈Γ(h) }.


Assuming that the height function h∈ L^∞((0,T^*)×Ω) describes the moving boundaries and taking any function (Φ,ϕ)∈𝒳[h](T^*), we introduce the weak form for the PDE system (<ref>):


    ∫_Ω∂_t(C(h)Θ)Φx+∫_Ω∫_Y(h)∂_tθϕyx+∫_ΩK(h)∇Θ·∇Φx
    +∫_Ω∫_Y(h)κ∇_yθ·∇_yϕyx
    
    =∫_Ω(F- L(h)∂_th)Φx
    +∫_Ω∫_Y(h)fϕyx


Please note that the q(h)[θ] term in <ref> cancels with the micro boundary integral due to ϕ=Φ on Γ(h).
With that in mind, we can now define a solution of the two-scale heat problem in moving coordinates:


We say that (h,Θ,θ) is a local-in-time weak solution in moving coordinates, when there is a time horizon S^*=(0,T^*), such that


    
  (i) The domain changes are characterized by the height function h∈ W^1,∞(S^*;L^∞(Ω)) which satisfy
    
    h_L^∞(S^*×Ω)≤ a^*.

    
  (ii) The velocity of the domain evolution is driven by the temperature:
    
    ∂_th=Θ-Θ^ref in L^2(S^*×Ω).

    
  (iii) The temperatures (Θ,θ)∈𝒳[h](T^*) with ∂_t(Θ,θ)∈ L^2(S^*×Ω)× L^2(Q[h](T^*)) satisfy
    
    
    ∫_S∫_Ω∂_t(C(h)Θ)Φx+∫_Ω∫_Y(h)∂_tθϕyx+∫_ΩK(h)∇Θ·∇Φx
        +∫_Ω∫_Y(h)κ∇_yθ·∇_yϕyx
    
        =∫_Ω(F- L(h)∂_th)Φx
        +∫_Ω∫_Y(h)fϕyx

    for all (Φ,ϕ)∈𝒳[h].
    
  (iv) The initial conditions are satisfied: (Θ(0),θ(0), h(0))=(Θ_0,θ_0,0).



This solution concept, however, is inconvenient for the mathematical analyses due to the changing domains.
We use the concept of the Hanzawa transformation to reformulate this problem in the fixed domain Y_0=Y(0).
The precise definition and properties of this transformation are introduced in <ref> and specifically <ref>.

Now, for any 0<T^*<T and height function h∈ W^1,∞(0,T^*;L^∞(Ω)) with |h|≤ a^*, let s_h∈ W^1,∞(0,T^*;L^∞(Ω;C^2(Y))) be the corresponding Hanwaza transformation.

















We introduce the Jacobian matrix F_h=D_ys_h, the Jacobian determinant J_h= F_h, and the velocity ∂_ts_h. 
Also, we denote the push-forward S_h and the pull-back S_h^-1 for any sufficiently regular function fY→ defined via S_hf=f∘ s_h and S_h^-1f=f∘ s_h^-1.
Finally, we introduce the transformed quantities 


    c(h)   =J_h,       κ(h)   =J_hF_h^-1κF_h^-T,

    f(h)   =J_hS_hf,       v(h)   =J_hF_h^-1∂_ts_h.  

With this transformation, we can alternatively formulate the problem in fixed coordinates where we use 𝒳_0(T^*):=𝒳[0](T^*):




We say that (h,Θ,ϑ) is a local-in-time weak solution in moving coordinates, when there is a time horizon S^*=(0,T^*), such that:


    
  (i) The domain changes are characterized via the height function h∈ W^1,∞(S^*;L^∞(Ω)) which satisfy
    
    h_L^∞(S^*×Ω)≤ a^*.

    
  (ii) The velocity is driven by the temperature:
    
    ∂_th=Θ-Θ^ref in L^2(S^*×Ω).

    
  (iii) The temperatures (Θ,ϑ)∈𝒳_0(T^*) with ∂_t(Θ,ϑ)∈ L^2(S^*×Ω)× L^2(S^*×Ω× Y_0) satisfy
    
    
    ∫_S∫_Ω∂_t(C(h)Θ)Φx+∫_Ω∫_Y_0∂_t(c(h)ϑ)ϕyx+∫_ΩK(h)∇Θ·∇Φx
        +∫_Ω∫_Y_0κ(h)∇_yϑ·∇_yϕyx
    
        +∫_Ω∫_Y_0ϑ v(h)·∇_yϕyx
        =∫_Ω(F- L(h)∂_th)Φx
        +∫_Ω∫_Y_0f(h)ϕyx

    for all (Φ,ϕ)∈𝒳_0.
    
  (iv) The initial conditions are satisfied: (Θ(0),ϑ(0),h)=(Θ_0,ϑ_0,0).



It is relatively straightforward to show that (h,Θ,θ) is a solution in the sense of <ref> if and only if (h,Θ,S_h^-1ϑ) is a solution in the sense of <ref> (see, e.g., <cit.>).
In <ref>, we will work with <ref> to show the existence of local-in-time weak solutions.







§ ANALYSIS OF THE MOVING BOUNDARY PROBLEM

In this section, we investigate the two-scale problem with changing micro-domains and show that there are weak solutions (h,Θ,ϑ) in the sense of <ref> over some possibly small time interval (0,T^*)⊂ S.
We also establish uniqueness in case of higher regularity and long-time existence under certain smallness conditions.
Our strategy and outline of this section is the following:


    
  (i) In <ref>, we introduce the Hanzawa transformation (<ref>) and establish some uniform positivity and Lipschitz estimates (<ref>).
    
  (ii) We go on, in <ref>, to look at the linearized problem by solving the heat problem for a prescribed height function and establishing a-priori estimates (<ref>).
    
  (iii) Finally, in <ref>, we employ a Schauder fixed-point argument to prove the existence of weak solutions (<ref>).
    We go on to show uniqueness (<ref>) and long-time existence (<ref>) when certain conditions are met.    




 §.§ Hanzawa transformation and analysis of the cell problems

In the following, we present some auxiliary results regarding the Hanzawa transformation which are necessary to estimate and control the impact of the changing geometry on the overall system.
For more details regarding this type of setup, including the case of nonuniform growth, we refer to <cit.>.

As a C^3-interface, Γ_0=∂ Y_0 has a tubular neighborhood U_Γ⊂⊂ Y with tubular radius a>0.
In U_Γ, every point is uniquely characterized by its projection onto Γ_0 (via P U_Γ→Γ_0), and its signed distance (via d U_Γ→(-a,a))[Here, we follow the convention that negative distances correspond to points inside Y_0.].
Moreover, the function


    ΛΓ_0×(-a,a)→ U_Γ,  Λ(γ,h)=γ+hn(γ)


is a C^2-isomorphism with inverse


    Λ^-1 U_Γ→Γ_0×(-a,a),    Λ^-1(y)=(P(y),d(y)).


We set a^*=a/8. 
For h∈[-a^*,a^*], we introduce


    Γ(h)={γ+h n(γ) :  γ∈Γ_0}, Y(h)=(Γ(h)).


Clearly, Γ(0)=Γ_0, Y(0)=Y_0, as well as ∂ Y(h)=Γ(h).
In the following, let χ→[0,1] be a smooth cut-off function that satisfies


(i)    χ(r)=1  for r∈(-1/2,1/2),        (ii)    χ(r)=0  for r∉(-1,1),

(iii)    sign(r)χ'(r)≤0,     (iv)    |χ'(r)|≤5/2

and introduce ψU_Γ→^d given by ψ(y)=n(P(y))χ(d(y)/a).
Note that ψ∈ C^2(U) with |D_yψ(y)|<4/a.[This follows as the mean curvature is bounded by a^-1.]
Then, the so-called Hanzawa transform s_hY→Y defined via


    s_h(y)=
         y+hψ(y)     y∈ U_Γ,
    
         y     y∈Y∖ U_Γ


is a C^2-transformation satisfying s_h(Γ)=Γ(h) and |D_ys_h|, |D_ys_h^-1|≤ 2 for all h∈[-a^*,a^*].
Please note that due to h being constant this is all a special case of the more general results for the Hanzawa transformation as outlined in <cit.>.
In the following, we denote the Jacobian and the Jacobian derivative by F_h=D_ys_h and J_h= F_h, respectively.


    
        
  (i) There are 0<c<C independent of h∈[-a^*,a^*] and y∈Y such that
        
        
    c≤ J_h(y)≤ C,   c|ξ|^2≤ F^-1_h(y)F^-T_h(y)ξ·ξ≤ C|ξ|^2   (ξ∈^d).

        
  (ii) Let h_1,h_2∈ [-a^*,a^*] and let s_i=s_h_i, F_i=F_h_i, J_i=J_h_i denote the corresponding transformation functions.
        Then,
        
    F_1-F_2_L^∞(Y)^d× d+F_1^-1-F_2^-1_L^∞(Y)^d× d
            +J_1-J_2_L^∞(Y)≤ C|h_1-h_2|

        
        where C>0 independent of h.
    


  (i). These bounds exist as |F_h(y)|, |F_h^-1(y)|≤ 2 uniformly and independently of h∈[-a^*,a^*].
  
  (ii). Outside the tubular neighborhood U_Γ, the transformation is trivial, i.e., F_i= and J_i=1, so we only have to look at y∈ U.
  We immediately get
  
  
    |F_1(y)-F_2(y)|=|(h_1-h_2)D_yψ(y)|≤ C|h_1-h_2|.

  
  We already know that the F_i are invertible with
  
  
    F_i^-1(y)=(+h_iD_yψ(y))^-1

  
  from where
  
  
    F_i^-1(y)=-h_iD_yψ(y)F_i^-1(y)

  follows.
  As a consequence,
  
  
    F_1^-1(y)-F_2^-1(y)=D_yψ(y)(h_2F_2^-1(y)-h_1F_1^-1(y))

  
  or,
  
    |F_1^-1(y)-F_2^-1(y)|≤ C|h_1-h_2|+|h_2D_yψ(y)||F_1^-1(y)-F_2^-1(y)|.

  
  Since |h_2D_yψ(y)|≤1/2, the Lipschitz estimate follows.
  Finally, for the determinant a second order Taylor approximation yields
  
  
    J_i(y)=(+h_i D_yψ(y))=1+h_i((D_y ψ(y))+𝒪(h_i))

  
  showing the Lipschitz estimate.


In the following, for any h∈[-a^*,a^*], we note that


    S_h H^1(Y∖ Y_0)→ H^1(Y∖ Y(h)),      S_h^-1 H^1(Y∖ Y(h))→ H^1(Y∖ Y_0)).


Based on the estimates in <ref>, there are constants c,C>0 such that


    cS_hϕ_H^1(Y∖ Y(h))≤ϕ_H^1(Y∖ Y_0)≤ CS_hϕ_H^1(Y∖ Y(h))     (ϕ∈ H^1(Y∖ Y_0)).




In the next step, we establish how the different coefficients depend on the height parameter and the corresponding transformation.
To that end, let h∈[-a^*,a^*] be given.
It is clear, that h↦|Y(h)| and h↦|Γ(h)| are C^1 and therefore also Lipschitz. 
We now look at the cell problem given via <ref>:



    For every h∈[-a^*,a^*], there is a unique zero-average solution ξ_j,h∈ H_#(Y∖ Y(h)), j=1,…,d, satisfying    
    
    
    ∫_Y∖Y(h)𝒦(∇_yξ_j,h+e_j)·∇_yϕy=0
    
    
    for all ϕ∈ H^1_#(Y∖ Y(h)) as well as ∇ξ_j,h_L^2(Y∖ Y(h))≤ C where C>0 is independent of h.
    
    Moreover, for h_1,h_2∈[-a^*,a^*], we have
    
    
    |∫_Y∖ Y(h_1)𝒦∇ξ_j,h_1· e_iy-∫_Y∖ Y(h_2)𝒦∇ξ_j,h_2· e_iy|≤ C |h_1-h_2|.



    The existence of a unique zero-average solution is standard and follows via Lax-Milgram <cit.>.
    Let h∈[-a^*,a^*].
    By testing with φ = ξ_j,h and by assumption (A1) we obtain the desired estimate
    
    c ∇ξ_j,h_L^2(Y∖ Y(h))^2 ≤ C e_j_L^2(Y∖ Y(h))∇ξ_j,h_L^2(Y∖ Y(h)) = C √(|Y∖ Y(h)|)_≤ 1∇ξ_j,h_L^2(Y∖ Y(h)).

    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    

    Now, let h_1,h_2∈[-a^*,a^*], take the difference of their respective weak forms, and test with ξ̅_j:=S_1^-1ξ_j,h_1-S_2^-1ξ_j,h_2:
    
    
    ∫_Y∖ Y_0J_1F_1^-1𝒦F^-T_1∇_yξ̅_j·∇_yξ̅_j
    
          =∫_Γ_0(J_1-J_2)e_j· nξ̅_jy
          +∫_Y∖ Y_0(J_2F_2^-1𝒦F^-T_2-J_1F_1^-1𝒦F^-T_1)∇_y(S_2^-1ξ_j,h_2)·∇_yξ̅_jy.

    
    With the uniform positivity of J_1F^-T_1 and the Lipschitz continuity of J_h and F_h^-T (<ref>), we estimate
    
    
    c∇_yξ̅_j^2_L^2(Y∖ Y_0)
    ≤ C|h_1-h_2|(ξ̅_j_L^1(Γ_0)+∇_y(S_2^-1ξ_j,h_2)_L^2(Y∖ Y_0)∇_yξ̅_j_L^2(Y∖ Y_0)).

    
    Via Poincare's inequality and (<ref>), we therefore get
    
    
    ξ̅_j_H^1(Y∖ Y_0)≤ C|h_1-h_2|.

    
    From here, we can conclude the desired Lipschitz estimate via
    
    
    ∫_Y∖ Y(h_1)𝒦∇ξ_j,h_1· e_iy
        -∫_Y∖ Y(h_2)𝒦∇ξ_j,h_2· e_iy
    
        =
        ∫_Y∖ Y_0(J_1F_1^-1𝒦 F^-T_1∇_y(S_1ξ_j,h_1)-J_2F_2^-1𝒦 F^-T_2∇_y(S_2ξ_j,h_2))· e_iy

    
    and <ref>.






    
  (i)  Let h_1,h_2∈[-a^*,a^*].
    There is a C>0 independent of h_1,h_2 such that
    
    
    |C(h_1)-C(h_2)|+|K(h_1)-K(h_2)|+| L(h_1)- L(h_2)|≤ C|h_1-h_2|.

    
  (ii)
    There is c>0 such that, for all h∈[-a^*,a^*] and all ρ∈^d,
    
    |C(h)|≥ c,   K(h)ρ·ρ≥ c|ρ|^2.




    (i). This is easy to check for C and L by directly utilizing the transformation <ref> and the Lipschitz estimates in <ref>.

    For the heat conductivity, we rely on additional estimates for the cell problem given in <ref>:
    Their difference can be estimated by
    
    
    K(h_1)_i,j-K(h_2)_i,j=∫_Y∖ Y(h_1)𝒦∇_yξ_j,h_1· e_iy
        -∫_Y∖ Y(h_2)𝒦∇_yξ_j,h_2· e_iy
    
        +∫_Y∖ Y(h_1)𝒦_ijy
        -∫_Y∖ Y(h_2)𝒦_ijy

    
    where we can apply <ref> to get the Lipschitz estimate.

    (ii). Trivially true for C(h)=1-|Y(h)| as (Γ_0,∂ Y)≥ 2a^*.
    For the macroscopic conductivity matrix, positivity can be shown easily for a fixed h with standard techniques (e.g., <cit.>, i.e., K(h)ρ·ρ≥ c_h|ρ|^2.
    Since K is continuous in h (shown in (i)) and [-a^*,a^*] is compact, uniform positivity follows with c=min_|h|≤ a^*c_h>0.    






 §.§ Analysis for prescribed height function

We take a look at the weak form for the heat system, i.e., <ref>, for a prescribed height function h and establish well-posedness and some further auxiliary results for this linearized problem.
For some arbitrary and later to be fixed M>0, we introduce the space of admissible height functions whose time derivatives are bounded by M:


    ℋ_M(t)={h∈ W^1,∞((0,t); L^∞(Ω)) :  h(0,x)=0, |∂_th(τ,x)|≤ M  for a.a. τ∈(0,t) and x∈Ω}.


Please note that every h∈ℋ_M(t) satisfies |h|≤ a^* as long as t≤a^*/M=:T_M; therefore the results from <ref> apply pointwise almost everywhere.
We set S_M=(0,T_M).


Let M>0 and h∈ℋ_M(T_M).
There exists a unique weak solution (Θ,ϑ)∈𝒳_0 with ∂_t(Θ,ϑ)∈ L^2(S_M×Ω)× L^2(S_M×Ω× Y_0) satisfying


    ∫_Ω∂_t(C(h)Θ)Φx+∫_Ω∫_Y_0∂_t(c(h)ϑ)ϕyx+∫_ΩK(h)∇Θ·∇Φx
        +∫_Ω∫_Y_0κ(h)∇_yϑ·∇_yϕyx
    
        +∫_Ω∫_Y_0ϑ v(h)·∇ϕyx
        =∫_Ω(F- L(h)∂_th)Φx
        +∫_Ω∫_Y_0f(h)ϕyx


for all (Φ,ϕ)∈𝒳_0 and almost all t∈ S_M.
In addition, there is C>0, which is independent of h and M, such that


    Θ_L^∞(S_M;L^2(Ω))+Θ_L^2(S_M;H^1(Ω))+ϑ_L^∞(S_M;L^2(Ω× Y_0))+ϑ_L^2(S_M×Ω;H^1(Y_0))≤ Cexp(CM)


Moreover, there is C>0 such that for all h∈ℋ_M(T_M):


    Θ_L^∞((0,t)×Ω) + ϑ_L^∞((0,t)×Ω× Y_0)≤ C(1+M^2√(t))




    For any such height function, all coefficients are well-defined and essentially bounded.
    Moreover, c(h) and C(h) are positive and κ(h) and K(h) are positive definite (see <ref>).
    As a consequence, this is a linear implicit evolution equation in the form of
    
    
    (ℬ(t)u(t))+𝒜(t)u(t)=f(t)

    
    which under the given regularity admits a unique solution.
    We refer to <cit.>.

    Regarding the estimates: First of all, the advective velocity v(h)=J_hF_h^-1∂_ts_h and the time derivatives of the coefficients c(h) and C(h) can be estimated by
    
    |v(h)|+|∂_tc(h)|+|∂_tC(h)|≤ C|∂_th|≤ CM.

    
    
    Now, testing with (Φ,ϕ)=(Θ,ϑ) shows:
    
    
    ∫_Ω1/2C(h)|Θ|^2x
        +∫_Ω∫_Y_01/2c(h)|ϑ|^2x
        +c∇Θ^2_L^2(Ω)
        +c∇_yϑ^2_L^2(Ω× Y_0)
    ≤ C∫_Ω M(|Θ|+|Θ|^2+∫_Y_0|ϑ|^2+|ϑ∇_yϑ|y)x
        +F_L^2(Ω)Θ_L^2(Ω)
        +f(h)_L^2(Ω× Y_0)ϑ_L^2(Ω× Y_0)

    
    for almost all t∈ S_M.
    Integrating over (0,t) for t∈ S_M and using standard energy techniques, we get 
    
    
    Θ(t)_L^2(Ω)^2
        +ϑ(t)_L^2(Ω× Y_0)^2
        +∫_0^t∇Θ^2_L^2(Ω)
        +∇_yϑ^2_L^2(Ω× Y_0)τ
    ≤ C∫_0^t∫_Ω M(|Θ|+|Θ|^2+(1+M)∫_Y_0|ϑ|^2y)xτ
    
        +∫_0^tF_L^2(Ω)Θ_L^2(Ω)
        +f(h)_L^2(Ω× Y_0)ϑ_L^2(Ω× Y_0)τ.

    
    On the first term on the right hand side, we can further estimate:
    
    
    ∫_0^t∫_Ω|∂_th|(|Θ|+|Θ|^2)xτ≤
        C∫_0^tM(Θ^2_L^2(Ω)+ 1 )τ

    
    As a result, we arrive at
    
    
    Θ(t)_L^2(Ω)^2
        +ϑ(t)_L^2(Ω× Y_0)^2
        +∫_0^t∇Θ^2_L^2(Ω)
        +∇_yϑ^2_L^2(Ω× Y_0)τ
    ≤ C(1+ ∫_0^t(1+M+M^2)(Θ^2_L^2(Ω)+ϑ_L^2(Ω× Y_0)^2)τ).

    
    With Grönwalls inequality, we are led to 
    
    
    Θ_L^∞(0,t;L^2(Ω))^2
        +ϑ_L^∞(0,t;L^2(Ω× Y_0))^2
        +∇Θ^2_L^2(0,t;L^2(Ω))
        +∇_yϑ^2_L^2(0,t;L^2(Ω× Y_0))
    ≤ Cexp(CtM^2).

    

    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    Regarding the boundedness of the solution, for k∈ with k≥max{Θ_0_L^∞(Ω),θ_0_L^∞(Ω× Y_0)}, we introduce the functions Θ^k∈ L^2(S_M;H^1(Ω)) and ϑ^k∈ L^2(S_M×Ω;H^1(Y_0)) via
    
    
    Θ^k=Θ-k    for Θ> k
    
                0   for |Θ|≤ k
    Θ+k    for Θ< -k
                ,    ϑ^k=ϑ-k    for ϑ> k
    
                0   for |ϑ|≤ k
    ϑ+k    for ϑ< -k
                ,

    
    as well as the sets
    
    
    A_Θ(t,k)   ={x∈Ω :  |Θ(τ,x)|≥ k},
     A_ϑ(t,x,k)   ={y∈  Y_0 :  |ϑ(τ,x,y)|≥ k}.

    
    Please note that ∂_tΘ^k=χ_A_Θ∂_tΘ, ∇Θ^k=χ_A_Θ∇Θ, and ΘΘ^k≥|Θ^k|^2 (similar for ϑ).
    With (Θ^k,ϑ^k) as a test function and the <ref>, standard energy estimates lead to (t∈ S_M)
    
    
    Θ^k(t)_L^2(Ω)^2+ϑ^k(t)^2_L^2(Ω× Y_0)
        +∇Θ^k^2_L^2((0,t)×Ω)
        +∇_yϑ^k^2_L^2((0,t)×Ω× Y_0)
    ≤ C∫_0^t∫_Ω(|∂_th||Θ||Θ^k|+|F- L(h)∂_th||Θ^k|)xτ
    
        +C∫_0^t∫_Ω∫_Y_0|∂_th||ϑ||ϑ^k|+|ϑ| |v(h)||∇_yϑ^k|
        +|f(h)||ϑ^k|yxτ

    
    where C>0 does not depend on h, k, or M.
    Using |Θ|≤ k+|Θ^k|, |ϑ|≤ k+|ϑ^k|, and |∂_th|≤ M, we estimate
    
    
    ∫_Ω|∂_th||Θ||Θ^k|x   ≤ 2M(Θ^k_L^2(Ω)^2+k^2|A_Θ(t,k)|),
    ∫_Y_0|∂_th||ϑ||ϑ^k|y   ≤ 2M(ϑ^k_L^2(Y_0)^2+k^2|A_ϑ(t,x,k)|).

    
    The latent heat term, we estimate via (with k≥max_h∈[-a^*,a^*]| L(h)|)
    
    
    ∫_Ω| L(h)∂_th||Θ^k|x≤M/2(Θ^k_L^2(Ω)^2+k^2|A_Θ(t,k)|).

    
    Looking at the convection term (v(h)≤ CM):
    
    
    ∫_Y_0|ϑ| |v(h)||∇_yϑ^k|y≤δ∇_yϑ^k_L^2(Y_0)^2+M^2/4δ(ϑ^k_L^2(Y_0)^2+k^2|A_ϑ(τ,x,k)|).

    
    for any δ>0.
    
    We set
    
    
    k^*=max{Θ_0_L^∞(Ω),θ_0_L^∞(Ω× Y_0),F_L^∞((0,T_M)×Ω), f_L^∞((0,T_M)×Ω× Y_0),max_h∈[-a,a]| L(h)|}

    
    and conclude, for all k≥ k^* and assuming M≥1,
    
    
    Θ^k(t)^2_L^2(Ω)+ϑ^k(t)^2_L^2(Ω× Y_0)
        +∇Θ^k^2_L^2((0,t)×Ω)
        +∇_yϑ^k^2_L^2((0,t)×Ω× Y_0)
    ≤ CM^2(θ^k_L^2((0,t)×Ω)^2+    
        ϑ^k_L^2((0,t)×Ω× Y_0)^2
        +k^2∫_0^t(|A_θ(τ,k)|+∫_Ω|A_ϑ(τ,x,k)|x)τ).

    

    
    Thus, Gronwall's inequality allows us to conclude for 0<t<T_M
    
    
    Θ^k_L^∞(0,t;L^2(Ω))^2+ϑ^k^2_L^∞(0,t;L^2(Ω× Y_0))
        +∇Θ^k_L^2(0,t×Ω)^2
        +∇_yϑ^k^2_L^2(0,t×Ω× Y_0)
    ≤ Cexp(tM^2)M^2k^2∫_0^t(|A_θ(τ,k)|+∫_Ω|A_ϑ(τ,x,k)|x)τ.

    
    where the constant C>0 is still independent of k, h, and M.
    From here, we apply <cit.> to conclude the existence of k_M, which is independent of h but does depend on M, such that |A_θ(τ,k_M)|,|A_ϑ(τ,x,k_M)|=0 almost everywhere.
    
    More precisely
    
    
    Θ_L^∞((0,t)×Ω)+ϑ_L^∞((0,t)×Ω)≤ 2k^*(1+Cexp(5/4M^2t)M^2 √(t))   (t∈(0,T_M))

    
    Moreover, tM^2≤ a^* for 0≤ t≤√(T_M).
    Over this time interval, we have
    
    
    Θ_L^∞((0,t)×Ω)+ϑ_L^∞((0,t)×Ω)≤ 2k^*(1+CM^2√(t))    (t∈ (0,√(T_M)).

    This implies uniformity in M of the L^∞ estimates for t≤a^*/M^4.
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    


























































 §.§ Fixed-point argument and long time behavior

We first note that <ref> induces a solution map ℒ:=(ℒ_1(h),ℒ_2(h))ℋ_M(T_M)→𝒳_0(T_M) for every M>0.
Here, ℒ_1(h)∈ L^2(S_M×Ω) denotes the macroscopic temperature and ℒ_2(h)∈ L^2(S_M×Ω× Y_0) the microscopic temperature, respectively.
Also, with <ref>, it holds


    C_L:=sup_h∈ℋ_M(t)ℒ_1(h)_L^∞((0,t)×Ω)<∞


where, importantly, C_L does not depend on M as long as t∈(0,a^*/M^4).
Setting


    𝒴(t):={(Θ,ϑ)∈𝒳_0(t) :  Θ_L^∞((0,t)×Ω)≤ C_L}⊂𝒳(t),

and T^*_M= a^*/M^-4, it is clear that ℒ[ℋ_M(T_M^*)]⊂𝒴(T_M^*).
Let (Θ,ϑ)∈𝒴(T_M^*).
To get to a fixed-point argument, we must be able to construct a new height function h̃ based on the macroscopic temperature Θ.
We introduce h̃(t,x)=∫_0^tΘ(τ,x)-Θ^refτ and note that h̃∈ W^1,∞((0,T_M^*);L^∞(Ω)) via the regularity of Θ.
Moreover, 

    |∂_th̃(t,x)|=|Θ(t,x)-Θ^ref|≤ C_L+|Θ^ref|

for almost all (t,x)∈ (0,T_M^*)×Ω.
As a consequence, h̃∈ℋ_M(T_M) as long as M≥ C_L+|Θ^ref|.
We fix M^*=C_l+|Θ^ref|, set T^*:=T_M^* and S^*=(0,T_M^*), and introduce


    𝒯𝒴(T^*)→𝒴(T^*),  𝒯(Θ,ϑ)=ℒ(∫_0^tΘ(τ,x)-Θ^refτ).




    There is at least one solution (h,Θ,ϑ)∈ℋ_M^*(T^*)×𝒳_0(T^*) for the moving boundary problem in fixed coordinates in the sense of <ref> over the time interval S^*=(0,T^*).


We wirst note, that (h,Θ,ϑ)∈ℋ_M^*(T^*)×𝒳_0(T^*) is a solution if, and only if, 𝒯(Θ,ϑ)=(Θ,ϑ) and h(t,x)=∫_0^tΘ(τ,x)-Θ^refτ.

(i) Continuity of fixed-point operator over L^2. Let (Θ_n,ϑ_n),(Θ_∞,ϑ_∞)∈𝒴(T^*) such that (Θ_n,ϑ_n)→(Θ_∞,ϑ_∞) in L^2(S^*×Ω)× L^2(S^*×Ω× Y_0).
Let h_n,h_∞∈ℋ_M^*(T^*) denote the corresponding height functions.
Due to ∂_th_k=Θ_k (k∈_∞), we immediately have ∂_th_n→∂_t h_∞ and h_n→ h_∞ in L^2(S^*×Ω).
Now, we set (Θ_k,ϑ_k):=𝒯(Θ_k,ϑ_k)∈𝒴(T^*) for k∈_∞.
Utilizing the a priori estimates established with <ref>, there exist (Θ,ϑ)∈𝒴(T^*) such that, at least along a subsequence, (Θ_n,ϑ_n)→ (Θ,ϑ) weakly in 𝒳_0(T^*) and, with Lions-Aubin, strongly in L^2.
If we can show that (Θ,ϑ)=(Θ_∞,ϑ_∞) and that the full sequence converges, continuity holds.


Looking at the most critical terms in the weak forms, namely the heat conductivities, we have the limits

    lim_j→∞∫_ΩK(h_n_j)∇Θ_n_j·∇Φx   =∫_ΩK(h_∞)∇Θ·∇Φx,
    lim_j→∞∫_Ω∫_Y_0κ(h_n_j)∇_yϑ_n_j·∇_yϕyx   =∫_Ω∫_Y_0κ(h)∇_yϑ·∇_yϕy


for all (Φ,φ)∈𝒳_0(T^*).
Here, we used the weak convergences of (∇Θ_n,∇_yϑ_n) along a subsequence n_j and the strong convergence of h_k together with the uniform bounds for K,κ.
After identifying all terms in the weak form in the same way, we can see that Θ=Θ_∞, which implies Θ_n_j→Θ_∞ strongly in L^2(S^*×Ω), similarly for ϑ̂_n_j.
Finally, since every subsequence (Θ_n,ϑ_n) has a converging subsequence which all converge to the same limit, the whole sequence converges implying L^2-continuity of 𝒯.

(ii) Existence of solutions. We showed that

    𝒯𝒴(T^*)⊂ L^2(S^*×Ω)× L^2(S^*×Ω× Y_0)→ L^2(S^*×Ω)× L^2(S^*×Ω× Y_0)


is continuous.
Moreover 𝒯[𝒴(T^*)] is compact in  L^2(S^*×Ω× Y_0) via Lions-Aubin and 𝒴(T^*) is closed and convex.
With Schauders fixed point theorem, there must therefore be at least one fixed point (Θ,ϑ)∈𝒴(T^*) satisfying 𝒯(Θ,ϑ)=(Θ,ϑ).



    Please note that all the arguments hold true when starting with a non-trivial height function, i.e., when h(0)=h_0 for some h_0∈ L^∞(Ω) with c_0:=h_0_L^∞(Ω)<a^*.
    It does however lead to a shorter time horizon, as now M^*=c_0+C_L+|Θ^ref| and T^*=(a^*-c_0)/(M^*)^4.





This argument does not guarantee uniqueness, however, this follows under slightly higher regularity via Lipschitz arguments (see <ref> below).
<Ref> ensures the existence of solutions for the moving boundary problem in fixed-coordinates, but using the push forward S_h the corresponding solutions in moving coordinates are given via (Θ,S_hϑ, h).



If 𝒯 has a fixed point (Θ,ϑ)∈𝒴(T^*) with the additional regularity ∂_tΘ, ∇Θ∈ L^∞((0,T^*)×Ω) and ∂_tϑ, ∇_yϑ∈ L^∞((0,T^*)×Ω;L^2(Y_0)).
Then, (Θ,ϑ) is the only fixed point.


    Let (Θ_i,ϑ_i)∈𝒴(T^*) (i=1,2) be fixed-points of 𝒯 with the higher regularity for i=2 with corresponding height functions h_i∈ℋ_M^*(T^*).
    Taking the difference of their weak forms and testing with (Θ̅,ϑ̅):=(Θ_1-Θ_2,ϑ_1-ϑ_2):
    
    
    ∫_Ω∂_t(C(h_1)Θ_1-C(h_2)Θ_2)Θ̅x_I_1+∫_Ω∫_Y_0∂_t(c(h_1)ϑ_1-c(h_2)ϑ_2)ϑ̅yx_I_2
    
        +∫_Ω(K(h_1)∇Θ_1-K(h_2)∇Θ_2)·∇Θ̅x_I_3
        +∫_Ω∫_Y_0(κ(h_1)∇_yϑ_1-κ(h_2)∇_yϑ_2)·∇_yϑ̅yx_I_4
    
        +∫_Ω∫_Y_0(ϑ_1 v(h_1)-ϑ_2 v(h_2))·∇_yϑ̅yx_I_5
    
        =∫_Ω( L(h_2)∂_th_2- L(h_1)∂_th_1)Θ̅x_I_6
        +∫_Ω∫_Y_0(f(h_1)-f(h_2))ϑ̅yx_I_7.

    
    Using the estimates from <ref>, we can now estimate the individual terms.
    Starting with the macroscopic time derivative, we get
    
    
    I_1   =
        ∫_Ω∂_t(C(h_1)Θ̅)Θ̅x+∫_Ω∂_t[(C(h_1)-C(h_2))Θ_2]Θ̅x
       =1/2∫_Ω C(h_1)Θ̅^2x
        -1/2∫_Ω∂_tC(h_1)Θ̅^2x
        +∫_Ω∂_t[(C(h_1)-C(h_2))Θ_2]Θ̅x.

    
    The second and third term on the right hand side satisfy
    
    
    |1/2∫_Ω∂_tC(h_1)Θ̅^2x|
        +|∫_Ω∂_t[(C(h_1)-C(h_2))Θ_2]Θ̅x|
        ≤ CΘ̅_L^2(Ω)(Θ̅^2_L^2(Ω)+∂_th̅_L^2(Ω)
        +h̅_L^2(Ω)).

    
    Similarly, we get for the microscopic time derivative:
    
    
    I_2≥1/2∫_Ω∫_Y_0c(h_1)ϑ̅^2yx
        -Cϑ̅_L^2(Ω× Y_0)(ϑ̅_L^2(Ω× Y_0)+∂_th̅_L^2(Ω)+h̅_L^2(Ω))

    
    Looking at the macroscopic heat conductivity term:
    
    
    I_3   =∫_ΩK(h_1)∇Θ̅·∇Θ̅x+∫_Ω(K(h_1)-K(h_2))∇Θ_2·∇Θ̅x,
       ≥ c∇Θ̅_L^2(Ω)^2-Ch̅_L^2(Ω)∇Θ̅_L^2(Ω)

    
    The same type of estimate also holds for the microscopic heat conductivity term I_4:
    
    
    I_4   =∫_Ω∫_Y_0κ(h_1)∇_yϑ̅·∇_yϑ̅yx
        +∫_Ω∫_Y_0(κ(h_1)-κ(h_2))∇_yϑ_2·∇_yϑ̅yx
       ≥ c∇_yϑ̅_L^2(Ω× Y_0)^2
        -Ch̅_L^2(Ω)∇_yϑ̅_L^2(Ω× Y_0)

    
    The convective part on the microscale can be estimated as
    
    
    |I_5|   ≤ C(ϑ̅_L^2(Ω× Y_0)∇_yϑ̅_L^2(Ω× Y_0)+h̅_L^2(Ω)∇_yϑ̅_L^2(Ω× Y_0)),

    
    and the macroscopic heat production via
    
    |I_6|   ≤ CΘ̅_L^2(Ω)(h̅_L^2(Ω)
        +∂_th̅_L^2(Ω))

    For I_7, we use the uniform Lipschitz continuity of f to estimate
    
    |I_7|≤ Ch̅_L^2(Ω)ϑ̅_L^2(Ω;L^1(Y)).

    
    Summarizing and sorting these individual estimates while subsuming the gradient differences via Young’s inequality, we get
    
    
    ∫_Ω C(h_1)Θ̅^2x+∫_Ω∫_Y_0c(h_1)ϑ̅^2yx
            +∇θ̅_L^2(Ω^2+∇_yϑ̅_L^2(Ω× Y_0)^2
    ≤ C(Θ̅^2_L^2(Ω)+ϑ̅^2_L^2(Ω× Y_0)+h̅^2_L^2(Ω)+∂_th̅^2_L^2(Ω)).

    
    Integrating over any time interval (0,t) and keeping in mind the uniform positivity of C(h_1) and c(h_1), we are led to
    
    
    Θ̅(t)^2_L^2(Ω)+ϑ̅(t)_L^2(Ω× Y_0)
            +∇θ̅_L^2((0,t)×Ω^2+∇_yϑ̅_L^2((0,t)×Ω× Y_0)^2
    ≤ C∫_0^t(Θ̅^2_L^2(Ω)+ϑ̅^2_L^2(Ω× Y_0)+h̅^2_L^2(Ω)+∂_th̅^2_L^2(Ω))τ.

    
    Now, looking at the terms with the height function, we first note that ∂_th̅=Θ.
    Similarly, we estimate 
    
    
    ∫_0^th̅(τ)^2_L^2(Ω)τ≤∫_0^t(∫_Ω∫_0^τ|Θ̅(s,x)|^2sx)^2τ
        =t∫_0^tΘ̅^2_L^2((0,τ)×Ω)τ≤ t^2Θ̅^2_L^2((0,t)×Ω).

    
    Applying Grönwall's inequality to <ref>, we conclude Θ̅=0, ϑ̅=0 which also implies h̅=0.    


The solution given by <ref> is only local-in-time, which is expected as the Hanzawa transformation can only ever track changes inside the tubular neighborhoods.
In the following lemma, we show that there are no other factors limiting the time horizon. 
This also means that in some very specific scenarios, i.e., whenever the data is very well prepared to ensure that the growth/shrinkage of the cells is controlled, global existence holds.
To that end, we introduce the supremal time horizon


    T_sup:=sup{t∈ :  there exists a solution (Θ,ϑ,h) in the sense of <ref>}.



    Let T∈, let (Θ,ϑ,h)∈ X_0(T)×ℋ_M^*(T) be a solution in the sense of <ref>, and let h_L^∞((0,T)×Ω)<a^*.
    Then, the solution can be extended, i.e., T_sup>T.

    Moreover, if h_L^∞((0,T')×Ω)<a^* for all 0<T'<T_sup, then the solution can be extended globally, i.e, T_sup=∞.


Let T∈ and let (Θ,ϑ,h)∈𝒳_0(T)×ℋ_M^*(T) be a solution with h_L^∞((0,T)×Ω)=a^*- for some >0.
Now, taking Y(a^*-) and Γ(a^*-) as the initial geometry and (Θ(T),ϑ(T)) as initial conditions, all results from this section can be applied.
In this case, we do not have a uniform initial geometry, i.e., the height function is not uniformly 0, but this does not pose problems (see <ref>).
Therefore, there is a time horizon T_>0 and a solution (Θ',ϑ',h') on (0,T_). 
Gluing (Θ',ϑ',h') to (Θ,ϑ,h) at time T results in a solution on (0,T+T_).

For the second proposition, assume that h_L^∞((0,T')×Ω)<a^* for all T'<T_sup.
There are two cases:[The limit must exist as the norm is monotonically increasing and bounded.]


    (i) lim_T'→ T_suph_L^∞((0,T')×Ω)=a-<a^*,   (ii) lim_T'→ T_suph_L^∞((0,T')×Ω)=a^*.


In the first case, T_sup=∞ because otherwise we could extend via the first part (choose T'<T_sup such that T_sup<T'+T_).
In the second case, if T_sup is finite the solution can not be extended to T_sup (since we assumed h_L^∞((0,T')×Ω)<a^*).

To show, on the contrary, that it can be extended, we first glue all extensions together to get a height function h∈ L^∞((0,T_sup)×Ω) which automatically satisfies h_L^∞((0,T_sup)×Ω)=a^*.
Similarly, we get the temperatures (Θ,ϑ).
Now, (Θ,ϑ,h) is a solution on (0,T') for all 0<T'<T_sup.
More specifically, it satisfies all conditions in <ref> for all smaller time intervals.
Since none of the estimates for (Θ,ϑ,h) blow-up for T'→ T_sup (note that h_L^∞((0,T_sup)×Ω)≤ a^* and T_sup<∞), if follows that (Θ,ϑ,h)∈𝒳(T_sup)×ℋ(T_sup) is a solution over (0,T_sup).
As a consequence, T_sup=∞.










§ NUMERICAL ANALYSIS

From a numerical perspective, the present model is quite challenging and computationally expensive to solve due to the different scales, changing domains, and non-linear coupling.
In this section, we introduce and investigate a precomputing strategy together with a semi-implicit time-stepping scheme to resolve these problems.

First note, that independent of the underlying numerical algorithm, the effective conductivity K has to be recomputed multiple times throughout the simulation process: after each discrete time step and at every macroscopic node since K depends on the local value of the height function h.
This is quite expensive since differential equations (cell problems) have to be solved to compute K.
Seeing that in the underlying setup, the general cell topology does not change for different heights and only depends on h, it would be advantageous if one could precompute the conductivity K for a discrete number of height values in an offline phase and utilize these values in the simulation by interpolation.
We start by analyzing this idea and the influence of the solution in more detail.



 §.§ Precomputing strategy

Given N ∈, we evenly split the interval [-a^*, a^*] into N sub-intervals [h_i,h_i+1] with h_i = -a^* + 2a^*iN, for i=0,…,N.
We then compute for each h_i the corresponding conductivity K_i = K(h_i) given by <ref>.
We also set Δ h = 1/N in the following.[
    Depending on the expected behavior of K(h) with respect to h, it might be advantageous to utilize a nonuniform grid (see e.g., <cit.>).
    In our case, this is unnecessary as the general behavior is quite smooth and uniform (for comparison, see <ref>).
]

For any h ∈ [-a^*, a^*] the precomputed conductivity K_pre is then defined via an interpolation scheme. This raises the question of what interpolation scheme is suitable given the properties of K.
In line with the Lipschitz continuity of K (<ref>), a piecewise linear interpolation is a natural candidate, e.g.,


    K_lin(h) = (1 - h - h_i/Δ h)K_i +  h - h_i/Δ h K_i+1 ,   if  h ∈ [h_i, h_i+1),


This Lipschitz continuity immediately guarantees at least a first-order interpolation error:

    |K(h) - K_lin(h)| ≤ C Δ h/2.













We need higher regularity results for the effect conductivity to ensure higher order errors, which we investigate in the following lemma.


    For the effective conductivity it holds K ∈ C^1,1([-a^*, a^*], ^d× d).



    Recall, that the effective conductivity is given by
    
    
    K_ij(h) = ∫_Y∖ Y(h)𝒦(∇ξ_h,j + e_j) · e_iy
            = ∫_Y∖ Y_0𝒦 (F_h^-1∇(S_h^-1ξ_h,j) + e_j) · e_i J_hy.

    
    Therefore, by the product rule, we obtain the following expression assuming that all parts are well-defined:
    
    
    ∂_h K_ij(h) 
                   = 
                ∫_Y∖ Y_0 K (∂_h F_h^-1∇(S_h^-1ξ_h,j)) · e_i J_h y
                + 
                ∫_Y∖ Y_0 K (F_h^-1∂_h ∇(S_h^-1ξ_h,j)) · e_i J_hy
       + 
                ∫_Y∖ Y_0 K (F_h^-1∇(S_h^-1ξ_h,j) + e_j) · e_i ∂_h J_hy.

    
    Now, we investigate if all the derivatives on the right-hand side are well-defined and Lipschitz with respect to h.
    For the Matrix F_h^-1 = (+h D_yψ)^-1, it holds ∂_h F_h^-1 = -F_h^-1 D_yψ F_h^-1. 
    
    
    
    
    
    
    
    
    
    
    
    Therefore, ∂_h F_h^-1 is well-defined and also Lipschitz in h.
    For the determinate J_h = (+hD_y ψ), it holds
    
    
    ∂_h J_h    = ∂_h∑_σ∈ S_dsgn(σ) ∏_i=1^d (δ_i,σ(i) + h D_yψ_i, σ(i)) 
            
            = ∑_k=1^d  ∑_σ∈ S_dsgn(σ) ∏_i=1^d p_k, i, σ(i),

    
    where S_d denotes the set of all permutations in dimension d, sgn(σ)=±1 depending on if the permutation is even or odd, and
    
    
    p_k, i, σ(i) = 
            
                (δ_i,σ(i) + h D_yψ_i, σ(i)),     if  i≠ k, 
    
                D_yψ_i, σ(i),     if  i = k.

    
    This is a sum of multiple determinants (each time removing the identity and h from a different column of the matrix +hD_y ψ), hence ∂_h J_h is also well-defined and by the same arguments as in <ref> also Lipschitz with respect to h.

    Next, we investigate the derivative of the cell solution S_h^-1ξ_h, j with respect to h. To this end, we define the operator E_j:[-a^*, a^*] × V → V^*, where 
    
    
    V {v ∈ H^1_#(Y∖ Y_0) : ∫_Y ∖ Y_0 v y  = 0 },

    
    and the operator is given by
    
    
    E_j(h, v) = ∫_Y ∖ Y_0 J_h F^-1_h 𝒦 F^-T_h ∇_y v ·∇_y ∗y - ∫_Γ_0 J_h e_j · n ∗y.

    
    Here ∗ denotes a placeholder, for the argument of the functional.
    Note, that the cell solutions S_h^-1ξ_h, j are implicitly defined by E_j(h, S_h^-1ξ_h, j) = 0.
    We want to utilize the implicit function theorem <cit.> to show the existence of the Fréchet-derivative of the cell solution and find a characterization of it.
    Given that E_j is linear in v∈ V and F_h, J_h are differentiable in h∈[-a^*,a^*], it is straightforward to check that E_j is Fréchet-differentiable. The derivative ∂_u E_j(h, S_h^-1ξ_h, j) is a bounded linear operator from V → V^* and defined by 
    
    
    ∂_u E_j(h, S_h^-1ξ_h, j)[v] = ∫_Y ∖ Y_0 J_h F^-1_h 𝒦 F^-T_h ∇_y v ·∇_y ∗y.

    
    For v = 0 ∈ V it holds ∂_u E_j(h, S_h^-1ξ_h, j)[v]_V^* = 0 and for v ≠ 0
    
    
    ∂_u E_j(h, S_h^-1ξ_h, j)[v]_V^* 
            = sup_φ∈ V, φ≠ 0|∂_u E_j(h, S_h^-1ξ_h, j)[v] (φ) |/φ_H^1(Y∖ Y_0)≥|∂_u E_j(h, S_h^-1ξ_h, j)[v] (v) |/v _H^1(Y∖ Y_0)≥ C v _H^1(Y∖ Y_0),

    
    where in the last inequality we applied assumption (A1), <ref> (i) and the Poincaré inequality. Therefore, ∂_u E_j(h, S_h^-1ξ_h, j)[v]_V^*≥ C v _H^1(Y∖ Y_0) for all v ∈ V. With <cit.> we can conclude that ∂_u E_j(h, S_h^-1ξ_h, j) is injective and the image is closed, hence the inverse exists. Additionally, the inverse is also bounded. By the implicit function theorem, we obtain that the solution operator h ↦ S_h^-1ξ_h, j is well-defined and Fréchet-differentiable, where the derivative is given by
    
    
    ∂_u E_j(h, S_h^-1ξ_h, j) [∂_h (S_h^-1ξ_h, j)]    = -∂_h E_j(h, S_h^-1ξ_h, j) 
                
       ⇔   ∫_Y∖ Y_0 J_h F^-1_h 𝒦 F^-T_h ∇_y (∂_h (S_h^-1ξ_h, j)) ·∇_y∗y   = - ∫_Y∖ Y_0∂_h(J_h F^-1_h 𝒦 F^-T_h) ∇_y (S_h^-1ξ_h, j) ·∇_y∗y
             + ∫_Γ_0∂_h J_h e_j· n ∗y,

    
    understood as an element in V^*. With the same arguments as for the estimate in <ref> one can show that ∂_h (S_h^-1ξ_h, j) is Lipschitz in h as well. 
    Finally, we conclude that ∂_h K_ij is well-defined and Lipschitz in h since all parts of the right-hand side of <ref> are bounded and Lipschitz. 


The above regularity ensures a quadratic error order if we utilize linear interpolation:


    |K(h) - K_lin(h)| ≤ C (Δ h)^2  for all  h ∈ [-a^*, a^*].



    Since F_h^-1 and J_h are smooth regarding h, the implicit function theorem can also be used to show the existence of higher derivatives of the solution operator. Therefore, higher-order interpolation schemes can be applied as well. But we have to note, that the Lipschitz constant grows exponentially with 1/a^*. E.g., for the first derivative we see in the above proof, that ∂_h F_h^-1≤ C D_y ψ≤5C/a^*, which leads to
    
    
    |∂_h K(h + Δ h) - ∂_h K(h)| ≤ C (1 + 1/(a^*)^2) Δ h.

    
    This means that high-order interpolations may need a small step size Δ h to ensure the desired convergence order.



    By assumption (A1) we have that the underlying conduction matrix 𝒦 is not dependent on the space variable. For some applications this may not hold and 𝒦 may be a function of y. In this case the results of <ref> are still valid under the condition that 𝒦 is Lipschitz in regard to y. The property K ∈ C^1,1([-a^*, a^*], ^d× d) requires the additional assumption that 𝒦∈ C^1,1(Y∖ Y_0). 


In the following, K_pre stands for the precomputed conductivity, while ℰ_pre:= sup_|h|≤ a^*|K(h)-K_pre(h)| denotes the corresponding interpolation error.
In the case that K_pre = K_lin, or a higher order interpolation, K_pre is a Lipschitz continuous function in h and the results of <ref> can be transferred to ensure the existence of solutions for the precomputed system.
The important question is how the interpolation error propagates over time, this is analyzed in the following theorem.


    Let (h, Θ, ϑ) denote the solution with the exact conductivity and (h_pre, Θ_pre, ϑ_pre) the solution with the precomputed values K_pre. Set T^* = min(T^*, T_pre^*) such that both solutions are defined on (0, T^*). Assume that 
    ∂_t Θ_pre, ∇Θ_pre∈ L^∞((0, T^*) ×Ω), 
    ∂_t ϑ_pre, ∇_y ϑ_pre∈ L^∞((0, T^*) ×Ω× Y_0). Then, there exists a C > 0 (depending on T^*) such that
    
    
    Θ_L^∞((0, T^*); L^2(Ω)) + ϑ_L^∞((0, T^*); L^2(Ω× Y_0)) 
            + ∇Θ_L^2((0, T^*) ×Ω)
            + ∇ϑ_L^2((0, T^*) ×Ω× Y_0)≤ C ℰ_pre,

    
    where (h, Θ, ϑ)  (h-h_pre, Θ - Θ_pre, ϑ - ϑ_pre)_|(0, T^*) denotes the difference of both solutions.



    The proof follows the same idea as the uniqueness proof in <ref>. Here, we only have to consider the macroscopic conductivity term in more detail, all other terms can be estimated as previously shown. For the integral I_3 in the proof of <ref>, we have
    
    
    I_3 = 
            ∫_ΩK(h)∇Θ̅·∇Θ̅x
            + 
            ∫_Ω[K(h) - K_pre(h_pre)] ∇Θ_pre∇Θ̅x.

    
    The integral on the right can be further estimated by
    
    
    ∫_Ω   | [K(h) - K_pre(h_pre)] ∇Θ_pre·∇Θ̅|x
       ≤∫_Ω|(K(h)  - K(h_pre )) ∇Θ_pre·∇Θ̅|x
            + ∫_Ω|(K(h_pre ) - K_pre(h_pre)) ∇Θ_pre·∇Θ̅|x
       ≤ C ∇Θ_pre_L^∞(Ω)∇Θ_L^2(Ω)(h_L^2(Ω) + ℰ_pre).

    
    Now plugging this estimate into the proof steps of <ref>, we obtain, for any time interval (0, t) ⊂ (0, T̅^*), the following estimate
    
    
    Θ̅(t, ·)_L^2(Ω)^2
            +ϑ̅(t, ·)_L^2(Ω× Y_0)^2
            +∇Θ̅^2_L^2((0,t) ×Ω)
            +∇_yϑ̅^2_L^2((0,t) ×Ω× Y_0)
    ≤ C ℰ_pre^2 + C (1 + t^2) (Θ̅_L^2((0, t) ×Ω)^2 + ϑ̅_L^2((0, t) ×Ω× Y_0)^2).

    
    Applying the Grönwall inequality leads to the desired result.    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    




 §.§ Resolving the nonlinearity.
 Even with the precomputing strategy at hand, we are still facing a nonlinear two-scale problem. We propose a semi-implicit time-stepping method to resolve the nonlinearity, which is often applied to nonlinear equations in numerical simulations. To this end, given a time step Δ t we define the discrete time point t_i = i Δ t. Then we are looking at each i > 0 for the time-discrete solution (Θ_i, ϑ_i) ∈X of


    ∫_Ω C(h_i) Θ_i - Θ_i-1/Δ tΦx
            + ∫_Ω∫_Y_0 c(h_i)  ϑ_i - ϑ_i-1/Δ tϕyx
            + ∫_Ω K_pre(h_i) ∇(Θ_i + Θ_i-1/2) ·∇Φx
       + ∫_Ω∫_Y_0κ(h_i) ∇_y ϑ_i + ϑ_i-1/2·∇_y ϕyx
            +  ∫_Ω∫_Y_0ϑ_i v(h_i)·∇_yϕyx
            = ∫_Ω(F_i- L(h_i)h_i - h_i-1/Δ t)Φx
       +∫_Ω∫_Y_0f(h_i)ϕyx 
            - ∫_Ω∂_h C(h_i) h_i - h_i-1/Δ tΘ_i-1Φx
            - ∫_Ω∂_h c(h_i) h_i - h_i-1/Δ tϑ_i-1ϕx


for all (Φ,ϕ) ∈X where 


    X{
            (Φ,ϕ) ∈ H^1(Ω) × L^2(Ω, H^1(Y_0)) : Φ(x) = ϕ(x, y)  a.e. on Ω×Γ_0 
        }.


To find (Θ_i, ϑ_i) a differential equation has to be solved at each time step. But we now linearize the problem by using an explicit scheme to compute the height function h, to be precise


    h_i = h_i-1 + Δ t (Θ_i-1 - Θ^ref)    for  i > 0   and   h_0 = 0.


This also leads to the simplification of (h_i - h_i-1)/Δ t = Θ_i-1 - Θ^ref in the above scheme. In (<ref>) we used the product rule to split the time derivative into the derivative of Θ and of C, respectively for ϑ and c. This simplifies the proof for the time stepping error. Note, that also the more natural discretization 

    ∂_t(C(h) Θ) ≈C(h_i) Θ_i - C(h_i-1) Θ_i-1/Δ t

could be used and only deviates from the above scheme by a term of order Δ t. For the gradients, we utilize an average between the current and previous time step since this fits more naturally in the following proof idea. Note, that this scheme requires ∇Θ_0 ∈ L^2(Ω) and ∇_y ϑ_0 ∈ L^2(Ω× Y_0). But again, just ∇Θ_i, and ∇_y ϑ_i, could be used.

If |h_i| ≤ a^*, the solution of the system (<ref>) exists and is unique since it is a linear time-discretized parabolic two-scale equation. Given that Θ_0 ∈ L^∞(Ω) we can always carry out one time step if Δ t is small enough. To show that the discrete solution exists on a larger time interval (0, T^*_dis), we need to show that the discrete solution Θ_i can be bounded in L^∞(Ω) independent of the height function. For this, we can proceed as in <ref>, e.g., we would need to carry out the following steps:


    
  * Choose a M ≥ 1 and a N ∈, then define the time horizon T^*_dis=a^*/M^4 and step Δ t = T^*_dis/N.
    
  * Start with a fixed height function {h_i}_i=0^N defined by 
    
    
    h_i = h_i-1 + Δ t b_i,    h_0 = 0    and   b_i_L^∞≤ M.

    
    
  * Derive a L^∞(Ω) bound for the solution of <ref> for the prescribed height function {h_i}_i=0^N independent of M. This follows with similar computations as in <ref> and by testing with Φ = Θ_i + Θ_i-1, ϕ = ϑ_i + ϑ_i-1 and summing over all time steps. As the proof is similar to the previous one, it will not be presented in detail here.
    
  * Choose the M corresponding to the solution bound and obtain that the discrete method is reasonable and the solution exists on a nonempty interval (0, T^*_dis).


Note, that the above procedure also ensures that for a smaller time step Δ t the discrete solution exists, at least, on the same time interval. The error we introduce through this scheme is analyzed in the next lemma.


    Let (ĥ, Θ̂, ϑ̂) denote the solution that is continuous in time, exists on the time interval (0, T^*) and uses the precomputed values K_lin. Denote with {h_i, Θ_i, ϑ_i}_i=0^N the time discrete solution of <ref> that exists on [0, T^*_dis], with K_pre = K_lin. Let the assumptions
    
    
    

    [t]0.49
      
        
  * Θ̂∈ W^2, ∞((0, T^*); W^1, ∞(Ω)),
        
  * F ∈ C(S ×Ω),
        
  * Δ t ≤C(a^*)/8K(a^*),
      
    
    
    [t]0.49
      
        
  * ϑ̂∈ W^2, ∞((0, T^*); W^1, ∞(Ω× Y_0)),
        
  * f ∈ C(S ×Ω; C^0, 1(Y_0)),
        
      
    
    
    

    
    be fulfilled. Let N_t ∈ be such that Δ t N_t < min(T^*, T^*_dis) and Δ t(1 + N_t) ≥min(T^*, T^*_dis). Then, there exists a C > 0 (depending on (Θ̂, ϑ̂)) such that
    
    
    sup_i=1,…, N_tΘ̂(t_i) - Θ_i_H^1(Ω) + ϑ̂(t_i) - ϑ_i_H^1(Ω) + ĥ(t_i) - h_i_L^2(Ω)≤ C (Δ t + Δ h).

    


 
    We follow the proof ideas of <cit.> where numerical approaches for similar nonlinear problems are analyzed. The main difference in our case is, that we consider a two-scale system, the nonlinearity is coupled to the height function h, a less regular conductivity, and a different right-hand side because of C(h). But the general idea from <cit.> can be transferred to our case. We denote by ·_i the difference between the continuous and discrete solution at time point t_i, e.g., Θ_i = Θ̂(t_i) - Θ_i. We start by investigating the difference between the height functions of both solutions. By definition of both functions, we obtain
    
    
    h_i_L^2(Ω)   = ĥ(t_i) - h_i_L^2(Ω)
       ≤ĥ(t_i-1) - h_i-1_L^2(Ω) 
            +  ∫_t_i-1^t_i (Θ̂(s) - Θ^ref) s - Δ t (Θ_i-1 - Θ^ref) _L^2(Ω)
       ≤ĥ(t_i-1) - h_i-1_L^2(Ω) 
            + Δ t Θ_i-1_L^2(Ω)
            + ∫_t_i-1^t_i∫_t_i-1^s ∂_t Θ̂(τ) τs_L^2(Ω)
       ≤ĥ(t_i-1) - h_i-1_L^2(Ω) 
            + Δ t Θ_i-1_L^2(Ω) + C (Δ t )^2

    
    in the estimates we used the fundamental theorem of calculus and that ∂_t Θ̂ is bounded. Iteratively applying the above argument leads to
    
    
    h_i_L^2(Ω)≤∑_n = 1^i-1[ Δ t Θ_n_L^2(Ω) + C (Δ t )^2 ].

    
    Next, we derive the estimates for the temperature. To this end, we only show the error estimate for the macroscopic function Θ. The reason for this is, that the estimates for ϑ follow with the same steps since the microscopic equation has a similar structure. This allows us to reduce the length of the proof considerably. Therefore, let the continuous solution Θ fulfill
    
    
    ∫_Ω C(h) ∂_t ΘΦx 
            + ∫_Ω∂_h C(h) ∂_t hΘΦx
            + ∫_Ω K_lin(h) ∇Θ·∇Φx 
            = ∫_Ω  L(h) ∂_t hΦx

    
    and the time-discrete solution 
    
    
    ∫_Ω C(h_i) Θ_i - Θ_i-1/Δ t   Φx
                + ∫_Ω∂_h C(h_i) (Θ_i-1 - Θ^ref)  Θ_i-1Φx
       + ∫_Ω K_lin(h_i) ∇(Θ_i + Θ_i-1/2) ·∇Φx  
                = ∫_Ω  L(h_i) (Θ_i-1 - Θ^ref) Φx.

    
    To derive an error estimate at  t_i, we consider (<ref>) localized at t_i, subtract  the <ref> and (<ref>) from each other and test with Φ = Θ_i - Θ_i-1. We obtain four terms which we estimate individually. Let us introduce the discrete time derivative d_t which is defined by d_t Θ_i = Θ_i - Θ_i-1/Δ t. Additionally, we abuse the notation and write d_t Θ̂_i = Θ̂(t_i) - Θ̂(t_i-1)/Δ t. We start with the time derivative term, which yields
    
    
    ∫_Ω   C(h_i) d_tΘ_i Φx - ∫_Ω C(h) ∂_t Θ(t_i) Φx 
            = ∫_Ω C(h_i) d_tΘ_i Φx 
            - ∫_Ω C(h_i) d_tΘ̂_i Φx_=I_1
       + ∫_Ω C(h_i) d_tΘ̂_i Φx 
            - ∫_Ω C(h(t_i)) d_tΘ̂_i Φx_=I_2
            + ∫_Ω C(h(t_i)) d_tΘ̂_i Φx
            - ∫_Ω C(h(t_i)) ∂_t Θ(t_i) Φx_=I_3.

    
    For terms on the right-hand side, it holds
    
    
    I_1    = Δ t ∫_Ω C(h_i) (d_t Θ_i)^2 x≥Δ t C(a^*)  d_t Θ_i_L^2(Ω)^2,
            
    
            |I_2|    ≤ C h_i_L^2(Ω)Θ_i - Θ_i-1_L^2(Ω),
            
    
            |I_3|    ≤ C Δ t ∂_ttΘ̂(s) s_L^∞((0, T^*) ×Ω)Θ_i - Θ_i-1_L^2(Ω)≤ C (Δ t)^3 + Δ t d_t Θ_i_L^2(Ω)^2.

    
    Here, we used for I_2 and I_3 that C is Lipschitz, the Hölder inequality, and the Young inequality with a small > 0. I_2 can be estimated further by utilizing <ref>,
    
    
    |I_2|
               ≤  C ∑_n = 1^i-1[ Δ t Θ_n_L^2(Ω) + C (Δ t )^2 ] Θ_i - Θ_i-1_L^2(Ω)
       = C ∑_n = 1^i-1[ Δ t Θ_n_L^2(Ω)Δ t d_t Θ_i_L^2(Ω) + C (Δ t )^2 Δ t d_t Θ_i_L^2(Ω)] 
            
       ≤ C ∑_n = 1^i-1[(Δ t)^2 Θ_n_L^2(Ω)^2 +  (Δ t)^2 d_t Θ_i_L^2(Ω)^2 + C (Δ t )^4 +  (Δ t)^2 d_t Θ_i_L^2(Ω)^2 ] 
            
       ≤  C (Δ t )^3 + C Δ t d_t Θ_i_L^2(Ω)^2 + C Δ t ∑_n = 1^i-1Δ t Θ_n_L^2(Ω)^2.

    
    Next, the term with ∂_h C, where we obtain
    
    
    ∫_Ω∂_h C(h_i)    (Θ_i-1 - Θ^ref)  Θ_i-1Φx - ∫_Ω∂_h C(h(t_i)) (Θ(t_i) - Θ^ref) Θ(t_i) Φx
       = ∫_Ω∂_h C(h_i) (Θ_i-1 - Θ^ref)  Θ_i-1Φx 
            - ∫_Ω∂_h C(h_i) (Θ_i-1 - Θ^ref)  Θ̂(t_i-1) Φx_=I_4
       + ∫_Ω∂_h C(h_i) (Θ_i-1 - Θ^ref)  Θ̂(t_i-1) Φx -
            ∫_Ω∂_h C(h_i) (Θ̂(t_i-1) - Θ^ref)  Θ̂(t_i-1) Φx_=I_5
       + ∫_Ω∂_h C(h_i) (Θ̂(t_i-1) - Θ^ref)  Θ̂(t_i-1) Φx - ∫_Ω∂_h C(h(t_i)) (Θ(t_i-1) - Θ^ref) Θ̂(t_i-1)Φx_=I_6
       + ∫_Ω∂_h C(h(t_i)) (Θ̂(t_i-1) - Θ^ref)  Θ̂(t_i-1) Φx - ∫_Ω∂_h C(h(t_i)) (Θ(t_i) - Θ^ref) Θ(t_i) Φx_=I_7

    Using that ∂_h C is well-defined and bounded (since ∂_h J_h is well-defined and bounded; see the proof of <ref>) and that the solutions are bounded, we can obtain with similar arguments as before
    
    
    |I_4| + |I_5| 
               ≤ C Θ_i-1_L^2(Ω)Θ_i - Θ_i-1_L^2(Ω)≤ C Δ t Θ_i-1_L^2(Ω)^2 + Δ t  d_t Θ_i_L^2(Ω)^2,
            
    
            |I_6|    ≤ C h_i_L^2(Ω)Θ_i - Θ_i-1_L^2(Ω),
            
    
            |I_7|    ≤ C Δ t ∂_t Θ̂_L^∞((0, T^*) ×Ω)Θ_i - Θ_i-1_L^2(Ω)≤ C (Δ t)^3 + Δ t d_t Θ_i_L^2(Ω)^2.

    
    Since the term containing L is comparable to the previous ones, we first handle it and afterward estimate the gradients. By introducing the mixed terms, we have the following equality 
    
    
    ∫_Ω  L(h_i) (Θ_i-1 - Θ^ref) Φx   - ∫_Ω  L(h(t_i)) (Θ̂(t_i) - Θ^ref) Φx
       =  ∫_Ω  L(h_i) (Θ_i-1 - Θ^ref) Φx - ∫_Ω  L(h_i) (Θ(t_i-1) - Θ^ref) Φx_=I_8
       + ∫_Ω  L(h_i) (Θ(t_i-1) - Θ^ref) Φx - ∫_Ω  L(h_i) (Θ(t_i) - Θ^ref) Φx_=I_9
       +
            ∫_Ω  L(h_i) (Θ(t_i) - Θ^ref) Φx - ∫_Ω  L(h(t_i)) (Θ̂(t_i) - Θ^ref) Φx_=I_10

    All of I_8, I_9 and I_10 are of the same structure as before and we can find the following estimates
    
    
    |I_8|    ≤  C Δ t Θ_i-1_L^2(Ω)^2 + Δ t  d_t Θ_i_L^2(Ω)^2,
            
    
            |I_9|    ≤ C (Δ t)^3 + Δ t d_t Θ_i_L^2(Ω)^2,
            
    
            |I_10|    ≤ C h_i_L^2(Ω)Θ_i - Θ_i-1_L^2(Ω).

    
    Let us next consider the gradient terms, here the estimates are more involved but follow the general idea outlined in <cit.>. Introducing mixed terms into the difference first leads to
    
    
    ∫_Ω   K_lin(h_i) ∇(Θ_i + Θ_i-1/2) ·∇Φx 
            - ∫_Ω K_lin(h(t_i)) ∇Θ(t_i) ·∇Φx
       = ∫_Ω K_lin(h_i) ∇(Θ_i + Θ_i-1/2) ·∇Φx 
            - ∫_Ω K_lin(h_i) ∇(Θ̂(t_i) + Θ̂(t_i-1)/2) ·∇Φx_=I_11, i
       + ∫_Ω K_lin(h_i) ∇(Θ̂(t_i) + Θ̂(t_i-1)/2) ·∇Φx 
            -  ∫_Ω K_lin(ĥ(t_i)) ∇(Θ̂(t_i) + Θ̂(t_i-1)/2) ·∇Φx_= I_12, i
       + ∫_Ω K_lin(ĥ(t_i)) ∇(Θ̂(t_i) + Θ̂(t_i-1)/2) ·∇Φx - ∫_Ω K_lin(h(t_i)) ∇Θ(t_i) ·∇Φx_=I_13, i.

    Starting with I_11, i, where we obtain
    
    
    2 I_11, i   = ∫_Ω K_lin(h_i) ∇ (Θ_i + Θ_i-1) ·∇ (Θ_i - Θ_i-1) x =  ∫_Ω K_lin(h_i) (∇Θ_i ·∇Θ_i - ∇Θ_i-1·∇Θ_i-1) x.

    
    To further estimate the error, we now sum over m=1, …, i, and using summation by parts we obtain
    
    
    ∑_m=1^i ∫_Ω K_lin(h_i) (∇Θ_i ·∇Θ_i - ∇Θ_i-1·∇Θ_i-1) x   = ∑_m=1^i-1∫_Ω∇Θ_m ·∇Θ_m (K_lin(h_m)  - K_lin(h_m+1)) x
        + ∫_Ω∇Θ_i ·∇Θ_i K_lin(h_i) x.

    
    The sum on the right can be further estimated by 
    
    
    | ∑_m=1^i-1∫_Ω∇Θ_m ·∇Θ_m (K_lin(h_m)  - K_lin(h_m+1)) x | ≤  C ∑_m=1^i-1Δ t  ∇Θ_m_L^2(Ω)^2,

    
    where we used that K_lin is Lipschitz. For I_12 we start with a similar strategy,
    
    
    ∑_m=1^i I_12, i = ∫_Ω (K_lin(h_i) - K_lin(h(t_i)))  ∇Θ_i ·∇(Θ̂(t_i) + Θ̂(t_i-1)/2) x_= I_12, a
       + ∑_m=1^i-1∫_Ω (K_lin(h_m)  - K_lin(ĥ(t_m))) ∇Θ_m  ·∇(Θ(t_m) + Θ(t_m-1)/2 - Θ(t_m+1) + Θ(t_m)/2) x_= I_12, b
       + ∑_m=1^i-1∫_Ω[K_lin(h_m)  - K_lin(ĥ(t_m)) - K_lin(h_m+1) + K_lin(ĥ(t_m+1))] ∇Θ_m  ·∇(Θ(t_m) + Θ(t_m-1)/2) x_= I_12, c.

    
    The first term I_12, a can be estimated with the same ideas as before, leading to
    
    
    |I_12, a| ≤∇Θ_i_L^2(Ω) + C (Δ t )^2 + C ∑_n=1^i-1Δ t Θ_n_L^2(Ω)^2.

    
    For I_12, b we need <ref> and that the time derivative of ∇Θ̂ exists, to get the estimate
    
    
    |I_12, b|    ≤ C ∑_m=1^i-1Δ t ∇Θ_m_L^2(Ω)[Δ t  + ∑_n=1^m-1Δ t Θ_n_L^2(Ω)] 
            
       = C ∑_m=1^i-1Δ t ∇Θ_m_L^2(Ω)[Δ t + (i-1 - m) Δ t Θ_m_L^2(Ω)] 
            
       ≤ C (Δ t)^2 + C ∑_m=1^i-1Δ t Θ_m_H^1(Ω)^2.

    
    To bound I_12, c we use that K_lin∈ H^1((-a^*, a^*)) and since the exact K is in C^1,1 we also have |∂_h K_lin(h_1) - ∂_h K_lin(h_2)| ≤ C |h_1 - h_2| + Δ h, for a.a. h_1, h_2 ∈ (-a^*, a^*). This yields
    
    
    |I_12, c|    ≤ C ∑_m=1^i-1Δ t ∫_Ω∇Θ_m∫_0^1 |∂_h K(h_m + τ (h_m+1 - h_m)) 
            
        - ∂_h K(ĥ(t_m) + τ (ĥ(t_m+1) - ĥ(t_m)))| τx
       ≤ C ∑_m=1^i-1Δ t ∫_Ω∇Θ_m∫_0^1  |Δ h + h(t_m) - h_m + τ[h_m+1 - h_m - ĥ(t_m+1) + ĥ(t_m)]  | τx
       ≤ C ∑_m=1^i-1Δ t ∇Θ_m_L^2(Ω) [ Δ h + Δ t + ∑_n=1^m-1Δ t Θ_n_L^2(Ω)].

    
    Next, we apply the Young-inequality and can proceed in the same way as for the estimate of I_12, a. Lastly, I_13 can again be estimated by summation, leading to
    
    
    | ∑_m=1^i  I_13, i | 
               ≤∫_Ω | K_lin(h(t_i)) ∇(Θ(t_i-1) - Θ(t_i)/2) ∇Θ_i x | 
            
       + ∑_m=1^i-1 | ∫_Ω (K_lin(h(t_m))  - K_lin(h(t_m+1))) ∇Θ_m  ·∇(Θ(t_m-1) - Θ(t_m)/2) x | 
            
       + ∑_m=1^i-1 | ∫_Ω K_lin(h(t_m+1)) ∇Θ_m  ·(Θ(t_m-1) - Θ(t_m)/2 - Θ(t_m) - Θ(t_m+1)/2) x | 
            
       ≤ C Δ t ∇Θ_i_L^2(Ω) + ∑_m=1^i-1 C ∇Θ_m_L^2(Ω) (Δ t )^2
            
       ≤ C (Δ t)^2 + ∇Θ_i_L^2(Ω)^2 + ∑_m=1^i-1Δ t ∇Θ_m_L^2(Ω)^2.

    In the first inequality above the Lipschitz continuity of K_lin, the Hölder inequality, and the time-regularity of ∇Θ̂ were utilized. In the second inequality, we used Young again.

    We can now combine all of the above estimates. To this end, we also have to sum I_1, … I_10 over 1,…,i to combine them with the gradient estimates, to be precise we would first sum and then carry out the above computations inside the sum. This leads to the estimate
    
    
    ∑_m=1^i    Δ t C(a^*) d_t Θ_m^2_L^2(Ω) + K(a^*) ∇Θ_i_L^2(Ω)^2 
                ≤ C (Δ t)^2 +  
                C ∇Θ_i_L^2(Ω)^2
                + ∑_m=1^i Δ t C d_t Θ_m^2_L^2(Ω)
       + C ∑_m=1^i-1Δ t [ Θ_m^2_H^1(Ω) + Θ_m^2_L^2(Ω) + ∇Θ_m^2_L^2(Ω)]
                + C ∑_m=1^iΔ t ∑_n=1^m-1Δ t Θ_n^2_L^2(Ω) +  C (Δ h)^2.

    
    We are missing the L^2-norm of Θ_i on the left for the final estimate. Therefore, we use the same computation as <cit.> to obtain
    
    
    K(a^*)(Θ̅_i^2_L^2(Ω) + Θ̅_i-1^2_L^2(Ω)) 
                   ≤ 2 K(a^*) Δ t ∫_Ω d_t Θ_i Θ_i-1x + K(a^*) (Δ t)^2 d_t Θ̅_i^2_L^2(Ω)
       ≤C(a^*)Δ t/4d_t Θ̅_i^2_L^2(Ω) + C Δ t Θ̅_i-1^2_L^2(Ω).

    
    For the last inequality, we used the assumption Δ t ≤C(a^*)/8K(a^*). Next, we sum <ref> from 1,…,i and add the estimate to <ref>. Then the sum over d_t Θ̅ which originates when summing  (<ref>) can be collected into the already present term in (<ref>). 
    Additionally, the  on the right side originates from the Young inequality and can be chosen such, that all terms connected to  can be incorporated into the left-hand side. All other sums on the right can be further bounded by the H^1-norm, which yields
    
    
    ∑_m=1^i Δ t C d_t Θ_m^2_L^2(Ω) + C Θ_i_H^1(Ω)^2 
            ≤ C (Δ t)^2 + C (Δ h)^2 + C ∑_m=1^i-1Δ t Θ_m^2_H^1(Ω).

    
    Finally, applying the discrete Grönwall lemma gives the desired result. As mentioned at the start the estimates for ϑ follow analogously. Plugging the estimate for Θ back into <ref>, we find the estimate for h. 



    The convergence of the time-stepping scheme requires that the precomputing also gets refined with smaller time steps. This condition appears because the linear interpolation has a piecewise constant derivative. This is not necessary for a smoother interpolation, for example, quadratic splines.




 §.§ Further error sources
 In addition to the discretization in time, we also have to discretize the function space.
To this end, we consider a finite element (FE) approach. 
Define the triangulation 𝒯_H_M,Ω of Ω and 𝒯_H_m,Y_0 of Y_0, where H_M and H_m denote the largest edge length of the simplices in 𝒯_H_M,Ω and 𝒯_H_m,Y_0 respectively.
Denote with ℙ_1 the space of linear polynomials. On the macro scale, we define the finite element space X_H_M,Ω⊂ H^1(Ω) by


    X_H_M,Ω = {Φ∈ C(Ω) : Φ_|T∈ℙ_1(T) 
             for all  T ∈𝒯_H_M,Ω}.


Similarly, on the microscale the space X_H_m, Y_0⊂ H^1(Y_0) is given by


    X_H_m, Y_0 = {ϕ∈ C(Y_0) : ϕ_|T∈ℙ_1(T) 
             for all  T ∈𝒯_H_m,Y_0}.


Let N_M denote the degrees of freedom (DOFs) on the macroscale and N_m DOFs on the microscale. Define the basis functions {Φ_n}_n=1^N_M such that span(Φ_n) = X_H_M,Ω and equally the micro basis functions {ϕ_k}_k=1^N_m. The time  discretized finite element approximation of the solutions (Θ, ϑ, h) are then given by (H=(H_M,H_m))


    Θ_i, H = ∑_n=1^N_Mα_i, nΦ_n(x),
          ϑ_i, H = ∑_n=1^N_M∑_k=1^N_mβ_i, n, kΦ_n(x) ϕ_k(y) 
           and    
        h_i, H = ∑_n=1^N_M h_i, nΦ_n(x),


with the additional condition Θ_i, H = ϑ_i, H   |Γ_0 for all i=1,…,N. For the coefficients it holds, α_i, n, β_i, n, k, h_i, n∈.

The above FE approach introduces another error in the numerical approximation. The error estimate from <ref> can also be extended to include the projection error in regards to the FE space. This was carried out for a comparable problem in <cit.> and could be transferred to our setup under regularity assumptions of the solutions. Here, we do not carry out the complete proof, instead, we demonstrate the error in the following section through numerical simulations. 

Note that the above FE approach is a natural candidate for discretizing the problem, but can become quite expensive due to the high dimensionality of ϑ, especially in three dimensions. We still use the above method since we only want to demonstrate the previous analytical results on a two-dimensional setup. Parallelization is used to speed up the calculations; see <ref> for details. However, in other applications, this procedure may not be feasible. In <ref> we gave an overview of possible extensions and different approaches. Note, that the presented precomputing method and theory could also be combined with the alternatives to reduce the computational effort.













Next to the discretization of (Θ, ϑ) we also need to solve the cell problems <ref> with a FE approach. To this end introduce the mesh size H_cell of the mesh for Y∖ Y(h). Let K_H denote the effective conductivity computed with the FE solution. Under the assumption that we use piecewise linear FEs for solving the cell problems and the cell solutions are sufficiently smooth, we can obtain the error estimate


    |K(h) - K_H(h)| ≤ C H_cell,


for a C < ∞ independent of H_cell.







§ SIMULATIONS

In this last section, we want to investigate and back up the previous analysis results with numerical simulations. To this end, we apply the time-discretized, finite element approach with precomputation as explained in <ref>. Before we study the convergence behavior, we introduce the general problem setup and explain some implementation details.
We use the FEM library FEniCS <cit.> and implement a custom parallelization for the micro problems. Our code is freely available online <cit.>. 

As mentioned in the previous section, we want to apply an approach where the time step for Θ and ϑ is carried out simultaneously. Therefore, at time step i, we have to solve the linear system


    [     A   C_1   C_2     … C_N_M;   D_1   B_1     0     …     0;   D_2     0     ⋱           ⋮;     ⋮     ⋮           ⋱     0; D_N_M     0     …     0 B_N_M ][      α_i;   β_i, 1;        ⋮;        ⋮; β_i, N_M ]
        = 
        [      F_i;   f_i, 1;        ⋮;        ⋮; f_i, N_M ]
        .


Here, we do not want to go too deep into the implementation details of each matrix block, for this, we point to the available source code. Instead, we only explain the role of each part:


    
  * The matrix A ∈^N_M × N_M encodes the macroscopic bilinear form.
    
  * The matrices B_k ∈^N_m × N_m represent the microscopic bilinear form, for k=1,… N_M.
    
  * C_k ∈^N_M × N_m describes the heat flow across the micro boundary Γ_0, and can be created by computing κ(h_i) ∇ϑ_i · n over Γ_0.
    
  * D_k ∈^N_m × N_m enforces the coupling that Θ = ϑ on Γ_0. Therefore this matrix only contains 0 and 1 on the corresponding entries. In particular, the row structure of all D_k is the same, only one column contains ones and for each k this column is shifted to a different position. In the corresponding rows of matrix B_k only a -1 is present and on the right-hand side a 0.
    
  * The right-hand side F_i ∈^N_M and f_i, k∈^N_m include the heat sources and information from the previous time step.


The role of C_k and D_k is the coupling between macro and micro scale. Note that there is no direct coupling between the different cells and the block containing the B_k matrices is diagonal. Additionally, all matrices are sparse.
The matrices A, B_k, C_k need to be recomputed in each time step, since they depend on the current height h_i. This is easily parallelizable once h_i is known.

Before we start presenting the simulation results, we collect here all information regarding the data and geometry. For the macro domain we use Ω = [0, 1]^2, the time interval is set to S=[0, 10], and the microdomain Y_0 is defined as a disk with radius r = 0.25. Note, that for a disk the Hanzawa transformation only is a scaling in the direction of the radius. The conductivity values are set to κ = 𝒦 = 0.1 and the initial temperature to Θ_0 = ϑ_0 = Θ^ref = 0. We only consider a macroscopic heat source, meaning f = 0, given by


    F(x, t) = 0.75 max{0,  min[1, g(t, x)]}max{0, min[1, 5 - t]}


with


    g(t, x) = 2 - 10 x - [ 0.2 + 0.6 t/5;           0.7 ]_∞.


F describes a smoothed step function that moves to the right and becomes 0 for t ≥ 5.
Additionally, we introduce a speed variable v=0.1 to better control the change of the microdomain, e.g., ∂_t h = v(Θ - Θ^ref). This does not influence the previous analysis. We remark, that we do not have a concrete application in mind for this setup. Instead, the values are chosen to observe interesting aspects of the model.

If not mentioned otherwise, we use Δ t = 0.1, H_M = 0.05, H_m = 0.06, H_cell = 5e-4 and Δ h = 1/320 in the following simulations. 

Since our microdomains are symmetric around the center of the domain, the effective conductivity K(h) reduces to a scalar value. The dependence of K of different radii r, is shown in <ref>. Note, that we characterize the change in radius by the height function h, a negative h equals shrinkage and a positive h growth of the disk radius. 





 §.§ Precomputing strategy


We start with the numerical investigation of the precomputing method. Both a piecewise linear interpolation as well as a quadratic spline interpolation will be studied. For the precomputation, we solve (Δ h)^-1 = 320 cell problems for different height values h that are taken from a uniform grid in [-0.245, 0.245]. Outside of this interval, we extend the interpolation by a constant value. Since no analytic solution is at hand, we compute a reference solution that uses all cell problems and the quadratic spline interpolation. The error is then computed between the reference solution and simulation results that utilize fewer cell problems for the interpolation.

The results are presented in <ref>. For the linear interpolation, we can observe the expected convergence order of 𝒪((Δ h)^2). In the case of quadratic interpolation, we note for a large Δ h, meaning the use of fewer precomputed values, we detect that the error decreases with the expected order of 𝒪((Δ h)^3). However, as soon as finer interpolation is used, the error stops decreasing and reaches a plateau. This is where the discretization error of the cell problems comes into play, affecting the accuracy of the interpolation. This effect is further described for different H_cell in <ref>. There we focus only on the function Θ to simplify the visualization. It can be seen that for larger H_cell, meaning a coarser cell mesh, the plateau occurs earlier. Therefore, to obtain the expected convergence in the precomputing scheme, an accurate solution of the cell problems is mandatory.



Another aspect we want to highlight is the interpolation property when the disk gets close to the outer boundary of the unit cell, meaning h → 0.25 or equally r → 0.5. Corresponding to the previous theory, for a radius close to 0.5 we would obtain a small a^* since the disk is not allowed to grow much further. Therefore, by <ref> the slope of K(h) may become large. This is also observable in <ref>. To better demonstrate the effect on the interpolation, we construct linear and quadratic interpolations for different Δ h and compare them on the interval I_h = [-0.25, h_max]. Again as a reference, we use the finest interpolation with Δ h = 1/320 and compare by computing the L^∞-difference on a uniform grid in I_h with 1000 points. This comparison is visualized in <ref>. We notice, that when h_max gets closer 0.25 the convergence order gets reduced. To circumvent this effect one could employ a nonuniform sampling of the height values used for the interpolation, e.g., solve more cell problems when h is close to 0.25. Note, that in the quadratic spline interpolation, we again observe the influence of H_cell once Δ h is small. From a theoretical point of view, this effect could also appear when h → 0, but there we observe that the slope of K(h) is rather flat. 

Finally, we want to briefly mention the computation time. On our setup, an AMD EPYC 7282 16-core CPU, it takes about 10 minutes to solve a cell problem for H_cell=5e-4, including the mesh generation. Since the precomputation is easily parallelizable, it took about 5 hours to compute the full dataset of 320 cell problems, as we were able to spread the work over several CPUs. Then, solving the entire problem in parallel on 8 cores, using the discretization mentioned above, takes about 13 seconds. If we had not used the precomputation scheme, the cell problems would need to be solved at each macro DOF and possibly at each time step, this would result in computation times of several days or weeks. This is also the reason why we do not use such a solution as a reference for the convergence study. 





 §.§ Time and space discretization error

In this second simulation section, we briefly present the convergence behavior with respect to the spatial discretization of the macro and micro scales and the time discretization. We study the influence of each discretization independently and fix the other parameters in each simulation. As before, no analytical solutions are available and a comparison is made with a finer numerical solution. For the fine reference solution, we use H_M=0.01, H_m=0.006, and Δ t = 0.003 respectively for each convergence study and apply quadratic spline interpolation for the precomputing.

The results are presented in <ref>. The order of error for the time-stepping method is consistent with the theoretical results of <ref>. Given the FE spaces of piecewise linear polynomials, both on the macro and micro scale, we also obtain the expected convergence behavior in regard to the space discretization.





§ CONCLUDING REMARKS

We have suggested, analyzed, and implemented a precomputing strategy for non-linearly coupled two-scale systems with evolving microstructures.
Utilizing the Hanzawa transformation, the existence of local-in-time, weak solutions was established.
Under more restrictive conditions on the data, it is expected that the solutions are unique and strong (see also <cit.>).
We showed that the additional interpolation error introduced via the precomputing can be controlled by the step size and the interpolation order given that the underlying relationship in the material parameters is smooth.
We also introduced a semi-implicit time-stepping method to resolve the nonlinearities inherent to the moving microsctructures and showed linear convergence.
We note that it is possible to improve convergence by using, e.g., a Crank-Nicolson-Galerkin
approximation (<cit.>).
The simulation results presented in <ref> confirm the analytical convergence.
The implementation was done in FEniCS and our code is freely available online <cit.>.

While the model in question in this paper is concerned with phase transitions on the microscale, we want to point out that the same kind of model with very similar analytical and numerical challenges appears in many different applications: precipitation in porous media, swelling of biological tissues, and degradation of concrete to name just a few.
The main advantage of our approach is the fact that a significant part of the computational effort is pushed into an offline phase in a way that is additionally perfectly parallelizable.
There are of course potential drawbacks as well: When the underlying functions are discontinuous or extremely volatile, the interpolation error might be difficult to control.
Moreover, in our case we only have a one-dimensional bounded parameter space h∈[-a^*,a^*], which is relatively easy to exhaust.
In cases where shape changes are possible (e.g., <cit.>) the parameter space becomes much larger.






§ ACKNOWLEDGEMENTS

The research activity of ME is funded by the European Union’s Horizon 2022 research and innovation program under the Marie Skłodowska-Curie fellowship project MATT (funding number 101061956, <https://doi.org/10.3030/101061956>).

TF acknowledges funding by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – project nr.  
281474342/GRK2224/2 and by project nr. 439916647.























































abbrv

















 Paper Title: Analysis of continuous data assimilation with large (or even infinite) nudging parameters.



































[label1]Department of Mathematics and Statistics, Mississippi State University, MS, 39762; partially supported by NSF grant DMS 2110768.
[label2]School of Mathematical and Statistical Sciences, Clemson University, Clemson, SC, 29364, xuejial@clemson.edu; partially supported by NSF grant DMS 2152623.
[label3]School of Mathematical and Statistical Sciences, Clemson University, Clemson, SC, 29364, rebholz@clemson.edu; partially supported by NSF grant DMS 2152623.



















§ ABSTRACT 
This paper considers continuous data assimilation (CDA) in partial differential equation (PDE) discretizations where nudging parameters can be taken arbitrarily large.  We prove that  solutions are long-time optimally accurate  for such parameters for the heat and Navier-Stokes equations (using implicit time stepping methods), with error bounds that do not grow as the nudging parameter gets large.  Existing theoretical results either prove optimal accuracy but with the error scaled by the nudging parameter, or suboptimal accuracy that is independent of it.  The key idea to the improved analysis is to decompose the error based on a weighted inner product that incorporates the (symmetric by construction) nudging term, and prove that the projection error from this weighted inner product is optimal and independent of the nudging parameter.  We apply the idea to BDF2 - finite element discretizations of the heat equation and Navier-Stokes equations to show that with CDA, they will admit optimal long-time accurate solutions independent of the nudging parameter, for nudging parameters large enough.  Several numerical tests are given for the heat equation, fluid transport equation, Navier-Stokes, and Cahn-Hilliard that illustrate the theory.















continuous data assimilation long time accuracy finite element method projection properties














	

§ INTRODUCTION

	Data assimilation is a broad topic that generally refers to incorporating observations into a given dynamical system to better estimate some relevant quantities of interest. One common use of data assimilation is to generate an accurate initial condition for the dynamical system so that its future state can be accurately predicted. Data assimilation was originally made popular in the weather forecasting community <cit.>, and various categories of data assimilation methods exist, such as Kalman filtering and particle filtering <cit.>, 3D/4D variational methods <cit.>, and nudging methods <cit.>. In the past decade, the continuous data assimilation (CDA) method developed by Azouani, Olson, and Titi <cit.>  has rapidly become popular due to its robustness, effectiveness, and strong mathematical foundation. 
	
	CDA works by adding a nudging term (also called feedback control or penalty term) that assimilates observations into   dynamical systems and attempts to drive the assimilated state to the true state. 
	In general, CDA applied to a time dependent PDE 
	
    u_t +F(u)     =f, 
    
    	u(0) = u_0,

	is written as (suppressing boundary conditions for the moment)
	
    ∂ w/∂ t+F(w)+μ (I_Hw-I_Hu)     =f, 
    
    		w(0)=0.
 
Here, μ (I_Hw-I_Hu) is the nudging term, u is the partially observed true solution, I_H is an interpolation or observation operator associated with the measurement location with H representing the sparsity of observations, 
μ is the nudging parameter emphasizing the accuracy of observations, and F is an operator specific to the PDE at hand. We note that an initial condition of 0 is chosen for w, although long-time accuracy is typically achieved in  many CDA analyses considering arbitrary initial conditions. Of course, the more accurate the initial condition, the quicker the convergence, in general.  While CDA is similar to traditional nudging methods, a fundamental difference is the use of the interpolation operator which is a minor modification but has a critical impact. Researchers have proved in a variety of settings and systems that by using sparse observations and appropriate I_H, applying CDA leads to an assimilated solution that converges to the true solution (up to optimal discretization error) exponentially fast in time, independent of the initial condition (see e.g.<cit.> and references therein).  

In addition to the strong mathematical foundation, another reason CDA is attractive is its straightforward implementation. The CDA computation is carried out the same way as solving the associated evolutionary PDE, only with extra attention to the nudging term that can be easily handled in most cases.  Studies have been performed using the CDA implementation based on different interpolants I_H and discretization schemes that make implementation simple (e.g. only altering the diagonal of the system matrix at each time step) <cit.>, and maintain the expected exponential convergence of discrete CDA to an optimally accurate solution.  



The goal of this paper is to prove optimal long-time convergence of discrete CDA with no upper restriction on the nudging parameter μ (i.e. no scaling of the error with μ).  There have been several studies that have shown excellent results with CDA with nudging parameter of 1000 and larger applied to Navier-Stokes and Cahn-Hilliard solvers where implicit time stepping is used <cit.>; some of these works also test varying μ and show no deterioration of accuracy as μ gets larger.  However, to date, no theoretical results exist to explain these good results; current known results for CDA that are optimal either depend on μ or have an upper bound restriction on μ.  Allowing μ to be very large can bring convenience of CDA implementation by allowing for a direct Dirichlet enforcement assimilation. Very large μ is numerically equivalent in the linear systems to a direct enforcement of w(x_i)=u(x_i) at all measurement nodes x_i associated with the determination of I_Hu and I_Hw. Such an implementation has been mentioned in <cit.>, but without any proof of accuracy. In this work, we provide a rigorous analysis to support this result in the context of CDA for evolutionary PDEs. We first propose projection operators tailored for use with CDA, and prove an optimal estimate for its approximation error, independent of the nudging parameter μ.  With these new projections, we are able to prove that CDA with arbitrarily large parameters find optimal long-time accurate solutions with respect to the L^2-norm with error bounds independent of μ for BDF2 - finite element discretizations of the heat equation and incompressible Navier-Stokes equations.  We note that the tools and techniques developed and used for the heat and Navier-Stokes equations are extendable to many dissipative PDEs.


The paper is organized as follows. In Section <ref>, we introduce notation and provide mathematical preliminaries for a smooth analysis to follow. In Section <ref>, we state the new CDA Poisson and CDA Stokes projections and analyze their properties. In Section <ref>, we present the CDA convergence analysis for the discretized heat equation. In Section <ref>, we focus on a CDA convergence analysis of discretized Navier-Stokes equations. In Section <ref>, we provide several numerical tests that support the theory. Finally, we draw conclusions in Section <ref>.

	
	

§ PRELIMINARIES


Throughout this paper, C will denote a generic constant, possibly changing at each instance, that is independent of mesh size h, time step Δ t, nudging parameter μ, and problem data. We consider Ω 
	 to be a generic polygonal or polyhedral domain with smooth boundary ∂Ω. Let H^k(Ω) denote the Sobolev space W^k,2(Ω) and  L^2(Ω)=W^0,2(Ω). We use (·,·) to denote the L^2 inner product that induces the L^2 norm ·,  while all other inner products or norms will be appropriately subscripted. Define the following function spaces:
	
    H^1_0(Ω):=
    		{v∈ H^1(Ω): v=0  on ∂Ω},
        X:= ( H^1_0(Ω))^d,    d=2,3,
       Q:={v∈L^2(Ω): ∫_Ωvdx=0},
        V:=
    		{v∈ X:  (∇· v,q)=0 ∀ q∈ Q}.



Recall that the Poincaré inequality holds on X: there exists a constant C_P dependent only on the domain Ω satisfying 

    v ≤ C_P ∇ v ,

 for all v∈ X. 




































	
	
	


  

	


 §.§ Finite element preliminaries

We consider finite element (FE) methods for the spatial discretization of the CDA algorithm. Denote by τ_h(Ω) a shape-regular and conforming triangulation of the interested domain Ω. We use Y_h=H^1_0(Ω)
∩ P_k(τ_h(Ω)) as the FE discretization space for the heat equation. Let X_h× Q_h⊂ X× Q 

be a pair of inf-sup stable FE spaces for Navier-Stokes equation, such as X_h=X∩ P_k(τ_h(Ω)) and Q_h=Q∩ P_k-1(τ_h(Ω)) as Taylor-Hood or Scott-Vogelius elements. 




 §.§ BDF2 preliminaries


The time stepping method used in our analysis and most of our numerical tests is BDF2, and we give here some preliminary results for BDF2 analysis.  We will utitlize the G-norm <cit.>: x^2_G:=(x,Gx)_ℝ^2, where G=[ 1/2  -1;  -1 5/2 ]. It is easy to show that for x∈ (L^2(Ω))^2, the G-norm is equivalent to the (L^2(Ω))^2 norm:

    C_ℓx_G≤x≤ C_ux_G.

Here, C_ℓ=3-2√(2) and C_u=3+2√(2). For v^i∈ L^2, i=n-1,n,n+1, we have the identity <cit.>:

    (1/2(3v^n+1-4v^n+v^n-1),v^n+1)   =1/2([v^n+1;v^n]_G^2-[v^n;v^n-1]_G^2)
       +1/4v^n+1-2v^n+v^n-1^2.




 §.§ CDA preliminaries


Denote by 
τ_H(Ω) a coarse mesh of Ω, whose nodes are locations of the measurements or observables of the true
solution. Assume h<<H and the nodes of mesh τ_H(Ω) are a subset of the vertices of the mesh τ_h(Ω)
 and that

if I_h is the nodal interpolant operator on τ_h, then 
I_H I_h u = I_H u for any u∈ H^1(Ω).
Also, I_H must satisfy the
usual CDA estimates: there exists a constant C_I independent of H satisfying

    I_Hv-v≤ C_IH ∇ v,    ∀ v∈ H^1(Ω),
    I_Hv≤ C_I v,    ∀ v∈ H^1(Ω).

We note that if I_H were the nodal interpolant, then C_I in (<ref>)  could be large due to an inverse dependence on h for v∈ X_h or Y_h and so this is not an ideal interpolant choice in most cases.  Instead, algebraic nudging <cit.>, the Scott-Zhang interpolant, or the L^2 projection onto the coarse space could be used <cit.>.


We also need the following lemma for the long-time analysis that is proven in <cit.>.

	Suppose constants r>1 and B≥ 0. If a real number sequence {a_n}_n≥ 0 satisfies
	
    ra_n+1≤ a_n+B,

	then 
	
    a_n+1≤ a_0(1/r)^n+1+B/r-1.







§ CDA POISSON AND CDA STOKES PROJECTIONS


In this section we will present and analyze Poisson and Stokes interpolants tailored to use with CDA.  The estimates derived in this section are the keys that allow for the optimal long-time accuracy results that are independent of the nudging parameter to hold in the sections that follow.




 §.§ CDA Poisson projection

The CDA Poisson projection is defined as follows: Given u∈ H^1_0(Ω) and I_H u, find u_h:= P_hu↦ u_h in Y_h satisfying

    κ (∇ u_h,∇ v_h ) + μ(I_H u_h, I_H v_h) = κ (∇ u,∇ v_h) + μ(I_H u,I_H v_h),   ∀ v_h ∈ Y_h,

where κ > 0 is a positive constant.


	Given u∈ H^1_0(Ω)∩ H^k+1(Ω) and μ≥ 0,  P_h u satisfies

    ∇ (u - P_h u)    ≤ Ch^k  u _H^k+1(Ω),
     u - P_h u    ≤ Ch^k+1 u _H^k+1(Ω),

	where C is independent of u, h, and μ.
	
	



We first rewrite the equation (<ref>) as follows:
	
    (∇ (u - u_h),∇ v_h) + μ/κ (I_H (u-u_h),I_H v_h)=0,   ∀ v_h∈ Y_h.


	Set e=u- u_h = (u - I_h u) + (I_h u - u_h) =: η + ϕ, where I_h u ∈ Y_h is the nodal interpolant of u.  Properties of I_h and I_H imply that 
	
    I_H η = I_H ( u - I_h u ) = I_Hu - I_HI_hu = I_H u - I_H u = 0.
  
	Using this, after choosing v_h = ϕ in (<ref>), then yields
	
    ∇ϕ^2 + μ/κ I_H ϕ^2 =  (∇η,∇ϕ) ≤1/2∇ϕ^2 + 1/2∇η^2,

	thanks to Young's inequality.  Reducing (<ref>) gives
	
    ∇ϕ^2 + 2μ/κ I_H ϕ^2 ≤∇η^2.

With the triangle inequality and standard approximation theory, we get
	
    ∇ e ^2 + μ/κ I_H e ^2 ≤ 4 ∇ (u - I_h u) ^2 ≤ Ch^2k u _H^k+1(Ω)^2,

	where C is independent of h and μ.  This proves the first result of the lemma.
	
	For the L^2 estimate, we begin by defining ξ̂∈ C^2(Ω)∩ C^0(Ω) satisfying
	
    -Δξ̂= e,   Ω.

	Let ψ be any harmonic function  (e.g. a harmonic polynomial <cit.>) that satisfies ψ(x_i)=ξ̂(x_i) for all the measurement points x_i.  Then we have for ξ:=ξ̂- ψ that
	
    -Δξ = e,  I_H ξ=0.

	Testing the first ξ equation with e yields
	
    e ^2 = (∇ξ,∇ e).

	From (<ref>) with v_h=I_h ξ, we have that
	
    (∇ e,∇ I_h ξ) + μ (I_H e,I_H I_h ξ)=0,

	and so combining this with (<ref>) gives
	
    e ^2 = (∇ e,∇ (ξ - I_h ξ)) - μ (I_H e,I_H I_h ξ).

	Since the measurement nodes { x_i }_i=1^M are nodes on τ_h and τ_H ⊂τ_h, I_H I_h ξ = I_H ξ=0.  Hence we have the estimate
	
    e    =  (∇ e,∇ (ξ - I_h ξ)) 
       ≤∇ e ∇ (ξ - I_h ξ ) 
       ≤ Ch ∇ e Δξ
        = Ch ∇ e  e,

	where C is an interpolation constant and independent of μ.  This reduces
	
    e ≤ Ch ∇ e,

	and using the bound (<ref>) for ∇ e proves the second result of the lemma.
	
	








 §.§ CDA Stokes projection


We now consider the CDA Stokes projection, which is defined as follows:  Given u∈ X and I_H u,
find u_h:= P_h^Su↦ u_h in V_h={v∈ X_h: (∇· v,q)=0  ∀ q∈ Q_h } satisfying

    ν (∇ u_h,∇ v_h) + μ(I_H u_h,I_H v_h)   =ν (∇ u,∇ v)+μ(I_Hu,I_Hv_h),   ∀ v_h∈ V_h.


At first, this projection may appear to be very similar to the CDA Poisson projection with Y_h replaced by V_h. However, there is a subtle difference regarding the fact that I_H does not necessarily map into the (discretely divergence free space) V_h.  Hence, there is a more delicate analysis needed to prove that this projection is optimal in L^2 and H^1.  

For simplicity, we will consider Scott-Vogelius elements with X_h = P_k(τ_h)∩ X and Q_h = P_k-1∩ L^2_0(Ω) with k=2, but the same idea can be applied for k>2 in 2D and for k≥ 3 in 3D (for lower order elements than these, the following construction would not work).  The key idea is to alter the nodal interpolation I_h similar to what was done in <cit.> as follows: I_h^S will be a nodal interpolant at all the vertices of τ_h (and therefore also at all the measurement nodes so that I_H I_h^S = I_H), but the value at the midedges is defined so that the total flux through any element boundary is 0.  Hence on each edge e, where {x_i}_i=1^3 are the nodes on e (i=1 and 3 correspond to vertices, i=2 to the midedge), and n⃗ is the unit outward normal, ψ_i(x) is the finite element nodal basis that belongs to support point x_i, e⃗_j is a d dimension unit vector with j^th component 1 and others 0, ψ_i^j(x) is a d dimension vector with j^th component as ψ_i(x) and others 0, we make the following correction to the midedge value:

    I_h^Su(x_2)    =
    		u(x_2) + ∫_eu(s)·n⃗ds-∑_i,j∫_e((e⃗_j· u(x_i))ψ_i^j(s))·n⃗ds/∑_j=1^d∫_eψ_2^j(s)·n⃗dsn⃗.

With this definition of I_h^S, one can calculate that the total flux through each element E is zero, and thus by the divergence theorem that ∫_E∇· I_h^S u dx=0.  With this, since ∇· X_h⊂ Q_h, we get that ∇· I_h^S u _L^2(E)=0 and thus ∇· I_h^S u =0.  Therefore, it holds that I_h^S: H^1→ V_h.

In addition to being divergence-free, it is proven in <cit.> that I_h^S has the following property.


	For u∈ L^∞(Ω), the interpolant I_h^S satisfies
	
    max_x∈ e|u(x)-I_h^Su(x)|≤ Ch^k+1.

  
Since all nodes of τ_h are on edges of the mesh, we immediately get that the max error at any node is Ch^k+1.  This can be used with standard approximation theory to get that

    u(x)-I_h^S(u)   ≤ Ch^k+1 u _H^k+1(Ω),
    ∇(u-I_h^Su)   ≤ Ch^k u _H^k+1(Ω).


With these approximation properties and with I_h^S mapping into V_h, we have the following result for the CDA-Stokes projection:

	Given u∈ X∩ H^k+1(Ω) and μ≥ 0,  the Stokes projection P_h^S u satisfies

    ∇ (u - P^S_h u)    ≤ Ch^k  u _H^k+1(Ω),
     u - P^S_h u    ≤ Ch^k+1 u _H^k+1(Ω),

	where C is independent of u, h, and μ.



Follow the proof of Lemma <ref> with I_h replaced by I_h^S and Y_h replaced by V_h.








	

§ ANALYSIS OF LARGE PARAMETER CDA FOR BDF2-FEM DISCRETIZATION OF HEAT EQUATION

	
We now consider an analysis of CDA with large nudging parameters applied to the BDF2-FEM discretization of the heat equation with form
	
    u_t - κΔ u     = f,
    
    	 u(·,0)   =u_0, 
    
    	 u|_∂Ω    = 0,

where κ > 0 is the thermal diffusivity constant.
	
Introducing the bilinear form a(·,·): H^1_0× H^1_0→ℝ defined by a(ϕ,v)=(κ∇ϕ,∇ v), the CDA algorithm we consider herein for the heat equation using a BDF2-FEM discretization is as follows:



	






We now prove that this method has temporally and spatially optimal long-time accuracy with respect to the L^2-norm for any μ>0.


	Let u∈ L^∞(0,∞;H^k+1(Ω)) be the solution of the heat equation (<ref>) where u_t, u_tt, u_ttt∈ L^∞(0,∞;L^2(Ω)), u^0=u_0∈ L^2(Ω), u^1∈ H_0^1(Ω), and f∈ L^∞(0,∞;L^2(Ω)). Then, for any given H>0 and μ≥1/2C_I^2H^2, the difference between the CDA solution of (<ref>) and the true solution satisfies: 

    w_h^n+1-u^n+1^2   ≤ C(1/1+λΔ t)^n+1([w_h^1-u^1;w_h^0-u^0]^2_G+κΔ t/4∇ (w_h^1-u^1)^2)
            +C/λ(Δ t^4+h^2k+2)

for any n≥ 1. Here, λ=min{Δ t^-1, κ C_ℓ^2 /2C_I^2H^2,κ C_ℓ^2 /4C_P^2} and the constant C is independent of μ, h, and Δ t.



	In (<ref>), the nudging parameter μ does not appear on any right side terms, which implies that there will be no adverse effect by considering arbitrarily large μ in the CDA algorithm. 



	Using Taylor's theorem, the solution of the heat equation (<ref>) satisfies
	
    (3u^n+1-4u^n+u^n-1/2Δ t, v_h)+a(u^n+1,v_h)   =(f,v_h)+Δ t^2/3(u_ttt(t^*),v_h),  ∀ v_h∈ Y_h,

where t^*∈[t^n-1,t^n+1].	
	Subtracting equation (<ref>) from (<ref>) gives the error equation

    (3e^n+1-4e^n+e^n-1/2Δ t, v_h)	   +a(e^n+1,v_h)+μ(I_He^n+1, I_Hv_h)
       =Δ t^2/3(u_ttt(t^*),v_h),  ∀ v_h∈ Y_h,

where e^n=u^n-w_h^n.
We decompose e^n as: e^n=(u^n-P_hu^n)+(P_hu^n-w_h^n)=η^n+ϕ^n. Then with the projection equation (<ref>), we have that a(η,v_h)+μ(I_H η,I_H v_h)=0, so we can rewrite (<ref>) as 

    (3ϕ^n+1-4ϕ^n+ϕ^n-1/2Δ t, v_h)	+a(ϕ^n+1,v_h)+μ(I_Hϕ^n+1, I_Hv_h)
       =-(3η^n+1-4η^n+η^n-1/2Δ t, v_h)+Δ t^2/3(u_ttt(t^*),v_h).

Taking v_h=ϕ^n+1 in equation (<ref>), using the BDF2 identity (<ref>), and multiplying by 2Δ t, we have 

    [ϕ^n+1;ϕ^n]^2_G-[ϕ^n;ϕ^n-1]^2_G+1/2ϕ^n+1-2ϕ^n+ϕ^n-1^2+2κΔ t∇ϕ^n+1^2
       +2μΔ t  I_Hϕ^n+1^2
    		=-(3η^n+1-4η^n+η^n-1, ϕ^n+1)+2Δ t^3/3(u_ttt(t^*),ϕ^n+1).

Following the analysis from <cit.> for the time derivative term and using the Cauchy-Schwarz and Young's inequalities on the right-hand side terms of (<ref>), we obtain

    -(3η^n+1-4η^n+η^n-1, ϕ^n+1)   ≤ Cκ^-1Δ t(η_t^2_L^∞(0,∞;L^2)+∫_t^n-1^t^n+1η_tt^2dt)+κΔ t/4∇ϕ^n+1^2,
    2Δ t^3/3(u_ttt(t^*),ϕ^n+1)   ≤  Cκ^-1Δ t^5u_ttt(t^*)^2    +κΔ t/4∇ϕ^n+1^2.

Using the interpolation property (<ref>), we  provide the lower bound of the left-hand side terms of equation (<ref>) by 

    2 κΔ t∇ϕ^n+1^2+   2μΔ tI_Hϕ^n+1^2  =κΔ t∇ϕ^n+1^2+2μΔ tI_Hϕ^n+1^2+κΔ t∇ϕ^n+1^2
       ≥κΔ t/C_I^2H^2ϕ^n+1-I_Hϕ^n+1^2+2μΔ tI_Hϕ^n+1^2+κΔ t∇ϕ^n+1^2
       ≥β(ϕ^n+1-I_Hϕ^n+1^2+I_Hϕ^n+1^2)+κΔ t∇ϕ^n+1^2
       ≥β/2ϕ^n+1^2+κΔ t∇ϕ^n+1^2,

where β=min{2μΔ t,κΔ t/C_I^2H^2}= κΔ t/C_I^2H^2 by allowing μ to be sufficient large, i.e., μ≥κ/2C_I^2H^2. 

Combining (<ref>), (<ref>), and (<ref>) results in

    [ϕ^n+1;ϕ^n]^2_G   +κΔ t/4∇ϕ^n+1^2+κΔ t/4∇ϕ^n+1^2+κΔ t/2C_I^2H^2ϕ^n+1^2+κΔ t/4∇ϕ^n^2
       ≤[ϕ^n;ϕ^n-1]^2_G+κΔ t/4∇ϕ^n^2
    	 + Cκ^-1Δ t^5u_ttt(t^*)^2 
       +Cκ^-1Δ t(η_t^2_L^∞(0,∞;L^2)+∫_t^n-1^t^n+1η_tt^2dt),

where we added term κΔ t/4∇ϕ^n^2 on both sides of (<ref>). 

After using the Poincaré inequality and norm equivalence (<ref>), we obtain

    [ϕ^n+1;ϕ^n]^2_G+κΔ t/4∇ϕ^n+1^2+λΔ t([ϕ^n+1;ϕ^n]^2_G+κΔ t/4∇ϕ^n+1^2)≤
         [ϕ^n;ϕ^n-1]^2_G +κΔ t/4∇ϕ^n^2+ Cκ^-1Δ t^5u_ttt(t^*)^2 
    		
           +Cκ^-1Δ t(η_t^2_L^∞(0,∞;L^2)+∫_t^n-1^t^n+1η_tt^2dt).

Here, λ=min{Δ t^-1, κ C_ℓ^2 /2C_I^2H^2,κ C_ℓ^2 /4C_P^2}.
Using Lemma <ref> and the estimate on η, we have

    [ϕ^n+1;ϕ^n]^2_G+κΔ t/4∇ϕ^n+1^2   ≤(1/1+λΔ t)^n+1([ϕ^1;ϕ^0]^2_G+κΔ t/4∇ϕ^1^2) 
    			
         +C/λ(Δ t^4+h^2k+2).

Finally, the proof of Theorem <ref> is completed by applying the triangle inequality and another application of the norm equivalence property (<ref>). 

    w_h^n+1-u^n+1^2≤ 2(η^n+1^2+	ϕ^n+1^2)
       ≤(1/(1+λΔ t))^n+1ϕ^0^2 +4C_I^2H^2/Δ t(4C_P^2Δ tη_t(t^**)^2+C_P^2Δ t^3u_tt(t^*)^2).

which completes the proof.










§ DISCRETE CDA ANALYSIS FOR NAVIER-STOKES EQUATIONS


We consider a BDF2-FEM algorithm with CDA for the NSE, and assume divergence-free elements (X_h,Q_h) that also satisfy the discrete inf-sup condition.  We assume u solves the Navier-Stokes equations:

    u_t + u·∇ u + ∇ p - νΔ u     = f, 
    ∇· u    =0,
    
    u(·,0)   = u_0,
    
    u|_∂Ω    = 0.

We define the following bilinear and trilinear forms, respectively:

    a^N(ϕ,v)=( ν∇ϕ,∇ v),  ∀ϕ,v∈ X,
    
    b(ϕ,w,v)=(  ϕ·∇ w, v),  ∀ϕ,w,v∈ X.








By construction, b(ϕ,v,v)=0 for all ϕ,v∈ X with ∇·ϕ=0,  and additionally have the following upper bounds <cit.>:
 for any ϕ,w,v∈ X it holds that for some C>0 dependent only on the size of Ω,

    b(ϕ,w,v)   ≤ C ∇ϕ∇ w∇ v,
    	b(ϕ,w,v)   ≤ C (w_L^∞+∇ w_L^3)ϕ∇ v.





 We now prove a long time optimal L^2 accuracy result for Algorithm <ref>.  This improves on the analysis in <cit.> by providing optimal accuracy in space and time that does not scale with μ when μ gets large.



		Let u∈ L^∞(0,T;H^k+1(Ω)) solve the Navier-Stokes equations (<ref>), u_tt∈ L^∞(0,T;H^1(Ω)), u_ttt∈ L^∞(0,T;L^2(Ω)), u^0 = u_0∈ L^2(Ω), u^1∈ X, and f∈ L^∞(0,T;L^2(Ω)). Assume the time step Δ t satisfies 
		
    1/2-Cν^-1Δ t(h^2k+2+h^2k+u^n+1^2_L^∞+∇ u^n+1^2_L^3)>0   and  μ≥ν/4C_I^2 H^2.
 If 
		
    ν/4C_I^2 H^2-Cν^-1Δ t(h^2k+2+h^2k+u^n+1^2_L^∞+∇ u^n+1^2_L^3)>0,

		then the difference between the CDA solution of (<ref>) and the true Navier-Stokes equations' solution satisfies: for any n≥ 1


    w_h^n+1-u^n+1^2   ≤ C(1/1+λΔ t)^n+1([w_h^1-u^1;w_h^0-u^0]^2_G+κΔ t/4∇ (w_h^1-u^1)^2)
           +Cν^-1(h^2k+2+h^4k+Δ t^4)/λ.

Here, λ=min{Δ t^-1,ν C_l^2/4C_P^2,(ν/4C_I^2 H^2-Cν^-1(h^2k+2+h^2k+u^n+1^2_L^∞+∇ u^n+1^2_L^3))C_l^2}, and note the constants are independent of μ. 




Note that for Δ t and H sufficiently small, λ=ν C_l^2/4C_P^2.  Then for n large enough, the L^2 accuracy is 𝒪(h^k+1 + Δ t^2).





	Using Taylor's theorem, it can be shown that the true solution of the Navier-Stokes equations (<ref>) satisfies 
	
    (3u^n+1-4u^n+u^n-1/2Δ t, v)	+a^N(u^n+1,v)+b(2u^n-u^n-1,u^n+1,v)
       =(f^n+1,v)+Δ t^2/3(u_ttt(t^*),v)+Δ t^2 b(u_tt(t^**),u^n+1,v),  ∀ v∈  X,

	where t^*,t^**∈[t^n-1,t^n+1].	Subtracting equation (<ref>) from (<ref>) gives an error equation:

    (3e^n+1-4e^n+e^n-1/2Δ t, v)	+a^N(e^n+1,v_h)-b(2e^n-e^n-1,e^n+1,v_h)
    	
         +b(2u^n-u^n-1,e^n+1,v_h)+b(2e^n-e^n-1,u^n+1,v_h)+μ(I_He^n+1, I_Hv_h)
    	
       =Δ t^2/3(u_ttt(t^*),v_h)+Δ t^2 b(u_tt(t^**),u^n+1,v_h),  ∀ v_h∈ X_h,

where e^n=u^n-w^n_h. We then decompose e^n as 

    e^n=(u^n-P_h^Su^n)+(P_h^Su^n-w_h^n)=η^n+ϕ^n

and, with the help of the projection equation (<ref>), we rewrite (<ref>) as:

    (3ϕ^n+1-4ϕ^n+ϕ^n-1/2Δ t, v)+(3η^n+1-4η^n+η^n-1/2Δ t, v)	+a^N(ϕ^n+1,v_h)
    			
         -b(2ϕ^n-ϕ^n-1,ϕ^n+1,v_h)-b(2ϕ^n-ϕ^n-1,η^n+1,v_h)-b(2η^n-η^n-1,ϕ^n+1,v_h)
    			
         -b(2η^n-η^n-1,η^n+1,v_h)+b(2u^n-u^n-1,ϕ^n+1,v_h)+b(2u^n-u^n-1,η^n+1,v_h)
    			
         +b(2ϕ^n-ϕ^n-1,u^n+1,v_h)+b(2η^n-η^n-1,u^n+1,v_h)+μ(I_H
    			ϕ^n+1, I_Hv_h)
    			
       =Δ t^2/3(u_ttt(t^*),v_h)+Δ t^2 b(u_tt(t^**),u^n+1,v_h),  ∀ v_h∈ X_h.

Taking v_h=ϕ^n+1 and using the identity b(·,v,v)=0, ∀ v∈ X, 
we have

    1/2Δ t([ϕ^n+1;ϕ^n]^2_G-[ϕ^n;ϕ^n-1]^2_G+1/2ϕ^n+1-2ϕ^n+ϕ^n-1^2)	+ν∇ϕ^n+1^2
       +μI_Hϕ^n+1^2
    		=-(3η^n+1-4η^n+η^n-1/2Δ t, ϕ^n+1)+Δ t^2b(u_tt(t^**),u^n+1,ϕ^n+1)
       
    		+b(2ϕ^n-ϕ^n-1,η^n+1,ϕ^n+1)+b(2η^n-η^n-1,η^n+1,ϕ^n+1)-b(2u^n-u^n-1,η^n+1,ϕ^n+1)
    		
         -b(2ϕ^n-ϕ^n-1,u^n+1,ϕ^n+1)-b(2η^n-η^n-1,u^n+1,ϕ^n+1)+Δ t^2/3(u_ttt(t^*),ϕ^n+1).

Applying standard inequalities such as Young's and the Cauchy-Schwarz inequalities, properties (<ref>)-(<ref>), we bound the right hand side term of (<ref>) as follows:


    Δ t^2/3(u_ttt(t^*),ϕ^n+1)   ≤ Cν^-1Δ t^4u_ttt(t^*)^2+ν/20∇ϕ^n+1^2
    
    Δ t^2b(u_tt(t^**),u^n+1,ϕ^n+1)   ≤ Cν^-1Δ t^4∇ u^n+1^2∇ u_tt(t^**)^2+ν/20∇ϕ^n+1^2
    
    
    -(3η^n+1-4η^n+η^n-1/2Δ t, ϕ^n+1)   ≤ Cν^-1(η_t^2_L^∞(0,∞;L^2)+∫_t^n-1^t^n+1η_tt^2dt)+ν/20∇ϕ^n+1^2
    
    
    b(2ϕ^n-ϕ^n-1,η^n+1,ϕ^n+1)   =-b(ϕ^n+1-2ϕ^n+ϕ^n-1,η^n+1,ϕ^n+1)+b(ϕ^n+1,η^n+1,ϕ^n+1)
    
       ≤ Cν^-1(η^n+1^2_L^∞+∇η^n+1^2_L^3)ϕ^n+1-2ϕ^n+ϕ^n-1^2
    
         +Cν^-1(η^n+1^2_L^∞+∇η^n+1^2)ϕ^n+1^2+ν/10∇ϕ^n+1^2
    
    
    b(2η^n-η^n-1,η^n+1,ϕ^n+1)   ≤ Cν^-1∇(2η^n-η^n-1)^2∇η^n+1^2+ν/20∇ϕ^n+1^2
    
    
    -b(2u^n-u^n-1,η^n+1,ϕ^n+1)   = b(2u^n-u^n-1,ϕ^n+1,η^n+1)
    
       ≤ Cν^-12u^n-u^n-1^2η^n+1^2_L^∞+ν/20∇ϕ^n+1^2
    
    
    -b(2ϕ^n-ϕ^n-1,u^n+1,ϕ^n+1)   =b(ϕ^n+1-2ϕ^n+ϕ^n-1,u^n+1,ϕ^n+1)-b(ϕ^n+1,u^n+1,ϕ^n+1)
    
       ≤ Cν^-1(u^n+1^2_L^∞+∇ u^n+1^2_L^3)ϕ^n+1-2ϕ^n+ϕ^n-1^2
    
         +Cν^-1(u^n+1^2_L^∞+∇ u^n+1^2)ϕ^n+1^2+ν/10∇ϕ^n+1^2
    
    
    -b(2η^n-η^n-1,u^n+1,ϕ^n+1)   = -b(2η^n-η^n-1,u^n+1,ϕ^n+1)
    
       ≤ Cν^-1(u^n+1^2_L^∞+∇ u^n+1^2_L^3)η^n-η^n-1^2
    
         +ν/20∇ϕ^n+1^2.

We use the interpolant inequality (<ref>) to lower bound ν∇ϕ^n+1^2+μI_Hϕ^n+1^2, following the analysis of <cit.>, as follows:

    ν∇ϕ^n+1^2+μI_Hϕ^n+1^2 ≥ν/4C_I^2H^2ϕ^n+1-I_Hϕ^n+1^2+μI_Hϕ^n+1^2+3ν/4∇ϕ^n+1^2
       ≥γ(ϕ^n+1-I_Hϕ^n+1^2+I_Hϕ^n+1^2)+3ν/4∇ϕ^n+1^2≥γ/2ϕ^n+1^2+3ν/4∇ϕ^n+1^2,

where γ=min{μ,ν/4C_I^2H^2}= ν/4C_I^2H^2 by allowing μ to be sufficient large, i.e., μ≥ν/4C_I^2H^2.

 Combining (<ref>), (<ref>), and (<ref>), multiplying by 2 Δ t, and taking into account the assumptions on the regularity of the solution u while utilizing standard approximation properties, leads to 

    [ϕ^n+1;ϕ^n]^2_G+νΔ t/4∇ϕ^n+1^2+νΔ t/4∇ϕ^n+1^2+νΔ t/4∇ϕ^n^2
    		
         +Δ t(ν/4C_I^2 H^2-Cν^-1(h^2k+2+h^2k+u^n+1^2_L^∞+∇ u^n+1^2_L^3))ϕ^n+1^2
    		
         +(1/2-Cν^-1Δ t(h^2k+2+h^2k+u^n+1^2_L^∞+∇ u^n+1^2_L^3))ϕ^n+1-2ϕ^n+ϕ^n-1^2
    		
       ≤[ϕ^n;ϕ^n-1]^2_G+νΔ t/4∇ϕ^n^2+Cν^-1(Δ th^2k+2+Δ th^4k+Δ t^5).

Choose H and Δ t so that ν/4C_I^2 H^2-Cν^-1Δ t(h^2k+2+h^2k+u^n+1^2_L^∞+∇ u^n+1^2_L^3)>0 and 1/2-Cν^-1Δ t(h^2k+2+h^2k+u^n+1^2_L^∞+∇ u^n+1^2_L^3)>0 are satisfied.  With these choices, using the Poincaré inequality, G-norm equivalence (<ref>), CDA Stokes projection error estimate, and proceeding similarly as (<ref>)-(<ref>), leads to

    [ϕ^n+1;ϕ^n]^2_G+νΔ t/4∇ϕ^n+1^2   ≤(1/1+λΔ t)^n+1([ϕ^1;ϕ^0]^2_G+νΔ t/4∇ϕ^1^2)
           +Cν^-1(h^2k+2+h^4k+Δ t^4)/λ,

where λ=min{Δ t^-1,ν C_l^2/4C_P^2,(ν/4C_I^2 H^2-Cν^-1(h^2k+2+h^2k+u^n+1^2_L^∞+∇ u^n+1^2_L^3))C_l^2}. Then, the proof is completed by applying the triangle inequality.




§ NUMERICAL EXPERIMENTS
 

We now illustrate the effectiveness of the proposed method for several test problems.  For all of our tests, we take the L^2 projection onto piecewise constants as I_H, implemented with algebraic nudging <cit.>.



 §.§ The heat equation




For our first test, we pick a problem with domain Ω=(0,1)^2,  κ=1, and analytical solution

    u = sin( t + 2π x + π y).

The solution then determines  f=u_t-Δ u, and the boundary conditions are taken to be Dirichlet on all of ∂Ω and equal to the solution.

We compute with the BDF2-FEM CDA algorithm above using H = 1/9 and μ=∞ (direct implementation) with P_2 elements on barycenter refinements of uniform triangular meshes.  We note that we also tested with μ=10^5, 10^8 and saw no difference in results between these solutions and those found using direct implementation.

The theory above from Theorem <ref> shows that for μ sufficiently large and for n large enough, 

    w_h^n - u(t^n) ≤ C(h^k+1 + Δ t^2).

We illustrate this theory by first taking small T=0.3 and Δ t=0.001, and varying h.  Errors and rates at the final time are shown in Table <ref>, and we observe third order convergence in space, which is consistent with the theory since we use P_2 elements.  To illustrate temporal convergence rates, we take h=1/256, T=1, and vary Δ t.  Errors and rates at the final time are shown in Table <ref>, and we observe rates consistent with second order temporal convergence  (the last error and rate are reduced due to temporal error no longer being completely dominant over spatial error with the smallest choice of Δ t).  Plots of the error measured in the L^2-norm for all of these tests are shown in Figure <ref>, and the results are consistent with the theory - exponential convergence in time up to the discretization error, then long-time optimal accuracy, i.e. O(h^3 + Δ t^2) since P_2 elements are used for this test, with respect to the L^2-norm.













 §.§ A contaminant transport model


In this numerical experiment, a CDA-FEM of the heat equation with added linear transport that models contaminant transport in a river is investigated. The linear transport equation takes the form

    u_t + U ·∇ u - κΔ u    =    f, 
    
    u(0)     =     u_0,

where U is a given divergence free velocity field, κ is the (material dependent) diffusion coefficient, and here u represents the concentration of a contaminant.  This system is similar to heat equation with an additional linear term U·∇ u, and thus CDA can be applied in the same way as above for the heat equation with (essentially) the same convergence results.

We follow a test problem from <cit.>, the domain for which is the area enclosed by the curves y=sin(x), y=1+sin(x), x=0 and x=4π as left and right boundaries.  Using the mesh τ_h shown in Figure
<ref>, we use (P_2,P_1) Taylor-Hood velocity-pressure elements (which gives 13,928 degrees of freedom (dof)) to compute the Stokes equations with viscosity 0.01 and f=0, using no-slip boundary conditions on the top and bottom boundaries, a plug inflow of u_in=3, and a zero-traction outflow enforced with the do-nothing condition.  We take our transport velocity U to be the velocity solution of this discrete Stokes problem.  









We use BDF2-FEM with CDA to solve (<ref>)-(<ref>) equipped with a boundary condition of 0 at the inflow (c_in=0), the transport velocity U coming from the discrete Stokes problem, transport term discretization 

    b^*(U,w_h^n+1,v):= b(U,w_h^n+1,v_h) + 1/2 ((∇· U)w_h^n+1,v_h)

where b(U,w_h^n+1,v_h) := (U·∇ w_h^n+1,v_h), κ=0.01, and 
zero Neumann conditions  ∇ c · n =0 at all other boundaries.  For the initial condition, we take zero contaminant except for two `blobs', which are represented with c=3 inside the circles  with radius 0.1 centered at (1,1.5) and (5,-0.5).  We first compute a direct numerical simulation (DNS) for the concentration c using BDF2-FEM, P_2 on the same mesh as is used for the Stokes problem, and time step Δ t=0.02.  







Finally, we compute the CDA solution, using BDF2-CDA with the same parameters as the DNS except zero initial conditions, taking the DNS solution u as the true solution, and testing the algorithm using several choices of μ including μ=∞ which we implement with direct enforcement.  The 37 measurement nodes are shown in Figure <ref> and we note that these nodes are also vertices on the finite element mesh.  Convergence of the CDA solution to the true (DNS) solution is shown in Figure <ref>, for varying μ.  We observe very similar convergence for each choice of μ=0.1, 1, 10000, and ∞, with the plots for μ=10000 and ∞ (direct enforcement) laying on top of each other.  For the case μ=∞, we show contour plots of the DA and true (DNS) solution at t=0, 1, 2, and 5 in Figure <ref>, and observe that the agreement between the solutions increases as time-stepping moves forward and that the solutions are visually indistinguishable at the final stopping time.





 §.§ The Navier-Stokes equations






To illustrate the theory for the Navier-Stokes equations, we choose a test problem from <cit.> for simulating 2D Kelvin-Helmholtz instability.  The domain is taken to be the unit square and enforce periodic
boundary conditions on x=0,1 (representing an infinite extension in the horizontal direction).  At y=0,1, we enforce a no penetration and free slip condition.  The initial condition for the benchmark problem is set as

    u_0(x,y) = ( [ u_∞tanh( 2y-1/δ_0);                  0 ]) + c_n ( [  ∂_y ψ(x,y); -∂_x ψ(x,y) ]),

where δ_0=1/28 is the initial vorticity thickness, u_∞=1 is set as the reference velocity, c_n is a noise/scaling factor that is taken to be 10^-3, and

    ψ(x,y) = u_∞exp( -(y-0.5)^2 /δ_0^2) ( cos(8π x) + cos(20π x) ).

The Reynolds number for this test is defined by Re=δ_0 u_∞/ν = 1/28 ν, and ν is defined by selecting Re.  We use Re=100 for this test.

We compute solutions for the BDF2-CDA for the NSE similar to the previous experiments: first compute a reference solution (using the initial condition above), and then use this reference solution as the true solution to nudge towards.  We use (P_2,P_1^disc) Scott-Vogelius elements on a barycenter refinement of an h=1/96 uniform triangular mesh, and a time step size of Δ t=0.01.  For the CDA, we take the initial condition to be w_h^0=w_h^1=0, and compute using Algorithm <ref> with μ=∞ (implemented with direct enforcement) and varying H (=1/25, 1/20, and 1/15) using N=1/H^2 measurement points.  Convergence of the CDA solutions to the true solutions are shown in Figure <ref>, and we observe that the method is converging in L^2 for all three choices of H with faster convergence as H decreases.  Contour plots of the solution's absolute vorticity magnitudes are shown in Figure <ref>, along with those of the true solution.  At t=2, none of the CDA solutions appear accurate.  By t=5, the N=20 and 25 solutions are already approaching the true solution, and by t=8 their solutions are visually indistinguishable from the true solution.  At t=9, there are still visible errors in the N=15 solution, demonstrating that the inclusion of more information yields faster convergence.
  








 §.§ The Cahn-Hilliard equation
 

In this subsection, we briefly demonstrate via a numerical experiment that similar CDA projections could be established for different types of evolutionary PDEs. Specifically, we consider the Cahn-Hilliard (CH) equation <cit.>

    ∂_t φ - Δ(φ^3 - φ) + ε^2  Δ^2 φ    = 0, 
    φ(·,0)    =φ_0,

where φ represents the order parameter which takes on values between -1 and 1 and is often interpreted as a concentration of one component in a two component system. The states φ = ± 1 indicate phases of pure concentration and ε > 0 is understood to represent an interfacial width between the two phases. The following CDA-FEM algorithm for the CH equation (<ref>) was analyzed in <cit.>: Given ϕ_h^n-1∈ Z_h and true solution φ∈ L^∞(0,T;H_N^2(Ω)), find ϕ_h^n∈  Z_h such that 

    (ϕ_h^n - ϕ_h^n-1/Δ t, ψ) + (∇((ϕ_h^n)^3 - ϕ_h^n-1),∇ψ) + ε^2 ϕ_h^nψ + μ(I_H (ϕ_h^n - φ^n),ψ) =0 ,

for all ψ∈ Z_h with Z_h := H^1(Ω)∩ P_k(τ_h(Ω)). Here, the bilinear form ·· has the following definition:

    wv :=    ∑_K ∈τ_h∫_K (∇^2 w : ∇^2 v)   dx + ∑_e ∈ℰ_h∫_e {-5mu{∂^2 w/∂ n_e^2}-5mu}∂ v/∂ n_e dS 
    
       + ∑_e ∈ℰ_h∫_e {-5mu{∂^2 v/∂ n_e^2}-5mu}∂ w/∂ n_e dS + σ∑_e ∈ℰ_h1/|e|∫_e  ∂ w/∂ n_e∂ v/∂ n_e dS,

with σ≥ 1 known as a penalty parameter. The jumps and averages that appear in (<ref>) are defined as follows. For an interior edge e shared by two triangles K_± where n_e points from K_- to K_+, we define on the edge e

    ∂ v/∂ n_e = n_e ·(∇ v_+ - ∇ v_-),
      {-5mu{∂^2 v/∂ n_e^2}-5mu} = 1/2( ∂^2 v_-/∂ n_e^2 +  ∂^2 v_+/∂ n_e^2),

where ∂^2 v/∂ n_e^2 = n_e ·(∇^2 v) n_e and v_± = v |_K_±. For a boundary edge e, we take n_e to be the unit normal pointing towards the outside of Ω and define on the edge e

    ∂ v/∂ n_e = - n_e ·∇ v_K ,
      {-5mu{∂^2 v/∂ n_e^2}-5mu} = n_e ·( ∇^2 v ) n_e.


In <cit.>, it was once again determined that an upper bound to the nudging parameter μ was required in the analysis of the FEM CDA scheme. However, the following numerical experiment suggests otherwise, in which we consider a square domain Ω = (0,1)^2 and take τ_h(Ω) to be a regular triangulation of Ω consisting of right isosceles triangles that are a quasi-uniform family. (We use a family of meshes τ_h(Ω) such that no triangle in the mesh has more than one edge on the boundary.) 

As the Firedrake Project <cit.> was utilized this experiment, we now describe how the data assimilation term μ(I_H(ϕ_h^n - φ^n),  I_H ψ) was implemented. Specifically, a true solution φ was obtained at all times by selecting a cross shaped region as initial conditions, setting the interfacial width parameter ε = 0.002 and the nudging parameter μ = 0, and solving the CH equation using the C^0 interior penalty FEM (<ref>). A data assimilation grid size H was chosen and grid points were identified and located on the finite element mesh. 





Finally, the initial conditions for the numerical solution ϕ_0 was set to random initial conditions.

Figure <ref> demonstrates the effectiveness of the CDA-FEM for various grid sizes H with a large nudging parameter of μ = 10^9. Specifically, we choose five different grid sizes H = 0.011049, 0.015625, 0.03125, 0.0625, and 0.125, which correspond respectively to 8,100, 4,096, 1,024, 256 and 64 grid points, while the fine mesh uses piecewise quadratics and has 33,025 grid points.   Theorem 4.3 from <cit.> provides a sufficient condition that the grid size should be chosen as H = 𝒪( ε^2) but our experiments suggest that a grid size much coarser than that will produce good results if a large enough nudging parameter is chosen. This also suggests that the new theory developed herein should allow for a relaxation of the restriction on the size of the data assimilation grid.




Figure <ref> demonstrates the effectiveness of the CDA-FEM for various values of the nudging parameter μ. For this experiment, we set the data assimilation grid to be H = 0.03125, and choose four different values for the nudging parameter μ = 2500, 5000, 10000, and 10^9. Theorem 4.3 from <cit.> admits a sufficient condition that the appropriate value for the nudging parameter μ is at least 1/ε^2 = 2500, but if μ is too large then H needs to be very small.  However, our experiments show that good results can also be obtained for much larger values of μ without requiring H to be very small.  








§ CONCLUSION

In this paper, we proved optimal long time convergence in L^2 for finite element methods with continuous data assimilation without requiring any upper restrictions on the nudging parameter μ, and without any negative influence of μ on the error estimate. The goal was achieved by proposing a CDA Poisson projection along with an optimal projection error estimate, in which a harmonic approximation and nodal interpolant played key roles in the the analysis. We then applied the CDA Poisson projection to CDA convergence  analysis for two typical evolution PDEs: the heat equation and the Navier-Stokes equations. The key contribution here is the fact that with these new projection, we were able to eliminate the scaling of the error with μ as μ gets large.   This in turn allows for choosing large or very large μ, with the latter being numerically equivalent to direct Dirichlet enforcement of the measurement data into the discrete solution.

Multiple numerical tests illustrate the theory.


Future work will consider the case of how to choose μ for noisy data.  In many applications of CDA, the data measurements will have error present, and it is an open question of how μ should be chosen if certain properties of the noise are known, such as signal to noise ratio or distribution of the noise.





§ ACKNOWLEDGEMENTS

Authors Xuejian Li and Leo G. Rebholz were partially supported by NSF grant No. DMS-2152623. Author Amanda E. Diegel was partially supported by NSF grant No. DMS-2110768. Also, the authors thank Dr. Quyuan Lin for helpful discussions regarding to this work.  


















10

Albanez_Nussenzveig_Lopes_Titi_2016
D. Albanez, H. Nussenzveig Lopes, and E. Titi.
Continuous data assimilation for the three-dimensional
  Navier–Stokes-α model.
Asymptotic Anal., 97(1-2):139–164, 2016.

Anthes_1974_JAS
R.A. Anthes.
Data assimilation and initialization of hurricane prediction models.
J. Atmos. Sci., 31(3):702–719, 1974.

AMGC2002
M.S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp.
A tutorial on particle filters for online nonlinear/non-Gaussian
  Bayesian tracking.
IEEE T. Signal Proces., 50(2):174–188, 2002.

AN2012
D. Auroux and M. Nodet.
The back and forth nudging algorithm for data assimilation problems:
  theoretical results on transport equations.
ESAIM Control Optim. Calc. Var., 18(2):318–342, 2012.

DJ2005
D. Aurouxa and J. Blum.
Back and forth nudging algorithm for data assimilation problems.
C. R. Math. Acad. Sci. Paris, 340(12):873–878, 2005.

ABR01
S. Axler, P. Bourdon, and W. Ramey.
Harmonic Function Theory.
Springer, New York, 2001.


AOT14
A. Azouani, E. Olson, and E.S. Titi.
Continuous data assimilation using general interpolant observables.
J. Nonlinear Sci., 24:277–304, 2014.

Bessaih_Olson_Titi_2015
H. Bessaih, E. Olson, and E.S. Titi.
Continuous data assimilation with stochastically noisy data.
Nonlinearity, 28(3):729–753, 2015.

Biswas_Hudson_Larios_Pei_2017
A. Biswas, J. Hudson, A. Larios, and Y. Pei.
Continuous data assimilation for the 2D magnetohydrodynamic
  equations using one component of the velocity and magnetic fields.
Asymptot. Anal., 108(1-2):1–43, 2018.

Cahn:61:spinodal
J.W. Cahn.
On spinodal decomposition.
Acta Metall Mater, 9(9):795–801, 1961.

CH:58:free
J.W. Cahn and J.E. Hilliard.
Free energy of a nonuniform system. I. interfacial free energy.
J. Chem. Phys., 28(2):258–267, 1958.

CGJP22
Y. Cao, A. Giorgini, M. Jolly, and A. Pakzad.
Continuous data assimilation for the 3D Ladyzhenskaya model:
  analysis and computations.
Nonlinear Anal. Real World Appl., 68(103659):1–29, 2022.

CHL20
E. Carlson, J. Hudson, and A. Larios.
Parameter recovery for the 2 dimensional Navier–Stokes equations via
  continuous data assimilation.
SIAM J.Sci. Comput., 42(1):A250–A270, 2020.

C1998
P. Courtier, E. Andersson, W. Heckley, J. Pailleux, D. Vasiljevic, M. Hamrud,
  A. Hollingsworth, F. Rabier, and M. Fisher.
The ECMWF implementation of three-dimensional variational
  assimilation (3D-Var). I:formulation.
Q. J. R. Meteorol. Soc., 124(550):1783–1807, 1998.

C1994
P. Courtier, J.N. Thepaut, and A Hollingsworth.
A strategy for operational implementation of 4D-var, using an
  incremental approach.
Q. J. Roy. Meteor. Soc., 120(519):1367–1387, 1994.

DR22
A.E. Diegel and L.G. Rebholz.
Continuous data assimilation and long-time accuracy in a C^0
  interior penalty method for the Cahn-Hilliard equation.
Appl. Math. Comput., 424:127042, 2022.

G2009
G. Evensen.
The ensemble Kalman filter for combined state and parameter
  estimation: Monte Carlo techniques for data assimilation in large systems.
IEEE Control Syst. Mag., 29(3):83–104, 2009.

FJT15
A. Farhat, M.S. Jolly, and E.S. Titi.
Continuous data assimilation for the 2d Bénard convection through
  velocity measurements alone.
Physica D., 303:59–66, 2015.

FK2018
P. Fearnhead and H.R. Künsch.
Particle filters and data assimilation.
Annu. Rev. Stat. Appl., 5(1):421–449, 2018.

GN20
B. Garcia-Archilla and J. Novo.
Error analysis of fully discrete mixed finite element data
  assimilation schemes for the Navier-Stokes equations.
Adv. Comput. Math., 46(61): 1–33, 2020.

GNT18
B. Garcia-Archilla, J. Novo, and E.S. Titi.
Uniform in time error estimates for a finite element method applied
  to a downscaling data assimilation algorithm.
SIAM J. Numer. Anal., 58:410–429, 2020.

GLRVZ21_CDA
M. Gardner, A. Larios, L.G. Rebholz, D. Vargun, and C. Zerfas.
Continuous data assimilation applied to a velocity-vorticity
  formulation of the 2D Navier-Stokes equations.
Electronic Research Archive, 29(3):2223–2247, 2021.

HairerWanner
E. Hairer and G. Wanner.
Solving ordinary differential equations. II, volume 14 of
  Springer Series in Computational Mathematics.
Springer-Verlag, Berlin, second edition, 1996.
Stiff and differential-algebraic problems.

R:16:firedrake
D.A. Ham, P.H.J. Kelly, L. Mitchell, C.J. Cotter, R.C. Kirby, K. Sagiyama,
  N. Bouziani, S. Vorderwuelbecke, T.J. Gregory, J. Betteridge, D.R. Shapero,
  R.W. Nixon-Hill, C.J. Ward, P.E. Farrell, P.D. Brubeck, I. Marsden, T.H.
  Gibson, M. Homolya, T. Sun, A.T.T. McRae, F. Luporini, A. Gregory, M. Lange,
  S.W. Funke, F. Rathgeber, G-T. Bercea, and G.R. Markall.
Firedrake User Manual.
Imperial College London and University of Oxford and Baylor
  University and University of Washington, first edition edition, 5 2023.

HRV24
E. Hawkins, L. Rebholz, and D. Vargun.
Removing splitting/modeling error in projection/penalty methods for
  Navier-Stokes simulations with continuous data assimilation.
Communications in Mathematical Research, 40(1):1–29, 2024.

X2016
T. Heister, L. G. Rebholz, and M.Y. Xiao.
Flux-preserving enforcement of inhomogeneous Dirichlet boundary
  conditions for strongly divergence-free mixed finite element methods for flow
  problems.
J. Math. Anal. Appl., 438(1):507–513, 2016.

Hoke_Anthes_1976_MWR
J.E. Hoke and R.A. Anthes.
The initialization of numerical models by a dynamic-initialization
  technique.
Monthly Weather Review, 104(12):1551–1556, 1976.

HJK18
J. Honeycutt, H. Johnson, and S. Kelly.
Using data assimilation to better predict contaminant transport in
  fluids.
SIAM Undergraduate Research Online, 11:1–16, 2018.

IMT20
A.H. Ibdah, C.F. Mondaini, and E.S. Titi.
Fully discrete numerical schemes of a data assimilation algorithm:
  uniform-in-time error estimates.
IMA J. Numer. Anal., 40(4):2584–2625, 11 2019.

Jolly_Martinez_Olson_Titi_2018_blurred_SQG
M.S. Jolly, V.R. Martinez, E.J. Olson, and E.S. Titi.
Continuous data assimilation with blurred-in-time measurements of the
  surface quasi-geostrophic equation.
Chin. Ann. Math. Ser. B, 40(5):721–764, 2019.

Kalman_1960_JBE
R.E. Kalman.
A new approach to linear filtering and prediction problems.
J. Basic Eng., 82(1):35–45, 1960.

LLC2019
A. Larios, L.G. Rebholz, and C. Zerfas.
Global in time stability and accuracy of IMEX-FEM data
  assimilation schemes for Navier-Stokes equations.
Comput. Methods Appl. Mech. Engrg., 345:1077–1093, 2019.

Larios_Victor_2019
A. Larios and C. Victor.
Continuous data assimilation with a moving cluster of data points for
  a reaction diffusion equation: A computational study.
Commun. Comp. Phys., 29(4):1273–1298, 2021.

laytonBook
W. Layton.
An Introduction to the Numerical Analysis of Viscous
  Incompressible Flows.
SIAM, Philadelphia, 2008.

Lewis_Lakshmivarahan_2008
J. Lewis and S. Lakshmivarahan.
Sasakiś pivotal contribution: calculus of variations applied to
  weather map analysis.
Mon. Weather Rev., 136(9):3553–3567, 2008.

L2023
X. Li, E.V. Hawkins, L.G. Rebholz, and D. Vargun.
Accelerating and enabling convergence of nonlinear solvers for
  Navier-Stokes equations by continuous data assimilation.
Comput. Methods Appl. Mech. Engrg., 416:1–17, 2023.

LGHL2023
X. Li, G. Wei, X.M. He, and T. Lin.
Variational data assimilation and its decoupled iterative numerical
  algorithms for Stokes–Darcy model.
SIAM J. Sci. Comput., 2023.

Miranville:19:CH
A. Miranville.
The Cahn–Hilliard Equation: Recent Advances and
  Applications.
SIAM, 2019.

OR24
M. Olshanskii and L. Rebholz.
Local conservation laws of continuous Galerkin method for the
  incompressible Navier-Stokes equations in EMAC form.
Computer Methods in Applied Mechanics and Engineering,
  418B(116583):1–14, 2024.

R2007
F. Rawlins, S.P. Ballard, K.J. Bovis, A.M. Clayton, D. Li, G.W. Inverarity,
  A.C. Lorenc, and T.J. Payne.
The met office global four-dimensional variational data assimilation
  scheme.
Q. J. R. Meteorol. Soc., 133(623):347–362, 2007.

RZ21
L.G. Rebholz and C. Zerfas.
Simple and efficient continuous data assimilation of evolution
  equations via algebraic nudging.
Numer. Methods Partial Differ. Equ., 37(3):2588–2612, 2021.

SJLLLS18
P. Schroeder, V. John, P. Lederer, C. Lehrenfeld, G. Lube, and J. Schoberl.
On reference solutions and the sensitivity of the 2D
  Kelvin-Helmholtz instability problem.
Computers and Mathematics with Applications, 77:1010–1028,
  2019.

ZRSI19
C. Zerfas, L. Rebholz, M. Schneier, and T. Iliescu.
Continuous data assimilation reduced order models of fluid flow.
Comput. Methods Appl. Mech. Eng., 357(112596):1–21, 2019.

zou_navon_ledimet_1992
X. Zou, I.M. Navon, and F.X. Ledimet.
An optimal nudging data assimilation scheme using parameter
  estimation.
Q. J. R. Meteorol. Soc, 118(508):1163–1186, 1992.





