Solving PDEs with Unmeasurable Source Terms Using Coupled
  Physics-Informed Neural Network with Recurrent Prediction for Soft Sensors
Partial differential equations (PDEs) are a model candidate for soft sensors
in industrial processes with spatiotemporal dependence. Although
physics-informed neural networks (PINNs) are a promising machine learning
method for solving PDEs, they are infeasible for the nonhomogeneous PDEs with
unmeasurable source terms. To this end, a coupled PINN (CPINN) with a recurrent
prediction (RP) learning strategy (CPINN- RP) is proposed. First, CPINN
composed of NetU and NetG is proposed. NetU is for approximating PDEs solutions
and NetG is for regularizing the training of NetU. The two networks are
integrated into a data-physics-hybrid loss function. Then, we theoretically
prove that the proposed CPINN has a satisfying approximation capability for
solutions to nonhomogeneous PDEs with unmeasurable source terms. Besides the
theoretical aspects, we propose a hierarchical training strategy to optimize
and couple NetU and NetG. Secondly, NetU-RP is proposed for compensating
information loss in data sampling to improve the prediction performance, in
which RP is the recurrently delayed outputs of well-trained CPINN and hard
sensors. Finally, the artificial and practical datasets are used to verify the
feasibility and effectiveness of CPINN-RP for soft sensors.
Aina Wang, Pan Qin, Xi-Ming Sun
Coupled Physics-informed Neural Networks for Inferring Solutions of Partial Differential Equations with Unknown Source Terms

###### Abstract

Physics-informed neural networks (PINNs) provide a transformative development for approximating the solutions to partial differential equations (PDEs). This work proposes a coupled physics-informed neural network (C-PINNs) for the nonhomogeneous PDEs with unknown dynamical source terms, which is used to describe the systems with external forces and cannot be well approximated by the existing PINNs. In our method, two neural networks, \(NetU\) and \(NetG\), are proposed. \(NetU\) is constructed to generate a quasi-solution satisfying PDEs under study. \(NetG\) is used to regularize the training of \(NetU\). Then, the two networks are integrated into a data-physics-hybrid cost function. Finally, we propose a hierarchical training strategy to optimize and couple the two networks. The performance of C-PINN is proved by approximating several classical PDEs.

 Coupled physics-informed neural network, hierarchical training strategy, partial differential equations, unknown source term

## I Introduction

Partial differential equations (PDEs) are one of the general representations for describing spatio-temporal dependence in physics [1], medicine [2], engineering [3], finance [4], and weather [5, 6]. Numerical approaches, like the finite difference method (FDM) [7] and finite element (FEM) [8, 9], have been widely investigated and applied. FDM used a topologically square lines network to construct PDEs' discretization. Thus, complex geometries in multiple dimensions challenge FDM [10]. On the other hand, complicated geometries can be treated with FEM [11]. The greatest difficulty of classical numerical approaches is balancing the accuracy and efficiency of forming meshes.

Among the numerical methods for solving PDEs, the Galerkin method is a famous computation method in which the linear combination of basis functions was employed to approximate the solutions to PDEs [12]. Motivated by this, several works have used machine learning models to replace the linear combination of basis functions to construct data-efficient and physics-informed learning methods for solving PDEs [13, 14, 15]. Successful applications of deep learning methods to various fields, like image [16], text [17], and speech recognition [18], ensure that they are excellent replacers of the linear combination of basis functions for solving PDEs [4]. Consequently, leveraging the well-known approximation capability of neural networks to solve PDEs is a natural idea and has been investigated in various forms previously [19, 20, 21]. The framework of physics-informed neural networks (PINNs) [22] was introduced to solve the forward problems while respecting any given physical laws governed by PDEs, including the nonlinear operator, initial, and boundary conditions. Within the PINNs framework, both the sparse measurements and the physical knowledge were fully integrated into cost function [23, 24]. The solution with respect to spatio-temporal dependence was obtained by training the cost function. Note that the approximation obtained by machine learning and deep learning is meshfree, which has no problem on balancing accuracy and efficiency of forming meshes.

Meanwhile, the potential of using PINNs to solve the inverse problem is promising [25]. A hybrid PINN was proposed to solve PDEs in [26], in which a local fitting method was combined with neural networks to solve PDEs. The hybrid PINN was used to identify unknown constant parameters in PDEs. The generative adversarial network (GAN) [27] was also physics-informed to solve the inverse problems. The stochastic physics-informed GAN was investigated for estimating the distributions of unknown parameters in PDEs. The recent work [28] encoded the physical laws governed by PDEs into the architecture of GANs to solve the inverse problems for stochastic PDEs. PINNs were also combined with the Bayesian method to solve inverse problems from noisy data [29].

PDEs can be classified into homogeneous and nonhomogeneous types. Systems without external forces can be described by the homogeneous PDEs. The nonhomogeneous PDEs can be applied to reveal the continuous energy propagation behavior of the source and hereby are effective for describing practical systems driven by external forces. The function forms of the solution and the source term were both assumed to be unknown in [30], in which the measurements of the source term should be obtained separately from the measurements of the solution. However, the independent measurements of the external forces cannot always be easily obtained from practical situations. The recent work [31] can directly solve the steady-state PDEs' forward and inverse problems, where the source terms were assumed to be constant. Thus, [31] was not feasible for systems with unsteady external forces, which should be described by dynamical functions.

Although the aforementioned methods have made great progress on unknown parameters, prior information or measurements on external forces cannot always be easily obtained
from practical situations. For example, the real distribution of the seismic wave field underground is unknown [32]; the vast of signals internal engine, indicating the operation state of the engine, cannot be isolated [33]. Furthermore, the existing methods with the assumption of the constant source term cannot be readily extended to describe the spatio-temporal dependence of complex dynamical systems. The determination of dynamical source terms with less prior information or even without any prior information is an under-investigated issue.

To this end, this paper proposes a coupled-PINN (C-PINN), using the sparse measurements and limited prior information of PDEs, to solve PDEs with unknown source terms. In our method, two neural networks, \(NetU\) and \(NetG\), are proposed. \(NetU\) is applied to generate a quasi-solution satisfying PDEs under study; \(NetG\) is used to regularize the training of \(NetU\). Then, the two networks are integrated into a data-physics-hybrid cost function. Furthermore, we propose a hierarchical training strategy to optimize and couple the two networks. Finally, the proposed C-PINN is applied to solve several classical PDEs to demonstrate its performance.

The rest of the paper is organized as follows. The classical PINNs is briefly reviewed in Section II. A C-PINN using the sparse measurements and limited prior knowledge to solve PDEs with unknown source terms is proposed in Section III. Meanwhile, the two neural networks, \(NetU\) and \(NetG\), are proposed in our method. Furthermore, a hierarchical training strategy is proposed to optimize and couple the two networks. In Section IV, our proposed C-PINN is validated with four case studies. In Section V, the concluding remarks and the future work are presented.

## II Brief Review of PINNs

In this section, we briefly review the basic idea of PINNs for data-driven solutions to PDEs and data-driven discovery of PDEs [22].

Data-driven solutions to PDEs describe that solve PDEs of the generalized form

\[u_{t}(\mathbf{x},t)+\mathcal{N}[u(\mathbf{x},t)]=0,\,\,x\in\Omega\subseteq\mathbb{R}^ {d},\,\,t\in[0,T]\subset\mathbb{R} \tag{1}\]

with known parameters. Here, \(\mathbf{x}\) is the spatial variable, \(t\) is the temporal variable with \(t=0\) being at the initial state, \(u:\mathbb{R}^{d}\times\mathbb{R}\rightarrow\mathbb{R}\) denotes the hidden solution, \(\mathcal{N}[\cdot]\) is a series of partial differential operators, the domain \(\Omega\subseteq\mathbb{R}^{d}\) is a spatial bounded open set with the boundary \(\partial\Omega\). Analytical or numerical methods have been widely investigated to find proper solution \(\psi(\mathbf{x},t)\) satisfying (1) [34]. The left-hand-side of (1) can be used to define a residual function as the following

\[f(\mathbf{x},t):=u_{t}(\mathbf{x},t)+\mathcal{N}[u(\mathbf{x},t)], \tag{2}\]

where a neural network is used to approximate the solution \(\psi(\mathbf{x},t)\) to PDEs. The inverse problem is focused on the data-driven discovery of PDEs of the generalized form (1), where unknown parameters of PDEs here turn into parameters of PINNs.

PINNs for both problems can be trained by minimizing the cost function

\[\text{MSE}=\text{MSE}_{D}+\text{MSE}_{PH}. \tag{3}\]

Here, \(\text{MSE}_{D}\) is formulated as the following

\[\text{MSE}_{D}=\sum_{(\mathbf{x},t)\in D}\left(\hat{u}\left(\mathbf{x},t;\hat{\mathbf{ \Theta}}_{U}\right)-u\left(\mathbf{x},t\right)\right)^{2}, \tag{4}\]

where \(\hat{u}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}\right)\) is the function of neural network with \(\hat{\mathbf{\Theta}}_{U}\) being its trained parameter set. Let \(D\) denote the training dataset. This mean squared error term can be considered as the data-driven loss. \(\text{MSE}_{PH}\) is as the following

\[\text{MSE}_{PH}=\sum_{(\mathbf{x},t)\in E}\hat{f}\left(\mathbf{x},t\right)^{2}, \tag{5}\]

which regularizes \(\hat{u}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}\right)\) to satisfy (1). Let \(E\) denote the set of collocation points. This regularization term can be considered as the physics-informed loss for the homogeneous PDEs. Here, \(\hat{f}\left(\mathbf{x},t\right)\) is defined as

\[\hat{f}\left(\mathbf{x},t\right):=\hat{u}_{t}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U }\right)+\mathcal{N}\left[\hat{u}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}\right) \right], \tag{6}\]

where \(\hat{u}_{t}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}\right)\) and \(\mathcal{N}\left[\hat{u}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}\right)\right]\) can be obtained using automatic differential [35].

## III Constructing C-PINN

C-PINN for solving PDEs with unknown source terms is presented in this section. The nonhomogeneous PDEs are of the following generalized form

\[u_{t}(\mathbf{x},t)+\mathcal{N}[u(\mathbf{x},t)]=g(\mathbf{x},t),\,\mathbf{x}\in\Omega \subseteq\mathbb{R}^{d},\,t\in[0,T]\subset\mathbb{R}, \tag{7}\]

where \(\mathbf{x}\) and \(t\) are the spatial and temporal variable, respectively, \(u:\mathbb{R}^{d}\times\mathbb{R}\rightarrow\mathbb{R}\) is similar to (1), \(g:\mathbb{R}^{d}\times\mathbb{R}\rightarrow\mathbb{R}\) denotes the general types of source terms including linear, nonlinear, state-steady, or dynamical, \(\Omega\) is a spatial bounded open set with the boundary \(\partial\Omega\). Without loss of generality, the spatial set of (7) is subjected to Dirichlet boundary, Neumann boundary, or the hybrid of Dirichlet and Neumann boundary conditions. In general, \(g(\mathbf{x},t)\) is used as source terms to describe the external forces for dynamical systems and cannot always be separately measured, as mentioned in Section I.

Different from (6), the residual function is defined for the nonhomogeneous case as the following

\[f_{N}(\mathbf{x},t):=f(\mathbf{x},t)-g(\mathbf{x},t)=u_{t}(\mathbf{x},t)+\mathcal{N}[u(\mathbf{x},t )]-g(\mathbf{x},t). \tag{8}\]

When \(g(\mathbf{x},t)\) is exactly known, \(\hat{f}_{N}(\mathbf{x},t)\), obtained with automatic differential from (8), can be directly used to regularize the approximation of \(u(\mathbf{x},t)\). However, the unknown \(g(\mathbf{x},t)\) will lead to unknown \(f_{N}(\mathbf{x},t)\), which makes the aforementioned regularization infeasible.

Therefore, the goal of C-PINN is to approximate the solution to PDEs with unknown source terms described by (7). To this end, there are two neural networks included in C-PINN: (a) \(NetU\) for approximating the solution satisfying (7); (b) \(NetG\) for regularizing the training of \(NetU\).

#### Iii-1 Cost function

To train C-PINN, the training dataset is uniformly sampled from the system governed by (7). The training dataset \(D\) divided into \(D=D_{B}\cup D_{I}\) with \(D_{B}\cap D_{I}=\varnothing\), where \(D_{B}\) denotes the boundary and initial training dataset and \(D_{I}\) is the training dataset of interior of \(\Omega\). Collocation points
\((\mathbf{x},t)\in E\) correspond to those of \((\mathbf{x},t,u)\in D_{I}\). Then, we adopt the following data-physics-hybrid cost function

\[\text{MSE}=\text{MSE}_{D}+\text{MSE}_{PN} \tag{9}\]

to train our proposed C-PINN. \(\text{MSE}_{D}\) and \(\text{MSE}_{PN}\) in (9) are the data-driven loss and physics-informed loss for the nonhomogeneous PDEs, respectively. \(\text{MSE}_{D}\) adopts the same form of (4). \(\text{MSE}_{PN}\) is as the following

\[\text{MSE}_{PN}=\sum_{(\mathbf{x},t)\in E}\left(\hat{f}\left(\mathbf{x},t \right)-\hat{g}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{G}\right)\right)^{2},\]

where \(\hat{g}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{G}\right)\) is the function of \(NetG\) with \(\hat{\mathbf{\Theta}}_{G}\) being its trained parameter set, \(\hat{f}(\mathbf{x},t)\) has been defined by (2). \(\text{MSE}_{PN}\) corresponds to the physics-informed loss for the nonhomogeneous PDEs obtained from (8) imposed at a finite set collocation points \((\mathbf{x},t)\in E\), which is used to regularize \(\hat{u}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}\right)\) of \(NetU\) to satisfy (7).

#### Iii-B2 Hierarchical training strategy

Considering the relation between \(NetU\) and \(NetG\) in (3), a hierarchical training strategy is proposed. In many cases, the exact formulation or even sparse measurements of \(g(\mathbf{x},t)\) are not available, while the sparse measurements \(D_{I}\) can be obtained to enforce the structure of (7) to achieve \(\hat{\mathbf{\Theta}}_{G}\). Thus, \(\mathbf{\Theta}_{U}\) and \(\mathbf{\Theta}_{G}\) should be iteratively estimated with mutual dependence. Assume \(k\) is the present iteration step, the core issue of the hierarchical training strategy is described by the following two optimization problems

\[\hat{\mathbf{\Theta}}_{G}^{(k+1)} =\underset{\mathbf{\Theta}_{G}}{\arg\min}\ \left\{\text{MSE}_{D}\left(\hat{\mathbf{\Theta}}_{U}^{(k)} \right)+\text{MSE}_{PN}\left(\mathbf{\Theta}_{G};\hat{\mathbf{\Theta}}_{U}^{(k)} \right)\right\} \tag{10}\] \[=\underset{\mathbf{\Theta}_{G}}{\arg\min}\ \ \text{MSE}_{PN}\left(\mathbf{\Theta}_{G};\hat{\mathbf{ \Theta}}_{U}^{(k)}\right)\]

and

\[\hat{\mathbf{\Theta}}_{U}^{(k+1)}=\underset{\mathbf{\Theta}_{U}}{\arg\min}\ \left\{\text{MSE}_{D}\left(\mathbf{\Theta}_{U} \right)+\text{MSE}_{PN}\left(\mathbf{\Theta}_{U};\hat{\mathbf{\Theta}}_{G}^{(k+1)} \right)\right\}, \tag{11}\]

where \(\hat{\mathbf{\Theta}}_{U}^{(k)}\) is the estimated parameter set of \(NetU\) at \(k^{th}\) step, \(\hat{\mathbf{\Theta}}_{U}^{(k+1)}\) is the estimated parameter set of \(NetG\) at \((k+1)^{th}\) step, \(\hat{\mathbf{\Theta}}_{U}^{(k+1)}\) is the estimated parameter set of \(NetU\) at \((k+1)^{th}\) step, which is used to describe the function \(\hat{u}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}^{(k+1)}\right)\). The details of the hierarchical training strategy are obtained by Algorithm 1.

Note that \(\mathbf{\Theta}_{U}^{(0)}\) and \(\mathbf{\Theta}_{G}^{(0)}\) are used as a given parameter set for \(NetU\) and the initialization of the parameter set for \(NetG\) at Step 0, respectively. Furthermore, the iterative transmission of parameter sets of \(NetG\) and \(NetU\) happens in the algorithm.

## IV Numerical experiments

In this section, our proposed C-PINN is applied to solve several classical PDEs to demonstrate its performance. All the examples are implemented with TensorFlow. The fully connected structure with a hyperbolic tangent activation function is applied, which is initialized by Xavier. These training dataset \((\mathbf{x},t,u)\in D\) and collocation points \((\mathbf{x},t)\in E\) are then input into \(NetU\) and \(NetG\). L-BFGS [36] is used to hierarchically solve the optimization problems (10) and (11) to couple the two networks.

```
-Initialize: Randomly sampled training dataset \((\mathbf{x},t,u)\in D\) and collocation points \((\mathbf{x},t)\in E\). Randomly generate initial parameter sets \(\mathbf{\Theta}_{U}^{(0)}\) and \(\mathbf{\Theta}_{G}^{(0)}\). - Step 0: Assume the \(k^{th}\) iteration has achieved \(\hat{\mathbf{\Theta}}_{U}^{(k)}\) and \(\hat{\mathbf{\Theta}}_{G}^{(k)}\). - Repeat: - Step \(k\)-1: Training for \(NetG\) by solving the optimization problem (10) to obtain \(\hat{\mathbf{\Theta}}_{U}^{(k+1)}\), where the estimations of \(\hat{u}_{t}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}^{(k)}\right)+\mathcal{N}\left( \hat{u}(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}^{(k)}\right)\) in \(\text{MSE}_{PN}\) is obtained from the former iteration result \(\hat{\mathbf{\Theta}}_{U}^{(k)}\). - Step \(k\)-2: Training for \(NetU\) by solving the optimization problem (11) to obtain \(\hat{\mathbf{\Theta}}_{U}^{(k+1)}\), which is used to estimate \(\hat{g}\left(\mathbf{x},t;\mathbf{\Theta}_{G}^{(k+1)}\right)\) in \(\text{MSE}_{PN}\). -Until the stop criterion is satisfied. -Return the solution function \(\hat{\mathbf{\Theta}}_{U}\rightarrow\hat{u}\left(\mathbf{x},t;\hat{\mathbf{\Theta}}_{U}\right)\), which can predict the solution (8) with any point \((\mathbf{x},t)\) in \(\Omega\).
```

**Algorithm 1** The hierarchical strategy of optimizing and coupling for C-PINN.

We evaluate the performance of our proposed C-PINN by means of root mean squared error (RMSE)

\[\text{RMSE}=\sqrt{\frac{1}{|T|}\sum_{(\mathbf{x},t)\in T}\left(u\left( \mathbf{x},t\right)-\hat{u}\left(\mathbf{x},t\right)\right)^{2}},\]

where \(|T|\) is the cardinality with respect to the collocation points \((\mathbf{x},t)\in T\), \(T\) is the set of testing collocation points. \(u\left(\mathbf{x},t\right)\) and \(\hat{u}\left(\mathbf{x},t\right)\) denote the ground truth and the corresponding predictions, respectively. To further validate the performance of C-PINN, the Pearson's correlation coefficient (CC)

\[\text{CC}=\frac{\text{cov}\left(u\left(\mathbf{x},t\right),\hat{u}\left(\mathbf{x},t \right)\right)}{\sqrt{\text{Var}\;u\left(\mathbf{x},t\right)}\sqrt{\text{Var}\;\hat{u} \left(\mathbf{x},t\right)}}\]

is also used to measure the similarity between ground truth and prediction, where CC is the correlation coefficient of \(u(\mathbf{x},t)\) and \(\hat{u}(\mathbf{x},t)\), \(\text{cov}\left(u\left(\mathbf{x},t\right),\hat{u}\left(\mathbf{x},t\right)\right)\) is the covariance between \(u(\mathbf{x},t)\) and \(\hat{u}(\mathbf{x},t)\), and Var \(\hat{u}\left(\mathbf{x},t\right)\) and Var \(\hat{u}\left(\mathbf{x},t\right)\) are variance of \(u(\mathbf{x},t)\) and \(\hat{u}(\mathbf{x},t)\), respectively.

### _Case 1: 1-D Heat Equation_

C-PINN is first applied to solve the heat equation with unknown external forces, where both Dirichlet and Neumann boundary conditions are conducted to demonstrate its performance.

#### Iv-A1 Dirichlet Boundary Condition

Here, we consider the heat equation with Dirichlet boundary condition as the following

\[\frac{\partial u}{\partial t}=a^{2}\frac{\partial^{2}u}{\partial x^{2}}+g(x,t), \quad 0<x<L,t>0 \tag{12}\] \[u|_{t=0}=\phi(x),\quad 0\leqslant x\leqslant L\] \[u|_{x=0}=0,\quad u|_{x=L}=0,\quad t>0,\]

where thermal diffusivity \(a=1\), \(u(x,t)\) is the primary variable and means the temperature at \((x,t)\), \(L=\pi\) is the length of bounded rod, \(\phi(x)=0\) is initial temperature, and
\(g(x,t)=xe^{-t}\) denotes the unknown external heat source at \((x,t)\). The analytical solution \(u\left(x,t\right)\) to (12) is obtained with respect to [37]. In this experiment, the setting-ups of C-PINN are as follows. There are eight hidden layers with 20 units in each of them for both \(NetU\) and \(NetG\). A total of 110 training data \((x,t,u(x,t))\) in \(D\) with \(t\in[0,6]\), including 10 training data in \(D_{\text{1}}\) and 100 training data in \(D_{\text{B}}\), are randomly sampled, 10 sparse collocation points are randomly sampled to enforce the structure of (12). Fig. 1 shows the sparse training dataset and the prediction results. Specifically, the magnitude of the predictions \(\hat{u}(x,t)\) using the training dataset is shown in Fig. 1(a) with a heat map. In this case, RMSE is \(4.225390e-02\) and the correlation coefficient is \(9.785444e-01\). Moreover, we compare the ground truths and the predictions at fixed-time \(t\)= 1.5, 3, and 4.5 in Fig. 1(b) to (d), respectively. The evaluation criteria in Table I are applied to further quantify the performance of our proposed C-PINN.

Subsequently, the experiment for PDE with Neumann boundary condition will be further explored to show the general performance of C-PINN.

_2) Neumann Boundary Condition_

Heat equation with Neumann boundary condition is defined as

\[\frac{\partial u}{\partial t}=a^{2}\frac{\partial^{2}u}{\partial x ^{2}}+g(x,t),\quad 0<x<L,t>0 \tag{13}\] \[u|_{t=0}=\phi(x),\quad\quad 0\leqslant x\leqslant L\] \[u|_{x=0}=0,\quad\left.\frac{\partial u}{\partial x}\right|_{x=L} =0,\quad t>0,\]

with the thermal diffusivity \(a=1\), the length of bounded rod \(L=\pi\), the initial temperature \(\phi(x)=\sin\left(x/2\right)\), and the external heat source is \(g(x,t)=\sin\left(x/2\right)\). The analytical solution \(u(x,t)\) to (13) is obtained according to [37]. In this example, \(NetU\) is of three hidden layers consisting of 30 neurons individually. \(NetG\) is of eight hidden layers consisting of 20 units individually. \((x,t,u(x,t))\) in \(D\) are considered with \(t\in[0,10]\). A total of 130 training data in \(D_{B}\), including 10 initial training data, 60 left boundary training data, and 60 right boundary training data are randomly sampled. Moreover, the 20 sparse collocation points are randomly sampled to enforce the structure of (13). The magnitude of the predictions \(\hat{u}(x,t)\) using the training dataset is shown in Fig. 2(a). RMSE is \(5.748950e-02\) and the correlation coefficient is \(9.988286e-01\). Moreover, we compare the ground truths and the predictions at fixed-time \(t\)= 3, 6, and 9 in Fig. 2(b) to (d), respectively. The evaluation criteria in Table II are applied to further evaluate the performance of our proposed C-PINN.

### _Case 2: 1-D Wave Equation_

The wave equation is as the following

\[\frac{\partial^{2}u}{\partial t^{2}}=a^{2}\frac{\partial^{2}u}{ \partial x^{2}}+g(x,t),\quad 0<x<L,t>0 \tag{14}\] \[u|_{x=0}=0,\quad u|_{x=L}=0,\quad t>0\] \[u|_{t=0}=0,\quad\left.\frac{\partial u}{\partial t}\right|_{t=0} =0,\quad 0\leqslant x\leqslant L,\]

Fig. 1: (a) Predictions \(\hat{u}(x,t)\) for the 1-D heat equation with Dirichlet boundary condition; (b), (c), and (d) Comparisons of the ground truths and predictions corresponding to the fixed-time \(t\)= 1.5, 3, and 4.5 snapshots depicted by the dashed vertical lines in (a), respectively.

Fig. 2: (a) Predictions \(\hat{u}\left(x,t\right)\) for the 1-D heat equation with Neumann boundary condition. (b), (c), and (d) Comparisons of the ground truths and predictions correspond to the fixed-time \(t\)= 3, 6, and 9 snapshots depicted by the dashed vertical lines in (a), respectively.
where the wave speed \(a\) is 1, the length of bounded string \(L\) is \(\pi\), the time of wave propagation \(t\) is 6, the external force is

\[g(x,t)=\sin\frac{2\pi x}{L}\sin\frac{2\pi t}{L}\]

at\((x,t)\) and displacement \(u(x,t)\) at \((x,t)\) according to [37] is further investigated.

In this experiment, \(NetU\) is of three hidden layers consisting of 30 neurons individually. \(NetG\) is of eight hidden layers consisting of 20 units individually. A total of 210 training data \((x,t,u\left(x,t\right))\) in \(D\), including 50 initial training data, 120 boundary training data, and 40 collation points are randomly sampled. Fig. 3(a) shows the sparse training dataset and the magnitude of displacement \(\hat{u}(x,t)\) at \((x,t)\). Fig. 3(b) to (d) show the comparisons of ground truths and predictions corresponding to the three fixed-time \(t\)=1.5, 3, and 4.5, which are depicted by the dashed vertical lines in Fig. 3(a), respectively. RMSE is \(7.068626e-02\) and the correlation coefficient is \(9.864411e-01\). The evaluation criteria for the three temporal snapshots are listed in Table III.

### _Case 3: 2-D Poisson Equation_

We further consider the following 2-D Poisson equation

\[\begin{split}&\frac{\partial^{2}u}{\partial x^{2}}+\frac{ \partial^{2}u}{\partial y^{2}}=T_{0},\quad 0<x<a,0<y<b\\ & u(x,0)=0,\quad u(x,b)=T,\quad 0\leqslant x\leqslant a\\ & u(0,y)=0,\quad u(a,y)=0,\quad 0\leqslant y\leqslant b,\end{split} \tag{15}\]

where \(T\) is 1, the constant source term \(T_{0}=1\) is unknown, and \(a=b=1\). The analytical solution \(u(x,y)\) to (15) is obtained according to [37]. In this experiment, the setting-ups of C-PINN are as follows. There are eight hidden layers with 20 units in each of them for both \(NetU\) and \(NetG\). Thirty training data in \(D_{\text{B}}\) and 3 collocation points in \(D_{\text{I}}\) are used. Fig. 4(a) shows the sparse training dataset and the predictions \(\hat{u}(x,y)\). Fig. 4(b) to (d) show the prediction performance of fixed-location \(y\)=0.2, 0.4, and 0.6 snapshots depicted in Fig. 4(a), respectively. RMSE is \(1.594000e-02\) and the correlation coefficient is \(9.997390e-01\). The corresponding evaluation criteria are listed in Table IV.

### _Case 4: 3-D Helmholtz Equation_

C-PINN is also applied to solve 3-D Helmholtz equation with an unknown source term. In particular, we consider the same test PDEs that were previously suggested in [26]

\[\begin{split}\Delta u(\mathbf{x})+p^{2}u(\mathbf{x})&=g( \mathbf{x})\text{ in }\Omega\subset\mathbb{R}^{3}\\ u(\mathbf{x})&=u_{0}(\mathbf{x})\text{ on }\partial\Omega, \end{split} \tag{16}\]

where \(\Delta=\frac{\partial}{\partial x^{2}}+\frac{\partial}{\partial y^{2}}+ \frac{\partial}{\partial z^{2}}\) is Laplacian operator, \(\mathbf{x}=(x,y,z)^{\top}\) is coordinates with \(x,y,z\in(0,1/4]\), \(p=5\) is the wavenumber,

Fig. 4: (a) Predictions \(\hat{u}\left(x,y\right)\) for the 2-D Poisson equation. (b), (c), and, (d) Comparisons of the ground truths and predictions corresponding to the fixed-location \(y=0.2\), 0.4, and 0.6 snapshots depicted by the dashed vertical lines in (a), respectively.

Fig. 3: (a) Predictions \(\hat{u}\left(x,t\right)\) for 1-D wave equation. (b), (c), and (d) Comparisons of the ground truths and predictions corresponding to the fixed-time \(t\)=1.5, 3, and 4.5 snapshots depicted by the dashed vertical lines in Fig. 3(a), respectively.
a suitable \(g(\mathbf{x})\) is the right-hand side of (16) so that

\[u(\mathbf{x})=(0.1\sin\left(2\pi x\right)+\tanh\left(10x\right))\sin\left(2\pi y \right)\sin\left(2\pi z\right)\]

is the analytical solution of (16) [26]. In this experiment, \(NetU\) is of three hidden layers consisting of 100, 50, and 50 neurons individually. \(NetG\) is of eight hidden layers consisting of 20 units individually. Sixty training data and 120 collocation points are sampled. Fig. 5(a) shows the solution of \((x,y,z=0.12)\) snapshot. Furthermore, Fig. 5(b) to (d) show the comparisons of ground truths and predictions, which are extracted at \((x=0.05,z=0.12)\), \((x=0.15,z=0.12)\), and \((x=0.2,z=0.12)\), respectively. The evaluation criteria for this extractions are listed in Table V. In this experiment, RMSE is \(1.192859e-01\), and the correlation coefficient is \(9.057524e-01\).

## V Conclusion

This paper proposes a novel PINN, called C-PINN, to solve PDEs with less prior information or even without any prior information for source terms. In our approach, two neural networks, \(NetU\) and \(NetG\), are proposed with a fully-connected structure. \(NetU\) for approximating the solution satisfying PDEs under study; \(NetG\) for regularizing the training of \(NetU\). Then, the two networks are integrated into a data-physics-hybrid cost function. Furthermore, the two networks are optimized and coupled by the proposed hierarchical training strategy. Finally, C-PINN is applied to solve several classical PDEs to testify to its performance. Note that C-PINN inherits the advantages of PINN, such as sparse property and automatic differential. C-PINN is proposed to solve such a dilemma as the governing equation of dynamical systems with unknown forces. Thus, C-PINN can be further applied to infer the unknown source terms. Meanwhile, C-PINN can be extended to identify the operators from the sparse measurements.

In the future, we will continue to use our C-PINN in various scenarios, like solving PDEs with unknown structure parameters and high-dimension PDEs. For the case, the structures of PDE are totally unknown, regularization method will be combined with C-PINN to select operators from the sparse measurements. Our proposed C-PINN has been shown to solve several classical PDEs successfully. For more complex situations, the features extraction, like convolution and pooling, will be added to C-PINN.
