Conductivity Imaging from Internal Measurements with Mixed Least-Squares
  Deep Neural Networks
In this work we develop a novel approach using deep neural networks to
reconstruct the conductivity distribution in elliptic problems from one
measurement of the solution over the whole domain. The approach is based on a
mixed reformulation of the governing equation and utilizes the standard
least-squares objective, with deep neural networks as ansatz functions to
approximate the conductivity and flux simultaneously. We provide a thorough
analysis of the deep neural network approximations of the conductivity for both
continuous and empirical losses, including rigorous error estimates that are
explicit in terms of the noise level, various penalty parameters and neural
network architectural parameters (depth, width and parameter bound). We also
provide multiple numerical experiments in two- and multi-dimensions to
illustrate distinct features of the approach, e.g., excellent stability with
respect to data noise and capability of solving high-dimensional problems.
Bangti Jin, Xiyao Li, Qimeng Quan, Zhi Zhou
# Conductivity Imaging from Internal Measurements with Mixed Least-Squares Deep Neural Networks+

###### Abstract

In this work we develop a novel approach using deep neural networks to reconstruct the conductivity distribution in elliptic problems from one internal measurement. The approach is based on a mixed reformulation of the governing equation and utilizes the standard least-squares objective to approximate the conductivity and flux simultaneously, with deep neural networks as ansatz functions. We provide a thorough analysis of the neural network approximations for both continuous and empirical losses, including rigorous error estimates that are explicit in terms of the noise level, various penalty parameters and neural network architectural parameters (depth, width and parameter bound). We also provide extensive numerical experiments in two- and multi-dimensions to illustrate distinct features of the approach, e.g., excellent stability with respect to data noise and capability of solving high-dimensional problems.

**Key words**: conductivity imaging, least-squares approach, neural network, generalization error, error estimate

## 1 Introduction

The conductivity value varies widely with the composition and type of materials and its accurate imaging can provide valuable structural information about the object. This observation underpins several important imaging modalities, e.g., electrical impedance tomography, current density impedance imaging, and acousto-electrical tomography; see the works [6, 41] for overviews on mathematical models and theory. In this work, we aim at identifying the conductivity distribution in elliptic problems from internal data using deep neural networks. Let \(\Omega\subset\mathbb{R}^{d}\) be a simply connected open bounded domain with a smooth boundary \(\partial\Omega\). Consider the following Neumann boundary value problem for the function \(u\)

\[\left\{\begin{aligned} -\nabla\cdot(q\nabla u)&=f, \quad\text{ in }\Omega,\\ q\partial_{\mathbf{n}}u&=g,\quad\text{ on }\partial \Omega,\end{aligned}\right. \tag{1.1}\]

where \(\mathbf{n}\) denotes the unit outward normal direction to the boundary \(\partial\Omega\) and \(\partial_{\mathbf{n}}\) denotes taking the normal derivative. The functions \(f\) and \(g\) in (1.1) are the given source and flux, respectively, and satisfy the standard compatibility condition \(\int_{\Omega}f\;\mathrm{d}x+\int_{\partial\Omega}g\;\mathrm{d}S=0\) in order to ensure solvability, and the solution \(u\) is unique under suitable normalization condition, e.g., \(\int_{\Omega}u\mathrm{d}x=0\). The conductivity \(q\) is assumed to belong to the following admissible set

\[\mathcal{A}=\{q\in H^{1}(\Omega):c_{0}\leq q\leq c_{1}\text{ a.e. in }\Omega\},\]

with the constants \(0<c_{0}<c_{1}<\infty\) being the lower and upper bounds on \(q\), respectively. We use the notation \(u(q)\) to explicitly indicate the dependence of the solution \(u\) to problem (1.1) on the coefficient \(q\).
The concerned inverse problem is to recover the conductivity \(q\) from an internal observation of the solution \(u\). It has been extensively studied in both engineering and mathematics communities. For example, the model (1.1) is often used to describe the behavior of a confined inhomogeneous aquifer, where the variable \(u\) represents the piezometric head, \(f\) is the recharge, and \(q\) is hydraulic conductivity (or transmissivity in the two-dimensional case); see the works [19, 44] for extensive discussions on parameter identifications in hydrology. Theoretically, Holder type stability estimates of the inverse problem were established under different settings [4, 10, 35]. Numerically, the reconstruction can be obtained using the regularized output least-squares approach [1, 12, 14], equation error approach [3, 18, 26] and mixed type formulation [29] etc. Error bounds on the numerical approximation obtained by the Galerkin finite element method (FEM) of the regularized formulation were established in [25, 40].

In this work, we develop a new approach for conductivity reconstruction using deep neural networks (DNNs). It is based on a least-squares mixed-type reformulation of the governing equation, with an \(H^{1}(\Omega)\) penalty on the unknown conductivity \(q\). The mixed least-squares formulation was first proposed by Kohn and Lowe [29] (and hence also known as the Kohn-Lowe approach), where both \(q\) and \(u\) were approximated using the Galerkin FEM. In our approach, we approximate both current density \(\sigma\) and conductivity \(q\) separately using two DNNs, adopt a least-squares objective for all the equality constraints, and minimize the overall loss with respect to the DNN parameters. The use of DNNs in place of FEM allows exploiting inductive bias and expressivity of DNNs for function approximations, which can be highly beneficial for numerical recovery. By leveraging the approximation theory of DNNs [20], nonstandard energy argument [25, 29] and statistical learning theory [5, 9], we derive novel error bounds on the approximations in terms of the accuracy of the observational data, DNN architecture (depth, width and parameter bound), and the numbers of sampling points in the domain and on the boundary etc. This is carried out for both population and empirical losses (resulting from Monte Carlo quadrature of the integrals). These error bounds provide theoretical underpinnings for the approach. In practice, the proposed approach is easy to implement, robust with respect to noise and can handle high-dimensional inverse problems (e.g., \(d=5\)). For example, the approach can still yield reasonable approximations in the presence of 10% noise in the data, as is confirmed by the extensive numerical experiments in both low and high-dimensional settings. These distinct features make the method highly attractive, and the numerical results clearly show its significant potential for solving the inverse problem. The development of the DNN formulations, error analysis and extensive numerical validation represent the main contributions of the present work.

In recent years, the use of DNNs for solving direct and inverse problems for PDEs has received a lot of attention; see [15, 39] for overviews. Existing neural inverse schemes using DNNs can roughly be divided into two groups: supervised approaches (see, e.g., [21, 27, 37]) and unsupervised approaches (see, e.g., [7, 8, 24, 32, 43]). Supervised methods exploit the availability of (abundant) paired training data to extract problem-specific features, and are concerned with learning forward operators or their (regularized) inverses. In contrast, unsupervised approaches exploit expressivity of DNNs as universal function approximators for the unknown coefficient and the state, i.e., using DNNs as ansatz functions in the approximation scheme, which enjoy excellent approximation properties for high-dimensional functions (in favorable situations), and the associated inductive biases [33]. The works [7] and [8] investigated image reconstruction in the classical EIT problem, using the weak and strong formulations (also with the \(L^{\infty}\) norm consistency for the latter), respectively. The work [24] applied the deep Ritz method to a least-gradient reformulation for the current density impedance imaging, and derived a generalization error for the loss function. The approach performs reasonably well for both full and partial interior current density data, and shows remarkable robustness against data noise. Pakravan et al [32] developed a hybrid approach, blending high expressivity of DNNs with the accuracy and reliability of traditional numerical methods for PDEs, and showed the approach for recovering the diffusion coefficient in one- and two-dimensional elliptic PDEs; see also [11] for a hybrid DNN-FEM approach for recovering the conductivity coefficient in elliptic and parabolic problems, where the conductivity and state are approximated using DNNs and Galerkin FEM, respectively. The work [11] aims at combining the strengths of neural network and classical Galerkin FEM, i.e., expressivity of DNNs and solid theoretical foundations of the FEM, and provides a thorough theoretical analysis. However, the approach is limited to low-dimensional problems. All these works have presented very encouraging empirical results for a range of PDE inverse problems, and clearly showed enormous potentials of DNNs in solving PDE inverse problems. Our approach follows the unsupervised paradigm, but unlike existing approaches, it employs a mixed formulation of the governing equation and thus differs greatly from the aforementioned ones. In addition, we have established rigorous error bounds on the approximation. Note that the theoretical analysis of neural inverse schemes is largely elusive, due to the outstanding challenges, mostly associated with nonconvexity of the objective function and a lack of linear structure of the approximation space.

The rest of the paper is organized as follows. In Section 2 we recall preliminaries on neural networks, especially approximation theory. In Section 3, we develop the reconstruction approach for a Neumann boundary value problem
(1.1) based on a mixed formulation of the governing equation. Further we present an error analysis of the approach for both population and empirical losses, using tools from partial differential equations and statistical learning theory. In Section 4, we describe the extension of the approach to the case of a Dirichlet boundary value problem. In Section 5, we present extensive numerical experiments to validate the effectiveness of the proposed approach, including highly challenging cases with large noise and high-dimensionality. Throughout, we denote by \(W^{k,p}(\Omega)\) and \(W_{0}^{k,p}(\Omega)\) the standard Sobolev spaces of order \(k\) for any integer \(k\geq 0\) and real \(p\geq 1\), equipped with the norm \(\|\cdot\|_{W^{k,p}(\Omega)}\). Further, we denote by \(W^{-k,p^{\prime}}(\Omega)\) the dual space of \(W_{0}^{k,p}(\Omega)\), with the pair \((p,p^{\prime})\) being the Holder conjugate exponents. We also write \(H^{k}(\Omega)\) and \(H_{0}^{k}(\Omega)\) with the norm \(\|\cdot\|_{H^{k}(\Omega)}\) if \(p=2\) and write \(L^{p}(\Omega)\) with the norm \(\|\cdot\|_{L^{p}(\Omega)}\) if \(k=0\). The spaces on the boundary \(\partial\Omega\) are defined similarly. The notation \((\cdot,\cdot)\) denotes the standard \(L^{2}(\Omega)\) inner product. For a Banach space \(B\), and the notation \(B^{d}\) represent the \(d\)-fold product space. We denote by \(c\) a generic constant not necessarily the same at each occurrence but it is always independent of the approximation accuracy \(\epsilon\) of the DNN, the noise level \(\delta\) and the penalty parameters (\(\gamma_{\sigma}\), \(\gamma_{b}\) and \(\gamma_{q}\)).

## 2 Preliminaries on DNNs

In this section, we describe useful notation and properties on fully connected feedforward neural networks. Let \(\{d_{\ell}\}_{\ell=0}^{L}\subset\mathbb{N}\) be fixed natural numbers with the input dimensionality \(d_{0}=d\) and output dimensionality \(d_{L}\), and a parameterization \(\Theta=\{(A^{(\ell)},b^{(\ell)})_{\ell=1}^{L}\}\) consisting of weight matrices and bias vectors, with \(A^{(\ell)}=[W_{ij}^{(\ell)}]\in\mathbb{R}^{d_{\ell}\times d_{\ell-1}}\) and \(b^{(\ell)}=[b_{i}^{(\ell)}]\in\mathbb{R}^{d_{\ell}}\) the weight matrix and bias vector at the \(\ell\)-th layer. Then a DNN function \(v_{\theta}:=v^{(L)}:\Omega\subset\mathbb{R}^{d}\rightarrow\mathbb{R}^{d_{L}}\) realized by the parameter \(\theta\in\Theta\) is defined recursively by

\[\text{DNN realization:}\quad\left\{\begin{aligned} & v^{(0)}=x,\quad x\in\Omega \subset\mathbb{R}^{d},\\ & v^{(\ell)}=\rho(A^{(\ell)}v^{(\ell-1)}+b^{(\ell)}),\quad\ell=1,2,\cdots,L-1,\\ & v^{(L)}=A^{(L)}v^{(L-1)}+b^{(L)},\end{aligned}\right. \tag{2.1}\]

where the nonlinear activation function \(\rho:\mathbb{R}\rightarrow\mathbb{R}\) is applied componentwise to a vector. The DNN has a depth \(L\) and width \(W:=\max_{\ell=0,\ldots,L}(d_{\ell})\) and the total number of parameters \(\sum_{\ell=1}^{L}d_{\ell}d_{\ell-1}+d_{\ell}\). Given the parametrization \(\Theta\) (i.e., architecture) and the DNN realization, we denote the associated DNN function class \(\mathcal{N}_{\Theta}\) by \(\mathcal{N}_{\Theta}:=\{v_{\theta},\ \theta\in\Theta\}\). Throughout, we fix \(\rho\equiv\tanh\): \(x\rightarrow\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}\) and denote the corresponding DNN as \(\tanh\)-DNN. The following approximation result holds [20, Proposition 4.8].

**Lemma 2.1**.: _Let \(s\in\mathbb{N}_{0}\) and \(p\in[1,\infty]\) be fixed, and \(v\in W^{k,p}(\Omega)\) with \(\mathbb{N}\ni k\geq s+1\). Then for any \(\epsilon>0\), there exists at least one \(\theta\in\Theta\) with depth \(O(\log(d+k))\) and number of nonzero weights \(O\big{(}\epsilon^{-\frac{d}{k-s-\mu(\epsilon=2)}}\big{)}\) and weight parameters bounded by \(O(\epsilon^{-2-\frac{2(d/p+d+k)+\mu(\epsilon=2)+d/p+d}{k-s-\mu(\epsilon=2)}})\) in the maximum norm, where \(\mu>0\) is arbitrarily small, such that the DNN realization \(v_{\theta}\in\mathcal{N}_{\Theta}\) satisfies_

\[\|v-v_{\theta}\|_{W^{s,p}(\Omega)}\leq\epsilon.\]

**Remark 2.1**.: _On the domain \(\Omega=(0,1)^{d}\), Guhring and Raslan [20, pp. 127-128] proved Lemma 2.1 using two steps. They first approximate a function \(v\in W^{k,p}(\Omega)\) by a localized Taylor polynomial \(v_{\mathrm{poly}}\):_

\[\|v-v_{\mathrm{poly}}\|_{W^{s,p}(\Omega)}\leq c_{\mathrm{poly}}N^{-(k-s-\mu(s=2 ))},\]

_where the construction \((\)see [20, Definition 4.4] for details\()\) of \(v_{\mathrm{poly}}\) relies on an approximate partition of unity \((\)depending on \(N\in\mathbb{N})\) and the constant \(c_{\mathrm{poly}}=c_{\mathrm{poly}}(d,p,k,s)>0\). Next they show that there exists a neural network parameter \(\theta\), satisfying the conditions in Lemma 2.1, such that [20, Lemma D.5]:_

\[\|v_{\mathrm{poly}}-v_{\theta}\|_{W^{s,p}(\Omega)}\leq c_{\mathrm{NN}}\|v\|_{W^{ s,p}(\Omega)}\bar{\epsilon},\]

_where the constant \(c_{\mathrm{NN}}=c_{\mathrm{NN}}(d,p,k,s)>0\) and \(\bar{\epsilon}\in(0,\frac{1}{2})\). Now for small \(\epsilon>0\), the desired estimate follows directly from the choice below_

\[N=\Big{(}\frac{\epsilon}{2c_{\mathrm{poly}}}\Big{)}^{-\frac{1}{k-s-\mu(\epsilon=2 )}}\quad\text{and}\quad\bar{\epsilon}=\frac{\epsilon}{2c_{\mathrm{NN}}\|v\|_{W^ {s,p}(\Omega)}}.\]
We denote the set of DNNs of depth \(L\), \(N_{\theta}\) nonzero weights, and maximum bound \(R\) on the parameter vector \(\theta\) by

\[\mathcal{N}(L,N_{\theta},R)=:\{v_{\theta}\text{ is an DNN of depth }L:\|\theta\|_{\ell^{0}}\leq N_{\theta},\|\theta\|_{\ell^{ \infty}}\leq R\},\]

where \(\|\cdot\|_{\ell^{0}}\) and \(\|\cdot\|_{\ell^{\infty}}\), denote respectively, the number of nonzero entries in and the maximum norm of a vector. Furthermore, for any \(\epsilon>0\) and \(p\geq 1\), we denote the DNN parameter set by \(\mathfrak{P}_{p,\epsilon}\) for the set

\[\mathcal{N}\Big{(}c\log(d+2),c\epsilon^{-\frac{d}{1-\rho}},c\epsilon^{-2- \frac{2p+3d+3p+2p\epsilon}{p(1-\mu)}}\Big{)}.\]

Below, for a vector-valued function, we use the DNN function class to approximate its components. This can be easily achieved by DNN parallelization, which combines multiple \(\tanh\)-DNNs into one larger DNN. Then an induction argument allow assembling multiple DNNs into one big DNN. Moreover, the new DNN does not change the depth, and its width equals to the sum of that of subnetworks.

**Lemma 2.2**.: _Let \(\bar{\theta}=\{(\bar{A}^{(\ell)},\bar{b}^{(\ell)})\}_{\ell=1}^{L},\tilde{ \theta}=\{(\bar{A}^{(\ell)},\tilde{b}^{(\ell)})\}_{\ell=1}^{L}\in\Theta\), let \(\bar{v}\) and \(\tilde{v}\) be their DNN realizations, and define \(\theta=\{(A^{(\ell)},b^{(\ell)})\}_{\ell=1}^{L}\) by_

\[A^{(1)} =\begin{bmatrix}\bar{A}^{(1)}\\ \tilde{A}^{(1)}\end{bmatrix}\in\mathbb{R}^{2d_{1}\times d_{0}},\ A^{(\ell)}= \begin{bmatrix}\bar{A}^{(\ell)}&0\\ 0&\bar{A}^{(\ell)}\end{bmatrix}\in\mathbb{R}^{2d_{\ell}\times 2d_{\ell-1}}, \quad\ell=2,\ldots,L;\] \[b^{(\ell)} =\begin{bmatrix}\bar{b}^{(\ell)}\\ \tilde{b}^{(\ell)}\end{bmatrix}\in\mathbb{R}^{2d_{\ell}},\quad\ell=1,\ldots,L.\]

_Then \(v_{\theta}=(\bar{v}_{\theta},\tilde{v}_{\tilde{\theta}})^{\top}\) is the DNN realization of \(\theta\), of depth \(L\) and width \(2W\)._

## 3 Inverse conductivity problem in the Neumann case

In this section we discuss the inverse conductivity problem for the Neumann boundary value problem (1.1).

### Mixed formulation and its DNN approximation

To develop the reconstruction method, we let \(\sigma=q\nabla u\) and rewrite problem (1.1) into a first-order system

\[\begin{cases}\sigma=q\nabla u,&\text{in }\Omega,\\ -\nabla\cdot\sigma=f,&\text{in }\Omega,\\ \sigma\cdot\mathbf{n}=g,&\text{on }\partial\Omega.\end{cases} \tag{3.1}\]

To recover the conductivity \(q\) from the observation \(z^{\delta}\), we employ a residual minimization scheme based on the first-order system (3.1). We approximate both \(\sigma\) and \(q\) using DNNs, and substitute the noisy data \(z^{\delta}\) for the scalar field \(u\) in the first equation. For the scalar field \(q\), we use a \(\tanh\)-DNN function class (of depth \(L_{q}\) and width \(W_{q}\)) with the parametrization \(\mathfrak{P}_{p,\epsilon_{q}}\) (with \(p\geq 2\) and tolerance \(\epsilon_{q}\)). Similarly, for the vector field \(\sigma:\Omega\to\mathbb{R}^{d}\), we employ \(d\) identical \(\tanh\)-DNN function classes (of depth \(L_{\sigma}\) and width \(W_{\sigma}\)) with the parametrizations \(\mathfrak{P}_{2,\epsilon_{\sigma}}\) (with tolerance \(\epsilon_{\sigma}\)), and stack them into one multi-output DNN (via the parallelization technique in Lemma 2.2).

To enforce the box constraint of the coefficient \(q\), we employ a cutoff operation \(P_{\mathcal{A}}:H^{1}(\Omega)\to\mathcal{A}\) defined by \(P_{\mathcal{A}}(v)=\min(\max(c_{0},v),c_{1})\). The operator \(P_{\mathcal{A}}\) is stable in the following sense [45, Corollary 2.1.8]

\[\|\nabla P_{\mathcal{A}}(v)\|_{L^{p}(\Omega)}\leq\|\nabla v\|_{L^{p}(\Omega)},\quad\forall v\in W^{1,p}(\Omega),p\in[1,\infty], \tag{3.2}\]

and moreover, for all \(v\in\mathcal{A}\), there holds

\[\|P_{\mathcal{A}}(w)-v\|_{L^{p}(\Omega)}\leq\|w-v\|_{L^{p}(\Omega)},\quad \forall w\in L^{p}(\Omega),\ p\in[1,\infty]. \tag{3.3}\]

Using the standard least-squares formulation on all equality constraints, we obtain the following objective

\[J_{\boldsymbol{\gamma}}(\theta,\kappa)=\|\sigma_{\kappa}-P_{\mathcal{A}}(q_{ \theta})\nabla z^{\delta}\|_{L^{2}(\Omega)}^{2}+\gamma_{\sigma}\|\nabla\cdot \sigma_{\kappa}+f\|_{L^{2}(\Omega)}^{2}+\gamma_{b}\|\sigma_{\kappa}\cdot \mathbf{n}-g\|_{L^{2}(\partial\Omega)}^{2}+\gamma_{q}\|\nabla q_{\theta}\|_{L^{2 }(\Omega)}^{2}.\]
Then the DNN reconstruction problem reads

\[\min_{(\theta,\kappa)\in(\mathfrak{P}_{p,\kappa_{q}},\mathfrak{P}_{2,\sigma_{q}}^{ \otimes d})}J_{\boldsymbol{\gamma}}(\theta,\kappa), \tag{3.4}\]

where the superscript \(\otimes d\) denotes the \(d\)-fold direct product, \(\gamma_{\sigma}>0\), \(\gamma_{b}>0\) and \(\gamma_{q}>0\) are penalty parameters that balance the different terms and have to be tuned suitably, and we write \(\boldsymbol{\gamma}=(\gamma_{\sigma},\gamma_{b},\gamma_{q})\in\mathbb{R}_{+}^{3}\) below. The \(H^{1}(\Omega)\) seminorm penalty on the approximation \(q_{\theta}\) is to stabilize the minimization process. It is essential for overcoming the ill-posedness of the inverse problem [17, 23]. \(z^{\delta}\) is the noisy measurement of the exact data \(u(q^{\dagger})\) in the domain \(\Omega\) and the noise level \(\delta\) is defined by

\[\delta:=\|\nabla(u(q^{\dagger})-z^{\delta})\|_{L^{2}(\Omega)}. \tag{3.5}\]

Note that in (3.5) we impose a mild regularity condition on the noisy data \(z^{\delta}\) (smoother than the popular \(L^{2}(\Omega)\)), which may be obtained by presmoothing the raw noisy data beforehand. This assumption is commonly used in energy type formulations, e.g., Kohn-Lowe [29] and equation error [3] formulations. The well-posedness of problem (3.4) follows by a standard argument in calculus of variation. Indeed, the compactness of the parametrizations \(\mathfrak{P}_{p,\epsilon_{q}}\) and \(\mathfrak{P}_{2,\epsilon_{\sigma}}\) holds due to the uniform boundedness on the parameter vectors and finite-dimensionality of the space. Meanwhile, the smoothness of the \(\tanh\) activation function implies the continuity of the loss \(J_{\boldsymbol{\gamma}}\) in the DNN parameters \((\theta,\kappa)\). These two properties imply the existence of a minimizer \((\theta^{*},\kappa^{*})\).

Note that the objective \(J_{\boldsymbol{\gamma}}\) involves high-dimensional integrals, and hence quadrature is needed in practice. This may be achieved using any standard quadrature rules, and predominantly Monte Carlo methods in high-dimension. In this work, we employ the Monte Carlo method. Let \(\mathcal{U}(\Omega)\) and \(\mathcal{U}(\partial\Omega)\) be the uniform distributions over the domain \(\Omega\) and the boundary \(\partial\Omega\), respectively, and \((q_{\theta},\sigma_{\kappa})\) be the DNN realization of \((\theta,\kappa)\). Using the expectation \(\mathbb{E}_{\mathcal{U}(\cdot)}[\cdot]\) with respect to \(\mathcal{U}(\cdot)\), we can rewrite the loss \(J_{\boldsymbol{\gamma}}\) as

\[J_{\boldsymbol{\gamma}}(\theta,\kappa) =|\Omega|\mathbb{E}_{X\sim\mathcal{U}(\Omega)}\Big{[}\|\sigma_{ \kappa}(X)-P_{\mathcal{A}}(q_{\theta}(X))\nabla z^{\delta}(X)\|_{\ell^{2}}^{2} \Big{]}+\gamma_{\sigma}|\Omega|\mathbb{E}_{X\sim\mathcal{U}(\Omega)}\Big{[} \big{(}\nabla\cdot\sigma_{\kappa}(X)+f(X)\big{)}^{2}\Big{]}\] \[\quad+\gamma_{b}|\partial\Omega|\mathbb{E}_{Y\sim\mathcal{U}( \partial\Omega)}\Big{[}\big{(}\sigma_{\kappa}(Y)\cdot\mathbf{n}-g(Y)\big{)}^{ 2}\Big{]}+\gamma_{q}|\Omega|\mathbb{E}_{X\sim\mathcal{U}(\Omega)}\Big{[}\| \nabla q_{\theta}(X)\|_{\ell^{2}}^{2}\Big{]}\] \[=:\mathcal{E}_{d}(\sigma_{\kappa},q_{\theta})+\gamma_{\sigma} \mathcal{E}_{\sigma}(\sigma_{\kappa})+\gamma_{b}\mathcal{E}_{b}(\sigma_{ \kappa})+\gamma_{q}\mathcal{E}_{q}(q_{\theta}),\]

where \(|\Omega|\) and \(|\partial\Omega|\) denote the Lebesgue measure of \(\Omega\) and \(\partial\Omega\), respectively, and \(\|\cdot\|_{\ell^{2}}\) denotes the Euclidean norm on \(\mathbb{R}^{d}\). Next let \(X=\{X_{j}\}_{j=1}^{n_{r}}\) and \(Y=\{Y_{j}\}_{j=1}^{n_{b}}\) be independent and identically distributed (i.i.d.) samples drawn from the uniform distributions \(\mathcal{U}(\Omega)\) and \(\mathcal{U}(\partial\Omega)\), respectively, where \(n_{r}\) and \(n_{b}\) are the numbers of sampling points in the domain \(\Omega\) and on the boundary \(\partial\Omega\), respectively. Then the empirical loss \(\widehat{J}_{\boldsymbol{\gamma}}(\theta,\kappa)\) is given by

\[\widehat{J}_{\boldsymbol{\gamma}}(\theta,\kappa)=:\widehat{\mathcal{E}}_{d}( \sigma_{\kappa},q_{\theta})+\gamma_{\sigma}\widehat{\mathcal{E}}_{\sigma}( \sigma_{\kappa})+\gamma_{b}\widehat{\mathcal{E}}_{b}(\sigma_{\kappa})+\gamma_ {q}\widehat{\mathcal{E}}_{q}(q_{\theta}), \tag{3.6}\]

where \(\widehat{\mathcal{E}}_{d}(\sigma_{\kappa},q_{\theta})\), \(\widehat{\mathcal{E}}_{\sigma}(\sigma_{\kappa})\), \(\widehat{\mathcal{E}}_{b}(\sigma_{\kappa})\) and \(\widehat{\mathcal{E}}_{q}(q_{\theta})\) are Monte Carlo approximations of \(\mathcal{E}_{d}(\sigma_{\kappa},q_{\theta})\), \(\mathcal{E}_{\sigma}(\sigma_{\kappa})\), \(\mathcal{E}_{b}(\sigma_{\kappa})\) and \(\mathcal{E}_{q}(q_{\theta})\), obtained by replacing the expectation with a sample mean, and are defined by

\[\widehat{\mathcal{E}}_{d}(\sigma_{\kappa},q_{\theta}) :=n_{r}^{-1}|\Omega|\sum_{j=1}^{n_{r}}\|\sigma_{\kappa}(X_{j})-P_{ \mathcal{A}}(q_{\theta}(X_{j}))\nabla z^{\delta}(X_{j})\|_{\ell^{2}}^{2}, \tag{3.7}\] \[\widehat{\mathcal{E}}_{\sigma}(\sigma_{\kappa}) :=n_{r}^{-1}|\Omega|\sum_{j=1}^{n_{r}}\big{(}\nabla\cdot\sigma_{ \kappa}(X_{j})+f(X_{j})\big{)}^{2},\] (3.8) \[\widehat{\mathcal{E}}_{b}(\sigma_{\kappa}) :=n_{b}^{-1}|\partial\Omega|\sum_{j=1}^{n_{b}}\big{(}\sigma_{ \kappa}(Y_{j})\cdot\mathbf{n}-g(Y_{j})\big{)}^{2},\] (3.9) \[\widehat{\mathcal{E}}_{q}(q_{\theta}) :=n_{r}^{-1}|\Omega|\sum_{j=1}^{n_{r}}\|\nabla q_{\theta}(X_{j})\|_ {\ell^{2}}^{2}. \tag{3.10}\]

Additionally, we define variants of \(\mathcal{E}_{b}\) and \(\widehat{\mathcal{E}}_{b}\) by

\[\mathcal{E}_{b^{\prime}}=|\partial\Omega|\mathbb{E}_{\mathcal{U}(\partial\Omega)}[ \|\sigma(Y)-q^{\dagger}(Y)\nabla z^{\delta}(Y)\|_{\ell^{2}}^{2}]\quad\text{and} \quad\widehat{\mathcal{E}}_{b^{\prime}}=n_{b}^{-1}|\partial\Omega|\sum_{j=1}^{n_{ b}}\|\sigma(Y_{j})-q^{\dagger}(Y_{j})\nabla z^{\delta}(Y_{j})\|_{\ell^{2}}^{2}. \tag{3.11}\]

These quantities will be needed in the investigation of the inverse problem in the Dirichlet case in Section 4. The estimation of Monte Carlo errors will be discussed in Section 3.3 below.
### Error analysis of the population loss

Now we derive (weighted) \(L^{2}(\Omega)\) error bounds on \(q^{\dagger}-q^{*}_{\theta}\) for the DNN realization \(q^{*}_{\theta}\) of a minimizer \((\theta^{*},\kappa^{*})\) to the population loss \(J_{\mathbf{\gamma}}(\theta,\kappa)\) under Assumption 3.1 below.

**Assumption 3.1**.: \(q^{\dagger}\in W^{2,p}(\Omega)\cap\mathcal{A}\)_, \(f\in H^{1}(\Omega)\cap L^{\infty}(\Omega)\) and \(g\in H^{\frac{3}{2}}(\partial\Omega)\cap L^{\infty}(\partial\Omega)\), with \(p=\max(2,d+\nu)\) for small \(\nu>0\)._

The following _a priori_ regularity holds under Assumption 3.1 for \(u^{\dagger}:=u(q^{\dagger})\) and \(\sigma^{\dagger}:=q^{\dagger}\nabla u(q^{\dagger})\).

**Lemma 3.1**.: _Under Assumption 3.1, the solution \(u^{\dagger}\) to problem (1.1) (with \(q=q^{\dagger}\)) satisfies \(u^{\dagger}\in H^{3}(\Omega)\cap H^{1}_{0}(\Omega)\) and \(\sigma^{\dagger}\in(H^{2}(\Omega))^{d}\)._

Proof.: By Sobolev embedding theorem [2, Theorem 4.12, p. 85], \(q^{\dagger}\in W^{2,p}(\Omega)\hookrightarrow W^{1,\infty}(\Omega)\) for \(p>\max(2,p+\nu)\). Since \(f\in L^{\infty}(\Omega)\) and \(g\in L^{\infty}(\partial\Omega)\), standard elliptic regularity theory implies \(u^{\dagger}\in H^{2}(\Omega)\cap W^{1,\infty}(\Omega)\). Next, upon expansion, we have

\[\begin{cases}-\Delta u^{\dagger}=\frac{f}{q^{\dagger}}+\frac{\nabla q^{\dagger }\cdot\nabla u^{\dagger}}{q^{\dagger}},&\text{ in }\Omega,\\ \partial_{n}u^{\dagger}=\frac{q}{q^{\dagger}},&\text{ on }\partial\Omega, \end{cases}\]

where \(\cdot\) denotes Euclidean inner product on \(\mathbb{R}^{d}\). Since \(q^{\dagger}\in W^{1,\infty}(\Omega)\cap\mathcal{A}\) and \(f\in L^{\infty}(\Omega)\cap H^{1}(\Omega)\), we have for \(i=1,\ldots,d\), \(\partial_{x_{i}}\Big{(}\frac{f}{q^{\dagger}}\Big{)}=-\frac{f\partial_{x_{i}}q ^{\dagger}}{(q^{\dagger})^{2}}+\frac{\partial_{x_{i}}f}{q^{\dagger}}\in L^{2}(\Omega)\), i.e., \(\frac{f}{q^{\dagger}}\in H^{1}(\Omega)\). Likewise, since \(q^{\dagger}\in W^{1,\infty}(\Omega)\cap W^{2,p}(\Omega)\) and \(u^{\dagger}\in W^{1,\infty}(\Omega)\cap H^{2}(\Omega)\), we have for any \(i=1,\ldots,d\),

\[\partial_{x_{i}}\Big{(}\frac{\nabla q^{\dagger}\cdot\nabla u^{\dagger}}{q^{ \dagger}}\Big{)}=-\frac{\partial_{x_{i}}q^{\dagger}(\nabla q^{\dagger}\cdot \nabla u^{\dagger})}{(q^{\dagger})^{2}}+\frac{\nabla(\partial_{x_{i}}q^{ \dagger})\cdot\nabla u^{\dagger}}{q^{\dagger}}+\frac{\nabla(\partial_{x_{i}}u ^{\dagger})\cdot\nabla q^{\dagger}}{q^{\dagger}}\in L^{2}(\Omega),\]

i.e., \(\frac{\nabla q^{\dagger}\cdot\nabla u^{\dagger}}{q^{\dagger}}\in H^{1}(\Omega)\) also. Since \(g\in L^{\infty}(\partial\Omega)\cap H^{\frac{3}{2}}(\partial\Omega)\), by the trace theorem [2, Theorem 5.36, p. 164], there exists \(\tilde{g}\in L^{\infty}(\Omega)\cap H^{2}(\Omega)\). Then repeating the preceding argument gives for any \(i,j=1,\ldots,d\)

\[\partial_{x_{i}x_{j}}\Big{(}\frac{\tilde{g}}{q^{\dagger}}\Big{)}=\frac{q^{ \dagger}\partial_{x_{i}x_{j}}\tilde{g}-\partial_{x_{i}}\tilde{g}\partial_{x_ {j}}q^{\dagger}}{(q^{\dagger})^{2}}-\frac{(\partial_{x_{j}}\tilde{g})(\partial _{x_{i}}q^{\dagger})+\tilde{g}\partial_{x_{i}x_{j}}q^{\dagger}}{(q^{\dagger} )^{2}}+2\frac{\tilde{g}(\partial_{x_{i}}q^{\dagger})(\partial_{x_{j}}q^{ \dagger})}{(q^{\dagger})^{3}}\in L^{2}(\Omega),\]

i.e., \(\frac{\tilde{g}}{q^{\dagger}}\in H^{2}(\Omega)\). This and trace theorem imply \(\frac{q}{q^{\dagger}}\in H^{\frac{3}{2}}(\partial\Omega)\). Applying the standard elliptic regularity theory again yields \(u^{\dagger}\in H^{3}(\Omega)\). Finally, it follows from \(q^{\dagger}\in W^{2,p}(\Omega)\) and \(u^{\dagger}\in H^{3}(\Omega)\cap W^{1,\infty}(\Omega)\) that \(\sigma^{\dagger}\equiv q^{\dagger}\nabla u^{\dagger}\in(H^{2}(\Omega))^{d}\). 

The next lemma gives an _a priori_ estimate on the DNN realization \((q^{*}_{\theta},\sigma^{*}_{\kappa})\) of the minimizer \((\theta^{*},\kappa^{*})\).

**Lemma 3.2**.: _Let Assumption 3.1 hold. Fix small \(\epsilon_{\sigma}\), \(\epsilon_{q}>0\), and let \((\theta^{*},\kappa^{*})\in(\mathfrak{P}_{p,\epsilon_{q}},\mathfrak{P}_{2, \epsilon_{\sigma}}^{\otimes d})\) be a minimizer of problem (3.4). Then the following estimate holds_

\[J_{\mathbf{\gamma}}(\theta^{*},\kappa^{*})\leq c\big{(}\epsilon_{q}^{2}+(\gamma_{ \sigma}+\gamma_{b}+1)\epsilon_{\sigma}^{2}+\delta^{2}+\gamma_{q}\big{)}.\]

Proof.: First, Assumption 3.1 and Lemma 3.1 imply \(\sigma^{\dagger}\in(H^{2}(\Omega))^{d}\). By Lemma 2.1, there exists one \((\theta_{\epsilon},\kappa_{\epsilon})\in(\mathfrak{P}_{p,\epsilon_{q}},\mathfrak{P}_ {2,\epsilon_{\sigma}}^{\otimes d})\) such that its DNN realization \((q_{\theta_{\epsilon}},\sigma_{\kappa_{\epsilon}})\) satisfies

\[\|q^{\dagger}-q_{\theta_{\epsilon}}\|_{W^{1,p}(\Omega)}\leq\epsilon_{q}\quad \text{and}\quad\|\sigma^{\dagger}-\sigma_{\kappa_{\epsilon}}\|_{H^{1}(\Omega)} \leq\epsilon_{\sigma}. \tag{3.12}\]

Then the triangle inequality leads to \(\|\nabla q_{\theta_{\epsilon}}\|_{L^{2}(\Omega)}\leq c\). The embedding \(W^{1,p}(\Omega)\hookrightarrow L^{\infty}(\Omega)\)[2, Theorem 4.12, p. 85] implies

\[\|q^{\dagger}-q_{\theta_{\epsilon}}\|_{L^{\infty}(\Omega)}\leq c\epsilon_{q}. \tag{3.13}\]

By the minimizing property of \((\theta^{*},\kappa^{*})\) and the triangle inequality, we obtain

\[J_{\mathbf{\gamma}}(\theta^{*},\kappa^{*})\leq J_{\mathbf{\gamma}}( \theta_{\epsilon},\kappa_{\epsilon})\] \[=\|\sigma_{\kappa_{\epsilon}}-P_{\mathcal{A}}(q_{\theta_{ \epsilon}})\nabla z^{\delta}\|_{L^{2}(\Omega)}^{2}+\gamma_{\sigma}\|\nabla \cdot\sigma_{\kappa_{\epsilon}}+f\|_{L^{2}(\Omega)}^{2}+\gamma_{b}\|\sigma_{ \kappa_{\epsilon}}\cdot\mathbf{n}-g\|_{L^{2}(\partial\Omega)}^{2}+\gamma_{q} \|\nabla q_{\theta_{\epsilon}}\|_{L^{2}(\Omega)}^{2}\]
\[\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}(\Omega)}^{2}= \big{(}(q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))\nabla u^{\dagger},\nabla v_{ \psi}\big{)}=(\sigma^{\dagger},\nabla v_{\psi})-(P_{\mathcal{A}}(q_{\theta}^{*} )\nabla u^{\dagger},\nabla v_{\psi})\] \[=-\big{(}\nabla\cdot\sigma^{\dagger},v_{\psi}\big{)}-\big{(}P_{ \mathcal{A}}(q_{\theta}^{*})\nabla u^{\dagger},\nabla v_{\psi}\big{)}+(g,v_{ \psi})_{L^{2}(\partial\Omega)}\] \[=\big{(}f+\nabla\cdot\sigma_{\kappa}^{*},v_{\psi}\big{)}+\big{(} \sigma_{\kappa}^{*}-P_{\mathcal{A}}(q_{\theta}^{*})\nabla z^{\delta},\nabla v_ {\psi}\big{)}+\big{(}P_{\mathcal{A}}(q_{\theta}^{*})\nabla(z^{\delta}-u^{ \dagger}),\nabla v_{\psi}\big{)}+(g-\sigma_{\kappa}^{*}\cdot\mathbf{n},v_{ \psi})_{L^{2}(\partial\Omega)}.\]

Then Cauchy-Schwarz inequality, the trace theorem [2, Theorem 5.36, p. 164], Lemma 3.2 and Condition 3.2 imply

\[\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}(\Omega)} \leq c\big{[}\|f+\nabla\cdot\sigma_{\kappa}^{*}\|_{L^{2}(\Omega)}+\| \sigma_{\kappa}^{*}-P_{\mathcal{A}}(q_{\theta}^{*})\nabla z^{\delta}\|_{L^{2}( \Omega)}+\|P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{\infty}(\Omega)}\|\nabla(z^{ \delta}-u^{\dagger})\|_{L^{2}(\Omega)}\] \[\quad+\|g-\sigma_{\kappa}^{*}\cdot\mathbf{n}\|_{L^{2}(\partial \Omega)}\big{]}\|v_{\psi}\|_{H^{1}(\Omega)}\leq c\big{(}1+\gamma_{\sigma}^{- \frac{1}{2}}+\gamma_{b}^{-\frac{1}{2}}\big{)}\eta\|q^{\dagger}-P_{\mathcal{A}} (q_{\theta}^{*})\|_{H^{1}(\Omega)}.\]

By the definition of the \((H^{1}(\Omega))^{\prime}\)-norm, we have

\[\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{(H^{1}(\Omega))^{\prime}}\leq c \big{(}1+\gamma_{\sigma}^{-\frac{1}{2}}+\gamma_{b}^{-\frac{1}{2}}\big{)}\eta.\]

This, duality pairing and the bound on \(\|\nabla q_{\theta}^{*}\|_{L^{2}(\Omega)}\) in Lemma 3.2 and the stability of \(P_{\mathcal{A}}\) in (3.2) imply

\[\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}(\Omega)} \leq\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{(H^{1}(\Omega))^{ \prime}}^{\frac{1}{2}}\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{H^{1}( \Omega)}^{\frac{1}{2}}\] \[\leq c\|q^{\dagger}-q_{\theta}^{*}\|_{(H^{1}(\Omega))^{\prime}}^{ \frac{1}{2}}\big{(}1+\|\nabla P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}(\Omega) }^{\frac{1}{2}}\big{)}\leq c\|q^{\dagger}-q_{\theta}^{*}\|_{(H^{1}(\Omega))^{ \prime}}^{\frac{1}{2}}\big{(}1+\|\nabla q_{\theta}^{*}\|_{L^{2}(\Omega)}^{ \frac{1}{2}}\big{)}\] \[\leq c(1+\gamma_{\sigma}^{-\frac{1}{4}}+\gamma_{b}^{-\frac{1}{4}})(1+ \gamma_{q}^{-\frac{1}{4}}\eta^{\frac{1}{2}})\eta^{\frac{1}{2}}.\]

Thus we complete the proof of the theorem. 

**Remark 3.1**.: _Theorem 3.3 provides useful guidelines for choosing the parameters: \(\gamma_{\sigma}=O(1)\), \(\gamma_{b}=O(1)\), \(\gamma_{q}=O(\delta^{2})\), \(\epsilon_{q}=O(\delta)\) and \(\epsilon_{\sigma}=O(\delta)\). Then we obtain the following error estimate \(\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}(\Omega)}\leq c\delta^{ \frac{1}{2}}\)._
### Error analysis of the empirical loss

Now we analyze the error of the approximation \(\widehat{q}_{\theta}^{*}\), i.e., the DNN realization of a minimizer \((\widehat{\theta}^{*},\widehat{\kappa}^{*})\) to the empirical loss \(\widehat{J}_{\gamma}(\theta,\kappa)\). The loss \(\widehat{J}_{\gamma}(\theta,\kappa)\) involves also the quadrature error arising from approximating the integrals via Monte Carlo methods. The analysis requires the following assumption. The \(L^{\infty}\) bound is needed in order to apply the standard Rademacher complexity argument (cf. Dudley's formula in Lemma A.4).

**Assumption 3.4**.: \(f\in L^{\infty}(\Omega)\)_, and \(z^{\delta}\in W^{1,\infty}(\Omega)\)._

The key of the analysis is to bound the error \(\sup_{q_{\theta}\in\mathcal{N}_{q},\sigma_{\kappa}\in\mathcal{N}_{\sigma}} \big{|}J_{\gamma}(q_{\theta},\sigma_{\kappa})-\widehat{J}_{\gamma}(q_{\theta},\sigma_{\kappa})\big{|}\) for suitable DNN function classes \(\mathcal{N}_{q}\) and \(\mathcal{N}_{\sigma}\) (corresponding to the sets \(\mathfrak{P}_{p,c_{q}}\) and \(\mathfrak{P}_{\mathcal{E}_{\sigma}^{\delta}}^{\delta\delta}\), respectively), which are also known as statistical errors in statistical learning theory [5, 38]. The starting point of the analysis is the following splitting:

\[\sup_{q_{\theta}\in\mathcal{N}_{q},\sigma_{\kappa}\in\mathcal{N}_{\sigma}} \big{|}J_{\gamma}(q_{\theta},\sigma_{\kappa})-\widehat{J}_{\gamma}(q_{\theta },\sigma_{\kappa})\big{|}\leq\Delta\mathcal{E}_{d}+\gamma_{\sigma}\Delta \mathcal{E}_{\sigma}+\gamma_{b}\Delta\mathcal{E}_{b}+\gamma_{q}\Delta \mathcal{E}_{q},\]

with the error components given respectively by

\[\Delta\mathcal{E}_{d} :=\sup_{\sigma_{\kappa}\in\mathcal{N}_{\sigma},q_{\theta}\in \mathcal{N}_{q}}\big{|}\mathcal{E}_{d}(\sigma_{\kappa},q_{\theta})-\widehat{ \mathcal{E}}_{d}(\sigma_{\kappa},q_{\theta})\big{|}, \Delta\mathcal{E}_{\sigma}:=\sup_{\sigma_{\kappa}\in\mathcal{N}_{ \sigma}}\big{|}\mathcal{E}_{\sigma}(\sigma_{\kappa})-\widehat{\mathcal{E}}_{ \sigma}(\sigma_{\kappa})\big{|},\] \[\Delta\mathcal{E}_{b} :=\sup_{\sigma_{\kappa}\in\mathcal{N}_{\sigma}}\big{|}\mathcal{E} _{b}(\sigma_{\kappa})-\widehat{\mathcal{E}}_{b}(\sigma_{\kappa})\big{|}, \Delta\mathcal{E}_{q}:=\sup_{q_{\theta}\in\mathcal{N}_{q}}\big{|} \mathcal{E}_{q}(q_{\theta})-\widehat{\mathcal{E}}_{q}(q_{\theta})\big{|}.\]

Further we define \(\Delta\mathcal{E}_{b^{\prime}}:=\sup_{\sigma_{\kappa}\in\mathcal{N}_{\sigma}} \big{|}\mathcal{E}_{b^{\prime}}(\sigma_{\kappa})-\widehat{\mathcal{E}}_{b^{ \prime}}(\sigma_{\kappa})\big{|}\).

Now we state the quadrature error for each term (in high probability). The proof is based on PAC-type generalization bounds in Lemma A.1, which in turn employs the Rademacher complexity of the associated function classes via Dudley's formula in Lemma A.4. The overall proof is lengthy and hence it is deferred to the appendix.

**Theorem 3.5**.: _Let Assumptions 3.1 and 3.4 hold, and \(q_{\theta}\in\mathcal{N}(L_{q},N_{\theta},R_{q})\) and \(\sigma_{\kappa}\in\mathcal{N}(L_{\sigma},N_{\kappa},R_{\sigma})\). Fix \(\tau\in(0,\frac{1}{8})\), and define the following bounds_

\[e_{d} :=c\frac{R_{\sigma}^{2}N_{\kappa}^{2}(N_{\kappa}+N_{\theta})^{ \frac{1}{2}}(\log^{\frac{1}{2}}R_{\sigma}+\log^{\frac{1}{2}}N_{\kappa}+\log^{ \frac{1}{2}}R_{q}+\log^{\frac{1}{2}}N_{\theta}+\log^{\frac{1}{2}}n_{r})}{\sqrt{ n_{r}}}+\tilde{c}R_{\sigma}^{2}N_{\kappa}^{2}\sqrt{\frac{\log\frac{1}{\tau}}{2n_{r}}},\] \[e_{\sigma} :=c\frac{R_{\sigma}^{2L_{\sigma}}N_{\kappa}^{2L_{\sigma}-\frac{ 3}{2}}\big{(}\log^{\frac{1}{2}}R_{\sigma}+\log^{\frac{1}{2}}N_{\kappa}+\log^{ \frac{1}{2}}n_{r}\big{)}}{\sqrt{n_{r}}}+\tilde{c}R_{\sigma}^{2L_{\sigma}}N_{ \kappa}^{2L_{\sigma}-2}\sqrt{\frac{\log\frac{1}{\tau}}{2n_{r}}},\] \[e_{b} :=c\frac{R_{\sigma}^{2}N_{\kappa}^{\frac{5}{2}}\big{(}\log^{ \frac{1}{2}}R_{\sigma}+\log^{\frac{1}{2}}N_{\kappa}+\log^{\frac{1}{2}}n_{b} \big{)}}{\sqrt{n_{b}}}+\tilde{c}R_{\sigma}^{2}N_{\kappa}^{2}\sqrt{\frac{\log \frac{1}{\tau}}{2n_{b}}},\] \[e_{b^{\prime}} :=c\frac{R_{\sigma}^{2}N_{\kappa}^{\frac{5}{2}}\big{(}\log^{ \frac{1}{2}}R_{\sigma}+\log^{\frac{1}{2}}N_{\kappa}+\log^{\frac{1}{2}}n_{b} \big{)}}{\sqrt{n_{b}}}+\tilde{c}R_{\sigma}^{2}N_{\kappa}^{2}\sqrt{\frac{\log \frac{1}{\tau}}{2n_{r}}},\] \[e_{q} :=c\frac{R_{q}^{2L_{q}}N_{\theta}^{2L_{q}-\frac{3}{2}}\big{(}\log ^{\frac{1}{2}}R_{q}+\log^{\frac{1}{2}}N_{\theta}+\log^{\frac{1}{2}}n_{r}\big{)} }{\sqrt{n_{r}}}+\tilde{c}R_{q}^{2L_{q}}N_{\theta}^{2L_{q}-2}\sqrt{\frac{\log \frac{1}{\tau}}{2n_{r}}},\]

_where the constants \(c\) and \(\tilde{c}\) may depend on \(|\Omega|\), \(|\partial\Omega|\), \(d\)\(\|z^{\delta}\|_{W^{1,\infty}(\Omega)}\), \(\|f\|_{L^{\infty}(\Omega)}\), and \(\|g\|_{L^{\infty}(\partial\Omega)}\) at most polynomially. Then, with a probability at least \(1-\tau\), each of the following statements holds_

\[\Delta\mathcal{E}_{i}\leq e_{i},\quad i\in\{d,\sigma,b,b^{\prime},q\}.\]

Now we can state an error estimate on the numerical approximation \(\widehat{q}_{\theta}^{*}\).

**Theorem 3.6**.: _Let Assumptions 3.1 and 3.4 and Condition 3.2 hold. Fix small \(\epsilon_{q}\), \(\epsilon_{\sigma}>0\), and let \((\widehat{\theta}^{*},\widehat{\kappa}^{*})\in(\mathfrak{P}_{p,c_{q}},\mathfrak{P }_{2,\epsilon_{\sigma}}^{\delta\delta})\) be a minimizer of the empirical loss (3.6), and \(\widehat{q}_{\theta}^{*}\) and \(\widehat{\sigma}_{\kappa}^{*}\) their NN realizations. Fix \(\tau\in(0,\frac{1}{8})\), let the bounds \(e_{d}\), \(e_{\sigma}\), \(e_{b}\) and \(e_{q}\) be defined in Theorem 3.5, and further define \(\eta\) by \(\eta:=e_{d}+\gamma_{\sigma}e_{\sigma}+\gamma_{b}e_{b}+\gamma_{q}e_{q}+\epsilon_{q}^ {2}+(\gamma_{\sigma}+\gamma_{b}+1)\epsilon_{\sigma}^{2}+\delta^{2}+\gamma_{q}\). Then the following error bound holds with probability at least \(1-4\tau\)_

\[\|q^{\dagger}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*})\|_{L^{2}(\Omega)}\leq c \big{(}(e_{d}+\eta)^{\frac{1}{2}}+(e_{\sigma}+\gamma_{\sigma}^{-1}\eta)^{ \frac{1}{2}}+(e_{b}+\gamma_{b}^{-1}\eta)^{\frac{1}{2}}+(e_{q}+\gamma_{q}^{-1} \eta)^{\frac{1}{2}}\delta\big{)}^{\frac{1}{2}}\big{(}1+(e_{q}+\gamma_{q}^{-1} \eta)^{\frac{1}{4}}\big{)}.\]
Proof.: Let \((\theta^{*},\kappa^{*})\in(\mathfrak{P}_{p,e_{q}},\mathfrak{P}_{2,e_{\sigma}}^{ \otimes d})\) be a minimizer of problem (3.4). Then the minimizing property of \((\widehat{\theta}^{*},\widehat{\kappa}^{*})\) to the empirical loss \(\widehat{J}_{\boldsymbol{\gamma}}(\theta,\kappa)\) implies the following decomposition

\[\widehat{J}_{\boldsymbol{\gamma}}(\widehat{\theta}^{*},\widehat{\kappa}^{*}) \leq[\widehat{J}_{\boldsymbol{\gamma}}(\theta^{*},\kappa^{*})-J_{ \boldsymbol{\gamma}}(\theta^{*},\kappa^{*})]+J_{\boldsymbol{\gamma}}(\theta^{* },\kappa^{*})\leq|J_{\boldsymbol{\gamma}}(\theta^{*},\kappa^{*})-\widehat{J}_ {\boldsymbol{\gamma}}(\theta^{*},\kappa^{*})|+J_{\boldsymbol{\gamma}}(\theta^{* },\kappa^{*}).\]

Consequently, we deduce

\[\widehat{J}_{\boldsymbol{\gamma}}(\widehat{\theta}^{*},\widehat{\kappa}^{*}) \leq J_{\boldsymbol{\gamma}}(\theta^{*},\kappa^{*})+\sup_{(\theta,\kappa)\in( \mathfrak{P}_{p,e_{q}},\mathfrak{P}_{2,e_{\sigma}}^{\otimes d})}|J_{ \boldsymbol{\gamma}}(\theta,\kappa)-\widehat{J}_{\boldsymbol{\gamma}}(\theta, \kappa)|.\]

The two terms represent the approximation error and statistical error, respectively, and the former was already bounded in Section 3.2. It follows directly from Lemma 3.2 and Theorem 3.5 that with a probability at least \(1-4\tau\),

\[\widehat{J}_{\boldsymbol{\gamma}}(\widehat{\theta}^{*},\widehat{\kappa}^{*}) \leq c\big{(}e_{d}+\gamma_{\sigma}e_{\sigma}+\gamma_{b}e_{b}+\gamma_{q}e_{q}+ \epsilon_{q}^{2}+(\gamma_{\sigma}+\gamma_{b}+1)\epsilon_{\sigma}^{2}+\delta^{ 2}+\gamma_{q}\big{)},\]

i.e., \(\widehat{J}_{\boldsymbol{\gamma}}(\widehat{\theta}^{*},\widehat{\kappa}^{*}) \leq c\eta\). This estimate and the triangle inequality imply that with probability at least \(1-4\tau\),

\[\|\widehat{\sigma}_{\kappa}^{*}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*}) \nabla z^{\delta}\|_{L^{2}(\Omega)}^{2}\leq\big{[}\mathcal{E}_{d}(\widehat{ \sigma}_{\kappa}^{*},\widehat{q}_{\theta}^{*})-\widehat{\mathcal{E}}_{d}( \widehat{\sigma}_{\kappa}^{*},\widehat{q}_{\theta}^{*})\big{]}+\widehat{ \mathcal{E}}_{d}(\widehat{\sigma}_{\kappa}^{*},\widehat{q}_{\theta}^{*})\leq c (e_{d}+\eta).\]

Similarly, the following estimates hold simultaneously with a probability at least \(1-4\tau\),

\[\|f+\nabla\cdot\widehat{\sigma}_{\kappa}^{*}\|_{L^{2}(\Omega)}^{2}\leq c(e_{ \sigma}+\gamma_{\sigma}^{-1}\eta),\quad\|g-\widehat{\sigma}_{\kappa}^{*}\cdot \mathbf{n}\|_{L^{2}(\partial\Omega)}^{2}\leq c(e_{b}+\gamma_{b}^{-1}\eta), \quad\|\nabla\widehat{q}_{\theta}^{*}\|_{L^{2}(\Omega)}^{2}\leq c(e_{q}+\gamma _{q}^{-1}\eta).\]

Replacing \(q_{b}^{*}\) by \(\widehat{q}_{\theta}^{*}\) and repeating the argument of Theorem 3.3 gives

\[\|q^{\dagger}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*})\|_{L^{2}( \Omega)}^{2}\leq c\big{(}\|f+\nabla\cdot\widehat{\sigma}_{\kappa}^{*}\|_{L^{2}( \Omega)}+\|\widehat{\sigma}_{\kappa}^{*}-P_{\mathcal{A}}(\widehat{q}_{\theta}^ {*})\nabla z^{\delta}\|_{L^{2}(\Omega)}\] \[+\|P_{\mathcal{A}}(\widehat{q}_{\theta}^{*})\|_{L^{\infty}( \Omega)}\|\nabla(z^{\delta}-u^{\dagger})\|_{L^{2}(\Omega)}+\|g-\widehat{ \sigma}_{\kappa}^{*}\cdot\mathbf{n}\|_{L^{2}(\partial\Omega)}\big{)}\|v_{ \psi}\|_{H^{1}(\Omega)}.\]

This and the preceding estimates and Condition 3.2 imply (with \(v_{\psi}\) from Condition 3.2 for \(\psi=q^{\dagger}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*})\))

\[\|q^{\dagger}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*})\|_{H^{-1}(\Omega)} \leq c\big{(}(e_{d}+\eta)^{\frac{1}{2}}+(e_{\sigma}+\gamma_{\sigma}^{-1}\eta)^{ \frac{1}{2}}+(e_{b}+\gamma_{b}^{-1}\eta)^{\frac{1}{2}}+(e_{q}+\gamma_{q}^{-1} \eta)^{\frac{1}{2}}\delta\big{)}.\]

The desired estimate now follows from the argument of Theorem 3.3 and the bound on \(\|\nabla\widehat{q}_{\theta}^{*}\|_{L^{2}(\Omega)}^{2}\). 

**Remark 3.2**.: _Under the assumptions of Theorem 3.5 and the parameter selections in Remark 3.1, we may choose the numbers \(n_{\tau}\) and \(n_{b}\) of sample points in the domain \(\Omega\) and on the boundary \(\partial\Omega\) by_

\[n_{\tau}=O\Big{(}\max\Big{(}\frac{R_{\sigma}^{4}N_{\kappa}^{4}(N_{\theta}+N_{ \kappa})}{\delta^{4+s}},\frac{R_{\sigma}^{4L_{\sigma}}N_{\kappa}^{4L_{\sigma}-3 }}{\delta^{4+s}},\frac{R_{q}^{4L_{\sigma}}N_{\kappa}^{4L_{\sigma}-3}}{\delta^{ 4}}\Big{)}\Big{)}\quad\text{and}\quad n_{b}=O\Big{(}\frac{R_{\sigma}^{4}N_{ \kappa}^{5}}{\delta^{4+s}}\Big{)},\]

_for any \(s>0\) (i.e., using the power \(s\) to absorb the \(\log\) factor). Note that by Lemma 2.1, \(R_{q}=O(\delta^{-2.\frac{2p+3d+3pd+2pn}{p(1-\mu)}})\), \(N_{\theta}=O(\delta^{-\frac{d}{1-\mu}})\), \(R_{\sigma}=O(\delta^{-2-\frac{4(1+\mu)+pd}{2(1-\mu)}})\) and \(N_{\kappa}=O(\delta^{-\frac{d}{1-\mu}})\). Then with probability at least \(1-4\tau\), we have_

\[\|q^{\dagger}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*})\|_{L^{2}(\Omega)}\leq c \delta^{\frac{1}{2}}.\]

## 4 Inverse conductivity problem in the Dirichlet case

In this section, we extend the approach to the Dirichlet boundary value problem:

\[\begin{cases}-\nabla\cdot(q\nabla u)=f,&\text{ in }\Omega,\\ \qquad\qquad u=0,&\text{ on }\partial\Omega.\end{cases} \tag{4.1}\]

We also provide relevant analysis of the reconstruction scheme.
### Mixed formulation and its DNN approximation

Like in Section 3, to develop a reconstruction algorithm, we rewrite (4.1) into a first-order system:

\[\begin{cases}\sigma=q\nabla u,&\text{in}\ \ \Omega,\\ -\nabla\cdot\sigma=f,&\text{in}\ \Omega,\\ u=0,&\text{on}\ \partial\Omega.\end{cases} \tag{4.2}\]

To recover the conductivity coefficient \(q\), we discretize the scalar field \(q\) and vector field \(\sigma\) by two DNN function classes \(\mathfrak{P}_{p,e_{q}}\) and \(\mathfrak{P}_{2,e_{\sigma}}^{\otimes d}\), respectively. Then with the notation in Section 3, the DNN approximation scheme reads

\[\min_{(\theta,\kappa)\in(\mathfrak{P}_{p,e_{q}},\mathfrak{P}_{2,e_{\sigma}}^{ \otimes d})}J_{\boldsymbol{\gamma}}(\theta,\kappa)=\|\sigma_{\kappa}-P_{ \mathcal{A}}(q_{\theta})\nabla z^{\delta}\|_{L^{2}(\Omega)}^{2}+\gamma_{ \sigma}\|\nabla\cdot\sigma_{\kappa}+f\|_{L^{2}(\Omega)}^{2}+\gamma_{q}\| \nabla q_{\theta}\|_{L^{2}(\Omega)}^{2}, \tag{4.3}\]

where \(z^{\delta}\) is a noisy measurement of the exact data \(u(q^{\dagger})\) with the noise level \(\delta\) satisfying \(\delta:=\|u(q^{\dagger})-z^{\delta}\|_{W^{\frac{3}{2},2}(\Omega)}\). In practice, however, the formulation (4.3) does not lend itself to high-quality reconstructions. This is attributed to the absence of the current density \(\sigma\) on the boundary \(\partial\Omega\) so that the first term does not allow learning the current density \(\sigma\) accurately. Hence, we supplement the loss (4.3) with an additional boundary term

\[\begin{split}\min_{(\theta,\kappa)\in(\mathfrak{P}_{p,e_{q}}, \mathfrak{P}_{2,e_{\sigma}}^{\otimes d})}J_{\boldsymbol{\gamma}}(\theta, \kappa)=&\|\sigma_{\kappa}-P_{\mathcal{A}}(q_{\theta})\nabla z^{ \delta}\|_{L^{2}(\Omega)}^{2}+\gamma_{\sigma}\|\nabla\cdot\sigma_{\kappa}+f \|_{L^{2}(\Omega)}^{2}\\ &+\gamma_{q}\|\nabla q_{\theta}\|_{L^{2}(\Omega)}^{2}+\gamma_{b} \|\sigma_{\kappa}-q^{\dagger}\nabla z^{\delta}\|_{L^{2}(\partial\Omega)}^{2}. \end{split} \tag{4.4}\]

This modified formulation requires a knowledge of the exact conductivity \(q^{\dagger}\) on the boundary \(\partial\Omega\). Note that this assumption is frequently made in existing uniqueness analysis [4, 35] and numerical studies [7, 8]. Similarly, one can ensure the existence of a minimizer of (4.4). Let \((\theta^{*},\kappa^{*})\) be a minimizer of problem (4.4) and \((q_{\theta}^{*},\sigma_{\kappa}^{*})\) be its DNN realization. In practice, we approximate the integrals using Monte Carlo methods. Using the uniform distributions \(\mathcal{U}(\Omega)\) and \(\mathcal{U}(\partial\Omega)\), we rewrite the population loss (4.4) as

\[J_{\boldsymbol{\gamma}}(\theta,\kappa) =|\Omega|\mathbb{E}_{X\sim\mathcal{U}(\Omega)}\Big{[}\|\sigma_{ \kappa}(X)-P_{\mathcal{A}}(q_{\theta}(X))\nabla z^{\delta}(X)\|_{\ell^{2}}^{2} \Big{]}+\gamma_{\sigma}|\Omega|\mathbb{E}_{X\sim\mathcal{U}(\Omega)}\Big{[} \big{(}\nabla\cdot\sigma_{\kappa}(X)+f(X)\big{)}^{2}\Big{]}\] \[\quad+\gamma_{b}|\partial\Omega|\mathbb{E}_{Y\sim\mathcal{U}( \partial\Omega)}\Big{[}\|\sigma_{\kappa}-q^{\dagger}\nabla z^{\delta}(Y)\|_{ \ell^{2}}^{2}\Big{]}+\gamma_{q}|\Omega|\mathbb{E}_{X\sim\mathcal{U}(\Omega)} \Big{[}\|\nabla q_{\theta}(X)\|_{\ell^{2}}^{2}\Big{]}\] \[=:\mathcal{E}_{d}(\sigma_{\kappa},q_{\theta})+\gamma_{\sigma} \mathcal{E}_{\sigma}(\sigma_{\kappa})+\gamma_{b}\mathcal{E}_{b^{\prime}}( \sigma_{\kappa})+\gamma_{q}\mathcal{E}_{q}(q_{\theta}).\]

Now we draw i.i.d. samples \(X=\{X_{j}\}_{j=1}^{n_{s}}\) and \(Y=\{Y_{j}\}_{j=1}^{n_{b}}\) from the uniform distributions \(\mathcal{U}(\Omega)\) and \(\mathcal{U}(\partial\Omega)\). The empirical loss \(\widehat{J}_{\boldsymbol{\gamma}}(\theta,\kappa)\) is given by

\[\widehat{J}_{\boldsymbol{\gamma}}(\theta,\kappa):=\widehat{\mathcal{E}}_{d}( \sigma_{\kappa},q_{\theta})+\gamma_{\sigma}\widehat{\mathcal{E}}_{\sigma}( \sigma_{\kappa})+\gamma_{q}\widehat{\mathcal{E}}_{q}(q_{\theta})+\gamma_{b} \widehat{\mathcal{E}}_{b^{\prime}}(\sigma_{\kappa}), \tag{4.5}\]

where \(\widehat{\mathcal{E}}_{d}(\sigma_{\kappa},q_{\theta})\), \(\widehat{\mathcal{E}}_{\sigma}(\sigma_{\kappa})\) and \(\widehat{\mathcal{E}}_{q}(q_{\theta})\) are given by (3.7)-(3.9), and \(\widehat{\mathcal{E}}_{b^{\prime}}(\sigma_{\kappa,i})\) is given in (3.11).

**Remark 4.1**.: _Instead of (4.4), there are alternative formulations. For example, one may use the loss_

\[J_{\boldsymbol{\gamma}}(\theta,\kappa)=\|\sigma_{\kappa}-P_{\mathcal{A}}(q_{ \theta})\nabla z^{\delta}\|_{L^{2}(\Omega)}^{2}+\gamma_{\sigma}\|\nabla \cdot\sigma_{\kappa}+f\|_{L^{2}(\Omega)}^{2}+\gamma_{q}\|\nabla q_{\theta}\| _{L^{2}(\Omega)}^{2}+\gamma_{b}\|q_{\theta}-q^{\dagger}\|_{L^{2}(\partial \Omega)}^{2}, \tag{4.6}\]

_which enforces the boundary condition \(q_{\theta}=q^{\dagger}\) on \(\partial\Omega\) directly. It can be analyzed analogously. However, numerically, it is less robust than (4.4) for noisy data. See Section 5 for numerical illustrations._

### Error analysis of the population loss

First we analyze the error of the DNN realization \(q_{\theta}^{*}\in\mathcal{N}_{q}\) of the minimizer \(\theta^{*}\) to the population loss (4.4) under the following assumption.

**Assumption 4.1**.: \(q^{\dagger}\in W^{2,p}(\Omega)\cap\mathcal{A}\)_, and \(f\in H^{1}(\Omega)\cap L^{\infty}(\Omega)\), with \(p=\max(2,d+\nu)\) for some small \(\nu>0\)._

The following regularity result holds for \(u^{\dagger}:=u(q^{\dagger})\) and \(\sigma^{\dagger}:=q^{\dagger}\nabla u^{\dagger}\). The proof follows exactly as Lemma 3.1, and hence it is omitted.
**Lemma 4.1**.: _Let Assumption 4.1 hold. Then the solution \(u^{\dagger}\left(\text{with }q=q^{\dagger}\right)\) to problem (4.1) satisfies \(u^{\dagger}\in H^{3}(\Omega)\cap W^{2,p}(\Omega)\cap H^{1}_{0}(\Omega)\) and \(\sigma^{\dagger}\in(H^{2}(\Omega))^{d}\)._

**Lemma 4.2**.: _Let Assumption 4.1 hold. Fix small \(\epsilon_{q}\), \(\epsilon_{\sigma}>0\), and let \((\theta^{*},\kappa^{*})\in(\mathfrak{P}_{p,\epsilon_{q}},\mathfrak{P}_{2, \epsilon_{\sigma}}^{\otimes d})\) be a minimizer of problem (4.4). Then the following estimate holds_

\[J_{\boldsymbol{\gamma}}(\theta^{*},\kappa^{*})\leq c\big{(}\epsilon_{q}^{2}+(1 +\gamma_{\sigma}+\gamma_{b})\epsilon_{\sigma}^{2}+(1+\gamma_{\sigma})\delta^{ 2}+\gamma_{q}\big{)}.\]

Proof.: Assumption 3.1 and Lemma 4.1 imply \(\sigma^{\dagger}\in(H^{2}(\Omega))^{d}\). Then by Lemma 2.1, there exists at least one \((\theta_{\epsilon},\kappa_{\epsilon})\in(\mathfrak{P}_{p,\epsilon_{q}}, \mathfrak{P}_{2,\epsilon_{\sigma}}^{\otimes d})\) such that its DNN realization \((q_{\theta_{\epsilon}},\sigma_{\kappa_{\epsilon}})\) satisfies

\[\|q^{\dagger}-q_{\theta_{\epsilon}}\|_{W^{1,p}(\Omega)}\leq\epsilon_{q}\quad \text{and}\quad\|\sigma^{\dagger}-\sigma_{\kappa_{\epsilon}}\|_{H^{1}(\Omega)} \leq\epsilon_{\sigma}.\]

Then by the minimizing property of \((\theta^{*},\kappa^{*})\) and the triangle inequality, we have

\[J_{\boldsymbol{\gamma}}(\theta^{*},\kappa^{*})\leq J_{\boldsymbol {\gamma}}(\theta_{\epsilon},\kappa_{\epsilon})\] \[=\|\sigma_{\kappa_{\epsilon}}-P_{\mathcal{A}}(q_{\theta_{\epsilon }})\nabla z^{\delta}\|_{L^{2}(\Omega)}^{2}+\gamma_{\sigma}\|\nabla\cdot \sigma_{\kappa_{\epsilon}}+f\|_{L^{2}(\Omega)}^{2}+\gamma_{b}\|\sigma_{\kappa_ {\epsilon}}-q^{\dagger}\nabla z^{\delta}\|_{L^{2}(\partial\Omega)}^{2}+\gamma_ {q}\|\nabla q_{\theta_{\epsilon}}\|_{L^{2}(\Omega)}^{2}\] \[\leq c\big{(}\|\sigma_{\kappa_{\epsilon}}-\sigma^{\dagger}\|_{L^{ 2}(\Omega)}^{2}+\|(q^{\dagger}-P_{\mathcal{A}}(q_{\theta_{\epsilon}}))\nabla u ^{\dagger}\|_{L^{2}(\Omega)}^{2}+\|P_{\mathcal{A}}(q_{\theta_{\epsilon}}) \nabla(u^{\dagger}-z^{\delta})\|_{L^{2}(\Omega)}^{2}\] \[\qquad+\gamma_{\sigma}\|\nabla\cdot(\sigma_{\kappa_{\epsilon}}- \sigma^{\dagger})\|_{L^{2}(\Omega)}^{2}+\gamma_{q}\|\nabla q_{\theta_{\epsilon }}\|_{L^{2}(\Omega)}^{2}+\gamma_{b}\|\sigma_{\kappa_{\epsilon}}-\sigma^{ \dagger}\|_{H^{1}(\Omega)}^{2}+\gamma_{b}\|q^{\dagger}\nabla(u^{\dagger}-z^{ \delta})\|_{H^{\frac{1}{2}}(\partial\Omega)}^{2}\big{)}.\]

Now the estimate (3.2) and Assumption 4.1 imply

\[J_{\boldsymbol{\gamma}}(\theta^{*},\kappa^{*}) \leq c\big{(}\epsilon_{\sigma}^{2}+\|q^{\dagger}-P_{\mathcal{A}}(q _{\theta_{\epsilon}})\|_{L^{\infty}(\Omega)}^{2}\|\nabla u^{\dagger}\|_{L^{2 }(\Omega)}^{2}\] \[\quad+\|P_{\mathcal{A}}(q_{\theta_{\epsilon}})\|_{L^{\infty}( \Omega)}^{2}\|\nabla(u^{\dagger}-z^{\delta})\|_{L^{2}(\Omega)}^{2}+\gamma_{ \sigma}\epsilon_{\sigma}^{2}+\gamma_{q}+\gamma_{b}\epsilon_{\sigma}^{2}+\gamma _{b}\delta^{2}\big{)}\] \[\leq c\big{(}\epsilon_{q}^{2}+(1+\gamma_{\sigma}+\gamma_{b}) \epsilon_{\sigma}^{2}+(1+\gamma_{b})\delta^{2}+\gamma_{q}\big{)}.\]

This completes the proof of the lemma. 

**Condition 4.2**.: _There exist some \(\beta>0\) and \(c\) such that \(q^{\dagger}|\nabla u^{\dagger}|^{2}+fu^{\dagger}\geq c\operatorname{dist}(x, \partial\Omega)^{\beta}\) almost every in \(\Omega\)._

This condition holds under mild assumptions [10, Lemmas 3.3 and 3.7]: it holds with \(\beta=2\) if \(q^{\dagger}\in\mathcal{A}\), and \(f\in L^{2}(\Omega)\) with \(f\geq c_{f}>0\) (with \(c_{f}\in\mathbb{R}\)) over a Lipschitz domain \(\Omega\), and with \(\beta=0\) when \(q^{\dagger}\in C^{1,\alpha}(\overline{\Omega})\cap\mathcal{A}\), and \(f\in C^{0,\alpha}(\overline{\Omega})\) and \(f\geq c_{f}>0\)on a \(C^{2,\alpha}\) domain \(\Omega\) for some \(\alpha>0\).

**Theorem 4.3**.: _Let Assumption 4.1 hold. Fix small \(\epsilon_{q}\), \(\epsilon_{\sigma}>0\), and let \((\theta^{*},\kappa^{*})\in(\mathfrak{P}_{p,\epsilon_{q}},\mathfrak{P}_{2, \epsilon_{\sigma}}^{\otimes d})\) be a minimizer of problem (4.4) and \(q_{\theta}^{*}\) the DNN realization of \(\theta^{*}\). Then with \(\eta:=\big{(}\epsilon_{q}^{2}+(1+\gamma_{\sigma}+\gamma_{b})\epsilon_{\sigma }^{2}+(1+\gamma_{b})\delta^{2}+\gamma_{q}\big{)}^{\frac{1}{2}},\) there holds_

\[\int_{\Omega}\Big{(}\frac{q^{\dagger}-q_{\theta}^{*}}{q^{\dagger}}\Big{)}^{2} \big{(}q^{\dagger}|\nabla u^{\dagger}|^{2}+fu^{\dagger}\big{)}\;\mathrm{d}x \leq c\big{(}\gamma_{\sigma}^{-\frac{1}{2}}+\gamma_{q}^{-\frac{1}{2}}\eta+1 \big{)}\eta.\]

_Moreover, if Condition 4.2 holds, then_

\[\|q^{\dagger}-q_{\theta}^{*}\|_{L^{2}(\Omega)}\leq c\big{[}\big{(}\gamma_{\sigma }^{-\frac{1}{2}}+\gamma_{q}^{-\frac{1}{2}}\eta+1\big{)}\eta\big{]}^{\frac{1}{2( \vartheta+1)}}.\]

Proof.: For any test function \(\varphi\in H^{1}_{0}(\Omega)\), we have

\[\big{(}(q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))\nabla u^{ \dagger},\nabla\varphi\big{)} =\big{(}\sigma^{\dagger}-\sigma_{\kappa}^{*},\nabla\varphi\big{)}+ \big{(}P_{\mathcal{A}}(q_{\theta}^{*})\nabla(z^{\delta}-u^{\dagger}),\nabla \varphi\big{)}+\big{(}\sigma_{\kappa}^{*}-P_{\mathcal{A}}(q_{\theta}^{*})\nabla z ^{\delta},\nabla\varphi\big{)}\] \[=-\big{(}\nabla\cdot(\sigma^{\dagger}-\sigma_{\kappa}^{*}),\varphi \big{)}+\big{(}P_{\mathcal{A}}(q_{\theta}^{*})\nabla(z^{\delta}-u^{\dagger}), \nabla\varphi\big{)}+\big{(}\sigma_{\kappa}^{*}-P_{\mathcal{A}}(q_{\theta}^{*}) \nabla z^{\delta},\nabla\varphi\big{)}.\]

Let \(\varphi\equiv\frac{q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})}{q^{\dagger}}u^{\dagger}\). Then by direct computation, we have

\[\nabla\varphi=\frac{\nabla(q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))}{q^{ \dagger}}u^{\dagger}+\frac{(q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))}{q^{ \dagger}}\nabla u^{\dagger}-\frac{(q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))}{(q ^{\dagger})^{2}}(\nabla q^{\dagger})u^{\dagger}.\]
Using Assumption 4.1, the box constraint on \(q^{\dagger}\) and \(P_{\mathcal{A}}(q_{\theta}^{*})\), and the stability estimate (3.2) of \(P_{\mathcal{A}}\), we arrive at

\[\left\|\frac{\nabla(q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))}{q ^{\dagger}}u^{\dagger}\right\|_{L^{2}(\Omega)} \leq c\big{(}1+\|\nabla P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}( \Omega)}\big{)}\leq c\big{(}1+\|\nabla q_{\theta}^{*}\|_{L^{2}(\Omega)}\big{)},\] \[\left\|\frac{(q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))}{q^{ \dagger}}\nabla u^{\dagger}\right\|_{L^{2}(\Omega)}+\left\|\frac{(q^{\dagger} -P_{\mathcal{A}}(q_{\theta}^{*}))}{(q^{\dagger})^{2}}(\nabla q^{\dagger})u^{ \dagger}\right\|_{L^{2}(\Omega)}\leq c.\]

This implies \(\varphi\in H^{1}_{0}(\Omega)\) with the following _a priori_ bound

\[\|\varphi\|_{L^{2}(\Omega)}\leq c\quad\text{and}\quad\|\nabla\varphi\|_{L^{2} (\Omega)}\leq c(1+\|\nabla q_{\theta}^{*}\|_{L^{2}(\Omega)}).\]

By Lemma 4.2 and the Cauchy-Schwarz inequality, we have

\[|(\nabla\cdot(\sigma^{\dagger}-\sigma_{\kappa}^{*}),\varphi)|\leq\|\nabla \cdot(\sigma^{\dagger}-\sigma_{\kappa}^{*})\|_{L^{2}(\Omega)}\|\varphi\|_{L^{2 }(\Omega)}\leq c\gamma_{\sigma}^{-\frac{1}{2}}\eta.\]

Similarly, we deduce

\[|\big{(}\sigma_{\kappa}^{*}-P_{\mathcal{A}}(q_{\theta}^{*})\nabla z^{\delta}, \nabla\varphi\big{)}|\leq\|\sigma_{\kappa}^{*}-q_{\theta}^{*}\nabla z^{\delta }\|_{L^{2}(\Omega)}\|\nabla\varphi\|_{L^{2}(\Omega)}\leq c(1+\gamma_{q}^{- \frac{1}{2}}\eta)\eta.\]

Meanwhile, it follows from the Cauchy-Schwartz inequality that

\[|\big{(}P_{\mathcal{A}}(q_{\theta}^{*})\nabla(z^{\delta}-u^{\dagger}),\nabla \varphi\big{)}|\leq c\|P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{\infty}(\Omega)} \|\nabla(z^{\delta}-u^{\dagger})\|_{L^{2}(\Omega)}\|\nabla\varphi\|_{L^{2}( \Omega)}\leq c(1+\gamma_{q}^{-\frac{1}{2}}\eta)\delta.\]

Upon repeating the argument in [10, 25], we obtain

\[\big{(}(q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))\nabla u^{\dagger},\nabla \varphi\big{)}=\frac{1}{2}\int_{\Omega}\Big{(}\frac{q^{\dagger}-P_{\mathcal{A }}(q_{\theta}^{*})}{q^{\dagger}}\Big{)}^{2}\big{(}q^{\dagger}|\nabla u^{ \dagger}|^{2}+fu^{\dagger}\big{)}\;\mathrm{d}x.\]

Combining the preceding estimates yields the first assertion. Next we bound \(\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}(\Omega)}.\) Upon fixing \(\rho>0\), we split the domain \(\Omega\) into two disjoint sets \(\Omega=\Omega_{\rho}\cup\Omega_{\rho}^{c}\), with \(\Omega_{\rho}=\{x\in\Omega:\mathrm{dist}(x,\partial\Omega)\geq\rho\}\) and \(\Omega_{\rho}^{c}=\Omega\setminus\Omega_{\rho}\). Then by the box constraint \(q^{\dagger}\in\mathcal{A}\), we have

\[\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}(\Omega_{ \rho})}^{2} =\rho^{-\beta}\int_{\Omega_{\rho}}(q^{\dagger}-P_{\mathcal{A}}(q_{ \theta}^{*}))^{2}\rho^{\beta}\mathrm{d}x\leq\rho^{-\beta}\int_{\Omega_{\rho}}( q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*}))^{2}\mathrm{dist}(x,\partial\Omega)^{ \beta}\mathrm{d}x\] \[\leq c\rho^{-\beta}\int_{\Omega_{\rho}}\Big{(}\frac{q^{\dagger}-P_ {\mathcal{A}}(q_{\theta}^{*})}{q^{\dagger}}\Big{)}^{2}\big{(}q^{\dagger}| \nabla u^{\dagger}|^{2}+fu^{\dagger}\big{)}\mathrm{d}x\leq c\rho^{-\beta}\big{(} \gamma_{\sigma}^{-\frac{1}{2}}+\gamma_{q}^{-\frac{1}{2}}\eta+1\big{)}\eta.\]

Meanwhile, using the box constraint \(q^{\dagger},P_{\mathcal{A}}(q_{\theta}^{*})\in\mathcal{A}\), we obtain

\[\|q^{\dagger}-P_{\mathcal{A}}(q_{\theta}^{*})\|_{L^{2}(\Omega_{ \rho}^{c})}^{2} \leq c\int_{\Omega_{\rho}^{c}}1\;\mathrm{d}x\|q^{\dagger}-P_{ \mathcal{A}}(q_{\theta}^{*})\|_{L^{\infty}(\Omega_{\rho}^{c})}^{2}\leq c\rho.\]

By combining the last two estimates and then optimizing in \(\rho\), we get the desired bound on \(\|q^{\dagger}-q_{\theta}^{*}\|_{L^{2}(\Omega)}\). 

### Error analysis of the empirical loss

Now we analyze the impact of quadrature error on the reconstruction, under the following condition.

**Assumption 4.4**.: \(f\in L^{\infty}(\Omega)\) _and \(\nabla z^{\delta}\in L^{\infty}(\Omega)\cap L^{\infty}(\partial\Omega)\)._

We have the following error bound on the DNN realization \(\widehat{q}_{\theta}^{*}\) of a minimizer \(\widehat{\theta}^{*}\) of the loss (4.5).

**Theorem 4.5**.: _Let Assumptions 4.1 and 4.4 hold. Fix small \(\epsilon_{q}\), \(\epsilon_{\sigma}>0\), and let \((\widehat{\theta}^{*},\widehat{\kappa}^{*})\in(\mathfrak{P}_{p,\epsilon_{q}}, \mathfrak{P}_{\infty,\epsilon_{\sigma}}^{\otimes d})\) be a minimizer of (4.5) and \(\widehat{q}_{\theta}^{*}\) the DNN realization of \(\widehat{\theta}^{*}\). Fix \(\tau\in(0,\frac{1}{8})\), let the quantities \(e_{d}\), \(e_{\sigma}\), \(e_{b^{\prime}}\) and \(e_{q}\) be defined in
_Theorem 3.5, and define \(\eta\) by \(\eta:=e_{d}+\gamma_{\sigma}e_{\sigma}+\gamma_{b}e_{\nu}+\gamma_{q}e_{q}+\epsilon _{q}^{2}+(1+\gamma_{\sigma}+\gamma_{b})\epsilon_{\sigma}^{2}+(1+\gamma_{b}) \delta^{2}+\gamma_{q}\). Then with probability at least \(1-4\tau\), there holds_

\[\int_{\Omega}\Big{(}\frac{q^{\dagger}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*} )}{q^{\dagger}}\Big{)}^{2}\big{(}q^{\dagger}|\nabla u^{\dagger}|^{2}+fu^{ \dagger}\big{)}\;\mathrm{d}x\leq c\big{(}(e_{d}+\eta)^{\frac{1}{2}}+(e_{ \sigma}+\gamma_{\sigma}^{-1}\eta)^{\frac{1}{2}}+\delta(1+e_{q}+\gamma_{q}^{-1 }\eta)^{\frac{1}{2}}\big{)}\big{(}1+e_{q}+\gamma_{q}^{-1}\eta\big{)}^{\frac{1 }{2}},\]

_where the constant \(c>0\) depends on \(|\Omega|\), \(|\partial\Omega|\), \(d\), \(\|z^{\delta}\|_{W^{1,\infty}(\partial\Omega)}\), \(\|f\|_{L^{\infty}(\Omega)}\) and \(\|q^{\dagger}\|_{L^{\infty}(\partial\Omega)}\) at most polynomially. Moreover, if Condition 4.2 holds, then with probability at least \(1-4\tau\),_

\[\|q^{\dagger}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*})\|_{L^{2}(\Omega)}\leq c \big{(}\big{(}(e_{d}+\eta)^{\frac{1}{2}}+(e_{\sigma}+\gamma_{\sigma}^{-1}\eta )^{\frac{1}{2}}+\delta(1+e_{q}+\gamma_{q}^{-1}\eta)^{\frac{1}{2}}\big{)}\big{(} 1+e_{q}+\gamma_{q}^{-1}\eta\big{)}^{\frac{1}{2}}\big{)}^{\frac{1}{2(\partial+1 )}}.\]

Proof.: The proof is similar to Theorem 3.6, using instead the estimate on \(e_{b^{\prime}}\). Indeed, the following estimate holds

\[\sup_{(\theta,\kappa)\in(\mathfrak{P}_{\eta,e_{q}},\mathfrak{P}_{\infty,e_{ \sigma}}^{\otimes d})}\big{|}J_{\mathbf{\gamma}}(q_{\theta},\sigma_{\kappa})- \widehat{J}_{\mathbf{\gamma}}(q_{\theta},\sigma_{\kappa})\big{|}\leq\Delta\mathcal{ E}_{d}+\gamma_{\sigma}\Delta\mathcal{E}_{\sigma}+\gamma_{b}\Delta\mathcal{E}_{b^{ \prime}}+\gamma_{q}\Delta\mathcal{E}_{q}.\]

Then for any minimizer \((\widehat{\theta}^{*},\widehat{\kappa}^{*})\) of the empirical loss (4.5), with probability at least \(1-4\tau\),

\[\widehat{J}_{\mathbf{\gamma}}(\widehat{\theta}^{*},\widehat{\kappa}^{*})\leq c \big{(}e_{d}+\gamma_{\sigma}e_{\sigma}+\gamma_{b}e_{\nu}+\gamma_{q}e_{q}+ \epsilon_{q}^{2}+(1+\gamma_{\sigma}+\gamma_{b})\epsilon_{\sigma}^{2}+(1+\gamma _{b})\delta^{2}+\gamma_{q}\big{)}.\]

Then by repeating the argument from Theorem 3.6 and replacing \(q_{\theta}^{*}\) by \(\widehat{q}_{\theta}^{*}\), we deduce that with probability at least \(1-4\tau\), the following three estimates hold simultaneously

\[\|\widehat{\sigma}_{\kappa}^{*}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*}) \nabla z^{\delta}\|_{L^{2}(\Omega)}^{2}\leq c(e_{d}+\eta),\quad\|f+\nabla \cdot\hat{\sigma}_{\kappa}^{*}\|_{L^{2}(\Omega)}^{2}\leq c(e_{\sigma}+\gamma_ {\sigma}^{-1}\eta),\quad\|\nabla\widetilde{q}_{\theta}^{*}\|_{L^{2}(\Omega)}^ {2}\leq c(e_{q}+\gamma_{q}^{-1}\eta).\]

Last, the desired estimates follow from the argument of Theorem 4.3. 

**Remark 4.2**.: _Under the assumptions in Theorem 4.5 and the choice of the numbers of sampling points in Remark 3.1, there holds with probability at least \(1-4\tau\), \(\|q^{\dagger}-P_{\mathcal{A}}(\widehat{q}_{\theta}^{*})\|_{L^{2}(\Omega)}\leq c \delta^{\frac{1}{2(1+\beta)}}\)._

## 5 Numerical experiments and discussions

Now we showcase the performance of the proposed approach. All computations are performed on TensorFlow 1.15.0 using Intel Core i7-11700K Processor with 16 CPUs. We measure the accuracy of a reconstruction \(\hat{q}\) (with respect to the exact one \(q^{\dagger}\)) by the relative \(L^{2}(\Omega)\) error \(e(\hat{q})\) defined by

\[e(\hat{q})=\|q^{\dagger}-\hat{q}\|_{L^{2}(\Omega)}/\|q^{\dagger}\|_{L^{2}( \Omega)}.\]

Throughout, for an elliptic problem in \(\mathbb{R}^{d}\), we use DNNs with an output dimension \(1\) and \(d\) to approximate the conductivity \(q\) and the flux \(\sigma\), respectively. Unless otherwise stated, both DNNs have 4 hidden layers (i.e., depth 5) with 26, 26, 26, and 10 neurons on each layer. The activation function \(\rho\) is taken to be \(\tanh\). The penalty parameters \(\gamma_{\sigma}\), \(\gamma_{b}\) and \(\gamma_{q}\) are for the divergence term, boundary term and \(H^{1}(\Omega)\)-penalty term, respectively, in the losses (3.4) and (4.4), and are determined in a trial-and-error way. The numbers of training points in the domain \(\Omega\) and on the boundary \(\partial\Omega\) are denoted by \(n_{r}\) and \(n_{b}\), respectively. The empirical loss \(\widehat{L}_{\mathbf{\gamma}}\) is minimized by ADAM [28]. We adopt an exponentially decaying learning rate schedule, which is determined by the starting learning rate (lr), decay rate (dr) and the epoch number at which the decay takes place (step). The epoch refers to the total number of epochs used for the reconstruction. Table 1 summarizes the algorithmic parameters for the experiments where the numbers in the brackets indicate the parameters used for noisy data (\(\delta=10\%\)).

### The Neumann problem

The first example is about recovering a smooth conductivity \(q^{\dagger}\) with three modes in 2D.

**Example 5.1**.: \(\Omega=(-1,1)^{2}\)_, \(q^{\dagger}=1+s_{1}(x_{1},x_{2})+s_{2}(x_{1},x_{2})+s_{3}(x_{1},x_{2})\), with \(s_{1}=0.3e^{-20(x_{1}-0.3)^{2}-15(x_{2}-0.3)^{2}}\), \(s_{2}=-0.3e^{-10x_{1}^{2}-10(x_{2}+0.5)^{2}}\) and \(s_{3}=0.2e^{-15(x_{1}+0.4)^{2}-15(x_{2}-0.35)^{2}}\), and \(u^{\dagger}=x_{1}+x_{2}+\frac{1}{3}(x_{1}^{3}+x_{2}^{3})\)._
\begin{table}
\begin{tabular}{c|c c c c|c c c c} \hline \hline para. \(\backslash\) Ex. No. & 5.1 & 5.2 & 5.3 & 5.4 & 5.5 & 5.6 & 5.7 & 5.8 \\ \hline \(\gamma_{\sigma}\) & 10(100) & 10(100) & 10 & 1(10) & 10 & 5(1) & 10 & 10 \\ \(\gamma_{b}\) & 10(100) & 10(100) & 10(100) & 10(100) & 10 & 10 & 10 & 10(50) \\ \(\gamma_{q}\) & 1e-5 & 1e-5 & 1e-5 & 1e-5 & 1e-5 & 1e-5 & 1e-5 & 1e-5 \\ \(n_{r}\) & 4e4 & 4e4 & 4e4 & 6e4 & 4e4 & 4e4 & 4e4 & 6e4 \\ \(n_{b}\) & 4e3 & 4e3 & 6e3 & 3e4 & 4e3 & 4e3 & 6e3 & 3e4 \\ lr & 2e-3 & 1e-3 & 3e-3 & 3e-3 & 2e-3 & 3e-3 & 3e-3 & 3e-3 \\ dr & 0.7 & 0.75 & 0.7 & 0.75 & 0.7 & 0.8 & 0.8 & 0.75 \\ step & 2000 & 1500 & 2500 & 3000 & 2000 & 2500 & 3000 & 3000 \\ epoch & 6e4(3e4) & 5e4(2e4) & 6e4(3e4) & 6e4(2e4) & 6e4(2e4) & 8e4(2e4) & 6e4(3e4) & 6e4(2e4) \\ \hline \hline \end{tabular}
\end{table}
Table 1: The algorithmic parameters used for the examples.

Figure 1: The reconstructions for Example 5.1 with exact data \((\text{top})\) and noisy data \((\delta=10\%\), bottom\()\).
This choice of \(u^{\dagger}\) ensures that both \(\nabla u^{\dagger}\) and \(\Delta u^{\dagger}\) do not vanish over the domain \(\Omega\), which are beneficial for the numerical construction and is also important for theoretical analysis [4]. Fig. 1 shows the reconstructions for exact data (top) and noisy data (bottom), where the pointwise errors \(|\hat{q}-q^{\dagger}|\) are also shown. The shape and location of the modes and the overall structure of \(q^{\dagger}\) are well resolved, except that the middle part between the top two bumps were slightly bridged. The method is observed to be very robust with respect to data noise, except very slight underestimation of the peak values of the top two bumps, even in the presence of 10% noise. Surprisingly, the approach seems fairly stable with respect to the iteration index, cf. Fig. 2. Indeed, the presence of the noise does not affect much the convergence behavior of the loss and the error, and the final values of the error are close for all the noise levels. This observation agrees well with that for the pointwise error in Fig. 1.

There are several algorithmic parameters influencing the overall accuracy of the DNN approximation \(\hat{q}\), e.g., the number of training points (\(n_{r}\) and \(n_{b}\)), DNN architectural parameters (width, depth, and activation function) and noise level \(\delta\). A practical yet provable guidance for choosing these parameters is unfortunately still missing. Instead, we explore the issue empirically. Table 2(a) shows the relative \(L^{2}(\Omega)\)-error of the reconstruction \(\hat{q}\) at different noise levels and different \(\gamma_{q}\). The method is observed to be very robust with respect to the presence of data noise - the results remain fairly accurate even for up to 10% data noise, and do not vary much with the penalty parameter \(\gamma_{q}\) (for the \(H^{1}(\Omega)\)-penalty), provided that it is properly chosen. However, there is also an inherent accuracy limitation of the approach, i.e., the reconstruction cannot be made arbitrarily accurate for exact data \(\nabla u^{\dagger}\). This may be attributed to the optimization error: due to the nonconvexity of the loss landscape, the optimizer may fail to find a global minimizer but instead only an approximate local minimizer. The saturation phenomenon has been consistently observed across a broad range of solvers based on DNNs [16, 24, 34]. Tables 2(b)-2(c) show that the relative \(L^{2}(\Omega)\) error \(e(\hat{q})\) of the reconstruction \(\hat{q}\) does not vary much with different DNN architectures and numbers of training points. This observation agrees with the convergence behavior of the optimization algorithm in Fig. 2, where the value of the loss \(\hat{J}_{\mathbf{\gamma}}\) and the error \(e(\hat{q})\) eventually stagnates at a certain level.

The second example is about recovering a discontinuous conductivity \(q^{\dagger}\). The notation \(\chi_{S}\) denotes the characteristic function of the set \(S\).

**Example 5.2**.: \(\Omega=(-1,1)^{2}\)_, \(q^{\dagger}=1+0.25\cdot\chi_{\{(x_{1}+0.15)^{2}+(x_{2}+0.3)^{2}\leq 0.25^{2}\}}\), \(f\equiv 0\) and \(g=x_{1}\)._

Given the exact conductivity \(q^{\dagger}\), the field \(u^{\dagger}\) is computed by solving the Neumann problem (1.1) using the public software package FreeFEM++ [22], and the (exact) observation \(\nabla u^{\dagger}\) at the training points is evaluated by numerical

Figure 2: The variation of the loss (top) and the relative \(L^{2}(\Omega)\) error \(e\) (bottom) during the training process for Example 5.1 at different noise levels.
\begin{table}

\end{table}
Table 2: The variation of the relative \(L^{2}(\Omega)\) error \(e(\hat{q})\) with respect to various algorithmic parameters.

Figure 3: The reconstructions for Example 5.2 with exact data \((\text{top})\) and noisy data \((\delta=10\%\), bottom\()\).
interpolation. Since \(q^{\dagger}\) is piece-wise constant, we may also include the popular total variation penalty [36], i.e., \(\gamma_{tv}|q|_{\mathrm{TV}}\), to the loss \(J_{\gamma}(\theta,\kappa)\) to promote piecewise constancy of the reconstruction. Fig. 3 shows the error plots without the total variation term, and Fig. 4 including the total variation term (\(\gamma_{tv}=0.01\) for both exact and noisy data). The reconstruction quality is improved by including the total variation term. However, the reconstruction \(\hat{q}_{\theta}\) still exhibits slight blurring effect around the discontinuous interface. Moreover, it is worth noting that the reconstruction \(\hat{q}\) remains accurate for up to \(10\%\) data noise, indicating high robustness of the approach with respect to noise.

The third example is about recovering a conductivity coefficient in 3D.

**Example 5.3**.: \(\Omega=(0,1)^{3}\)_, \(q^{\dagger}=1+0.3e^{-20(x_{1}-0.5)^{2}-20(x_{2}-0.5)^{2}-20(x_{3}-0.5)^{2}}\), and \(u^{\dagger}=x_{1}+x_{2}+x_{3}+\frac{1}{3}(x_{1}^{3}+x_{2}^{3}+x_{3}^{3})\)._

Fig. 5 shows the reconstruction at a 2D cross section, i.e., the surface \(x_{3}=0.5\), using the exact data \(\nabla u^{\dagger}\) and noisy data (\(\delta=10\%\)). The shape and the overall structure of \(q^{\dagger}\) are well recovered in both cases. The relative \(L^{2}(\Omega)\)-error \(e(\hat{q})\) is 1.49e-2 and 3.42e-2, for exact and noisy data, respectively. The error in the noisy case is higher than that for exact data, but still quite acceptable. This again shows the high robustness of the approach with respect to noise.

The last example is about recovering a conductivity function in 5D.

**Example 5.4**.: \(\Omega=(0,1)^{5}\)_, \(q^{\dagger}=1-(x_{1}-0.5)^{2}-(x_{2}-0.5)^{2}+\cos(\pi(x_{3}+1.5))+\cos(\pi(x _{4}+1.5))+\cos(\pi(x_{5}+1.5))\), and \(u^{\dagger}=x_{1}+x_{2}+x_{3}+x_{4}+x_{5}+\frac{1}{3}(x_{1}^{3}+x_{2}^{3}+x_{3 }^{3}+x_{4}^{3}+x_{5}^{3})\)._

Fig. 6 shows the reconstruction on a 2D cross section of the domain \(\Omega\), i.e., the surface (\(x_{3}=x_{4}=x_{5}=0.5\)). The relative \(L^{2}(\Omega)\)-error \(e(\hat{q})\) for the exact and noisy data is 3.44e-3 and 2.27e-2, respectively. The DNN approximations do capture the overall structure of the exact conductivity \(q^{\dagger}\) for both exact and noisy data, showing again the remarkable robustness of the method with respect to the presence of noise. Moreover, it also shows the advantage of using DNNs: it can solve inverse problems for high-dimensional PDEs, which is not easily tractable for more traditional approaches, e.g., FEM. Surprisingly, the approach can produce high quality results without using too many training points in the domain \(\Omega\), despite the high-dimensionality of the problem.

### The Dirichlet problem

The first example is about recovering a smooth conductivity in 2D.

Figure 4: The reconstructions for Example 5.2 using a loss function including the total variation term, with exact data (top) and noisy data (\(\delta=10\%\), bottom).
Figure 5: The reconstructions for Example 5.3 with exact data \((\text{top})\) and noisy data \((\delta=10\%,\text{bottom})\).

Figure 6: The reconstructions for Example 5.4 with exact data \((\text{top})\) and noisy data \((\delta=10\%,\text{bottom})\).
**Example 5.5**.: \(\Omega=(-1,1)^{2}\)_, \(q^{\dagger}=1+s_{1}(x_{1},x_{2})+s_{2}(x_{1},x_{2})\) with \(s_{1}=0.4e^{-15(x_{1}-0.5)^{2}-15x_{2}^{2}}\) and \(s_{2}=-0.4e^{-15(x_{1}+0.5)^{2}-15x_{2}^{2}}\), and \(u^{\dagger}=x_{1}+x_{2}+\frac{1}{3}(x_{1}^{3}+x_{2}^{3})\)._

Fig. 7 shows the reconstruction \(\hat{q}\) using the loss (4.4) with exact data (top) and noisy data (bottom, \(\delta=10\%\)). The relative \(L^{2}(\Omega)\)-error of the reconstructions is 9.72e-3 and 3.78e-2 for exact and noisy data, respectively. Again we observe the robustness of the proposed approach in the presence of noise - there is almost no degradation in the reconstruction quality, except slight underestimation of the peak values of the two bumps and very mild oscillations near the boundary. The locations and shape of the bumps were well captured. The convergence behavior of the optimizer in Fig. 8 shows that the loss \(J_{\boldsymbol{\gamma}}\) and error \(e(\hat{q})\) stagnate at a comparable level regardless of the noise level. In light of Remark 4.1, Fig. 9 shows the results by the loss (4.6) with exact (top), \(1\%\) noise (middle) and \(\delta=10\%\) (bottom). The accuracy of reconstructions with exact data and data with \(1\%\) noise is satisfactory, but the neural network fails to learn accurately using the loss (4.6) when the data is very noisy.

The second example is concerned with recovering a nearly piecewise constant conductivity function in 2D.

**Example 5.6**.: \(\Omega=(0,1)^{2}\)_, \(q^{\dagger}=1+0.3/(1+\tau(x,y))\), with \(\tau(x,y)=e^{400((x-0.65)^{2}+2(y-0.7)^{2}-0.15^{2})}\), and \(u^{\dagger}=\sin(\pi x)\sin(\pi y)\)._

Note that in this example, \(\nabla u\) vanishes at the four corners of the domain and the point \((\frac{1}{2},\frac{1}{2})\). Similar to Example 5.2, an additional total variation penalty, i.e., \(\gamma_{te}|q|_{\mathrm{TV}}\) (\(\gamma_{te}=0.01\)) is added to the loss \(J_{\boldsymbol{\gamma}}(\theta,\kappa)\) in order to promote piecewise constancy of the reconstruction. Fig. 10 shows the reconstruction using the loss (4.4) with exact (top) and noisy (bottom, \(\delta=10\%\)) data. It is observed that the reconstruction is accurate for exact data, and in the presence of \(10\%\) data noise, the reconstruction quality deteriorates only near the points where \(\nabla u\) vanishes. This is expected since substituting \(\nabla u=0\) back into the formulation (4.4) leads to a loss essentially independent of \(q_{\theta}\), thus the inaccuracy in the reconstruction in a small neighbourhood around these points.

The third example is about recovering a conductivity in 3D.

**Example 5.7**.: \(\Omega=(0,1)^{3}\)_, \(q^{\dagger}=1+e^{-(12(x_{1}-0.5)(x_{2}-0.5))^{2}}+x_{3}\), and \(u^{\dagger}=x_{1}+x_{2}+x_{3}+\frac{1}{3}(x_{1}^{3}+x_{2}^{3}+x_{3}^{3})\)._

Fig. 11 show the reconstruction using the loss (4.4) on a 2D cross section at \(x_{3}=0.5\), with exact (top) and noisy (bottom, \(\delta=10\%\)) data, with the relative \(L^{2}(\Omega)\)-error being 3.28e-3 and 3.48e-2, respectively, indicating the excellent robustness of the approach with respect to data noise: there is only very mild deterioration in the reconstruction \(\hat{q}\).

The last example is concerned with recovering a conductivity function in high-dimension.

Figure 7: The reconstructions for Example 5.5 with exact data (top) and noisy data (\(\delta=10\%\), bottom).
**Example 5.8**.: \(\Omega=(0,1)^{5}\)_, \(q^{\dagger}=1+0.5(x_{1}x_{5}+x_{2}x_{4}+x_{3}^{2})-0.3e^{-25(x_{1}-0.5)^{2}-25(x_{2 }-0.5)^{2}}\), and \(u^{\dagger}=x_{1}+x_{2}+x_{3}+x_{4}+x_{5}+\frac{1}{3}(x_{1}^{3}+x_{2}^{3}+x_{3 }^{3}+x_{4}^{3}+x_{5}^{3})\)._

Fig. 12 shows the reconstructions using the loss (4.4) on a 2D cross section with \(x_{3}=x_{4}=x_{5}=0.5\), with exact (top) and noisy (bottom, \(\delta=10\%\)) data. The relative \(L^{2}(\Omega)\)-error is 5.78e-3 and 2.86e-2 for exact and noisy data, respectively. The features of the true conductivity have been successfully recovered and visually there is almost no difference between the reconstructions from exact and noisy data. This again showcases the significant potential of NN approximations compared to more traditional approaches for solving high-dimensional inverse problems.

## Appendix A The proof of Theorem 3.5

In this appendix, we prove Theorem 3.5, which gives high probability bounds on the statistical errors \(\Delta\mathcal{E}_{i}\), \(i\in\{d,b,b^{\prime},q,\sigma\}\). These bounds play a crucial role in deriving error estimates in Theorems 3.6 and 4.5. By slightly abusing the notation, let \(\mathcal{N}_{\sigma}\equiv\mathcal{N}(L_{\sigma},W_{\sigma},R_{\sigma})\) and \(\mathcal{N}_{q}=\mathcal{N}(L_{q},W_{q},R_{q})\) be two DNN function classes of given depth, width and parameter bound for approximating the current density \(\sigma\) and the conductivity \(q\), respectively. Then we define the following function classes

\[\mathcal{H}_{d} =\{h:\Omega\rightarrow\mathbb{R}|\ h(x)=\|\sigma_{\kappa}(x)-P_{ \mathcal{A}}(q_{\theta}(x))\nabla z^{\delta}(x)\|_{\ell^{2}}^{2},\sigma_{\kappa }\in\mathcal{N}_{\sigma},q_{\theta}\in\mathcal{N}_{q}\},\] \[\mathcal{H}_{\sigma} =\{h:\Omega\rightarrow\mathbb{R}|\ h(x)=|\nabla\cdot\sigma_{ \kappa}(x)+f(x)|^{2},\sigma_{\kappa}\in\mathcal{N}_{\sigma}\},\] \[\mathcal{H}_{b} =\{h:\partial\Omega\rightarrow\mathbb{R}|\ h(x)=|\mathbf{n}\cdot \sigma_{\kappa}(x)-g(x)|^{2},\sigma_{\kappa}\in\mathcal{N}_{\sigma}\},\] \[\mathcal{H}_{\psi} =\{h:\partial\Omega\rightarrow\mathbb{R}|\ h(x)=\|\sigma_{\kappa }(x)-q^{\dagger}\nabla z^{\delta}\|_{\ell^{2}}^{2},\sigma_{\kappa}\in \mathcal{N}_{\sigma}\},\] \[\mathcal{H}_{q} =\{h:\Omega\rightarrow\mathbb{R}|\ h(x)=\|\nabla q_{\theta}(x) \|_{\ell^{2}}^{2},q_{\theta}\in\mathcal{N}_{q}\}.\]

To bound these errors, we employ Rademacher complexity [9] and boundedness and Lipschitz continuity of DNN functions and their derivatives with respect to the DNN parameters. Rademacher complexity measures the complexity of a collection of functions by the correlation between function values with Rademacher random variables.

**Definition A.1**.: _Let \(\mathcal{F}\) be a real-valued function class defined on the domain \(\Omega\) (or the boundary \(\partial\Omega\)) and \(\xi=\{\xi_{j}\}_{j=1}^{n}\) be i.i.d. samples from the distribution \(\mathcal{U}(\Omega)\) (or the distribution \(\mathcal{U}(\partial\Omega)\)). Then the Rademacher complexity \(\mathfrak{R}_{n}(\mathcal{F})\) of

Figure 8: The variation of the loss (top) and relative \(L^{2}(\Omega)\) error \(e(\hat{q})\) (bottom) during the training process for Example 5.5 at three different noise levels.
Figure 9: The reconstructions for Example 5.5 with exact data \((\text{top})\) and noisy data \((\delta=1\%\), middle and \(\delta=10\%\), bottom).
Figure 11: The reconstructions for Example 5.7 with exact data \((\text{top})\) and noisy data \((\delta=10\%,\text{bottom})\).

Figure 10: The reconstructions for Example 5.6 with exact data \((\text{top})\) and noisy data \((\delta=10\%,\text{bottom})\).
the class \(\mathcal{F}\) is defined by_

\[\mathfrak{R}_{n}(\mathcal{F})=\mathbb{E}_{\xi,\omega}\bigg{[}\sup_{v\in\mathcal{F} }\ n^{-1}\bigg{|}\ \sum_{j=1}^{n}\omega_{j}v(\xi_{j})\ \bigg{|}\bigg{]},\]

_where \(\omega=\{\omega_{j}\}_{j=1}^{n}\) are i.i.d Rademacher random variables with probability \(P(\omega_{j}=1)=P(\omega_{j}=-1)=\frac{1}{2}\)._

We use the following PAC-type generalization bound [31, Theorem 3.1] via Rademacher complexity. Note that the statement in [31, Theorem 3.1] only discusses the case of the function class ranging in \([0,1]\). For a general bounded function, applying McDiarmid's inequality and the original argument yields the following result.

**Lemma A.1**.: _Let \(X_{1},\ldots,X_{n}\) be a set of i.i.d. random variables. Let \(\mathcal{F}\) be a function class defined on \(D\) such that \(\sup_{v\in\mathcal{F}}\|v\|_{L^{\infty}(D)}\leq M_{\mathcal{F}}<\infty\). Then for any \(\tau\in(0,1)\), with probability at least \(1-\tau\):_

\[\sup_{v\in\mathcal{F}}\bigg{|}n^{-1}\sum_{j=1}^{n}v(X_{j})-\mathbb{E}[v(X)] \bigg{|}\leq 2\mathfrak{R}_{n}(\mathcal{F})+2M_{\mathcal{F}}\sqrt{\frac{ \log\frac{1}{\tau}}{2n}}.\]

To apply Lemma A.1, we have to bound the Rademacher complexity of the function classes \(\mathcal{H}_{i}\), \(i\in\{d,\sigma,b,b^{\prime},q\}\). This is achieved by combining Lipschitz continuity of DNN functions (or its derivatives) with the target function class in the DNN parameter, and Dudley's formula in Lemma A.4. The next lemma gives useful boundedness and Lipschitz continuity of a \(\tanh\)-DNN function class with respect to the DNN parameters. Note that Lemma A.2 also holds with the \(L^{\infty}(\partial\Omega)\) norm in place of the \(L^{\infty}(\Omega)\) norm, since the overall argument depends only on the boundedness of the activation function \(\rho=\tanh\) and its derivative on \(\mathbb{R}\).

**Lemma A.2**.: _Let \(\Theta\) be a parametrization with depth \(L\) and width \(W\), and \(\theta=\{(A^{(\ell)},b^{(\ell)})_{\ell=1}^{L}\},\tilde{\theta}=\{(\tilde{A}^ {(\ell)},\tilde{b}^{(\ell)})_{\ell=1}^{L}\}\in\Theta\). Then for the DNN realizations \(v,\tilde{v}:\Omega\to\mathbb{R}\) of \(\theta,\tilde{\theta}\) with \(\|\theta\|_{\ell^{\infty}},\|\tilde{\theta}\|_{\ell^{\infty}}\leq R\), the following estimates hold_

* \(\|v\|_{L^{\infty}(\Omega)}\leq R(W+1),\quad\|\nabla v\|_{L^{\infty}(\Omega; \mathbb{R}^{d})}\leq\sqrt{d}R^{L}W^{L-1}\)_;_
* \(\|v-\tilde{v}\|_{L^{\infty}(\Omega)}\leq 2LR^{L-1}W^{L}\|\theta-\tilde{\theta}\|_{ \ell^{\infty}},\quad\|\nabla(v-\tilde{v})\|_{L^{\infty}(\Omega;\mathbb{R}^{d })}\leq\sqrt{d}L^{2}R^{2L-2}W^{2L-2}\|\theta-\tilde{\theta}\|_{\ell^{\infty}}\)_._

Proof.: All the estimates are already contained in [24]; see [24, p. 19 and p. 22 of Lemma 3.4] and [24, Remark 3.3].

Figure 12: The reconstructions for Example 5.8 with exact data \((\text{top})\) and noisy data \((\delta=10\%\), \(\text{bottom})\).
Next we appeal to reduction to parameterisation. This is achieved using the Lipschitz continuity of the functions in the function classes \(\mathcal{H}_{i}\) with respect to the DNN parameters, following from Lemma A.2.

**Lemma A.3**.: _Let \(c_{z}=\|\nabla z^{\delta}\|_{L^{\infty}(\Omega)}\) and \(c^{\prime}_{z}=\|\nabla z^{\delta}\|_{L^{\infty}(\Omega)}\). For the function classes \(\mathcal{H}_{i}\), \(i\in\{d,\sigma,b,b^{\prime},q\}\), the functions are uniformly bounded:_

\[\|h\|_{L^{\infty}(\Omega)}\leq\begin{cases}M_{d}=2(dR_{\sigma}^{2}(W_{\sigma} +1)^{2}+c_{1}^{2}c_{z}^{2}),&h\in\mathcal{H}_{d},\\ M_{\sigma}=2(d^{2}R_{\sigma}^{2L_{\sigma}}W_{\sigma}^{2L_{\sigma}-2}+\|f\|_{L^{ \infty}(\Omega)}^{2}),&h\in\mathcal{H}_{\sigma},\\ M_{q}=dR_{q}^{2L_{q}}W_{q}^{2L_{q}-2},&h\in\mathcal{H}_{q},\\ M_{b}=2(dR_{\sigma}^{2}(W_{\sigma}+1)^{2}+\|g\|_{L^{\infty}(\partial\Omega)}^{ 2}),&h\in\mathcal{H}_{b},\\ M_{b^{\prime}}=2(dR_{\sigma}^{2}(W_{\sigma}+1)^{2}+c_{1}^{2}(c^{\prime}_{z})^ {2}),&h\in\mathcal{H}_{b^{\prime}}.\end{cases}\]

_Moreover, the following Lipschitz continuity estimates in the DNN parameters hold_

\[\|h-\tilde{h}\|_{L^{\infty}(\Omega)}\leq\Lambda_{d}(\|\theta- \tilde{\theta}\|_{\ell^{\infty}}+\|\kappa-\tilde{\kappa}\|_{\ell^{\infty}}), \quad\forall h,\tilde{h}\in\mathcal{H}_{d},\] \[\|h-\tilde{h}\|_{L^{\infty}(\Omega)}\leq\Lambda_{\sigma}\|\kappa- \tilde{\kappa}\|_{\ell^{\infty}},\quad\forall h,\tilde{h}\in\mathcal{H}_{ \sigma},\] \[\|h-\tilde{h}\|_{L^{\infty}(\partial\Omega)}\leq\Lambda_{b}\| \kappa-\tilde{\kappa}\|_{\ell^{\infty}},\quad\forall h,\tilde{h}\in\mathcal{H} _{b},\] \[\|h-\tilde{h}\|_{L^{\infty}(\partial\Omega)}\leq\Lambda_{b^{\prime }}\|\kappa-\tilde{\kappa}\|_{\ell^{\infty}},\quad\forall h,\tilde{h}\in \mathcal{H}_{b^{\prime}},\] \[\|h_{\theta}-h_{\tilde{\varrho}}\|_{L^{\infty}(\Omega)}\leq \Lambda_{q}\|\theta-\tilde{\theta}\|_{\ell^{\infty}},\quad\forall h,\tilde{h} \in\mathcal{H}_{q},\]

_with the Lipschitz constants \(\Lambda_{i}\), \(i\in\{d,\sigma,b,b^{\prime},q\}\), given by_

\[\Lambda_{d}=2\big{(}\sqrt{d}R_{\sigma}(W_{\sigma}+1)+c_{1}c_{z} \big{)}\max(2\sqrt{d}L_{\sigma}R_{\sigma}^{L_{\sigma}-1}W_{\sigma}^{L_{\sigma} },c_{z}L_{q}R_{q}^{L_{q}-1}W_{q}^{L_{q}}),\] \[\Lambda_{\sigma}=2(dR_{\sigma}^{L_{\sigma}}W_{\sigma}^{L_{\sigma} -1}+\|f\|_{L^{\infty}(\Omega)})dL_{\sigma}^{2}R_{\sigma}^{2L_{\sigma}-2}W_{ \sigma}^{2L_{\sigma}-2},\] \[\Lambda_{b}=4(\sqrt{d}R_{\sigma}(W_{\sigma}+1)+\|g\|_{L^{\infty}( \partial\Omega)})\sqrt{d}L_{\sigma}R_{\sigma}^{L_{\sigma}-1}W_{\sigma}^{L_{ \sigma}},\] \[\Lambda_{b^{\prime}}=4\big{(}\sqrt{d}R_{\sigma}(W_{\sigma}+1)+c_ {1}c^{\prime}_{z}\big{)}\sqrt{d}L_{\sigma}R_{\sigma}^{L_{\sigma}-1}W_{\sigma}^{ L_{\sigma}},\] \[\Lambda_{q}=2dL_{q}^{2}R_{q}^{3L_{q}-2}W_{q}^{3L_{q}-3}.\]

Proof.: The assertions follow directly from Lemma A.2. Indeed, for \(h_{\theta,\kappa}\in\mathcal{H}_{d}\), we have

\[|h_{\theta,\kappa}(x)|\leq 2(\|\sigma_{\kappa}(x)\|_{\ell^{2}}^{2}+\|P_{ \mathcal{A}}(q_{\theta}(x))\nabla z^{\delta}(x)\|_{\ell^{2}}^{2})\leq 2(dR_{ \sigma}^{2}(W_{\sigma}+1)^{2}+c_{1}^{2}c_{z}^{2}).\]

For any \(h_{\theta,\kappa},h_{\tilde{\theta},\tilde{\kappa}}\in\mathcal{H}_{d}\), by completing the squares and Cauchy-Schwarz inequality, meanwhile noting the stability of \(P_{\mathcal{A}}\) in (3.2) we have

\[h_{\theta,\kappa}(x)-h_{\tilde{\theta},\tilde{\kappa}}(x)=\| \sigma_{\kappa}(x)-P_{\mathcal{A}}(q_{\theta}(x))\nabla z^{\delta}(x)\|_{\ell^{ 2}}^{2}-\|\sigma_{\tilde{\kappa}}(x)-P_{\mathcal{A}}(q_{\tilde{\theta}}(x)) \nabla z^{\delta}(x)\|_{\ell^{2}}^{2}\] \[= \big{(}\sigma_{\kappa}(x)-P_{\mathcal{A}}(q_{\theta}(x))\nabla z^{ \delta}(x)+\sigma_{\tilde{\kappa}}(x)-P_{\mathcal{A}}(q_{\tilde{\theta}}(x)) \nabla z^{\delta}(x),\] \[\quad\sigma_{\kappa}(x)-\sigma_{\tilde{\kappa}}(x)+(-P_{ \mathcal{A}}(q_{\theta}(x))+P_{\mathcal{A}}(q_{\tilde{\theta}}(x)))\nabla z^{ \delta}(x)\big{)}\] \[\leq 2\big{(}\sup_{\sigma_{\kappa}\in\mathcal{N}_{\kappa}}\|\sigma_{ \kappa}(x)\|_{L^{\infty}(\Omega;\mathbb{R}^{d})}+c_{1}c_{z}\big{)}\big{(}\| \sigma_{\kappa}(x)-\sigma_{\tilde{\kappa}}(x)\|_{L^{\infty}(\Omega;\mathbb{R}^{ d})}+c_{z}\|q_{\theta}(x)-q_{\tilde{\theta}}(x)\|_{L^{\infty}(\Omega)}\big{)}.\]

Then by Lemma A.2, we deduce

\[|h_{\theta,\kappa}(x)-h_{\tilde{\theta},\tilde{\kappa}}(x)|\] \[\leq 2\big{(}\sqrt{d}R_{\sigma}(W_{\sigma}+1)+c_{1}c_{z}\big{)}\times( 2\sqrt{d}L_{\sigma}R_{\sigma}^{L_{\sigma}-1}W_{\sigma}^{L_{\sigma}}\|\kappa- \tilde{\kappa}\|_{\ell^{\infty}}+c_{z}L_{q}R_{q}^{L_{q}-1}W_{q}^{L_{q}}\|\theta- \tilde{\theta}\|_{\ell^{\infty}})\] \[\leq 2\big{(}\sqrt{d}R_{\sigma}(W_{\sigma}+1)+c_{1}c_{z}\big{)}\max(2 \sqrt{d}L_{\sigma}R_{\sigma}^{L_{\sigma}-1}W_{\sigma}^{L_{\sigma}},c_{z}L_{q}R_{q}^{L _{q}-1}W_{q}^{L_{q}})(\|\kappa-\tilde{\kappa}\|_{\ell^{\infty}}+\|\theta- \tilde{\theta}\|_{\ell^{\infty}}).\]

The remaining estimates follow similarly. This completes the proof of the lemma. 

Next we bound the Rademacher complexities \(\mathcal{R}_{n}(\mathcal{H}_{i})\), \(i\in\{d,\sigma,b,b^{\prime},q\}\), using the concept of the covering number. Let \(\mathcal{G}\) be a real-valued function class equipped with the metric \(\rho\). An \(\epsilon\)-cover of the class \(\mathcal{G}\) with respect to the metric \(\rho\) is a collection of points \(\{x_{i}\}_{i=1}^{n}\subset\mathcal{G}\) such that for every \(x\in\mathcal{G}\), there exists at least one \(i\in\{1,\dots,n\}\) such that \(\rho(x,x_{i})\leq\epsilon\). The \(\epsilon\)-covering number \(\mathcal{C}(\mathcal{G},\rho,\epsilon)\) is the minimum cardinality among all \(\epsilon\)-cover of the class \(\mathcal{G}\) with respect to the metric \(\rho\). Then we can state the well-known Dudley's theorem [30, Theorem 9] and [42, Theorem 1.19].
**Lemma A.4**.: _Let \(M_{\mathcal{F}}:=\sup_{f\in\mathcal{F}}\|f\|_{L^{\infty}(\Omega)},\) and \(\mathcal{C}(\mathcal{F},\|\cdot\|_{L^{\infty}(\Omega)},\epsilon)\) be the covering number of the set \(\mathcal{F}\). Then the Rademacher complexity \(\mathfrak{R}_{n}(\mathcal{F})\) is bounded by_

\[\mathfrak{R}_{n}(\mathcal{F})\leq\inf_{0<s<M_{\mathcal{F}}}\bigg{(}4s\;+\;12n^ {-\frac{1}{2}}\int_{s}^{M_{\mathcal{F}}}\big{(}\log\mathcal{C}(\mathcal{F},\| \cdot\|_{L^{\infty}(\Omega)},\epsilon)\big{)}^{\frac{1}{2}}\;\mathrm{d} \epsilon\bigg{)}.\]

Now we can state the proof of Theorem 3.5.

Proof.: By the Lipschitz continuity of DNN functions with respect to the DNN parameters, the covering number of the corresponding function class can be bounded by that of the parametrization. For any \(n\in\mathbb{N}\), \(R\in[1,\infty)\), \(\epsilon\in(0,1)\), and \(B_{R}:=\{x\in\mathbb{R}^{n}:\;\|x\|_{\ell^{\infty}}\leq R\}\), then [13, Proposition 5]

\[\log\mathcal{C}(B_{R},\|\cdot\|_{\ell^{\infty}},\epsilon)\leq n\log(2R \epsilon^{-1}).\]

It follows directly from the Lipschitz continuity in Lemma A.2 that

\[\mathcal{C}(\mathcal{H}_{d},\|\cdot\|_{L^{\infty}(\Omega)},\epsilon)\leq \mathcal{C}(\Theta_{\sigma}\otimes\Theta_{q},\|\cdot\|_{\ell^{\infty}},\Lambda _{d}^{-1}\epsilon)\leq(N_{\kappa}+N_{\theta})\log(2\max(R_{q},R_{\sigma}) \Lambda_{d}\epsilon^{-1}),\]

with \(\Theta_{\sigma}\) and \(\Theta_{q}\) denoting the neural network parameter sets for \(\sigma\) and \(q\), respectively, \(\Lambda_{d}:=2\big{(}\sqrt{d}R_{\sigma}(W_{\sigma}+1)\,+\,c_{1}c_{z}\big{)} \max(2\sqrt{d}L_{\sigma}R_{\sigma}^{L_{\sigma}-1}W_{\sigma}^{L_{q}},c_{z}L_{q }R_{q}^{L_{q}-1}W_{q}^{L_{q}}\big{)}\), cf. Lemma A.3. By Lemma A.3, we also have \(M_{d}=2(dR_{\sigma}^{2}(W_{\sigma}+1)^{2}+c_{1}^{2}c_{z}^{2})\). Then letting \(s=n^{-\frac{1}{2}}\) in Lemma A.4 gives

\[\mathfrak{R}_{n}(\mathcal{H}_{d})\leq 4n^{-\frac{1}{2}}+12n^{- \frac{1}{2}}\int_{n^{-\frac{1}{2}}}^{M_{d}}\big{(}(N_{\kappa}+N_{\theta})\text {log}(2\max(R_{q},R_{\sigma})\Lambda_{d}\epsilon^{-1})\big{)}^{\frac{1}{2}} \,\mathrm{d}\epsilon\] \[\leq 4n^{-\frac{1}{2}}+12n^{-\frac{1}{2}}M_{d}\big{(}(N_{\kappa}+N _{\theta})\text{log}(2\max(R_{q},R_{\sigma})\Lambda_{d}n^{\frac{1}{2}})\big{)} ^{\frac{1}{2}}\] \[\leq 4n^{-\frac{1}{2}}+24n^{-\frac{1}{2}}(dR_{\sigma}^{2}(W_{ \sigma}+1)^{2}+c_{1}^{2}c_{z}^{2})(N_{\kappa}+N_{\theta})^{\frac{1}{2}}\big{(} \log\max(R_{q},R_{\sigma})+\log\Lambda_{d}+\log n+\log 2\big{)}^{\frac{1}{2}}.\]

Since \(1\leq R_{q},R_{\sigma}\), \(1\leq W_{q}\leq N_{\theta}\), \(1\leq W_{\sigma}\leq N_{\kappa}\) and \(2\leq L_{q},L_{\sigma}\leq c\log(d+2)\) (due to Lemma 2.1), we can bound the term \(\log\Lambda_{d}\) by

\[\log\Lambda_{d}\leq c(\log R_{\sigma}+\log N_{\kappa}+\log R_{q}+\log N_{ \theta}+\tilde{c}),\]

with the constants \(c\) and \(\tilde{c}\) depending on \(c_{1}\), \(c_{z}\), \(d\), \(L_{q}\) and \(L_{\sigma}\) at most polynomially. Hence, we have

\[\mathfrak{R}_{n}(\mathcal{H}_{d})\leq c_{d}n^{-\frac{1}{2}}R_{\sigma}^{2}N_{ \kappa}^{2}(N_{\kappa}+N_{\theta})^{\frac{1}{2}}(\log^{\frac{1}{2}}R_{\sigma}+ \log^{\frac{1}{2}}N_{\kappa}+\log^{\frac{1}{2}}R_{q}+\log^{\frac{1}{2}}N_{ \theta}+\log^{\frac{1}{2}}n),\]

where \(c_{d}>0\) depends on \(d\), \(c_{1}\) and \(c_{z}\) at most polynomially. Similarly, repeating the preceding argument leads to

\[\mathfrak{R}_{n}(\mathcal{H}_{\sigma}) \leq c_{\sigma}n^{-\frac{1}{2}}R_{\sigma}^{2L_{\sigma}}N_{\kappa}^{ 2L_{\sigma}-\frac{3}{2}}\big{(}\log^{\frac{1}{2}}R_{\sigma}+\log^{\frac{1}{2}} N_{\kappa}+\log^{\frac{1}{2}}n\big{)},\] \[\mathfrak{R}_{n}(\mathcal{H}_{b}) \leq c_{b}n^{-\frac{1}{2}}R_{\sigma}^{2}N_{\kappa}^{\frac{5}{2}} \big{(}\log^{\frac{1}{2}}R_{\sigma}+\log^{\frac{1}{2}}N_{\kappa}+\log^{\frac{1}{ 2}}n\big{)},\] \[\mathfrak{R}_{n}(\mathcal{H}_{b^{\prime}}) \leq c_{b^{\prime}}n^{-\frac{1}{2}}R_{\sigma}^{2}N_{\kappa}^{\frac{5 }{2}}\big{(}\log^{\frac{1}{2}}R_{\sigma}+\log^{\frac{1}{2}}N_{\kappa}+\log^{ \frac{1}{2}}n\big{)},\] \[\mathfrak{R}_{n}(\mathcal{H}_{q}) \leq c_{q}n^{-\frac{1}{2}}R_{q}^{2L_{q}}N_{\theta}^{2L_{q}-\frac{ 3}{2}}\big{(}\log^{\frac{1}{2}}R_{q}+\log^{\frac{1}{2}}N_{\theta}+\log^{ \frac{1}{2}}n\big{)},\]

where the involved constants depend on \(d\) at most polynomially. Last, the desired estimates follow from the PAC-type generalization bound in Lemma A.1. 
