# On the asymptotical regularization with convex constraints for inverse
problems

Min Zhong  School of Mathematics, Southeast University, Nanjing, Jiangsu
210096, P. R. China<EMAIL_ADDRESS>and  Wei Wang  Corresponding author.
College of Mathematics, Physics and Information Engineering, Jiaxing
University, Zhejiang 314001, P. R. China<EMAIL_ADDRESS>
###### Abstract.

In this paper, we consider the asymptotical regularization with convex
constraints for nonlinear ill-posed problems. The method allows to use non-
smooth penalty terms, including the L1-like and the total variation-like
penalty functionals, which are significant in reconstructing special features
of solutions such as sparsity and piecewise constancy. Under certain
conditions we give convergence properties of the methods. Moreover, we propose
Runge-Kutta type methods to discrete the initial value problems to construct
new type iterative regularization methods.

##  1\. Introduction

Consider the ill-posed operator equation

$F(x)=y\,,$ (1.1)

where $F:\mathcal{D}(F)\subset\mathcal{X}\rightarrow\mathcal{Y}$ is a
nonlinear operator, $\mathcal{X},\mathcal{Y}$ are infinite dimensional Hilbert
spaces. Instead of exact data $y$, in practice we are given only noisy data
$y^{\delta}$ satisfying

$\|y-y^{\delta}\|\leqslant\delta$ (1.2)

with known noise level $\delta$.

In order to find the solution of (1.1) with desired feature such as sparsity
and piecewise constancy, one may introduce non-smooth penalty functions, such
as the $L^{1}$ and total variation like functions [4, 5]. In recent years, the
relative iterative regularization has received tremendous attention, while
Landweber type iteration is one of the most fundamental methods [2, 16].

For a proper, lower semi-continuous, uniformly convex function as penalty term
$\Theta:\mathcal{X}\to(-\infty,+\infty)$ given the initial pairs
$(\xi_{0}^{\delta},x_{0}^{\delta})=(\xi_{0},\nabla\Theta^{*}(\xi_{0}))$, the
Landweber iteration with penalty $\Theta$ takes the form

$\displaystyle\xi_{n+1}^{\delta}$
$\displaystyle=\xi_{n}^{\delta}-\mu_{n}^{\delta}F^{\prime}(x_{n}^{\delta})^{*}(F(x_{n}^{\delta})-y^{\delta})\,,$
(1.3) $\displaystyle x_{n+1}^{\delta}$
$\displaystyle=\nabla\Theta^{*}(\xi_{n+1}^{\delta})\,,\quad n=0,1,2,\ldots$

where $F^{\prime}(\cdot)^{*}$ denotes the adjoint of $F^{\prime}$,
$\mu_{n}^{\delta}$ is the step-size and
$\Theta^{*}:\mathcal{X}\to(-\infty,\infty]$ is the Legendre-Fenchel conjugate
of $\Theta$. The advantage of this method is the freedom on the choice of
$\Theta$ so that the method can be utilized in detecting different features of
the sought solution. Extensive researches have been carried out to study above
Landweber-type iteration, its accelerated version and its modification. The
convergence analysis of the iterative methods has been given under discrepancy
principle or Heuristic choice rule [15, 19, 10, 29, 23, 28, 27].

In this paper, we investigate the methods of asymptotical regularization,
which can be considered as the continuous analogue of iteration (1.3). By
introducing an artificial time variable, a regularized approximation pair
$(\xi^{\delta}(T),x^{\delta}(T))$ is obtained by solving the initial value
problem

$\displaystyle\frac{d\xi^{\delta}(t)}{dt}$
$\displaystyle=-F^{\prime}(x^{\delta}(t))^{*}(F(x^{\delta}(t))-y^{\delta}),\quad
0<t\leqslant T,$ (1.4) $\displaystyle x^{\delta}(t)$
$\displaystyle=\nabla\Theta^{*}(\xi^{\delta}(t)),$

with initial value
$(\xi^{\delta}(0),x^{\delta}(0))=(\xi_{0},\nabla\Theta^{*}(\xi_{0}))$. Note
that, when take $\Theta(x)=\frac{1}{2}\|x\|^{2}$, then (1.4) became the
asymptotical regularization in [24]

$\displaystyle\frac{dx^{\delta}(t)}{dt}$
$\displaystyle=-F^{\prime}(x^{\delta}(t))^{*}(F(x^{\delta}(t))-y^{\delta}),\quad
0<t\leqslant T.$ (1.5)

with initial value $x^{\delta}(0)=x_{0}.$ In [24], Ulrich Tautenhahn give
convergence properties of method (1.5) and derive stability estimates. It also
has been show that $T$ plays the role of the regularization parameter and can
be chosen by a generalized discrepancy principle. For the autonomous ODE
(1.5), Runge-Kutta method can be used and the new type iterative
regularization methods can be constructed. The theory of the RK regularization
method has been well-developed for linear inverse problems [22, 26], and for
nonlinear inverse problems [17, 3], as some special cases, such as explicit
Euler method(Landweber iteration)[13], explicit 2-stage R-K Landweber method
[17], implicit Euler method (Levenberg-Marquardt iteration) [12].

Recently, there has been a great interest in analysing such dynamical flows.
In [26], variational source conditions have been established for general
spectral regularization methods which yield convergence rates. In [18], the
authors interpreted steady linear statistical inverse problems with white
noise, and introduced a stochastic differential equation system. In [1], the
authors applied the general theory of convergence rates to the three, well
studied examples, including Showalter’s method, heavy ball method and the
vanishing viscosity method. Besides general spectral analysis and source
condition, the conditional stability assumption was also a efficient tool to
analyze the convergence rate, which was fist proposed in [7], where the
convergence rate of the Tikhonov regularization in Hilbert spaces was proved,
then, the results have been extended to Banach spaces in [6]. In [11], the
authors analyzed a nonlinear Landweber iteration and proved its local
convergence rates under the Hölder continuity of the inverse mapping. The
latest related work was [27], the authors discussed the convergence rate of
Landweber-type iteration in Banach spaces.

In this manuscript, we will analyze the regularization property of
asymptotical regularization (1.4) through a proper choice of the terminating
time. The paper is organized as follows. In section 2, we give some
preliminaries from convex analysis. In section 3, we present the detailed
analysis of convergence and regularization properties of the asymptotical
regularization (1.4) including the convergence rate under conditional
stability assumption.

##  2\. Tools from convex analysis

Given a convex function $\Theta:\mathcal{X}\to(-\infty,\infty]$, we use
$\partial\Theta(x)$ to denote the subdifferential of $\Theta$ at
$x\in\mathcal{X}$, i.e.

$\partial\Theta(x):=\\{\xi\in\mathcal{X}:\Theta(\bar{x})-\Theta(x)-\langle\xi,\bar{x}-x\rangle\geqslant
0\mbox{ for all }\bar{x}\in\mathcal{X}\\}.$

Let $\mathcal{D}(\Theta):=\\{x\in\mathcal{X}:\Theta(x)<\infty\\}$ be its
effective domain and let

$\mathcal{D}(\partial\Theta):=\\{x\in\mathcal{D}(\Theta):\partial\Theta(x)\neq\varnothing\\}.$

The Bregman distance induced by $\Theta$ at $x$ in the direction
$\xi\in\partial\Theta(x)$ is defined by

$D_{\xi}\Theta(\bar{x},x):=\Theta(\bar{x})-\Theta(x)-\langle\xi,\bar{x}-x\rangle,\qquad\forall\bar{x}\in\mathcal{X}$

which is always nonnegative and satisfies the identity

$D_{\xi_{2}}\Theta(x,x_{2})-D_{\xi_{1}}\Theta(x,x_{1})=D_{\xi_{2}}\Theta(x_{1},x_{2})+\langle\xi_{2}-\xi_{1},x_{1}-x\rangle$
(2.1)

for all $x\in\mathcal{D}(\Theta),x_{1},x_{2}\in\mathcal{D}(\partial\Theta)$,
and $\xi_{1}\in\partial\Theta(x_{1})$, $\xi_{2}\in\partial\Theta(x_{2})$.

A proper convex function $\Theta:\mathcal{X}\to(-\infty,\infty]$ is called
uniformly convex if there exists a strictly increasing function
$h:[0,\infty)\rightarrow[0,\infty)$ with $h(0)=0$ such that

$\Theta(\lambda\bar{x}+(1-\lambda)x)+\lambda(1-\lambda)h(\|x-\bar{x}\|)\leqslant\lambda\Theta(\bar{x})+(1-\lambda)\Theta(x)$
(2.2)

for all $\bar{x},x\in\mathcal{X}$ and $\lambda\in[0,1]$. If $h(t)=c_{0}t^{p}$
for some $c_{0}>0$ and $p>1$ in (2.2), then $\Theta$ is called $p$-convex.

For a proper lower semi-continuous convex function
$\Theta:\mathcal{X}\to(-\infty,\infty]$, its Legendre-Fenchel conjugate is
defined by

$\Theta^{*}(\xi):=\sup_{x\in\mathcal{X}}\left\\{\langle\xi,x\rangle-\Theta(x)\right\\},\quad\xi\in\mathcal{X}$

which is also proper, lower semi-continuous, and convex. If $\mathcal{X}$ is
reflexive, then

$\xi\in\partial\Theta(x)\Longleftrightarrow
x\in\partial\Theta^{*}(\xi)\Longleftrightarrow\Theta(x)+\Theta^{*}(\xi)=\langle\xi,x\rangle.$
(2.3)

For detailed introduction of convex analysis, we refer to [25], here we list
some properties.

######  Proposition 2.1.

For a proper lower semi-continuous convex function $\Theta$ and $p>1$, the
following statements hold true:

  1. (i)

The $\Theta$ is $p$-convex if and only if

$D_{\xi}\Theta(\bar{x},x)\geqslant
c_{0}\|x-\bar{x}\|^{p},\quad\forall\bar{x}\in\mathcal{D}(\Theta),\
x\in\mathcal{D}(\partial\Theta),\ \xi\in\partial\Theta(x)\,.$ (2.4)

  2. (ii)

If $\Theta$ is $p$-convex with $p\geqslant 2$, then
$\mathcal{D}(\Theta^{*})=\mathcal{X}$, $\Theta^{*}$ is Fréchet differentiable
and its gradient $\nabla\Theta^{*}:\mathcal{X}\to\mathcal{X}$ satisfies

$\|\nabla\Theta^{*}(\xi_{1})-\nabla\Theta^{*}(\xi_{2})\|\leqslant\left(\frac{\|\xi_{1}-\xi_{2}\|}{2c_{0}}\right)^{\frac{1}{p-1}},\quad\forall\xi_{1},\xi_{2}\in\mathcal{X}.$

  3. (iii)

If $\mathcal{X}$ is reflexive and $\Theta$ is $p$-convex with $p\geqslant 2$,
then

$x=\nabla\Theta^{*}(\xi)\Longleftrightarrow\xi\in\partial\Theta(x)\Longleftrightarrow
x=\arg\min_{z\in\mathcal{X}}\left\\{\Theta(z)-\langle\xi,z\rangle\right\\}.$

  4. (iv)

If $\Theta$ is $p$-convex, then for any pairs $(x,\xi)$ and
$(\bar{x},\bar{\xi})$ with
$x,\bar{x}\in\mathcal{D}(\partial\Theta),\xi\in\partial\Theta(x),\bar{\xi}\in\partial\Theta(\bar{x})$,
then

$D_{\xi}\Theta(\bar{x},x)\leqslant\frac{1}{p^{*}(2c_{0})^{p*-1}}\|\xi-\bar{\xi}\|^{p*}\,.$
(2.5)

where $p^{*}=p/(p-1)$ be the conjugate exponent of $p$.

######  Assumption 2.2.

During this manuscript, we provide the following assumptions

  1. (a)

Let $\Theta:\mathcal{X}\to(-\infty,\infty]$ be a proper, weak lower semi-
continuous, and $p$-convex functional with $p>1$ such that the condition (2.4)
is satisfied for some $c_{0}>0$.

  2. (b)

There exists $\rho>0$, $x_{0}\in\mathcal{X}$ and
$\xi_{0}\in\partial\Theta(x_{0})$ such that
$B_{2\rho}(x_{0})\subset\mathcal{D}(F)$, where $B_{2\rho}(x_{0})$ denotes the
closed ball around $x_{0}$ with radius $2\rho$. In addition, assume (1.1) has
a solution $x^{*}\in\mathcal{D}(\Theta)$ with

$D_{\xi_{0}}\Theta(x^{*},x_{0})\leqslant c_{0}\rho^{p}\,.$

  3. (c)

The operator $F$ is continuous and weakly closed on $\mathcal{D}(F)$.

  4. (d)

There exists a family of bounded linear operators
$\\{L(x):\mathcal{X}\to\mathcal{Y}\\}_{x\in
B_{2\rho}(x_{0})\cap\mathcal{D}(\Theta)}$ such that $x\to L(x)$ is continuous
on $B_{2\rho}(x_{0})\cap\mathcal{D}(\Theta)$ and there is $0\leqslant\eta<1$
such that tangential cone condition

$\|F(x)-F(\bar{x})-L(\bar{x})(x-\bar{x})\|<\eta\|F(x)-F(\bar{x})\|$

holds true for all $x,\bar{x}\in B_{2\rho}(x_{0})\cap\mathcal{D}(\Theta)$.
Moreover, there is a constant $C_{0}>0$ such that

$\|L(x)\|_{\mathcal{X}\to\mathcal{Y}}\leqslant C_{0},\quad\forall x\in
B_{2\rho}(x_{0}).$

##  3\. Asymptotical regularization with convex penalty

In this paper, we investigate the method of asymptotical regularization, which
is the continuous analogue of (1.3). In this section a regularized
approximation $x^{\delta}(T)$ of $x^{*}$ is obtained by solving the coupling
initial value problem

$\displaystyle\frac{d\xi^{\delta}(t)}{dt}$
$\displaystyle=-L(x^{\delta}(t))^{*}(F(x^{\delta}(t))-y^{\delta}),\quad
0<t\leqslant T,$ (3.1) $\displaystyle x(t)$
$\displaystyle=\nabla\Theta^{*}(\xi^{\delta}(t)),$ $\displaystyle
x^{\delta}(0)$
$\displaystyle=x_{0},\quad\xi^{\delta}(0)=\partial\Theta(x_{0})\,.$

Here and subsequently, the $L(x)$ is a family of bounded linear operators from
$\mathcal{X}$ to $\mathcal{Y}$ satisfying Assumption 2.2 (d) and functional
$\Theta$ satisfying Assumption 2.2 (a). The $T$ plays the role of the
regularization parameter.

###  3.1. Convergence results

In this subsection, we study some convergence properties of (3.1), the basic
principle for proof have been developed in [24], where corresponding results
have been proved for classical Landweber asymptotical regularization in
Hilbert space.

######  Lemma 3.1.

Let Assumption 2.2 hold true. Let $(\xi^{\delta}(T),x^{\delta}(T))$ be the
solution of (3.1) for $T>0$. Then the approximated solution $x^{\delta}(T)$
and the residual $\|F(x^{\delta}(T))-y^{\delta}\|$ are continuous with respect
to $T$.

Proof. Referring to Proposition 2.1 (ii),

$\displaystyle\lim_{\Delta T\to 0}\left\|x^{\delta}(T+\Delta
T)-x^{\delta}(T)\right\|$ $\displaystyle=\lim_{\Delta T\to
0}\left\|\nabla\Theta^{*}(\xi^{\delta}(T+\Delta
T))-\nabla\Theta^{*}(\xi^{\delta}(T))\right\|$
$\displaystyle\leqslant\lim_{\Delta T\to 0}\left(\frac{\|\xi^{\delta}(T+\Delta
T)-\xi^{\delta}(T)\|}{2c_{0}}\right)^{\frac{1}{p-1}}$
$\displaystyle=\lim_{\Delta T\to 0}\left(\left\|\frac{\xi^{\delta}(T+\Delta
T)-\xi^{\delta}(T)}{\Delta T}\right\|\cdot\frac{\Delta
T}{2c_{0}}\right)^{\frac{1}{p-1}}\,.$

Since

$\lim_{\Delta T\to 0}\left\|\frac{\xi^{\delta}(T+\Delta
T)-\xi^{\delta}(T)}{\Delta T}\right\|=\frac{d\xi^{\delta}(t)}{dt}|_{t=T}$
(3.2)

exists and $\lim_{\Delta T\to 0}\frac{\Delta T}{2c_{0}}=0$. This implies
$x^{\delta}(T)$ is continuous. With the continuity of $F$ the residual is
continuous as well. ∎

######  Proposition 3.2.

Let Assumption 2.2 hold true and $(\xi^{\delta}(T),x^{\delta}(T))$ be the
solution of (3.1) for $T>0$. Let $\hat{x}$ be any solution of (1.1) satisfying
$D_{\xi_{0}}\Theta(\hat{x},x_{0})\leqslant c_{0}\rho^{p}$. Define

$\varphi(T):=D_{\xi^{\delta}(T)}\Theta(\hat{x},x^{\delta}(T)).$ (3.3)

If $x^{\delta}(T)\in B_{2\rho}(x_{0})$, then $\varphi(T)$ is differentiable
and

$\varphi^{\prime}(T)\leqslant-(1-\eta)\|F(x^{\delta}(T))-y\|^{2}+\delta(1+\eta)\|F(x^{\delta}(T))-y^{\delta}\|\,.$
(3.4)

In the case $\delta=0$,

$\int_{0}^{\infty}\|F(x(\tau))-y\|^{2}d\tau<\frac{1}{1-\eta}D_{\xi_{0}}\Theta(\hat{x},x_{0})\,,$
(3.5)

and

$\lim_{T\rightarrow\infty}\|F(x(T))-y\|=0\,.$ (3.6)

Proof. Referring to (2.1), we have

$\displaystyle\quad\lim_{\Delta T\to 0}\frac{\varphi(T+\Delta
T)-\varphi(T)}{\Delta T}$ $\displaystyle=\lim_{\Delta T\to
0}\frac{D_{\xi^{\delta}(T+\Delta T)}\Theta(\hat{x},x^{\delta}(T+\Delta
T))-D_{\xi^{\delta}(T)}\Theta(\hat{x},x^{\delta}(T))}{\Delta T}$
$\displaystyle=\lim_{\Delta T\to 0}\frac{D_{\xi^{\delta}(T+\Delta
T)}\Theta(x^{\delta}(T),x^{\delta}(T+\Delta T))+\langle\xi^{\delta}(T+\Delta
T)-\xi^{\delta}(T),x^{\delta}(T)-\hat{x}\rangle}{\Delta T}\,.$

The application of Proposition 2.1(iv) provides the estimate of the first term

$\displaystyle\quad\lim_{\Delta T\to 0}\frac{D_{\xi^{\delta}(T+\Delta
T)}\Theta(x^{\delta}(T),x^{\delta}(T+\Delta T))}{\Delta T}$
$\displaystyle=\lim_{\Delta T\to 0}\frac{(\Delta
T)^{p^{*}-1}}{p^{*}(2c_{0})^{p^{*}-1}}\left\|\frac{\xi^{\delta}(T+\Delta
T)-\xi^{\delta}(T)}{\Delta T}\right\|^{p^{*}}\,.$

Referring to (3.2) again, the above estimate converges to $0$. For the second
term

$\displaystyle\quad\lim_{\Delta T\to 0}\left\langle\frac{\xi^{\delta}(T+\Delta
T)-\xi^{\delta}(T)}{\Delta T},x^{\delta}(T)-\hat{x}\right\rangle$
$\displaystyle=\left\langle\frac{d\xi^{\delta}(T)}{dT},x^{\delta}(T)-\hat{x}\right\rangle$
$\displaystyle=-\left\langle
L(x^{\delta}(T))^{*}(F(x^{\delta}(T))-y^{\delta}),x^{\delta}(T)-\hat{x}\right\rangle$
$\displaystyle=-\left\langle(F(x^{\delta}(T))-y^{\delta}),L(x^{\delta}(T))(x^{\delta}(T)-\hat{x})\right\rangle$
$\displaystyle=-\|F(x^{\delta}(T))-y^{\delta}\|^{2}+\left\langle
F(x^{\delta}(T))-y^{\delta},F(x^{\delta}(T))-y^{\delta}-L(x^{\delta}(T))(x^{\delta}(T)-\hat{x})\right\rangle$
$\displaystyle=-\|F(x^{\delta}(T))-y^{\delta}\|^{2}+\delta\|F(x^{\delta}(T))-y^{\delta}\|+\langle
F(x^{\delta}(T))-y^{\delta},y-F(x^{\delta}(T))-L(x^{\delta}(T))(\hat{x}-x^{\delta}(T))\rangle\,.$

Since, assume $x^{\delta}(T)\in B_{2\rho}(x_{0})$, we have

$\|y-F(x^{\delta}(T))-L(x^{\delta}(T))(\hat{x}-x^{\delta}(T))\|\leqslant\eta\|y-F(x^{\delta}(T))\|.$
(3.7)

Putting it into above estimates, it is consequently that $\varphi(t)$ is
differentiable and

$\displaystyle\varphi^{\prime}(T)$ $\displaystyle=\lim_{\Delta T\to
0}\frac{\varphi(T+\Delta T)-\varphi(T)}{\Delta T}$
$\displaystyle\leqslant-\|F(x^{\delta}(T))-y^{\delta}\|^{2}+\delta\|F(x^{\delta}(T))-y^{\delta}\|+\eta\|F(x^{\delta}(T))-y^{\delta}\|\left(\delta+\|F(x^{\delta}(T))-y^{\delta}\|\right)$
$\displaystyle=-(1-\eta)\|F(x^{\delta}(T))-y^{\delta}\|^{2}+\delta(1+\eta)\|F(x^{\delta}(T))-y^{\delta}\|\,.$

In the case $\delta=0$, by integration from $0$ to $\infty$, we have

$(1-\eta)\int_{0}^{\infty}\|F(x^{\delta}(\tau))-y\|^{2}d\tau\leqslant-\int_{0}^{\infty}\varphi^{\prime}(\tau)d\tau$

which gives (3.5) and consequently (3.6). ∎

Proposition 3.2 shows that, the Bregman distance
$D_{\xi^{\delta}(T)}\Theta(\hat{x},x^{\delta}(T))$ as a function of $T$ is
monotonically decreasing, as far as
$\|F(x^{\delta}(T))-y^{\delta}\|>\tau\delta$ is holding with
$\tau=(1+\eta)/(1-\eta)$. Hence, it makes sense to choose the regularization
parameter by following discrepancy principle, i.e., $T=T^{*}$ is a solution of
the nonlinear equation

$h(T)=\|F(x^{\delta}(T))-y^{\delta}\|-\tau\delta=0\,\ \textrm{with }\
\tau>(1+\eta)/(1-\eta)\,.$ (3.8)

In the following proposition, we prove equation (3.8) has a solution.

######  Proposition 3.3.

Under the condition of Proposition 3.2, if
$\|F(x_{0})-y^{\delta}\|>\tau\delta$, then the nonlinear equation $h(T)=0$
exists at least one solution $T=T^{*}$.

Proof. Referring to Lemma 3.1, the function $h(T)$ is continuous, and
$h(0)=\|F(x_{0})-y^{\delta}\|-\tau\delta>0$. It is sufficient to show there
exists a $\bar{T}$, such that $h(\bar{T})\leqslant 0$. In contrast, we assume
$h(T)>0$ for all $T<\infty$, i.e.,

$\|F(x^{\delta}(T))-y^{\delta}\|>\tau\delta,\quad{\rm for~{}~{}all}\quad
T\geqslant 0.$

Combining this with Proposition 3.2, for all $T\geqslant 0$,

$\displaystyle\varphi^{\prime}(T)$
$\displaystyle\leqslant-(1-\eta)\|F(x^{\delta}(T))-y\|^{2}+\delta(1+\eta)\|F(x^{\delta}(T))-y^{\delta}\|$
$\displaystyle<-\left(1-\eta-\frac{1+\eta}{\tau}\right)\|F(x^{\delta}(t))-y^{\delta}\|^{2}\,.$

By integration, it follows that,

$\displaystyle\varphi(T)-\varphi(0)\leqslant-\left(1-\eta-\frac{1+\eta}{\tau}\right)\int_{0}^{T}\|F(x^{\delta}(\tau))-y^{\delta}\|^{2}d\tau\,,$
(3.9)

this implies

$\displaystyle\left(1-\eta-\frac{1+\eta}{\tau}\right)\int_{0}^{T}\|F(x^{\delta}(t))-y^{\delta}\|^{2}dt\leqslant\varphi(0).$
(3.10)

Thus,

$\left(1-\eta-\frac{1+\eta}{\tau}\right)T\tau^{2}\delta^{2}<D_{\xi_{0}}\Theta(\hat{x},x_{0})\leqslant
c_{0}\rho^{p}\,.$

This contradicts with the assumption when $T$ is large. ∎

######  Remark 3.4.

Since $\varphi^{\prime}(T)\leqslant 0$ for $T\leqslant T^{*}$, we obtain that

$D_{\xi^{\delta}(T)}\Theta(\hat{x},x^{\delta}(T))\leqslant
D_{\xi_{0}}\Theta(\hat{x},x_{0})\leqslant c_{0}\rho^{p}\,.$

Applying $p-$convexity of $\Theta$, it is consequently

$\|\hat{x}-x^{\delta}(T)\|\leqslant\rho\,.$

Then, referring to triangle inequality, we find the solution $x^{\delta}(T)$
remains in $B_{2\rho}(x_{0})\subset D(F)$. Furthermore, in the case
$\delta=0$, $x(T)$ remains in $B_{2\rho}(x_{0})$ for all $T<\infty$, this is
because $\frac{d}{dT}D_{\xi(T)}\Theta(\hat{x},x(T))<0$ for all $T<\infty$.

In the next, we prove that the solution $x(T)$ of (3.1) converges (for
$T\rightarrow\infty$) to a solution of (1.1).

######  Theorem 3.5.

Let Assumption 2.2 hold true and let $(\xi(T),x(T))$ be the solution of (3.1)
in the noisefree case for $T>0$. Then, there exists a solution $\bar{x}\in
B_{2\rho}(x_{0})\cap D(\Theta)$ of (1.1) such that

$\lim_{T\rightarrow\infty}\|x(T)-\bar{x}\|=0$

Proof. We define function $\theta(T)$ such that

$\|F(x(\theta(T)))-y\|=\min_{0\leqslant t\leqslant T}\|F(x(t))-y\|\,.$

It is obvious that the function $\theta(T)$ is continuous, monotonically
nondecreasing and can be bounded by $T$. Moreover $\theta(T)\rightarrow\infty$
as $T\rightarrow\infty$. First, we prove the sequence $x(\theta(T))$ converges
as $T\rightarrow\infty$. To this end, we will prove that for any
$\varepsilon>0$, there exists $T_{0}>0$, such that for arbitrary
$T_{2}>T_{1}>T$, there holds

$D_{\xi(\theta(T_{1}))}\Theta(x(\theta(T_{2})),x(\theta(T_{1})))<\varepsilon\,.$

Let $\hat{x}$ be any solution of (1.1) satisfying
$D_{\xi_{0}}\Theta(\hat{x},x_{0})\leqslant c_{0}\rho^{p}$, referring to (2.1),
we can rewrite the term
$D_{\xi(\theta(T_{1}))}\Theta(x(\theta(T_{2})),x(\theta(T_{1})))$ as

$D_{\xi(\theta(T_{2}))}\Theta(\hat{x},x(\theta(T_{2})))-D_{\theta(T_{1})}\Theta(\hat{x},\theta(T_{1}))+\langle\xi(\theta(T_{1}))-\xi(\theta(T_{2})),x(\theta(T_{2}))-\hat{x}\rangle\,.$
(3.11)

Since $\\{D_{\xi(T)}\Theta(\hat{x},x(T))\\}$ is strictly decreasing for $T>0$,
there exists a constant $a\geqslant 0$ and $\overline{T}>0$, such that for all
$T>\overline{T}$, the first and second terms can be bounded as

$a-\frac{\varepsilon}{3}<D_{\xi(\theta(T))}\Theta(\hat{x},x(\theta(T)))<a+\frac{\varepsilon}{3}\,.$

For the third term of (3.11), the fundamental theorem of calculus provides

$\displaystyle\left|\langle\xi(\theta(T_{1}))-\xi(\theta(T_{2})),x(\theta(T_{2}))-\hat{x}\rangle\right|$
$\displaystyle\leqslant\left|\left\langle\int_{\theta(T_{1})}^{\theta(T_{2})}L(x(s))^{*}(y-F(x(s)))ds,x(\theta(T_{2}))-\hat{x}\right\rangle\right|$
(3.12)
$\displaystyle\leqslant\int_{\theta(T_{1})}^{\theta(T_{2})}\|y-F(x(s))\|\|L(x(s))(x(\theta(T_{2}))-\hat{x})\|ds\,.$

Applying Assumption (2.2) (c), and the definition of the function $\theta(T)$,
it is easy to obtain

$\displaystyle\|L(x(s))(x(\theta(T_{2}))-\hat{x})\|$
$\displaystyle=\|L(x(s))(x(\theta(T_{2}))-x(s))\|+\|L(x(s))(x(s)-\hat{x})\|$
(3.13) $\displaystyle\leqslant 3(1+\eta)\|y-F(x(s))\|\,.$

Therefore, combining the property $\lim_{T\to\infty}\|F(x(T))-y\|=0$, there
exists a constant (still denote as) $\overline{T}$, for
$T_{2}>T_{1}>\overline{T}$,

$\displaystyle\left|\langle\xi(\theta(T_{1}))-\xi(\theta(T_{2})),x(\theta(T_{2}))-\hat{x}\rangle\right|\leqslant
3(1+\eta)\int_{\theta(T_{1})}^{\theta(T_{2})}\|y-F(x(s))\|^{2}ds<\frac{\varepsilon}{3}\,,$
(3.14)

and consequently,

$D_{\xi(\theta(T_{1}))}\Theta(x(\theta(T_{2})),x(\theta(T_{1})))<(a+\frac{\varepsilon}{3})-(a-\frac{\varepsilon}{3})+\frac{\varepsilon}{3}=\varepsilon\,.$

The application of the $p$-convexity of $\Theta$ implies
$\lim_{T\to\infty}x(\theta(T))$ exists, we denote it as $\bar{x}$.

Similarly, we consider

$\displaystyle D_{\xi(\theta(T))}\Theta(x(\theta(T)),x(T))$ (3.15)
$\displaystyle=D_{\xi(\theta(T))}\Theta(\hat{x},x(\theta(T)))-D_{\xi(T)}\Theta(\hat{x},x(T))+\langle\xi(T)-\xi(\theta(T)),x(\theta(T))-\hat{x}\rangle$

Since $\theta(T)\leqslant T$ and
$\|F(x(\theta(T)))-y\|\leqslant\|F(x(T))-y\|$, we have

$\displaystyle\left|\langle\xi(T)-\xi(\theta(T)),x(\theta(T))-\hat{x}\rangle\right|$
$\displaystyle\leqslant\left|\left\langle\int_{\theta(T)}^{T}L(x(s))^{*}(F(x(s))-y)ds,x(\theta(T))-\hat{x}\right\rangle\right|$
(3.16)
$\displaystyle\leqslant\int_{\theta(T)}^{T}\|F(x(s))-y\|\|L(x(s))(x(\theta(T))-\hat{x})\|ds$
$\displaystyle\leqslant 3(1+\eta)\int_{\theta(T)}^{T}\|y-F(x(s))\|^{2}ds.$

The same technique provides

$\lim_{T\to\infty}\left|\langle\xi(\theta(T))-\xi(T),x(\theta(T))-\hat{x}\rangle\right|=0$

and consequently,

$\lim_{T\to\infty}D_{\xi(\theta(T))}\Theta(x(\theta(T)),x(T))\to 0$

and $\lim_{T\to\infty}\|x(T)-x(\theta(T))\|=0$. The application of triangle
inequality gives

$\|x(T)-\bar{x}\|\leqslant\|x(\theta(T))-\bar{x}\|+\|x(T)-x(\theta(T))\|,$

therefore,

$\lim_{T\to\infty}x(T)=\bar{x}.$

By the weakly compactness of $F$ and continuity of $x(T)$, we can conclude
$\bar{x}$ is a solution to (1.1) and $\bar{x}\in B_{2\rho}(x_{0})\bigcap
D(\Theta)$. ∎

Finally, we perform the analysis from the Hölder continuity of the inverse
mapping instead of the source and non-linearity conditions and obtain the
convergence rates.

######  Assumption 3.6.

Assume the Hölder type conditional stability, i.e., denote the
$\nu\in[\frac{p}{2},p]$, there exists a constant $R_{F}$ such that

$D_{\bar{\xi}}\Theta(x,\bar{x})\leqslant
R_{F}\|F(x)-F(\bar{x})\|^{\nu},\quad\forall x,\bar{x}\in
B_{2\rho}(x_{0}),\bar{x}\in\mathcal{D}(\partial\Theta),\bar{\xi}\in\partial\Theta(\bar{x})\,.$

######  Theorem 3.7.

Let Assumption 2.2, in which the tangential cone condition is replaced by
Lipschitz continuity of $F^{\prime}$

$\|F^{\prime}(x)-F^{\prime}(\bar{x})\|\leqslant L^{\prime}\|x-\bar{x}\|,\quad
x,\bar{x}\in B_{2\rho}(x_{0}).$ (3.17)

and Assumption 3.6 hold true. Assume $\|F(x_{0})-y^{\delta}\|>\tau\delta$, let
$(\xi^{\delta}(T),x^{\delta}(T))$ be the solution of (3.1) with
$L(x)=F^{\prime}(x),$ where $T=T^{*}$ is the solution of (3.8). Then, there is
a solution $\bar{x}$ of (1.1) in $B_{2\rho}(x_{0})$, there holds

$D_{\xi^{\delta}(T^{*})}^{\delta}\Theta(\bar{x},x^{\delta}(T^{*}))\to
0,\quad\mbox{as }\delta\rightarrow 0\,,$

and the convergence rate is given by

$D_{\xi^{\delta}(T^{*})}^{\delta}\Theta(\bar{x},x^{\delta}(T^{*}))\leqslant
R_{F}(\tau+1)^{\nu}\delta^{\nu}\,.$

Proof. Assume $x^{\delta}(T)\in B_{2\rho}(x_{0})$ for $T<T^{*}$, we have

$\displaystyle\|y-F(x^{\delta}(T))-F^{\prime}(x^{\delta}(T))(\hat{x}-x^{\delta}(T))\|$
$\displaystyle\leqslant\frac{L}{2}\|x^{\delta}(T)-\hat{x}\|^{2}$
$\displaystyle\leqslant\frac{L}{2}\|x^{\delta}(T)-\hat{x}\|^{2-\frac{p}{\nu}}\|x^{\delta}(T)-\hat{x}\|^{\frac{p}{\nu}}$
$\displaystyle\leqslant\frac{L}{2}(4\rho)^{2-\frac{p}{\nu}}\left(\frac{1}{c_{0}}D_{\xi^{\delta}(T_{n})}\Theta(\hat{x},x^{\delta}(T_{n}))\right)^{\frac{1}{\nu}}$
$\displaystyle\leqslant\frac{L}{2}(4\rho)^{2-\frac{p}{\nu}}\left(\frac{R_{F}}{c_{0}}\right)^{\frac{1}{\nu}}\|y-F(x^{\delta}(T_{n}))\|\,.$

Denote

$\rho=\frac{1}{4}\left(\frac{2\eta}{L}\right)^{\frac{\nu}{2\nu-p}}\left(\frac{c_{0}}{R_{F}}\right)^{\frac{1}{2\nu-p}}$
(3.18)

then Assumption 3.6 with $p/2\leqslant\nu\leqslant p$ and (3.17) imply the
general tangential cone condition. The conclusion is directly obtained by
utilizing the conditional stability in Assumption 3.6,

$\displaystyle
D_{\xi^{\delta}(T^{*})}^{\delta}\Theta(\bar{x},x^{\delta}(T^{*}))$
$\displaystyle\leqslant R_{F}\|y-F(x^{\delta}(T^{*}))\|^{\nu}$ (3.19)
$\displaystyle\leqslant
R_{F}\left(\delta+\|y^{\delta}-F(x^{\delta}(T^{*}))\|\right)^{\nu}$
$\displaystyle\leqslant R_{F}(\tau+1)^{\nu}\delta^{\nu}\,.$

∎

######  Remark 3.8.

Note that one obtains for finite $T<\infty$, the existence of a unique
solution $\xi(t)\in C(0,T;\mathcal{X})$ of (3.1) if the operator
$G(\xi)=F^{\prime}(\nabla\Theta^{*}(\xi))^{*}[y^{\delta}-F(\nabla\Theta^{*}(\xi))]$
is locally Lipschitz continuous in $\mathcal{X}$ [8]. Actually, for
$(\xi_{1},x_{1})$ and $(\xi_{2},x_{2})$ with
$x_{i}=\nabla\Theta^{*}(\xi_{i})$, $i=1,2$, and $x_{1},x_{2}\in
B_{2\rho}(x_{0})$, we have

$\displaystyle G(\xi_{1})-G(\xi_{2})$
$\displaystyle=F^{\prime}(x_{1})^{*}(y^{\delta}-F(x_{1}))-F^{\prime}(x_{2})^{*}(y^{\delta}-F(x_{2}))$
$\displaystyle=\left(F^{\prime}(x_{1})^{*}-F^{\prime}(x_{2})^{*}\right)\left(y^{\delta}-F(x_{1})\right)+F^{\prime}(x_{2})^{*}\left(F(x_{2})-F(x_{1})\right)dt,$

Applying the Lipschitz continuity of $F^{\prime}$ (3.17) and the bounded
property of $F^{\prime}$, then we have

$\displaystyle\|G(\xi_{1})-G(\xi_{2})\|$
$\displaystyle\leqslant\|\left(F^{\prime}(x_{1})^{*}-F^{\prime}(x_{2})^{*}\right)\left(y^{\delta}-F(x_{1})\right)\|\|F^{\prime}(x_{2})^{*}\left(F(x_{2})-F(x_{1})\right)\|$
$\displaystyle\leqslant
L^{\prime}\|y^{\delta}-F(x_{1})\|\|x_{1}-x_{2}\|+C_{0}^{2}\|x_{1}-x_{2}\|$

Take the $\Theta$ as strongly convex (p=2) then

$\|x_{1}-x_{2}\|=\|\nabla\Theta^{*}(\xi_{1})-\nabla\Theta^{*}(\xi_{2})\|\leqslant\frac{\|\xi_{1}-\xi_{2}\|}{2c_{0}}.\quad\forall\xi_{1},\xi_{2}\in\mathcal{X}.$

Combining above estimates, we obtain that $G(\xi)$ is locally Lipschitz
continuous in $\mathcal{X}$, which implies that the existence of a unique
solution $\xi(t)\in C(0,T;\mathcal{X})$ of (3.1).

##  4\. Discretization of asymptotical method

In this section we recall the Runge-Kutta (RK) type iterative methods which
are regarded as the discretization of asymptotical method.

RK methods are a class of one-step methods for the numerical solutions of the
initial value problems for ODEs,

$\frac{dw(t)}{dt}=\Psi(t,w(t)),\quad t>0,\quad w(t)=w_{0}\,,$

with given $\Psi:[0,\infty]\times X\to X$ and $w_{0}\in X$.

A RK method of $s$-stages provides approximations $w_{n}$ to $w(t_{n})$ with
$t_{n}=\sum_{k=1}^{n}\triangle t_{k}$ such that

$\displaystyle w_{n+1}=w_{n}+\triangle
t_{n}\sum_{i=1}^{s}b_{i}\Psi(t+c_{i}\triangle t_{n},k_{i}),$ (4.1)
$\displaystyle k_{i}=w_{n}+\triangle
t_{n}\sum_{j=1}^{s}a_{i,j}\Psi(t+c_{j}\triangle t_{n},k_{j}).$

The given coefficients $A=(a_{ij})\in\mathbb{R}^{s\times s}$,
$b=(b_{1},b_{2},\cdots,b_{s})^{T}$ and $c=(c_{1},c_{2},\cdots,c_{s})^{T}$
determine the particular method. The method is called explicit if $A$ is
strictly lower triangular matrix. Otherwise, the method is called implicit,
since linear or nonlinear algebraic equations have to be solved to calculate
$k_{i},i=1,\cdots,s$. Usually, the specific instance of RK methods can be
presented by Butcher tableau, see Table 1 or by triple $(A,b,c)$ in [8]. We
only consider consistent $s-$stage RK method, i.e., $\sum_{i=1}^{s}b_{i}=1$.
However, an inconsistent $s-$stage RK method can be considered in the same
manner.

Table 1. The general Butcher tableau of RK method.

c | A
---|---
| $b^{T}$

$c_{1}$ | $a_{11}$ | $\cdots$ | $a_{1s}$
---|---|---|---
$\vdots$ | $\vdots$ | $\vdots$ | $\vdots$
$c_{s}$ | $a_{s1}$ | $\cdots$ | $a_{ss}$
| $b_{1}$ | $\cdots$ | $b_{s}$

A RK method applied to autonomous ODE (3.1), that is,

$\Psi(\xi^{\delta}(t))=L(x^{\delta}(t))^{*}(y^{\delta}-F(x^{\delta}(t)))\,,\quad
x^{\delta}(t)=\nabla\Theta^{*}(\xi^{\delta}(t))\,.$

Then, it is consequently

$\displaystyle\xi_{n+1}^{\delta}=\xi_{n}^{\delta}+\triangle
t_{n}\sum_{i=1}^{s}b_{i}L(\nabla\Theta^{*}(k_{i}))^{*}(y^{\delta}-F(\nabla\Theta^{*}(k_{i})))\,,$
(4.2) $\displaystyle k_{i}=\xi_{n}^{\delta}+\triangle
t_{n}\sum_{j=1}^{s}a_{ij}L(\nabla\Theta^{*}(k_{j}))^{*}(y^{\delta}-F(\nabla\Theta^{*}(k_{j})))\,.$

Use the explicit Euler method, i.e., $s=1$, $c=0$, $b=1$, $A=0$, it follow
that

$\displaystyle\xi_{n+1}^{\delta}=\xi_{n}^{\delta}+\triangle
t_{n}L(x_{n}^{\delta})^{*}(y^{\delta}-F(x_{n}^{\delta})),$ (4.3)
$\displaystyle x_{n+1}^{\delta}=\nabla\Theta(\xi_{n+1}^{\delta}).$

It is a Landweber-type iteration. Use the implicit Euler method with $s=1$,
$c=1$, $b=1$ and $A=1$, we have the implicit Landweber-type iteration

$\displaystyle\xi_{n+1}^{\delta}$ $\displaystyle=\xi_{n}^{\delta}+\triangle
t_{n}L(x_{n+1}^{\delta})^{*}(F(x_{n+1}^{\delta})-y^{\delta}),$ (4.4)
$\displaystyle x_{n+1}^{\delta}$
$\displaystyle=\nabla\Theta^{*}(\xi_{n+1}^{\delta}).$

For $s=2$, the explicit $2-$stage RK methods provides

$\displaystyle k_{2}=\xi^{\delta}_{n}+\triangle
t_{n}a_{21}L(x_{n}^{\delta})^{*}(y^{\delta}-F(x_{n}^{\delta})),$ (4.5)
$\displaystyle z_{n}^{\delta}=\nabla\Theta^{*}(k_{2}),$
$\displaystyle\xi_{n+1}^{\delta}=\xi_{n}+\triangle
t_{n}\left(b_{1}(y^{\delta}-F(x_{n}^{\delta}))+b_{2}L(z_{n}^{\delta})^{*}(y^{\delta}-F(z_{n}^{\delta}))\right),$
$\displaystyle x_{n+1}^{\delta}=\nabla\Theta^{*}(\xi_{n+1}^{\delta})\,,$

in which the coefficients should satisfy

$b_{1}+b_{2}=1\,,\quad a_{21}b_{2}=\frac{1}{2}\,.$ (4.6)

## Acknowledgements

The work of M Zhong is supported by the National Natural Science Foundation of
China (No. 11871149) and supported by Zhishan Youth Scholar Program of SEU.
The work of W Wang is supported by the National Natural Science Foundation of
China (No. 12071184).

## References

  * [1] R. Bo̧t, G. Dong, P. Elbau and O. Scherzer, Convergence rates of first-and higher-order dynamics for solving linear ill-posed problems. Foundations of Computational Mathematics. 2021 https://doi.org/10.1007/s10208-021-09536-6.
  * [2] R. Boţ and T. Hein, Iterative regularization with a geeral penalty term: theory and applications to $L^{1}$ and TV regularization, Inverse Problems 28 (2012) 104010.
  * [3] C. Bckmann, P. Pornsawad, Iterative Runge-Kutta-type methods for nonlinear ill-posed problems. Inverse Problems 24(2) (2008) 025002.
  * [4] A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imag. Sci. 2 (2009) 183–202.
  * [5] A. Beck and M. Teboulle, Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems, IEEE Trans. Image Process. 18 (2009) 2419–2434.
  * [6] J. Cheng, B. Hofmann and S. Lu, The index function and Tikhonov regularization for ill-posed problems, J. Comput. Appl. Math. 265 (2014) 110–119.
  * [7] J. Cheng and Y. Yamamoto, One new strategy for a priori choice of regularizing parameters in Tikhonov’s regularization, Inverse Problems 16 (2008) L31-L38.
  * [8] P. Deuflhard and F. Bornemann, Scientific Computing with Ordinary Differential Equations, New York: Springer, 2002.
  * [9] H. Engl, M. Hanke, A. Neubauer, Regularization of Inverse Problems, New York (NY): Springer, 1996.
  * [10] R. Gu, B. Han and Y. Chen, Fast subspace optimization method for nonlinear inverse problems in Banach spaces with uniformly convex penalty terms, Inverse Problems 35 (2019) 125011.
  * [11] M. V. de Hoop, L. Qiu, O. Scherzer, Local analysis of inverse problems: Hölder stability and iterative reconstruction, Inverse Problems 28 (2012) 045001.
  * [12] M. Hanke, A regularizing Levenberg Marquardt scheme with applications to inverse groundwater filtration problems, Inverse Problems 13 (1997) 79–95.
  * [13] M. Hanke, A. Neubauer and O. Scherzer, A convergence analysis of the Landweber iteration for nonlinear ill-posed problems, Numer. Math. 72 (1995) 21–37.
  * [14] Q. Jin, Inexact Newton-Landweber iteration in Banach spaces with nonsmooth convex penalty terms, SIAM J. Numer. Anal. 53 (5) (2015) 2389–2413.
  * [15] Q. Jin, Landweber-Kaczmarz method in Banach spaces with inexact inner solvers, Inverse Problems 32 (2016) 104005.
  * [16] Q. Jin, and W. Wang, Landweber iteration of Kaczmarz type with general non-smooth convex penalty functionals, Inverse Problems 29 (2013) 085011.
  * [17] L. Li, B. Han and W. Wang, R-K type Landweber method for nonlinear ill-posed problems, J. Comput. Appl. Math. 206 (2007) 341–357.
  * [18] S. Lu, P. Niu and F. Werner, On the asymptotical regularization for linear inverse problems in presence of white noise, SIAM/ASA Journal on Uncertainty Quantification 9(1) (2021) 1–28.
  * [19] P. Maaß and R. Strehlow, An iterative regularization method for nonlinear problems based on Bregman projections, Inverse Problems 32 (2016) 115013.
  * [20] A. Rieder, On the regularization of nonlinear ill-posed problems via inexact Newton iterations, Inverse Problems, 15 (1999) 309-327.
  * [21] L. Rudin, S. Osher and C. Fatemi, Nonlinear total variation based noise removal algorithm, Phys. D 60 (1992) 259-268.
  * [22] A. Rieder, Runge-Kutta integrators yield optimal regularization schemes, Inverse Problems 21 (2005) 453–471.
  * [23] R. Real and Q. Jin, A revisit on Landweber iteration, Inverse Problems 36 (2020) 075011.
  * [24] U. Tautenhahn, On the asymptotical regularization of nonlinear ill-posed problems, Inverse Problems 10 (1994) 1405-1418.
  * [25] C. Z$\breve{a}$linscu, Convex Analysis in General Vector Spaces World Scientific Publishing Co., Inc., River Edge. 2002.
  * [26] Y. Zhao, P. Mathé, S. Lu, Convergence analysis of asymptotical regularization and Runge-Kutta integrators for linear inverse problems under variational source conditions, CSIAM Transactions on Applied Mathematics 1(4) (2020) 693–714.
  * [27] M. Zhong, L. Qiu, W. Wang, Landweber-type method with uniformly convex constraints under conditional stability assumptions, preprint.
  * [28] M. Zhong, W. Wang, The two-point gradient methods for nonlinear inverse problems based on Bregman projections, Inverse Problems 36 (2020) 045012.
  * [29] M. Zhong, W. Wang and Q. Jin, Regularization of inverse problems by two-point gradient methods in Banach spaces, Numer. Math. 143 (2019) 713–747.