# Existence and Uniqueness of Weak solution of $\displaystyle p(x)$\-
laplacian in Sobolev spaces with variable exponents in complete manifolds

O. Benslimane1, A. Aberqi2 and J. Bennouna1 O. Benslimane, A. Aberqi and J.
Bennouna
1 Sidi Mohamed Ben Abdellah University, Faculty of Sciences Dhar Al Mahraz,
Department of Mathematics, B.P 1796 Atlas Fez, Morocco.
2 Sidi Mohamed Ben Abdellah University, National School of Applied Sciences
Fez, Morocco<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Key words and phrases:

Non-trivial solution; Lebesgue space with variable exponent; Sobolev spaces
Riemannian manifolds; Mountain pass Theorem.

###### 2010 Mathematics Subject Classification:

35J47, 35J60.

Abstract. The paper deals with the existence and uniqueness of a non-trivial
solution to non-homogeneous $\displaystyle p(x)-$laplacian equations, managed
by non polynomial growth operator in the framework of variable exponent
Sobolev spaces on Riemannian manifolds. The mountain pass Theorem is used.

##  1\. Introduction

Let $\displaystyle(M,g)$ be a complete non-compact Riemannian manifold, we
consider the following equation

$-\Delta_{p(x)}u(x)+h(x,u(x),\nabla u(x))+|\,u(x)\,|^{p(x)-2}u(x)=f(x,u(x)),$
(1.1)

where $\displaystyle-\Delta_{p(x)}u(x)=-\,div\,(|\,\nabla
u(x)\,|^{p(x)-2}\,.\,\nabla u(x))$ is the $\displaystyle p(x)$-laplacian in
$\displaystyle(M,g)$. The conditions assumed on the functions $\displaystyle
f$ and $\displaystyle h$ are:
$\displaystyle(f_{1}):\,f(x,0)=0$ and $\displaystyle f$ is measurable to the
first variable and continuous to the second variable.
$\displaystyle(f_{2}):$ There exists $\displaystyle\beta\in[\,p^{-},p^{+}]$
such as

$\displaystyle
0<\int_{M}F(x,\alpha)\,\,dv_{g}(x)\leq\int_{M}f(x,\alpha)\,.\,\frac{\alpha}{\beta}\,\,dv_{g}(x)\,\,\mbox{a.e}\,\,x\in
M,$

where $\displaystyle\displaystyle F(x,\alpha)=\int_{0}^{\alpha}f(x,t)\,\,dt$
being the primitive of $\displaystyle f(x,\alpha)$.
$\displaystyle(f_{3}):\,\displaystyle\lim_{|\,\alpha\,|\rightarrow\infty}\frac{f(x,\alpha)}{|\,\alpha\,|^{p(x)-1}}=0$
uniformly a.e $\displaystyle x\in M$.

######  Example 1.

$\displaystyle f(x,\alpha)=c\,|\,\alpha\,|^{K(x)-1}\,\,\forall
c>0,\,\,\beta\,<\,K(x)\,<p(x)$

is a function satisfying the above conditions.

$\displaystyle(h_{1}):\,h(x,s,\xi):M\times\mathbb{R}\times\mathbb{R}^{N}\longrightarrow\mathbb{R}^{N}$
be a Carathéodory function such as for a.e $\displaystyle x\in M$ and for all
$\displaystyle s\in\mathbb{R}$, $\displaystyle\xi\in\mathbb{R}^{N}$,

$\displaystyle|\,h(x,s,\xi)\,|\leq\gamma(x)+l(s)\,.\,|\,\xi\,|^{p(x)},$

where $\displaystyle l:\,\mathbb{R}\longrightarrow\mathbb{R}^{+}$ is a
continuous increasing positive function that belongs to $\displaystyle
L^{\infty}(M)$ and $\displaystyle\gamma(x)\in L^{1}(M)$.

The study of variational problems with non-standard growth conditions is an
interesting topic in recent years. $\displaystyle p(x)-$growth condition can
be regarded as an important case of non-standard $\displaystyle(p,q)-$growth
conditions. Recently, the study of these kind of problems has attracted more
and more attention. For example Fan Xian-Ling and Zhang Qi-Hu, in [9], proved
the existence of solutions to the Dirichlet problem of $\displaystyle
p(x)-$laplacian

$\displaystyle\begin{cases}\,-\mbox{div}(\,|\,\nabla u\,|^{p(x)-2}\,\nabla
u\,)\,=\,f(x,u)&\text{$\displaystyle x\in\Omega$ },\\\\[8.5359pt]
\,u\,=\,0&\text{$\displaystyle x\in\partial\Omega$ },\end{cases}$

with several sufficient conditions, and a criterion of existence for an
infinite number of pairs of solutions to this problem. For more result we
refer the reader to [5, 16]. The typical applications of variable exponent
equations include models for electrorheological fluids [3], image restoration
processing [7], non-Newtonian fluid dynamics [10], Poisson equation [8],
elasticity equations [13, 18], and thermistor model [19].
Moving on to another field undergoing great development; the Sobolev space on
Riemannian manifolds. The theory of Sobolev space for non compact manifold
arose in the 1970s with the work of Aubin, Cantor Hoffman, and Spruck, many of
the results presented in their lecture notes have been collected between the
1980s and the 1990s. It has been studied very intensively for over fifty years
see [4, 14, 11] and also e.g [12, 15].This is also the case for the
applications already mentioned to scalar curvature and generalized scalar
curvature equation, we quote [6]. Additionally, Yamabe problem for conformal
metrics with prescribed scalar curvature [17], and to obtain isoperimetric
type inequalities [14].

Considering that some basic properties of the standard Lebesgue space are not
valid in the variable exponent case. For example, Zhikov [19] observed that in
general smooth functions are not dense in $\displaystyle W^{K,p(.)}(\Omega)$.
Besides, the challenges coming are due to the absence of topological
properties like convergence and embedding.

In this paper, we will be applying the Sobolev spaces with their variable
exponents on the non-compact Riemannian manifolds theory to our equation
(1.1). As for the structure of the paper, we will be sectioning it into:
Recalling some definitions and Lemmas. We will then move to prove the
existence of non-trivial solution using the mountain pass Theorem, and we will
finish it by a demonstration of the uniqueness of non-trivial solution with
$\displaystyle f$ as the Contraction Lipschitz Continuous function.

##  2\. Framework Space: Notations and Basic Properties

First of all, we must recall the most important and pertinent properties and
notations, by that, referring to [4, 11, 16] for more details.

######  Definition 2.1.

Let $\displaystyle\nabla$ be the Levi-Civita connection. If $\displaystyle u$
is a smooth function on $\displaystyle M$, then $\displaystyle\nabla^{k}u$
denotes the $\displaystyle k-$th covariant derivative of $\displaystyle u$,
and $\displaystyle|\,\nabla^{k}u\,|$ the norm of $\displaystyle\nabla^{k}u$
defined in local coordinates by

$\displaystyle|\,\nabla^{k}u\,|^{2}=g^{i_{1}j_{1}}\cdots
g^{i_{k}j_{k}}\,(\nabla^{k}u)_{i_{1}\cdots i_{k}}\,(\nabla^{k}u)_{j_{1}\cdots
j_{k}}$

where Einstein’s convention is used.
$\displaystyle\hskip 28.45274pt\bullet$ Given a variable exponent
$\displaystyle p$ in $\displaystyle\mathcal{P}(M)$ and a natural number
$\displaystyle k$, introduce

$\displaystyle C^{p(.)}_{k}(M)=\\{\,u\in C^{\infty}(M)\,\,\mbox{such that
}\,\,\forall j\,\,0\leq j\leq k\,\,|\,\nabla^{k}u\,|\in L^{p(.)}\,\\}$

on $\displaystyle C^{p(.)}_{k}(M)$ define the norm

$\displaystyle||\,u\,||_{L^{p(.)}_{k}}=\sum_{j=0}^{k}||\,\nabla^{j}u\,||_{L^{p(.)}}$

######  Definition 2.2.

Given $\displaystyle(M,g)$ a smooth Riemannian manifold, and
$\displaystyle\gamma:\,[\,a,\,b\,]\longrightarrow M$ a curve of class
$\displaystyle C^{1}$, the length of $\displaystyle\gamma$ is

$\displaystyle
l(\gamma)=\int_{a}^{b}\sqrt{g\,(\,\frac{d\gamma}{dt},\,\frac{d\gamma}{dt}\,)}\,\,dt,$

and for a pair of points $\displaystyle x,\,y\in M$, we define the distance
$\displaystyle d_{g}(x,y)$ between $\displaystyle x$ and $\displaystyle y$ by

$\displaystyle
d_{g}(x,y)=\inf\,\\{\,l(\gamma):\,\gamma:\,[\,a,\,b\,]\rightarrow
M\,\,\mbox{such that}\,\,\gamma(a)=x\,\,\mbox{and}\,\,\gamma(b)=y\,\\}$

######  Definition 2.3.

A function $\displaystyle s:\,M\longrightarrow\mathbb{R}$ is log-Hölder
continuous if there exists a constant $\displaystyle c$ such that for every
pair of points $\displaystyle\\{x,\,y\\}$ in $\displaystyle M$ we have

$\displaystyle|\,s(x)-s(y)\,|\leq\frac{c}{log(e+\frac{1}{d_{g}(x,y)}\,)}.$

We note by $\displaystyle\mathcal{P}^{log}(M)$ the set of log-Hölder
continuous variable exponents.

######  Proposition 2.1.

Let $\displaystyle p\in\mathcal{P}^{log}(M)$, and let
$\displaystyle(\Omega,\phi)$ be a chart such that

$\displaystyle\frac{1}{2}\delta_{ij}\leq g_{ij}\leq 2\,\delta_{ij}$

as bilinear forms, where $\displaystyle\delta_{ij}$ is the delta Kronecker
symbol. Then $\displaystyle po\phi^{-1}\in\mathcal{P}^{log}(\phi(\Omega)).$

######  Definition 2.4.

We say that the n-manifold $\displaystyle(M,g)$ has property $\displaystyle
B_{vol}(\lambda,v)$ if its geometry is bounded in the following sense:
$\displaystyle\hskip 28.45274pt\bullet\,\,Rc(g)\geq\lambda(n-1)\,g$ for some
$\displaystyle\lambda$
$\displaystyle\hskip 28.45274pt\bullet$ There exists some $\displaystyle v>0$
such that $\displaystyle|\,B_{1}(x)\,|_{g}\geq v\,\,\forall x\in M.$

######  Proposition 2.2.

Let $\displaystyle(M,g)$ be a complete Riemannian n-manifold. Then, if the
embedding $\displaystyle L^{1}_{1}(M)\hookrightarrow L^{\frac{n}{n-1}}(M)$
holds, then whenever the real numbers $\displaystyle p$ and $\displaystyle q$
satisfy

$\displaystyle 1\leq p<n,$

and

$\displaystyle p\leq q\leq p*=\frac{np}{n-p},$

the embedding $\displaystyle L^{p}_{1}(M)\hookrightarrow L^{q}(M)$ also holds.

######  Proposition 2.3.

Assume that the complete n-manifold $\displaystyle(M,g)$ has property
$\displaystyle B_{vol}(\lambda,v)$ for some $\displaystyle(\lambda,v).$ Then
there exist positive constants
$\displaystyle\delta_{0}=\delta_{0}(n,\,\lambda,\,v)$ and $\displaystyle
A=A(n,\,\lambda,\,v)$, we have, if $\displaystyle R\leq\delta_{0}$, if
$\displaystyle x\in M$ if $\displaystyle 1\leq p\leq n$, and if $\displaystyle
u\in L^{p}_{1,0}(\,B_{R}(x)\,)$ the estimate

$\displaystyle||\,u\,||_{L^{q}}\leq A_{q}\,||\,\nabla u\,||_{L^{p}},$

where $\displaystyle\frac{1}{q}=\frac{1}{p}-\frac{1}{n}.$

######  Proposition 2.4.

Assume that for some $\displaystyle(\lambda,v)$ the complete n-manifold
$\displaystyle(M,g)$ has property $\displaystyle B_{vol}(\lambda,v)$. Let
$\displaystyle q\in\mathcal{P}(M)$ be uniformly continuous with $\displaystyle
p^{+}<n.$ Then $\displaystyle L^{p(.)}_{1}(M)\hookrightarrow
L^{q(.)}(M)\,\,\forall p\in\mathcal{P}(M)$ such that $\displaystyle p\ll q\ll
p*=\frac{np}{n-p}.$ In fact, for $\displaystyle||\,u\,||_{L^{p(.)}_{1}}$
sufficiently small we have the estimate

$\displaystyle\varrho_{q(.)}(u)\leq
G\,(\,\varrho_{p(.)}(u)+\varrho_{p(.)}(|\,\nabla u\,|)\,),$

where $\displaystyle G$ is a positive constant depend on $\displaystyle
n,\,\lambda,\,v,\,p$ and $\displaystyle q$.

##  3\. Existence of Non-trivial Solution

######  Definition 3.1.

$\displaystyle u\in L^{p(.)}_{1}(M)$ is said to be a weak solution of the
problem (1.1) if for every $\displaystyle\phi\in D(M)$ we have

$\displaystyle\displaystyle\int_{M}$ $\displaystyle\displaystyle|\,\nabla
u(x)\,|^{p(x)-2}g(\,\nabla
u(x),\nabla\phi(x))\,\,dv_{g}(x)+\int_{M}h(x,\,u(x),\,\nabla
u(x))\,.\,\phi(x)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}|\,u(x)\,|^{p(x)-2}u(x)\,\phi(x)\,\,dv_{g}(x)=\int_{M}f(x,u(x))\,\phi(x)\,\,dv_{g}(x)$
(3.1)

######  Theorem 3.1.

Suppose that $\displaystyle(f_{1})-(f_{3})$ and $\displaystyle(h_{1})$ are
true. Then the problem (1.1) possesses a non-trivial weak solution.

To access our main premises, the ones shown in the first Theorem, we have to
initially demonstrate few lemmas related to the mountain pass Theorem and
Palais-Smale condition. Considering the functional

$\displaystyle\displaystyle A(u)=$
$\displaystyle\displaystyle\int_{M}\frac{1}{p(x)}(\,|\,\nabla
u(x)\,|^{p(x)}+|\,u(x)\,|^{p(x)}\,)\,\,dv_{g}(x)+\int_{M}H(x,\,u(x),\,\nabla
u(x))\,\,dv_{g}(x)$
$\displaystyle\displaystyle-\int_{M}F(x,\,u(x))\,\,dv_{g}(x),$

with $\displaystyle\displaystyle H(x,\,u(x),\,\nabla
u(x))=\int_{0}^{\alpha}h(x,\,t,\,\nabla t)\,\,dt$ being the primitive of
$\displaystyle h(x,\,\alpha,\,\nabla\alpha)$. And $\displaystyle H(x,0,0)=0$.
It’s follow from (3.1) and the hypothesis of
$\displaystyle(f_{1})-(f_{3}),\,(h_{1})$ and the above definition of
$\displaystyle H$, that $\displaystyle A$ is $\displaystyle C^{1}$ functional.
Let $\displaystyle DA=DB-DC$ the differential of $\displaystyle A=B-C$ with

$\displaystyle\displaystyle DB=$
$\displaystyle\displaystyle\int_{M}\,|\,\nabla u(x)\,|^{p(x)-2}g(\,\nabla
u(x),\nabla\phi(x))\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}h(x,\,u(x),\,\nabla
u(x))\,.\,\phi(x)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}|\,u(x)\,|^{p(x)-2}u(x)\,\phi(x)\,\,dv_{g}(x),$

and

$\displaystyle DC=\int_{M}f(x,u(x))\,\phi(x)\,\,dv_{g}(x).$

######  Lemma 3.1.

The functional $\displaystyle A$ satisfies mountain pass geometry in the sense
that:
$\displaystyle i/\,\,A(0)=0.$
$\displaystyle ii/\,$ There exists $\displaystyle r,\eta>0$ such that
$\displaystyle A(u)\geq\eta$ if $\displaystyle||\,u\,||>r.$
$\displaystyle iii/\,$ There exists $\displaystyle u,\,||\,u\,||>r$ such that
$\displaystyle A(u)\leq 0.$

###### Proof.

$\displaystyle i/\,\,A(0)=0$ is obvious.
$\displaystyle ii/\,$ we need the assumptions $\displaystyle(f_{2}),\,(f_{3})$
and $\displaystyle(h_{1})$, then we obtain

$\displaystyle\displaystyle\int_{M}F(x,u(x))\,\,dv_{g}(x)$
$\displaystyle\displaystyle\leq\frac{\epsilon}{\beta}\,\int_{M}|\,u(x)\,|^{p(x)}\,\,dv_{g}(x)+\frac{m(\epsilon)}{\beta}$
$\displaystyle\displaystyle\leq\big{(}\,\frac{\epsilon+m(\epsilon)}{\beta}\,\big{)}\,\varrho_{p(.)}(u),$

and

$\displaystyle\displaystyle\bigg{|}\,\int_{M}H(x,\,u(x),\,\nabla
u(x))\,\,dv_{g}(x)\,\bigg{|}$
$\displaystyle\displaystyle\leq\int_{M}\bigg{(}\,\int_{0}^{u}|\,h(x,\,t,\,\nabla
t)\,|\,\,dt\,\bigg{)}\,\,dv_{g}(x)$ $\displaystyle\displaystyle\leq
c_{1}\,\varrho_{p(.)}(u)+c_{2}+l(u)\,\varrho_{p(.)}(\,|\,\nabla u\,|\,).$

Choose $\displaystyle||\,u\,||=r$ sufficiently small, so that
$\displaystyle\varrho_{p(.)}(u)\leq||\,u\,||_{p(x)}^{p}$ since
$\displaystyle||\,u\,||=r<1.$ Now using the Poincaré inequality, we get

$\displaystyle\displaystyle A(u)=$
$\displaystyle\displaystyle\int_{M}\frac{1}{p(x)}(\,|\,\nabla
u(x)\,|^{p(x)}+|\,u(x)\,|^{p(x)}\,)\,\,dv_{g}(x)+\int_{M}H(x,\,u(x),\,\nabla
u(x))\,\,dv_{g}(x)$ $\displaystyle\displaystyle\hskip
5.69046pt-\int_{M}F(x,\,u(x))\,\,dv_{g}(x)$
$\displaystyle\displaystyle\geq\frac{1}{p^{+}}\,(\,\varrho_{p(.)}(\,|\,\nabla
u\,|\,)+\varrho_{p(.)}(u)\,)-c_{1}\,\varrho_{p(.)}(u)-c_{2}-l(u)\,.\,\varrho_{p(.)}(\,|\,\nabla
u\,|)$ $\displaystyle\displaystyle\hskip
5.69046pt-\big{(}\,\frac{\epsilon+m(\epsilon)}{\beta}\,\big{)}\,\varrho_{p(.)}(u)$
$\displaystyle\displaystyle\geq\bigg{(}\,\frac{1}{p^{+}}-c_{1}-\frac{\epsilon+m(\epsilon)}{\beta}\,\bigg{)}\,\varrho_{p(.)}(u)+\bigg{(}\,\frac{1}{p^{+}}-l(u)\,\bigg{)}\,\varrho_{p(.)}\,(\,|\,\nabla
u\,|\,)$ $\displaystyle\displaystyle\geq
K_{1}\,\varrho_{p(.)}(u)+k_{2}\,\varrho_{p(.)}(\,|\,\nabla u\,|\,)$
$\displaystyle\displaystyle\geq\inf(K_{1},\,K_{2}\,)\,(\,\varrho_{p(.)}\,(u)+\varrho_{p(.)}\,(\,|\,\nabla
u\,|\,),$

with

$\displaystyle
K_{1}=\frac{1}{p^{+}}-c_{1}-\frac{\epsilon+m(\epsilon)}{\beta},\,\,\,\,K_{2}=\frac{1}{p^{+}}-l(u),$

and

$\displaystyle\varrho_{p(.)}(\,|\,\nabla u\,|\,)+\varrho_{p(.)}(u)\geq
2^{1-p^{+}}\,||\,u\,||_{L^{q(.)}(M)}^{q^{+}}.$

Hence,

$\displaystyle
A(u)\geq\inf(K_{1},\,K_{2})\,2^{1-p^{+}}\,||\,u\,||_{L^{q(.)}(M)}^{q^{+}}.$

So,

$\displaystyle A(u)\geq\eta\,\,\mbox{for some}\,\,\eta>0.$

We can prove $\displaystyle iii/$ by using $\displaystyle(f_{2})$ and
$\displaystyle(h_{1}),$ for $\displaystyle t>0$ and $\displaystyle u\neq 0.$
So,

$\displaystyle\displaystyle A(tu)$
$\displaystyle\displaystyle=\int_{M}\frac{1}{p(x)}\,(\,|\,\nabla
tu\,|^{p(x)}-|\,tu\,|^{p(x)}\,)\,\,dv_{g}(x)+\int_{M}H(x,\,tu,\,\nabla
tu)\,\,dv_{g}(x)$ $\displaystyle\displaystyle\hskip
5.69046pt-\int_{M}F(x,tu)\,\,dv_{g}(x)$
$\displaystyle\displaystyle=\int_{M}\frac{t^{p(x)}}{p(x)}\,(\,|\,\nabla
tu\,|^{p(x)}-|\,tu\,|^{p(x)}\,)\,\,dv_{g}(x)+\int_{M}H(x,\,tu,\,\nabla
tu)\,\,dv_{g}(x)$ $\displaystyle\displaystyle\hskip
5.69046pt-\int_{M}F(x,tu)\,\,dv_{g}(x)$
$\displaystyle\displaystyle\leq\int_{M}\frac{t^{p(x)}}{p(x)}\,(\,|\,\nabla
tu\,|^{p(x)}-|\,tu\,|^{p(x)}\,)\,\,dv_{g}(x)+c_{1}\,t^{p(x)}\,\int_{M}|\,tu\,|^{p(x)}\,\,dv_{g}(x)+c_{2}$
$\displaystyle\displaystyle\hskip
5.69046pt+l(tu)\,.\,t^{p(x)}\,\int_{M}|\,\nabla tu\,|^{p(x)}\,\,dv_{g}(x).$

This implies:

$\displaystyle\displaystyle A(tu)$
$\displaystyle\displaystyle\leq\int_{M}\frac{t^{p^{+}-p^{-}}}{p(x)}\,(\,|\,\nabla
u\,|^{p(x)}-|\,u\,|^{p(x)}\,)\,\,dv_{g}(x)+c_{1}\,.\,t^{p^{-}}\,\int_{M}|\,u\,|^{p(x)}\,\,dv_{g}(x)+c_{2}$
$\displaystyle\displaystyle\hskip
5.69046pt+l(tu)\,.\,t^{p^{+}}\,\int_{M}|\,\nabla u\,|^{p(x)}\,\,dv_{g}(x),$
(3.2)

dividing (3) by $\displaystyle t^{p^{+}}$ and passing the limit $\displaystyle
t\longrightarrow\infty$ we get $\displaystyle A(tu)\longrightarrow-\infty$,
since $\displaystyle p^{+}>p^{-}$.
Hence, $\displaystyle A(u)$ satisfies the hypothesis of mountain pass Theorem.
∎

######  Lemma 3.2.

The functional $\displaystyle A$ satisfies Palais-Smale condition.

###### Proof.

Let $\displaystyle\\{\,u_{n}\,\\}$ be a Palais-Smale sequence, such as the
associated sequence of real numbers $\displaystyle\\{\,A(u_{n})\,\\}$ is
bounded, and $\displaystyle DA(u_{n})\longrightarrow 0$ in
$\displaystyle(\,L^{p(.)}_{1}(M)\,)^{\prime}$. We will first demonstrate that
$\displaystyle(\,u_{n}\,)$ is bounded in $\displaystyle L^{p(.)}_{1}(M).$ And
prove it throughout contradiction.
Let $\displaystyle||\,u_{n}\,||_{L^{p(.)}_{1}(M)}\longrightarrow\infty$ as
$\displaystyle n\longrightarrow\infty$. Then, we have

$\displaystyle\displaystyle A(u_{n})-\frac{1}{\beta}<\,DA(u_{n}),\,u_{n}\,>\,$
$\displaystyle\displaystyle=\big{(}\,\frac{1}{p(x)}-\frac{1}{\beta}\,\big{)}\,\int_{M}(\,|\,\nabla
u_{n}\,|^{p(x)}-|\,u_{n}\,|^{p(x)}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle\hskip 5.69046pt+\int_{M}\big{(}\,H(x,u_{n},\nabla
u_{n})-h(x,u_{n},\nabla u_{n})\,.\,\frac{u_{n}}{\beta}\,\big{)}\,\,dv_{g}(x)$
$\displaystyle\displaystyle\hskip
5.69046pt+\int_{M}\big{(}\,f(x,u_{n})\,.\,\frac{u_{n}}{\beta}-F(x,u_{n})\,\big{)}\,\,dv_{g}(x),$

by $\displaystyle(f_{2})$, we obtain

$\displaystyle\displaystyle A(u_{n})-\frac{1}{\beta}<\,DA(u_{n}),\,u_{n}\,>\,$
$\displaystyle\displaystyle\geq\big{(}\,\frac{1}{p(x)}-\frac{1}{\beta}\,\big{)}\,\int_{M}(\,|\,\nabla
u_{n}\,|^{p(x)}-|\,u_{n}\,|^{p(x)}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}\big{(}\,H(x,u_{n},\nabla
u_{n})-h(x,u_{n},\nabla u_{n})\,.\,\frac{u_{n}}{\beta}\,\big{)}\,\,dv_{g}(x)$
$\displaystyle\displaystyle\geq J(u_{n})+I(u_{n}),$

with,

$\displaystyle
J(u_{n})=\big{(}\,\frac{1}{p(x)}-\frac{1}{\beta}\,\big{)}\,\int_{M}(\,|\,\nabla
u_{n}\,|^{p(x)}-|\,u_{n}\,|^{p(x)}\,)\,\,dv_{g}(x),$

and

$\displaystyle I(u_{n})=\int_{M}\big{(}\,H(x,u_{n},\nabla
u_{n})-h(x,u_{n},\nabla u_{n})\,.\,\frac{u_{n}}{\beta}\,\big{)}\,\,dv_{g}(x).$
$\displaystyle\displaystyle
J(u_{n})\geq\big{(}\,\frac{1}{p^{+}}-\frac{1}{\beta}\,\big{)}\,(\,\varrho_{p(.)}(u_{n})+\varrho_{p(.)}(\,|\,\nabla
u_{n}\,|\,)\,),$

or $\displaystyle\varrho_{p(.)}(\,|\,\nabla
u_{n}\,|\,)+\varrho_{p(.)}(u_{n})\geq
2^{1-p^{+}}\,||\,u_{n}\,||_{L^{q(.)}(M)}^{q^{+}}.$ So,

$\displaystyle\displaystyle J(u_{n})$
$\displaystyle\displaystyle\geq\big{(}\,\frac{1}{p^{+}}-\frac{1}{\beta}\,\big{)}\,2^{1-p^{+}}\,||\,u_{n}\,||^{q^{+}}_{L^{q(.)}(M)}$
$\displaystyle\displaystyle\geq\big{(}\,\frac{1}{p^{+}}-\frac{1}{\beta}\,\big{)}\,.\,\frac{2^{1-p^{+}}}{G^{q^{+}}}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}^{p^{+}},$

with $\displaystyle G$ being the positive constant of the embedding
$\displaystyle L^{p(.)}_{1}(M)\hookrightarrow L^{q(.)}(M)$. And by
$\displaystyle(h_{1})$, we get

$\displaystyle\displaystyle\int_{M}h(x,u_{n},\nabla
u_{n})\,.\,u_{n}\,\,dv_{g}(x)$
$\displaystyle\displaystyle\leq\int_{M}(\,\gamma(x)u_{n}+l(u_{n})\,.\,|\,\nabla
u_{n}\,|^{p(x)}\,.\,u_{n}\,)\,\,dv_{g}(x)$ $\displaystyle\displaystyle\leq
c_{1}\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}+l(u_{n})\,||\,|\,\nabla
u_{n}\,|\,||_{L^{p(.)}_{1}(M)}^{p(x)}\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}.$

And

$\displaystyle\displaystyle\int_{M}H(x,u_{n},\nabla u_{n})\,\,dv_{g}(x)$
$\displaystyle\displaystyle=\int_{M}\,\bigg{(}\,\int_{0}^{u_{n}}f(x,t,\nabla
t)\,\,dt\,\bigg{)}\,\,dv_{g}(x)$ $\displaystyle\displaystyle\geq-
c_{1}\,\varrho_{p(.)}(u_{n})-c_{2}-l(u_{n})\,.\,\varrho_{p(.)}(|\,\nabla
u_{n}\,|).$

Thus,

$\displaystyle\displaystyle I(u_{n})$ $\displaystyle\displaystyle\geq
c_{1}\,\varrho_{p(.)}(u_{n})-c_{2}-l(u_{n})\,.\,\varrho_{p(.)}(|\,\nabla
u_{n}\,|)$ $\displaystyle\displaystyle\hskip
8.5359pt-\frac{c_{1}}{\beta}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}-\frac{l(u_{n})}{\beta}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}\,.\,\varrho_{p(.)}(|\,\nabla
u_{n}\,|)$
$\displaystyle\displaystyle\geq-\bigg{(}\,c_{1}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}^{p(x)-1}-\frac{c_{1}}{\beta}\,\bigg{)}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}-c_{2}$
$\displaystyle\displaystyle\hskip
8.5359pt-\bigg{(}\,l(u_{n})+\frac{l(u_{n})}{\beta}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}\,\bigg{)}\,.\,\varrho_{p(.)}(|\,\nabla
u_{n}\,|).$

Hence,

$\displaystyle\displaystyle A(u_{n})-\frac{1}{\beta}<\,DA(u_{n}),\,u_{n}\,>\,$
$\displaystyle\displaystyle\geq\big{(}\,\frac{1}{p^{+}}-\frac{1}{\beta}\,)\,.\,2^{1-p^{+}}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}^{p^{+}}$
$\displaystyle\displaystyle\hskip
5.69046pt-\bigg{(}\,c_{1}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}^{p^{+}-1}-\frac{c_{1}}{\beta}\,\bigg{)}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}-c_{2}$
$\displaystyle\displaystyle\hskip
5.69046pt-\bigg{(}\,l(u_{n})+\frac{l(u_{n})}{\beta}\,.\,||\,u_{n}\,||_{L^{p(.)}_{1}(M)}\,\bigg{)}\,.\,\varrho_{p(.)}(|\,\nabla
u_{n}\,|).$ (3.3)

Now, dividing both sides of (3) by
$\displaystyle||\,u_{n}\,||_{L^{p(.)}_{1}(M)}$ and passing to the limit
$\displaystyle n\longrightarrow\infty$, we get $\displaystyle 0\geq\infty$ as
$\displaystyle p^{+}>1$ which is absurd.
Hence, $\displaystyle(u_{n})$ is bounded in $\displaystyle L^{p(.)}_{1}(M).$
$\displaystyle\bullet$ Since $\displaystyle\\{\,u_{n}\,\\}$ is bounded in
$\displaystyle L^{p(.)}_{1}(M)$, there exists a subsequence of
$\displaystyle\\{\,u_{n}\,\\},$ noted again by $\displaystyle\\{\,u_{n}\,\\},$
that converges in $\displaystyle L^{p(.)}_{1}(M)$. We will prove that
$\displaystyle\\{\,u_{n}\,\\}$ is Cauchy in $\displaystyle L^{p(.)}_{1}(M)$
i.e that

$\lim_{m,n\rightarrow\infty}\varrho_{p(.)}(u_{m}-u_{n})=0=\lim_{m,n\rightarrow\infty}\varrho_{p(.)}(|\,\nabla(\,u_{m}-u_{n})\,|\,).$
(3.4)

And to finish the demonstration of the Theorem 1, it suffice to show that the
subsequence of $\displaystyle\\{\,u_{n}\,\\}$ still noted by
$\displaystyle\\{\,u_{m}\,\\}$ is a Cauchy sequence in $\displaystyle
L^{p(.)}_{1}(M).$
Consider the following functionals

$B_{1}(u)=\int_{p(x)<2}(\,|\,\nabla
u(x)\,|^{p(x)}+|\,u(x)\,|^{p(x)}+H(x,u,\nabla u)\,)\,\,dv_{g}(x),$ (3.5)

and

$B_{2}(u)=\int_{p(x)\geq 2}(\,|\,\nabla
u(x)\,|^{p(x)}+|\,u(x)\,|^{p(x)}+H(x,u,\nabla u)\,)\,\,dv_{g}(x),$ (3.6)

on $\displaystyle L^{p(.)}_{1}(M),$ and note that

$\varrho_{p(.)}(u_{m}-u_{n})+\varrho_{p(.)}(\,|\,\nabla(\,u_{m}-u_{n}\,)\,|\,)=B_{1}(u_{m}-u_{n})-B_{2}(u_{m}-u_{n}),$
(3.7)

consider also the following inequalities

$|\,\zeta-\mu\,|^{p}\leq\frac{2}{p-1}\,\big{[}\,(\,|\,\zeta\,|^{p-2}\,\zeta-|\,\mu\,|^{p-2}\,\mu\,)\,.\,(\,\zeta-\mu\,)\,\big{]}^{\frac{p}{2}}\,.\,(\,|\,\zeta\,|-|\,\mu\,|\,)^{\frac{2q-q^{2}}{2}}\,\,\,\,\mbox{for}\,\,1\leq
p\leq 2,$ (3.8)

and

$|\,\zeta-\mu\,|^{p}\leq
2^{p}\,(\,|\,\zeta\,|^{p-2}\,\zeta-|\,\mu\,|^{p-2}\,\mu\,)\,.\,(\,\zeta-\mu\,)\,\,\,\mbox{for}\,\,\,p\geq
2,$ (3.9)

where, $\displaystyle\zeta,\,\mu\in\mathbb{R}^{N}$ and (3.1) we deduce that

$\displaystyle\displaystyle B_{2}(u_{m}-u_{n})$
$\displaystyle\displaystyle\leq\int_{p(x)\geq
2}(\,|\,\nabla(\,u_{m}-u_{n}\,)\,|^{p(x)}+|\,u_{m}-u_{n}\,|^{p(x)}+H(x,u_{m},\nabla
u_{m})$ $\displaystyle\displaystyle\hskip 5.69046pt-H(x,u_{n},\nabla
u_{n})\,)\,\,dv_{g}(x)$ $\displaystyle\displaystyle\leq
2^{p}\,\bigg{[}\,\int_{p\geq 2}(\,|\,\nabla u_{m}\,|^{p-2}\,\nabla
u_{m}-|\,\nabla u_{n}\,|^{p-2}\,\nabla
u_{n}\,)\,.\,\nabla(\,u_{m}-u_{n})\,\,dv_{g}(x)$
$\displaystyle\displaystyle\hskip 5.69046pt+\int_{p\geq
2}(\,|\,u_{m}\,|^{p-2}\,u_{m}-|\,u_{n}\,|^{p-2}\,u_{n}\,)\,.\,(u_{m}-u_{n})\,\,dv_{g}(x)\,\bigg{]}$
$\displaystyle\displaystyle\hskip 5.69046pt+c_{1}\,\int_{p\geq
2}|\,u_{m}\,|^{p}\,\,dv_{g}(x)+c_{2}$ $\displaystyle\displaystyle\hskip
5.69046pt+l(u_{m})\,\int_{p\geq 2}|\,\nabla
u_{m}\,|^{p}\,\,dv_{g}(x)-c_{1}\,\int_{p\geq
2}|\,u_{n}\,|^{p}\,\,dv_{g}(x)-c_{3}$ $\displaystyle\displaystyle\hskip
5.69046pt-l(u_{n})\,\int_{p\geq 2}|\,\nabla u_{n}\,|^{p}\,\,dv_{g}(x).$

Thus,

$2^{-p^{+}}\,B_{2}(u_{m}-u_{n})\leq<\,u_{m}-u_{n},\,DB(u_{m})-DB(u_{n})\,>.$
(3.10)

And using (3.5) and (3.8), we obtain

$\displaystyle\displaystyle\frac{p^{-}-1}{2}\,B_{1}(u_{m}-u_{n})$
$\displaystyle\displaystyle\leq\int_{p<2}g(\,|\,\nabla u_{m}\,|^{p-2}\nabla
u_{m}-|\,\nabla u_{n}\,|^{p-2}\,\nabla u_{n},\,\nabla(u_{m}-u_{n})\,)$
$\displaystyle\displaystyle\hskip 5.69046pt\times\,(|\,\nabla
u_{m}\,|+|\,\nabla u_{n}\,|)^{\frac{2q-q^{2}}{2}}\,\,dv_{g}(x)$
$\displaystyle\displaystyle\hskip 5.69046pt+\int_{p<2}(\,|\,\nabla
u_{m}\,|^{p-2}u_{m}-|\,u_{n}\,|^{p-2}\,u_{n}\,)\,.\,(u_{m}-u_{n}\,)^{\frac{p}{2}}$
$\displaystyle\displaystyle\hskip
5.69046pt\times\,(|\,u_{m}\,|+|\,u_{n}\,|)^{\frac{2q-q^{2}}{2}}\,\,dv_{g}(x)$
$\displaystyle\displaystyle\hskip 5.69046pt+\int_{p<2}H(x,u_{m},\nabla
u_{m})\,\,dv_{g}(x)-\int_{p<2}H(x,u_{n},\nabla u_{n})\,\,dv_{g}(x).$ (3.11)

######  Remark 3.1.

If $\displaystyle a$ and $\displaystyle b$ are two positive functions on
$\displaystyle M$, then by Hölder’s inequality

$\int_{p<2}a^{\frac{p}{2}}\,b^{\frac{2p-p^{2}}{2}}\leq
2\,||\,\mathbb{1}_{p<2}\,\,a^{\frac{p}{2}}\,||_{L^{\frac{2}{p}}}\,.\,||\,\mathbb{1}_{p<2}\,\,b^{\frac{2p-p^{2}}{2}}\,||_{L^{\frac{2}{2-p}}}.$
(3.12)

where $\displaystyle\mathbb{1}$ is the indicator function of $\displaystyle
M$, moreover, since

$\displaystyle||\,\mathbb{1}_{p<2}\,\,a^{\frac{2}{p}}\,||_{L^{\frac{2}{p}}}\leq\max\\{\,\varrho_{1}(a),\,\varrho_{1}(a)^{\frac{p^{-}}{2}}\,\\}$

and

$\displaystyle||\,\mathbb{1}_{p<2}\,\,b^{\frac{2p-p^{2}}{2}}\,||_{L^{\frac{2}{2-p}}}\leq\max\\{\,\varrho_{p}(b)^{\frac{2-p^{-}}{2}},\,1\,\\},$

we get,

$\int_{p<2}a^{\frac{p}{2}}\,.\,b^{\frac{2p-p^{2}}{2}}\leq
2\,\max\,\\{\,\varrho_{1}(a),\varrho_{1}(a)^{\frac{p^{-}}{2}}\,\\}\,\max\,\\{\,\varrho_{p}(b)^{\frac{2p-p^{2}}{2}},\,1\,\\}.$
(3.13)

Thus, using (3.13) twice in (3) we infer that

$(\,P^{-}-1\,)\,B_{1}(u_{m}-u_{n})\leq
P(\,<\,u_{m}-u_{n},DB(u_{m})-DB(u_{n})\,>\,)\,Q(\,||\,u_{m}\,||_{L^{p}_{1}},\,||\,u_{n}\,||_{L^{p}_{1}}\,),$
(3.14)

for some constructible continuous functions $\displaystyle P(x)$ and
$\displaystyle Q(y,z)$ with $\displaystyle P(0)=0$.
Since the sequence $\displaystyle\\{\,||\,u_{n}\,||_{L^{p(.)}_{1}}\,\\}$ is
bounded, from (3.14), (3.10) and (3.7) we conclude that if

$\displaystyle\lim_{m,n\rightarrow\infty}<\,u_{m}-u_{n},\,DB(u_{m})-DB(u_{n})\,>=0.$

To obtain such that a limit, recall that $\displaystyle DA=DB-DC$ hence

$\displaystyle\displaystyle<\,u_{m}-u_{n},DB(u_{m})-DB(u_{n})\,>\,=$
$\displaystyle\displaystyle\,<\,u_{m}-u_{n},\,DA(u_{m})-DA(u_{n})\,>$
$\displaystyle\displaystyle+<\,u_{m}-u_{n},\,DC(u_{m})-DC(u_{n})\,>,$

we estimate

$\displaystyle\displaystyle<\,u_{m}-u_{n},\,DC(u_{m})-DC(u_{n})\,>\leq$
$\displaystyle\displaystyle\frac{1}{G}\,||\,N_{f}(u_{m})-N_{f}(u_{n})\,||_{(\,L^{p(.)}_{1}(M)\,)^{\prime}}$
$\displaystyle\displaystyle\times||\,u_{m}-u_{n}\,||_{L^{q(.)}_{1}(M)},$

where $\displaystyle G$ being the constant of the embedding $\displaystyle
L^{p(.)}_{1}(M)\hookrightarrow L^{q(.)}(M)$
and $\displaystyle
N_{f}:\,L^{p(.)}(M)\longrightarrow(\,L^{p(.)}(M)\,)^{\prime}$ is the Nemytskii
operator induced by $\displaystyle f$, with
$\displaystyle N_{f}[u](x)=f(x,u(x))$.
Hence, using the continuity of Nemytskii map, the convergence of
$\displaystyle\\{\,u_{n}\,\\}$ in $\displaystyle L^{p(.)}(M)$ the boundedness
of $\displaystyle\\{\,||\,u_{n}\,||_{L^{q(.)}(M)}\,\\}$ and the hypothesis on
$\displaystyle DA(u_{n})\longrightarrow\infty$ as $\displaystyle
n\longrightarrow\infty$, we deduce that the functional $\displaystyle A$
satisfies the Palais-Smale Condition. ∎

##  4\. Uniqueness of Non-trivial Solution

Now, let us show the uniqueness of solution.

######  Theorem 4.1.

Assume that condition $\displaystyle(f_{1})-(f_{3})$ and
$\displaystyle(h_{1})$ are true. With $\displaystyle f\in
L^{p^{\prime}(x)}(M)$ is a contraction Lipschitz continuous function such that
$\displaystyle\exists\alpha$ with $\displaystyle 0\leq\alpha<1$

$|\,f(\,x,u_{1}\,)-f(\,x,u_{2}\,)\,|\leq\alpha\,|\,u_{1}-u_{2}\,|.$ (4.1)

Then, the problem (1.1) has a unique non-trivial solution.

###### Proof.

Suppose that $\displaystyle u_{1}$ and $\displaystyle u_{2}$ satisfy (1.1).
Then we have

$\displaystyle\displaystyle\int_{M}|\,$ $\displaystyle\displaystyle\nabla
u_{1}\,|^{p(x)-2}\,.\,\nabla u_{1}\,.\,(\,\nabla u_{1}-\nabla
u_{2}\,)\,\,dv_{g}(x)+\int_{M}h(\,x,u_{1},\nabla
u_{1}\,)\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}|\,u_{1}\,|^{p(x)-2}\,.\,u_{1}\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x)=\int_{M}f(\,x,u_{1}\,)\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x),$

and

$\displaystyle\displaystyle\int_{M}|\,$ $\displaystyle\displaystyle\nabla
u_{2}\,|^{p(x)-2}\,.\,\nabla u_{2}\,.\,(\,\nabla u_{2}-\nabla
u_{1}\,)\,\,dv_{g}(x)+\int_{M}h(\,x,u_{2},\nabla
u_{2}\,)\,.\,(\,u_{2}-u_{1}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}|\,u_{2}\,|^{p(x)-2}\,.\,u_{2}\,.\,(\,u_{2}-u_{1}\,)\,\,dv_{g}(x)=\int_{M}f(\,x,u_{2}\,)\,.\,(\,u_{2}-u_{1}\,)\,\,dv_{g}(x).$

Adding the above two equations yields.

$\displaystyle\displaystyle\int_{M}$
$\displaystyle\displaystyle\bigg{(}\,|\,\nabla u_{1}\,|^{p(x)-2}\,.\,\nabla
u_{1}-|\,\nabla u_{2}\,|^{p(x)-2}\,.\,\nabla u_{2}\,\bigg{)}\,.\,(\,\nabla
u_{1}-\nabla u_{2}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\,\int_{M}\bigg{(}\,h(\,x,u_{1},\nabla
u_{1}\,)-h(\,x,u_{2},\nabla
u_{2}\,)\,\bigg{)}\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}\bigg{(}\,|\,u_{1}\,|^{p(x)-2}\,.\,u_{1}-|\,u_{2}\,|^{p(x)-2}\,.\,u_{2}\,\bigg{)}\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle=\int_{M}(\,f(\,x,u_{1}\,)-f(\,x,u_{2}\,)\,)\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x),$

by $\displaystyle(f_{2}),\,(f_{3})$ and (4.1) we obtain

$\displaystyle\displaystyle\int_{M}$
$\displaystyle\displaystyle\bigg{(}\,|\,\nabla u_{1}\,|^{p(x)-2}\,.\,\nabla
u_{1}-|\,\nabla u_{2}\,|^{p(x)-2}\,.\,\nabla u_{2}\,\bigg{)}\,.\,(\,\nabla
u_{1}-\nabla u_{2}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\,\int_{M}\bigg{(}\,h(\,x,u_{1},\nabla
u_{1}\,)-h(\,x,u_{2},\nabla
u_{2}\,)\,\bigg{)}\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}\bigg{(}\,|\,u_{1}\,|^{p(x)-2}\,.\,u_{1}-|\,u_{2}\,|^{p(x)-2}\,.\,u_{2}\,+\alpha\,(\,u_{1}-u_{2}\,)\,\bigg{)}\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x)\geq
0.$

Then,

$\displaystyle\displaystyle\int_{M}$
$\displaystyle\displaystyle\bigg{(}\,|\,h(\,x,u_{1},\nabla
u_{1}\,)\,|-|\,h(\,x,u_{2},\nabla
u_{2}\,)\,|\,\bigg{)}\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x)\geq$
$\displaystyle\displaystyle\int_{M}\bigg{(}\,|\,\nabla
u_{1}\,|^{p(x)-2}\,.\,|\,\nabla u_{1}\,|-|\,\nabla
u_{2}\,|^{p(x)-2}\,.\,|\,\nabla u_{2}\,|\,\bigg{)}\,.\,(\,|\,\nabla
u_{1}\,|-|\,\nabla u_{2}\,|\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}\bigg{(}\,|\,u_{1}\,|^{p(x)-2}\,.\,|\,u_{1}\,|-|\,u_{2}\,|^{p(x)-2}\,.\,|\,u_{2}\,|\,\bigg{)}\,.\,(\,|\,u_{1}\,|-|\,u_{2}\,|\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle\geq\int_{M}\big{(}\,|\,\nabla
u_{1}\,|^{p(x)-1}-|\,\nabla u_{2}\,|^{p(x)-1}\,\big{)}\,.\,(\,|\,\nabla
u_{1}\,|-|\,\nabla u_{2}\,|\,)\,\,dv_{g}(x)$ $\displaystyle\displaystyle\hskip
8.5359pt+\int_{M}(\,|\,u_{1}\,|^{p(x)-1}-|\,u_{2}\,|^{p(x)-1}\,)\,.\,(\,|\,u_{1}\,|-|\,u_{2}\,|\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle\hskip
8.5359pt+\alpha\,\int_{M}(\,|\,u_{1}\,|-|\,u_{2}\,|\,)^{2}\,\,dv_{g}(x).$

So, by $\displaystyle(h_{1})$ we have

$\displaystyle\displaystyle\int_{M}$
$\displaystyle\displaystyle\bigg{(}\,|\,h(\,x,u_{1},\nabla
u_{1}\,)\,|-|\,h(\,x,u_{2},\nabla
u_{2}\,)\,|\,\bigg{)}\,.\,(\,u_{1}-u_{2}\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle\leq\int_{M}(\,l(u_{1})\,.\,|\,\nabla
u_{1}\,|^{p(x)}-l(u_{2})\,.\,|\,\nabla
u_{2}\,|^{p(x)}\,)\,.\,(\,|\,u_{1}\,|-|\,u_{2}\,|\,)\,\,dv_{g}(x).$

Hence,

$\displaystyle\displaystyle\int_{M}$
$\displaystyle\displaystyle(\,l(u_{1})\,.\,|\,\nabla
u_{1}\,|^{p(x)}-l(u_{2})\,.\,|\,\nabla
u_{2}\,|^{p(x)}\,)\,.\,(\,|\,u_{1}\,|-|\,u_{2}\,|\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle\geq\int_{M}\big{(}\,|\,\nabla
u_{1}\,|^{p(x)-1}-|\,\nabla u_{2}\,|^{p(x)-1}\,\big{)}\,.\,(\,|\,\nabla
u_{1}\,|-|\,\nabla u_{2}\,|\,)\,\,dv_{g}(x)$ $\displaystyle\displaystyle\hskip
8.5359pt+\int_{M}(\,|\,u_{1}\,|^{p(x)-1}-|\,u_{2}\,|^{p(x)-1}\,)\,.\,(\,|\,u_{1}\,|-|\,u_{2}\,|\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle\hskip
8.5359pt+\alpha\,\int_{M}(\,|\,u_{1}\,|-|\,u_{2}\,|\,)^{2}\,\,dv_{g}(x).$

Which implies that

$\displaystyle\displaystyle 0\geq$
$\displaystyle\displaystyle\int_{M}\big{(}\,|\,\nabla
u_{1}\,|^{p(x)-1}-|\,\nabla u_{2}\,|^{p(x)-1}\,\big{)}\,.\,(\,|\,\nabla
u_{1}\,|-|\,\nabla u_{2}\,|\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\int_{M}(\,|\,u_{1}\,|^{p(x)-1}-l(u_{1})\,.\,|\,\nabla
u_{1}\,|^{p(x)}-|\,u_{2}\,|^{p(x)-1}-l(u_{2})\,.\,|\,\nabla
u_{2}\,|^{p(x)}\,)$ $\displaystyle\displaystyle\hskip
5.69046pt\times(\,|\,u_{1}\,|-|\,u_{2}\,|\,)\,\,dv_{g}(x)$
$\displaystyle\displaystyle+\alpha\,\int_{M}(\,|\,u_{1}\,|-|\,u_{2}\,|\,)^{2}\,\,dv_{g}(x)$
$\displaystyle\displaystyle\geq\int_{M}\big{(}\,|\,\nabla
u_{1}\,|^{p(x)-1}-|\,\nabla u_{2}\,|^{p(x)-1}\,\big{)}\,.\,(\,|\,\nabla
u_{1}\,|-|\,\nabla u_{2}\,|\,)\,\,dv_{g}(x).$

Hence, we obtain that $\displaystyle u_{1}=u_{2}$ almost everywhere. This
complete the proof of uniqueness. ∎

##  5\. Conclusion

In sum, the first and second Theorems, are valid and substantiated. We can
then say that the solution of the problem (1.1) is a unique and non-trivial
one.

## References

  * [1] A. Aberqi, J. Bennouna, M. Elmassoudi, and Hammoumi, M.Existence and uniqueness of a renormalized solution of parabolic problems in Orlicz spaces, Monatshefte für Mathematik. 189 (2019), no. 2, 195–219.
  * [2] A. Aberqi, J. Bennouna, M. Mekkour, and H. Redwane.Nonlinear parabolic inequalities with lower order terms, Applicable Analysis. 96 (2017), no. 12, 2102–2117.
  * [3] E. Acerbi, and G. Mingione.Regularity results for stationary electro-rheological fluids, Archive for Rational Mechanics and Analysis. 164 (2002), no. 3, 213–259.
  * [4] Th. Aubin, Nonlinear analysis on manifolds. Monge-Ampere equations, pringer Science & Business Media, 1982.
  * [5] S. Barnaś, Existence result for hemivariational inequality involving p (x)-Laplacian, Opuscula Mathematica. 32 (2012), no. 3.
  * [6] M. Benalili, and Y. Maliki.Multiplicity of solutions for elliptic quasilinear equations with critical exponent on compact manifolds, Nonlinear Analysis: Theory, Methods & Applications. 71 (2009), no. 12, 5946–5960.
  * [7] Y. Chen, S. Levine, and M. Rao.Variable exponent, linear growth functionals in image restoration, SIAM journal on Applied Mathematics. 66 (2006), no. 4, 1383–1406.
  * [8] L. Diening, P. Harjulehto, P. Hästö, and M. Ruzicka.Lebesgue and Sobolev spaces with variable exponents, Springer, 2011.
  * [9] X.L. Fan, and Q.H. Zhang.Existence of solutions for p (x)-Laplacian Dirichlet problem, Nonlinear Analysis: Theory, Methods & Applications. 52 (2003), no. 8, 1843–1852.
  * [10] M. Gaczkowski, P. Górka, and D.J. Pons.Monotonicity methods in generalized Orlicz spaces for a class of non-Newtonian fluids, Mathematical methods in the applied sciences. 33 (2010), no. 2, 125–137.
  * [11] P.M. Gadea, J.M. Masqué, and I.V. Mykytyuk.Analysis and algebra on differentiable manifolds: a workbook for students and teachers, Springer Science & Business Media, 2012.
  * [12] S. Gala, Q. Liu, and M. A. Ragusa.A new regularity criterion for the nematic liquid crystal flows, Applicable Analysis. 91 (2012), no. 9, 1741–1747.
  * [13] P. Gwiazda, F.Z. Klawe, and A. Świerczewska-Gwiazda.Thermo-viscoelasticity for Norton–Hoff-type models, Nonlinear Analysis: Real World Applications. 26 (2015), 199–228.
  * [14] E. Hebey.Nonlinear Analysis on Manifolds: Sobolev Spaces and Inequalities: Sobolev Spaces and Inequalities, SAmerican Mathematical Society, 2000.
  * [15] S. Polidoro, M. A. Ragusa and others.Harnack inequality for hypoelliptic ultraparabolic equations with a singular lower order term, Revista Matematica Iberoamericana. 24 (2008), 1011–1046.
  * [16] V. D. Radulescu, and D. D. Repovs. Partial differential equations with variable exponents: variational methods and qualitative analysis, CRC press. 9 (2015).
  * [17] N. S. Trudinger. Remarks concerning the conformal deformation of Riemannian structures on compact manifolds, Annali della Scuola Normale Superiore di Pisa-Classe di Scienze. 22 (1968), no. 2, 265–274.
  * [18] V.V. Zhikov.Meyer-type estimates for solving the nonlinear Stokes system, Differentsial’nye Uravneniya. 33 (1997), no. 1, 107–114.
  * [19] V. V. Zhikov.On some variational problems, Russian Journal of Mathematical Physics. 5 (1997), no. 1, 105–116.


$\displaystyle\leq\frac{2T^{4}L_{f}^{8}\left\|\mathcal{H}\right\|_{\mathsf{op}}\log(\tilde{K}+1)}{\lambda^{3}(\tilde{K}+1)}+\frac{TL_{f}^{2}\left\|\mathcal{H}\right\|_{\mathsf{op}}}{\lambda^{2}}\left(\frac{T^{2}L_{f}D(\mathcal{B})}{\sigma_{w}}+\sqrt{\frac{8\log\frac{2\tilde{K}}{\delta}}{\tilde{N}}}\right)$
$\displaystyle\leq\frac{2T^{4}L_{f}^{8}\left\|\mathcal{H}\right\|_{\mathsf{op}}\log(N+1)}{\lambda^{3}(\lfloor
N^{1/3}\rfloor+1)}+\frac{TL_{f}^{2}\left\|\mathcal{H}\right\|_{\mathsf{op}}}{\lambda^{2}}\left(\frac{T^{2}L_{f}R}{N^{\alpha}\sigma_{w}}+\sqrt{\frac{8\log\frac{2N}{\delta}}{\lfloor
N^{2/3}\rfloor}}\right)$
$\displaystyle\leq\frac{d_{\phi}}{\lambda_{\min}^{\star}}\left\|\mathcal{H}\right\|_{\mathsf{op}},$

where the final inequality follows from the lower bound on $N$. By Assumption
2.6 paired with Lemma C.3, we have that

$\displaystyle\min_{\pi\in\Pi_{\exp}}\Phi\left(\operatorname{\mathbb{E}}_{\pi,\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{1},U_{t}^{1};\hat{\phi})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi})\right]\right)$
$\displaystyle=\operatorname{\mathrm{tr}}\left(\mathcal{H}\left(\operatorname{\mathbb{E}}_{\pi,\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi})\right]+\lambda
I\right)^{-1}\right)\leq\frac{d_{\phi}}{\lambda_{\min}^{\star}}\left\|\mathcal{H}\right\|_{\mathsf{op}}.$

We may lower bound

$\displaystyle\Phi\left(\frac{1}{\tilde{N}(\tilde{K}+1)}\sum_{t,n=1}^{T,(\tilde{N}\times(\tilde{K}+1))}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})\right)$
$\displaystyle\geq\frac{\lambda_{\min}\left(\mathcal{H}\right)}{\lambda_{\min}\left(\frac{1}{\tilde{N}\tilde{K}}\sum_{t,n=1}^{T,(\tilde{N}\times\tilde{K})}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})\right)+\lambda}.$

We conclude by combining these inequalities, and using the assumption that
$\lambda\leq\frac{\lambda_{\min}(\mathcal{H})\lambda_{\min}^{\star}}{4\left\|\mathcal{H}\right\|_{\mathsf{op}}d_{\phi}}$.
∎

######  Lemma C.5.

Consider a policy $\pi\in\Pi_{\exp}$. Suppose
$\left\|\hat{\phi}-\phi^{\star}\right\|\leq\min\mathopen{}\left\\{\frac{\tilde{\lambda}}{4TL_{f}^{2}},1\right\\}\mathclose{}$
for some $\tilde{\lambda}>0$. Define

$\displaystyle\Sigma^{\pi}_{\hat{\phi}}=\operatorname{\mathbb{E}}_{\pi,\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})\right],$

and suppose that $\Sigma^{\pi}_{\hat{\phi}}\succeq\tilde{\lambda}I$. Then
$\Sigma^{\pi}\succeq\frac{\tilde{\lambda}}{2}I$, and

  1. 1.

$\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}\right)^{-1}\right)\leq\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi})\left(\Sigma^{\pi}\right)^{-1}_{\hat{\phi}}\right)+\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},T,d_{\mathsf{X}},d_{\phi},\left\|\mathcal{H}(\phi^{\star})\right\|,\frac{1}{\tilde{\lambda}}\right)\left\|\hat{\phi}-\phi^{\star}\right\|$

  2. 2.

$\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi})\left(\Sigma^{\pi}_{\hat{\phi}}\right)^{-1}\right)\leq\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}\right)^{-1}\right)+\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},T,d_{\mathsf{X}},d_{\phi},\left\|\mathcal{H}(\phi^{\star})\right\|,\frac{1}{\tilde{\lambda}}\right)\left\|\hat{\phi}-\phi^{\star}\right\|$

###### Proof.

To begin, we observe that by a first order Taylor expansion of
$Df(x,u;\phi^{\star})$ about $\hat{\phi}$, we have that

$\displaystyle\Sigma^{\pi}$
$\displaystyle\succeq\operatorname{\mathbb{E}}_{\pi,\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})\right]+G,$

where $\left\|G\right\|\leq 2TL_{f}^{2}\left\|\hat{\phi}-\phi^{\star}\right\|$
by the boundedness assumptions on the derivatives of $f$ from Assumption 2.1.
By the assumption that
$\left\|\hat{\phi}-\phi^{\star}\right\|\leq\frac{\tilde{\lambda}}{4TL_{f}^{2}}$,
it holds that $\left\|G\right\|\leq\frac{\tilde{\lambda}}{2}$. Then it follows
that
$\lambda_{\min}(\Sigma^{\pi})\geq\lambda_{\min}(\Sigma^{\pi}_{\hat{\phi}})-\left\|G\right\|\geq\frac{\tilde{\lambda}}{2}$.

We proceed to bound
$\left\|\left(\Sigma^{\pi}\right)^{-1}-\left(\hat{\Sigma}^{\pi}\right)^{-1}\right\|$.
We immediately have that
$\left\|\left(\Sigma^{\pi}\right)^{-1}-\left(\hat{\Sigma}^{\pi}\right)^{-1}\right\|\leq\frac{1}{\lambda_{\min}\left(\Sigma^{\pi}\right)\lambda_{\min}\left(\Sigma^{\pi}_{\hat{\phi}}\right)}\left\|\Sigma^{\pi}_{\hat{\phi}}-\Sigma^{\pi}\right\|\leq\frac{2}{\tilde{\lambda}^{2}}\left\|\Sigma^{\pi}_{\hat{\phi}}-\Sigma^{\pi}\right\|$.
By Taylor expanding $Df(x,u;\hat{\phi})$ about $\phi^{\star}$, we have from
the boundedness assumptions of Assumption 2.1 that
$\left\|\Sigma^{\pi}_{\hat{\phi}}-\Sigma^{\pi}\right\|\leq
2TL_{f}^{2}\left\|\hat{\phi}-\phi^{\star}\right\|+TL_{f}\left\|\hat{\phi}-\phi^{\star}\right\|^{2}\leq
3TL_{f}^{2}\left\|\hat{\phi}-\phi^{\star}\right\|$. Furthermore, by Lemma B.2,
$\left\|\mathcal{H}(\hat{\phi})-\mathcal{H}(\phi^{\star})\right\|\leq\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},T,d_{\mathsf{X}}\right)\left\|\phi-\phi^{\star}\right\|.$

To conclude, we bound

$\displaystyle{\left|\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}\right)^{-1}\right)-\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}_{\hat{\phi}}\right)^{-1}\right)\right|}$
$\displaystyle\leq\left\|\mathcal{H}(\phi^{\star})\right\|d_{\phi}\left\|\left(\Sigma^{\pi}\right)^{-1}-\left(\hat{\Sigma}^{\pi}\right)^{-1}\right\|\leq\left\|\mathcal{H}(\phi^{\star})\right\|d_{\phi}\frac{6TL_{f}^{2}\left\|\hat{\phi}-\phi^{\star}\right\|}{\tilde{\lambda}^{2}},$

and

$\displaystyle{\left|\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}_{\hat{\phi}}\right)^{-1}\right)-\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi})\left(\Sigma^{\pi}_{\hat{\phi}}\right)^{-1}\right)\right|}$
$\displaystyle\leq\frac{d_{\phi}}{\tilde{\lambda}}\left\|\mathcal{H}(\hat{\phi})-\mathcal{H}(\phi^{\star})\right\|$
$\displaystyle\leq\frac{d_{\phi}}{\tilde{\lambda}}\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},T,d_{\mathsf{X}}\right)\left\|\phi-\phi^{\star}\right\|.$

Combining these inequalities with the triangle inequality provides the desired
result. ∎

##  Appendix D Proof of Main Excess Cost Bound, Theorem 3.2

Armed with the above results, we are ready to prove Theorem 3.2.

See 3.2

###### Proof.

We have that $\frac{N}{4}\geq\tau_{\mathbf{Err}}(\delta/4)$. Then by
Assumption 2.5, we may invoke Corollary A.1 to show that with probability at
least $1-\delta/4$,
$\left\|\hat{\phi}^{-}-\phi^{\star}\right\|\leq\frac{R}{N^{\alpha}}$, for $R$
defined in (17). This in turn implies that $\phi^{\star}\in\mathcal{B}^{-}$
for $B^{-}$ define in (16). The diameter of the ball is given as
$D(\mathcal{B}^{-})=\frac{2R}{N^{\alpha}}$. Denote the event that
$\phi^{\star}\in\mathcal{B}^{-}$ as $\mathcal{E}_{\mathsf{coarse}}$. We
condition on the event $\mathcal{E}_{\mathsf{coarse}}$ for the remainder of
the proof.

Let $C_{\mathsf{Hpert}}$ be the system theoretic constant defined in Lemma
B.2. As long as
$\frac{N}{4}\geq\frac{2RC_{\mathsf{Hpert}}}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))}^{\frac{1}{\alpha}}$,
it follows that

$\displaystyle\left\|\hat{\phi}^{-}-\phi^{\star}\right\|\leq\frac{\lambda_{\min}(\mathcal{H}(\phi^{\star}))}{2C_{\mathsf{Hpert}}}.$
(34)

Then by Lemma B.2, we have
$\lambda_{\min}(\mathcal{H}(\hat{\phi}^{-}))\geq\frac{1}{2}\lambda_{\min}(\mathcal{H}(\phi^{\star}))$
and $\left\|\mathcal{H}(\hat{\phi}^{-})\right\|\leq
2\left\|\mathcal{H}(\phi^{\star})\right\|$.

Condition on the event $\mathcal{E}_{\mathsf{coarse}}$. By the choice of
$\lambda$ in 5 in Algorithm 1, the assumption that

$\frac{N}{4}\geq\tau_{\mathsf{PD\,data}}(\delta/4),$

where

$\tau_{\mathsf{PD\,data}}(\delta)=\max\mathopen{}\left\\{\left\lceil\frac{4\lambda_{\min}^{\star}T^{4}L_{f}^{8}\log(N+1)}{\lambda^{3}d_{\phi}}\right\rceil^{3},\left\lceil\frac{8\lambda_{\min}^{\star}TL_{f}^{2}\sqrt{2\log(2N/\delta)}}{\lambda^{2}d_{\phi}}\right\rceil^{3},\left(\frac{4T^{3}L_{f}^{3}R\lambda_{\min}^{\star}}{\lambda^{2}\sigma_{w}d_{\phi}}\right)^{1/\alpha}\right\\}\mathclose{},$

and Assumption 2.6, we may guarantee that with probability at least
$1-\frac{\delta}{4}$ the success event of Lemma C.4 holds. Denote this event
$\mathcal{E}_{\mathsf{doed}}$. $\texttt{DOED}_{+}$ collects
$\tilde{K}\tilde{N}$ episodes of data data
$\mathopen{}\left\\{X_{t}^{n},U_{t}^{n}\right\\}\mathclose{}_{t,k=1}^{T+1,(\tilde{N}\times\tilde{K})}$,
where $\tilde{K}=\lfloor\frac{N}{4}^{1/3}\rfloor$ and
$\tilde{N}=\lfloor\frac{N}{4}^{2/3}\rfloor$. Under the success event, this
collected data satisfies

$\displaystyle\lambda_{\min}\left(\frac{1}{(\tilde{K}+1)\tilde{N}}\sum_{t,n=1}^{T,((\tilde{K}+1)\times\tilde{N})}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi})\right)\geq\frac{\lambda_{\min}(\mathcal{H}(\hat{\phi}^{-}))\lambda_{\min}^{\star}}{4\left\|\mathcal{H}(\hat{\phi}^{-})\right\|_{\mathsf{op}}d_{\phi}}.$
(35)

By the law of total expectation, the mixture policy $\pi^{\mathsf{mix}}$
defined in 9 of Algorithm 1 satisfies

$\displaystyle\operatorname{\mathbb{E}}_{\pi_{\mathsf{mix}},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})\right]$
$\displaystyle=\frac{1}{2}\operatorname{\mathbb{E}}_{\pi^{0},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})\right]$
(36)
$\displaystyle+\frac{1}{2\tilde{K}}\sum_{i=1}^{\tilde{K}}\operatorname{\mathbb{E}}_{\pi^{i},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})\right]$
$\displaystyle\succeq\frac{1}{2(\tilde{N}+1)}\sum_{i=0}^{\tilde{K}}\operatorname{\mathbb{E}}_{\pi^{i},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})\right].$

It follows from Lemma C.10 of Wagenmaker et al. (2023) that the empirical
covariance of the data collected from Algorithm 2 concentrates to the true
covariance. In particular, with probability at least $1-\delta/4$,

$\displaystyle\left\|\frac{1}{\tilde{K}+1}\sum_{i=0}^{\tilde{K}}\operatorname{\mathbb{E}}_{\pi^{i},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})\right]-\frac{1}{(\tilde{K}+1)\tilde{N}}\sum_{t,n=1}^{T,((\tilde{K}+1)\times\tilde{N})}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})\right\|$
(37) $\displaystyle\leq
d_{\phi}TL_{f}^{2}\sqrt{\frac{8d_{\phi}\log\left(1+8\sqrt{\tilde{K}\tilde{N}}\right)+8\log(4/\delta)}{\tilde{K}\tilde{N}}}.$

Denote this concentration event as $\mathcal{E}_{\mathsf{doed\,conc}}$. Under
the burn-in requirement,

$\displaystyle\frac{N}{4}\geq\tau_{\mathsf{doed\,conc}}(\delta)=\left\lceil\left(\frac{16\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))\lambda_{\min}^{\star}}\right)^{2}d_{\phi}^{4}T^{2}L_{f}^{4}\left(8d_{\phi}\log\left(1+8\sqrt{N/4}\right)+8\log(4/\delta)\right)\right\rceil^{3},$
(38)

this event implies

$\displaystyle\frac{1}{\tilde{K}+1}\sum_{i=0}^{\tilde{K}}\operatorname{\mathbb{E}}_{\pi^{i},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})\right]$
$\displaystyle\succeq\frac{1}{(\tilde{K}+1)\tilde{N}}\sum_{t,n=1}^{T,((\tilde{K}+1)\times\tilde{N})}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})$
$\displaystyle-\frac{\lambda_{\min}(\mathcal{H}(\phi^{\star}))\lambda_{\min}^{\star}}{16\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}d_{\phi}^{2}}I.$

Combining this inequality with (36) and (35) and using the fact that
$\lambda_{\min}(\mathcal{H}(\hat{\phi}^{-}))\geq\frac{1}{2}\lambda_{\min}(\mathcal{H}(\phi^{\star}))$
and that $\left\|\mathcal{H}(\hat{\phi}^{-})\right\|\leq
2\left\|\mathcal{H}(\phi^{\star})\right\|$, we find that

$\displaystyle\operatorname{\mathbb{E}}_{\pi_{\mathsf{mix}},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})\right]\succeq\frac{\lambda_{\min}(\mathcal{H}(\hat{\phi}^{-}))\lambda_{\min}^{\star}}{16\left\|\mathcal{H}(\hat{\phi}^{-})\right\|_{\mathsf{op}}d_{\phi}}I.$
(39)

Denote the event that Algorithm 2 collects data satisfying (35) and (37) as
$\mathcal{E}_{\mathsf{doed\,covariance}}$. As long as

$\displaystyle\frac{N}{4}\geq\left(\frac{64RTL_{f}^{2}\left\|\mathcal{H}(\phi^{\star})\right\|d_{\phi}}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))\lambda_{\min}^{\star}}\right)^{\frac{1}{\alpha}},$

it follows that

$\displaystyle\left\|\hat{\phi}^{-}-\phi^{\star}\right\|\leq\frac{\lambda_{\min}\left(\operatorname{\mathbb{E}}_{\pi_{\mathsf{mix}},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})\right]\right)}{4TL_{f}^{2}}$
(40)

from the bound
$\left\|\hat{\phi}^{-}-\phi^{\star}\right\|\leq\frac{R}{N^{\alpha}}$. Then by
Lemma C.5,

$\displaystyle\Sigma^{\pi_{\mathsf{mix}}}\succeq\frac{\lambda_{\min}(\mathcal{H}(\hat{\phi}^{-}))\lambda_{\min}^{\star}}{32\left\|\mathcal{H}(\hat{\phi}^{-})\right\|_{\mathsf{op}}d_{\phi}}I\succ
0.$

Additionally, the policy $\pi_{\mathsf{mix}}$ is
$(2C_{\mathsf{Loja}},\alpha)$-Lojasiewicz. This can be seen by noting that by
the law of total expectation. In particular, for all
$\phi\in\mathbb{R}^{d_{\phi}}$,

$\displaystyle\mathbf{Err}_{\pi_{\mathsf{mix}},\phi^{\star}}(\phi)\geq\frac{1}{2}\mathbf{Err}_{\pi^{0},\phi^{\star}}(\phi).$

Then under the events $\mathcal{E}_{\mathsf{coarse}}$,
$\mathcal{E}_{\mathsf{doed}}$ and $\mathcal{E}_{\mathsf{doed\,covariance}}$ in
conjunction with the burn-in requirement

$\displaystyle\frac{N}{4}\geq\max\mathopen{}\left\\{\tau_{\mathbf{Err}}(\delta/32),\tau_{\mathsf{small\,error}}(\delta/16),\tau_{\mathsf{h.o.t}}(\delta/16),\tau_{\mathsf{cov\,conc}}(\delta/16)\right\\}\mathclose{},$

the conditions are set to apply Theorem 3.1 on the data collected by rerunning
$\pi^{\mathsf{mix}}$ for $\frac{N}{4}$ episodes. Doing so ensures that with
probability at least $1-\delta/4$,

$\displaystyle\left\|\hat{\phi}^{-}-\phi^{\star}\right\|_{\mathcal{H}(\phi^{\star})}^{2}\leq
160\sigma_{w}^{2}\log\frac{92d_{\phi}}{\delta}\frac{\operatorname{\mathrm{tr}}(\mathcal{H}(\phi^{\star})(\Sigma^{\pi_{\mathsf{mix}}})^{-1})}{N}.$
(41)

Denote this event as $\mathcal{E}_{\mathsf{ID}}$.

Then under the event
$\mathcal{E}_{\mathsf{coarse}}\cap\mathcal{E}_{\mathsf{doed\,conc}}\cap\mathcal{E}_{\mathsf{ID}}$,
by using the covariance bound (39) and the estimation bound (40), it follows
from Lemma C.5

$\displaystyle\operatorname{\mathrm{tr}}(\mathcal{H}(\phi^{\star})(\Sigma^{\pi_{\mathsf{mix}}})^{-1})$
$\displaystyle\leq\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi}^{-})\left(\operatorname{\mathbb{E}}_{\pi_{\mathsf{mix}},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})\right]\right)^{-1}\right)$
(42)
$\displaystyle+\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},T,d_{\mathsf{X}},d_{\phi},\left\|\mathcal{H}(\phi^{\star})\right\|,\frac{1}{\lambda_{\min}^{\star}},\frac{1}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))}\right)\left\|\hat{\phi}-\phi^{\star}\right\|.$

By the lower bound (39), and the choice of $\lambda$ in 5 in Algorithm 1, it
follows that

$\displaystyle\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi}^{-})\left(\operatorname{\mathbb{E}}_{\pi_{\mathsf{mix}},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})\right]\right)^{-1}\right)$
(43) $\displaystyle\leq
2\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi}^{-})\left(\operatorname{\mathbb{E}}_{\pi_{\mathsf{mix}},\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})\right]+\lambda
I\right)^{-1}\right)$ $\displaystyle\leq
4\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi}^{-})\left(\frac{1}{(\tilde{K}+1)\tilde{N}}\sum_{t,n=1}^{T,((\tilde{K}+1)\times\tilde{N})}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})^{\top}Df(X_{t}^{1},U_{t}^{1};\hat{\phi}^{-})+\lambda
I\right)^{-1}\right)$ ((37) combined with (38)) $\displaystyle\leq
4\min_{\pi\in\Pi_{\exp}}\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi}^{-})\left(\operatorname{\mathbb{E}}_{\pi,\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})\right]+\lambda
I\right)^{-1}\right)$
$\displaystyle+\frac{8T^{4}L_{f}^{8}\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}\log(\tilde{K}+1)}{\lambda^{3}(\tilde{K}+1)}+\frac{4TL_{f}^{2}\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}}{\lambda^{2}}\left(\frac{T^{2}L_{f}D(\mathcal{B})}{\sigma_{w}}+\sqrt{\frac{8\log\frac{\tilde{K}}{2\delta}}{\tilde{N}}}\right)$
( (33) holds under the event $\mathcal{E}_{\mathsf{doed}}$)

By invoking Lemma C.5,

$\displaystyle\min_{\pi\in\Pi_{\exp}}\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi}^{-})\left(\operatorname{\mathbb{E}}_{\pi,\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})\right]+\lambda
I\right)^{-1}\right)$ (44)
$\displaystyle\leq\min_{\pi\in\Pi_{\exp}}\operatorname{\mathrm{tr}}\left(\mathcal{H}(\hat{\phi}^{-})\left(\operatorname{\mathbb{E}}_{\pi,\phi^{\star}}\left[\sum_{t=1}^{T}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})^{\top}Df(X_{t}^{n},U_{t}^{n};\hat{\phi}^{-})\right]\right)^{-1}\right)\quad\mbox{(regularization
decreases optimal value)}$
$\displaystyle\leq\min_{\pi\in\Pi_{\exp}}\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}\right)^{-1}\right)+\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},T,d_{\mathsf{X}},d_{\phi},\left\|\mathcal{H}(\phi^{\star})\right\|,\frac{1}{\lambda_{\min}^{\star}},\frac{1}{\lambda_{\min}(\mathcal{H}(\phi^{\star})}\right)\left\|\hat{\phi}-\phi^{\star}\right\|,$

where the final inequality follows from Lemma C.5. Combining (42) with (43)
and (44) and using the bound
$\left\|\hat{\phi}^{-}-\phi^{\star}\right\|\leq\frac{R}{N^{\alpha}}$ yields

$\displaystyle\operatorname{\mathrm{tr}}(\mathcal{H}(\phi^{\star})(\Sigma^{\pi_{\mathsf{mix}}})^{-1})$
$\displaystyle\leq
4\min_{\pi\in\Pi_{\exp}}\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}\right)^{-1}\right)$
(45)
$\displaystyle+\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},\sigma_{w},T,d_{\mathsf{X}},d_{\phi},\left\|\mathcal{H}(\phi^{\star})\right\|,\frac{1}{\lambda_{\min}^{\star}},\frac{1}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))}\right)\frac{R}{N^{\alpha}}$
$\displaystyle+\frac{8T^{4}L_{f}^{8}\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}\log(\tilde{K}+1)}{\lambda^{3}(\tilde{K}+1)}+\frac{4TL_{f}^{2}\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}}{\lambda^{2}}\left(\frac{T^{2}L_{f}D(\mathcal{B})}{\sigma_{w}}+\sqrt{\frac{8\log\frac{\tilde{K}}{2\delta}}{\tilde{N}}}\right)$

As a result of the above identfiication bounds, the excess cost of the
returned policy $\pi^{\star}(\hat{\phi}^{+})$ is bounded via Footnote 3 as

$\displaystyle\mathcal{J}(\phi^{+})-\mathcal{J}(\phi^{\star})$
$\displaystyle\leq\left\|\hat{\phi}^{+}-\phi^{\star}\right\|^{2}_{\mathcal{H}(\phi^{\star})}+C_{\mathsf{cost}}\left\|\hat{\phi}^{+}-\phi^{\star}\right\|^{3}$
$\displaystyle\leq\left(1+\frac{C_{\mathsf{cost}}\left\|\hat{\phi}^{+}-\phi^{\star}\right\|}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))}\right)\left\|\hat{\phi}^{+}-\phi^{\star}\right\|^{2}_{\mathcal{H}(\phi^{\star})}$
$\displaystyle\leq\left(1+\frac{2C_{\mathsf{cost}}R}{N^{\alpha}\lambda_{\min}(\mathcal{H}(\phi^{\star}))}\right)\frac{160\sigma_{w}^{2}\log\frac{92d_{\phi}}{\delta}}{N}\operatorname{\mathrm{tr}}(\mathcal{H}(\phi^{\star})(\Sigma^{\pi_{\mathsf{mix}}})^{-1})\quad\mbox{(\eqref{eq:
fine identification bound} and event $\mathcal{E}_{\mathsf{coarse}}$)}$
$\displaystyle\leq\left(1+\frac{2C_{\mathsf{cost}}R}{N^{\alpha}\lambda_{\min}(\mathcal{H}(\phi^{\star}))}\right)\frac{160\sigma_{w}^{2}\log\frac{92d_{\phi}}{\delta}}{N}\Bigg{(}4\min_{\pi\in\Pi_{\exp}}\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}\right)^{-1}\right)$
$\displaystyle+\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},\sigma_{w},T,d_{\mathsf{X}},d_{\phi},\left\|\mathcal{H}(\phi^{\star})\right\|,\frac{1}{\lambda_{\min}^{\star}},\frac{1}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))}\right)\frac{R}{N^{\alpha}}$
$\displaystyle+\frac{8T^{4}L_{f}^{8}\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}\log(\tilde{K}+1)}{\lambda^{3}(\tilde{K}+1)}+\frac{4TL_{f}^{2}\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}}{\lambda^{2}}\left(\frac{T^{2}L_{f}D(\mathcal{B})}{\sigma_{w}}+\sqrt{\frac{8\log\frac{\tilde{K}}{2\delta}}{\tilde{N}}}\right)\Bigg{)}.$

To conclude, we leverage the burn-in condition

$\displaystyle\frac{N}{4}\geq\max\Bigg{\\{}\Bigg{\lceil}\left(\frac{4T^{4}L_{f}^{8}\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}\log(N+1)\lambda^{-3}+2TL_{f}^{2}\left\|\mathcal{H}(\phi^{\star})\right\|_{\mathsf{op}}\lambda^{-2}\left(2T^{2}L_{f}R\sigma_{w}^{-1}+\sqrt{8\log\frac{2N}{\delta}}\right)+C_{\mathsf{H\,pert}}R}{4\min_{\pi\in\Pi_{\exp}}\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}\right)^{-1}\right)}\right)\bigg{\rceil}^{3/\alpha},$
$\displaystyle\left(2\frac{C_{\mathsf{cost}}R}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))}\right)^{1/\alpha}\Bigg{\\}},$

where

$\displaystyle
C_{\mathsf{H\,pert}}=\mathsf{poly}\left(L_{\pi^{\star}},L_{f},L_{\theta},L_{\mathsf{cost}},\sigma_{w}^{-1},\sigma_{w},T,d_{\mathsf{X}},d_{\phi},\left\|\mathcal{H}(\phi^{\star})\right\|,\frac{1}{\lambda_{\min}^{\star}},\frac{1}{\lambda_{\min}(\mathcal{H}(\phi^{\star}))}\right)$

Under this condition, the excess cost bound simplifies to

$\displaystyle\mathcal{J}(\phi^{+})-\mathcal{J}(\phi^{\star})$
$\displaystyle\leq\frac{2560\sigma_{w}^{2}\log\frac{92d_{\phi}}{\delta}}{N}\min_{\pi\in\Pi_{\exp}}\operatorname{\mathrm{tr}}\left(\mathcal{H}(\phi^{\star})\left(\Sigma^{\pi}\right)^{-1}\right).$

∎

##  Appendix E Aditional Experiments and Experiment Details

In Section E.1, we complement our illustrative example with an implementation
of ALCOI on a toy physical system. In Section E.2 we provide further
experimental details.

###  E.1 Cartpole Experiments

We additionally run our proposed algorithm on the cartpole system defined by
the dynamics

$\displaystyle(M+m)(\ddot{p}+b_{p}\dot{p})+m\ell\cos(\theta)(\ddot{\theta}+b_{\theta}\dot{\theta})$
$\displaystyle=m\ell\dot{\theta}^{2}\sin(\theta)+u$ $\displaystyle
m\cos(\theta)(\ddot{p}+b_{p}\dot{p})+m\ell(\ddot{\theta}+b_{\theta}\dot{\theta})$
$\displaystyle=mg\sin(\theta),$

where $p$ is the position of the cart, $\theta$ is the angle of the pole from
the upright position, and $u$ is the control input. Here, $M$ represents the
mass of the cart, $m$ the mass of the pole, $\ell$ the length of the pole, $g$
the acceleration due to gravity, $b_{x}$ is the friction coefficient for the
cart, and $b_{\theta}$ is the friction coefficient for the pole. We discretize
the system using the Euler approach using a timestep of $dt=0.1$. We simluate
the discretized system under additive zero mean Gaussian noise with covariance
$0.01I_{4}$. The unknown parameters are
$\phi_{\star}=\begin{bmatrix}M,m,\ell,g,b_{x},b_{\theta}\end{bmatrix}=[1,0.1,1,10,0.5,0.5]$
For every episode, the system starts from the hanging position with state
$\begin{bmatrix}0&0&\pi&0\end{bmatrix}$. The desired behavior is to swing the
pole to the upright position with the cart positioned at the origin within the
time horizon of $T=30$ timesteps. This is described by the cost functions
$c_{t}(x,u)=0$, $c_{T+1}(x)=\left\|x\right\|^{2}$.

To simplify the policy synthesis, rather than solving for the certainty
equivalent policy which directly minimize the cost, we deploy energy shaping
controllers which switch to an LQR controller about the upright position
(Tedrake, 2009). In particular, the certainty equivalent controller under
estimates
$\hat{\phi}=\begin{bmatrix}\hat{M},\hat{m},\hat{\ell},\hat{g},\hat{b}_{x},\hat{b}_{\theta}\end{bmatrix}$
selects the input as follows. If ${\left|\theta\right|}<\pi/4$, set $u$
according to the LQR controller synthesized by linearizing the dynamics under
the estimated parameters about the upright equilibrium point with weights
$Q=I$, $R=1$. Otherwise, set

$\displaystyle u$
$\displaystyle=(M+m-m\cos(\theta))\ddot{x}_{d}+mg\cos(\theta)\sin(\theta)-m\ell\dot{\theta}^{2}\sin(\theta)$
(46) $\displaystyle\ddot{x}_{d}$
$\displaystyle=5\dot{\theta}\cos(\theta)\left(\frac{1}{2}m\ell^{2}\dot{\theta}^{2}2+mg\ell\cos(\theta)-mg\ell\right)-0.01p-0.01\dot{p}.$
(47)

During exploration, the input energy budget is restricted to $0.1T$.

The comparison between the three approaches, random exploration, approximate
$A$-optimal exploration, and ALCOI is shown in Figure 3. As in the
illustrative example, we see the benefit of control-oriented exploration over
both random exploration.

Figure 3: Excess cost versus number of exploration episodes for the proposed
control-oriented identification procedure, approximate $A$-optimal design, and
random exploration for the cartpole swing-up task. The mean over $100$ runs is
shown, and the standard error is shaded.

###  E.2 Implementation Details

For both the cartpole experiments and the illustrative example, the initial
exploration policy used by ALCOI and approximate $A$-optimal exploration is
set to the random exploration policy.

We deviate from the proposed algorithm in the following respects.

  * •

Rather than running the initial policy for $\lfloor N/4\rfloor$ episodes and
the algorithm $\texttt{DOED}_{+}$ with a budget of $\lfloor N/4\rfloor$ before
running the mixture policy for $\lfloor N/4\rfloor$ episodes, we run the
initial policy for $\lfloor N/2\rfloor$ episodes and the algorithm
$\texttt{DOED}_{+}$ with a budget of $\lfloor N/2\rfloor$ episodes, and use
all of the collected data to fit the refined estimate.

  * •

We treat the regularization parameter $\lambda$ as a hyperparameter, which we
fix to $0.001$ in all experiments, rather than setting it according to 5 from
Algorithm 1.

  * •

The approach of optimism in the face of uncertainty from 8 of Algorithm 2 is
replace by certainty equivalent synthesis of the exploration policy. As the
policy class in both experiments consists of all trajectory dependent policies
with bounded input energy, we approximate this certainty equivalent synthesis
via a receding horizon control procedure. In particular, at time step $t$ if
the current state is $x_{t}$, the learner solves the problem:

$\displaystyle\underset{\tilde{x}_{t},\dots,\tilde{x}_{T+1},\tilde{u}_{t},\dots,\tilde{u}_{T}}{\mathsf{minimize:}}$
$\displaystyle\quad\sum_{k=t}^{T}c(\tilde{x}_{k},\tilde{u}_{k})$
$\displaystyle\mathsf{s.t.:}$
$\displaystyle\quad\tilde{x}_{k+1}=f(\tilde{x}_{k},\tilde{u}_{k};\hat{\phi}),\quad\tilde{x}_{t}=x_{t}\quad\sum_{k=1}^{t}\left\|u_{k}\right\|^{2}+\sum_{k=t+1}^{T}\left\|\tilde{u}_{k}\right\|^{2}\leq\mathsf{total\,budget},$

where $\hat{\phi}$ is the learner’s current esimate of the system parameters
and $(\tilde{x}_{k}$, $\tilde{u}_{k})$ constitute the planned trajectory. The
learner plays the input $\tilde{u}_{t}$, on the true system, and re-evaluates
the optimal control after observing the next state. The solution to the
optimization problem defining the above receding horizon controller is
computed via random shooting with a warm start. In particular, before applying
each control input, $100$ new control sequences are generated from a standard
normal distribution, and are normalized to satisfy the energy constraints. The
seqeuence which minimizes the objective is then selected. This procedure is
warm-started with the solution at the previous timestep by setting that as an
additional candidate solution.

  * •

The certainty equivalent policies intended to minimize the objective
$\mathcal{J}(\pi,\phi^{\star})$ are replaced with feedback linearization
policies defined in terms of the parameter estimate. In particular, for the
illustrative example and cartpole example, the policies are synthesized as

    1. 1.

Illustrative example:
$\hat{\pi}(\hat{\phi})=K\left(X_{t}-\begin{bmatrix}-5.5\\\
0\end{bmatrix}\right)-\sum_{i=1}^{4}\psi(X_{t}-\hat{\phi}^{(i)})$, where $K$
is the LQR controller synthesized via a linearization of the dynamics under
the parameter $\hat{\phi}$.

    2. 2.

Cartpole: the swing up controller is defined using feedback linearization as
in (46).

Note that a consequence of this is that there may exist policies in the policy
class $\Pi_{\star}$ which achieve lower cost than the policy defined in terms
of the optimal parameter $\phi_{\star}$.

The first modification is made because discarding the half of the data is
wasteful. It is included in the algorithm only in order to simplify the
analysis, as the constant factor of $2$ is not a concern of this analysis. The
second modification is made to avoid requiring knowledge of the parameter
$\lambda_{\min}^{\star}$ by the algorithm. The third modification is made to
avoid the computational intractability of OFU. The sampling based approach is
used since the optimization problem defining the receding horizon controller
is intractable. The final modification is made to avoid the computational
burden of solving the policy optimization problem via a Bellman equation in
both policy synthesis and in computation of the model-task Hessian. These
modifications result in a gap between the theoretical guarantees, and the
numerical validation. Future work could reduce this gap, e.g. by removing the
approach of optimism in the face of uncertainty from the analysis.