# Entry and leaving arcs of turnpikes: their exact computation in the calculus
of variations

L. Bayón, P. Fortuny Ayuso, J.M. Grau, M.M. Ruiz  All authors’ address: Dpto.
de Matemáticas, Universidad de Oviedo. Spain<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

We settle the question of how to compute the entry and leaving arcs for
turnpikes in autonomous variational problems, in the one-dimensional case
using the phase space of the vector field associated to the Euler equation,
and the initial/final and/or the transversality condition. The results hinge
on the realization that extremals are the contours of a well-known function
and that that the transversality condition is (generically) a curve. An
approximation algorithm is presented and an example included for completeness.

###### 1991 Mathematics Subject Classification:

49J15, 49M05, 49M99

##  1\. Introduction

The idea of _turnpike_ in the Calculus of Variations or in Optimal Control
describes the (usual) phenomenon which takes place when, in problems with
finite but arbitrarily large time, the optimal solutions _spend most of their
time_ near a specific point. Even more, these solutions tend to be composed of
three parts: an entry arc, the turnpike arc, and the leaving arc. The first
and last ones are _transitory_ arcs which take little time of the solution,
whereas the middle arc (the turnpike) is a long arc which is essentially
stationary, and tends to be exponentially near an equilibrium (see [1]).
Roughly speaking, in the long term, approximate solutions to problems having a
turnpike are determined essentially by the integrand function of the objective
functional, and are —again, essentially— independent of their endpoints and
time interval.

Although the first works on the topic investigated specific problems arising
in the context of economics and econometrics [2], [3], today the turnpike
property has become of interest in other areas [4, 5]. Recent studies have
proposed its use in applications varying from membrane-filtration systems [6]
to control of chemical reactors with uncertain models [7] or shape
optimization in aircraft design [8].

The property has also been noticed in Optimal Control Problems of almost any
type: with/without terminal constraints [1], [9]; with/without discounted cost
functionals [10], [11]; discrete-time problems with constraints [5], [12]; and
in a continuous-time problems without constraints [1]… Of course, no work on
the turnpike property can omit referencing Zaslavski’s exhaustive studies,
whose results and complete references are collected in [13, 14].

From a practical point of view, the interest of the turnpike phenomenon arises
from the fact that under this condition, the computation of (approximate)
optimal trajectories in all areas of optimal control and variational problems
becomes trivial for long enough time spans. In this sense, one of the first
applications is [15], where a time-invariant linear quadratic optimal control
problem is studied. They prove that the optimal trajectory is approximately
composed of two solutions of two infinite-horizon optimal control problems.
With $x(0)$ fixed, the solution for the interval $(0,+\infty)$ defines the
part of the trajectory for the original problem from $x=0$ to the turnpike.
With $x(T)$ fixed, the solution for the interval $(-\infty,T)$ defines the
part of the trajectory of the original problem from the turnpike to $t=T$. The
two parts are then pieced together and exhibit a similar transient behavior.
Their approach is elementary and points out very clearly that the
hyperbolicity phenomenon is the heart of the turnpike property.

Recently, in [1], the authors investigate the relation between the turnpike
propierty and numerical methods (direct and indirect) for a general nonlinear
optimal control problem, without any specific assumption, and for very general
terminal conditions. In the context of turnpike theorem, they provide a new
method to ensure the successful initialization of numerical methods. Assuming
that the Pontryagin maximum principle has been applied, the usual shooting
method can be used. However, this is in general very hard to initialize. As a
solution, they propose a variant: as the extremal is approximately known along
the interval $[\varepsilon,T-\varepsilon]$, for some $\varepsilon>0$ (i.e. the
turnpike), but it not at the endpoints $t=0$ and $t=T,$ the idea is choose
some arbitrary point of $[\varepsilon,T-\varepsilon]$ (for instance $t=T/2$),
and then integrate backwards over $[0,T/2]$ to get an approximation to $x(0)$,
and forward over $[T/2,T]$ to get the approximation to $x(T)$. The unknown
value of $x(T/2)$ must be adjusted, for instance, through a Newton method, so
that transversality conditions are satisfied.

Even more recently, in the same spirit, in [16] the authors use the turnpike
property in the numerical computation of optimal trajectories, splitting the
optimization horizon at the turnpike. They proceed as follows: given the
turnpike equilibrium $x^{e}$, the optimization horizon $T>0$, an initial
condition $x(0)$ and, a terminal condition $x(T)$, they compute an optimal
trajectory $x_{1}(\cdot)$ with finite horizon $T_{1}<T$ and initial and
terminal conditions $x_{1}(0)=x(0)$, $x_{1}(T_{1})=x^{e}$; and an optimal
trajectory $x_{2}(\cdot)$ with horizon $T_{2}<T-T_{1}$ and initial and
terminal conditions $x_{2}(0)=x^{e}$, $x_{2}(T_{2})=x(T)$. Finally, an
approximation of the optimal trajectory is then obtained by concatenating the
three arcs: $x_{1}(t),$ $t\in[0,T_{1}];$ $x^{e},$ $t\in[T_{1},T-T_{2}]$ and
$x_{2}(t-T+T_{2}),$ $t\in[T-T_{2},T].$ The resulting error can be estimated if
the speed of convergence towards the turnpike is known (as in the case of
exponential turnpike). They also use a second approach via Model Predictive
Control (MPC) which may have has some advantages.

To illustrate the turnpike and their methods, they consider a well known
harvest example [17], with both bilinear and quadratic objective. Remarkably
enough, the authors do not seem to notice that in the free-endpoint case, the
leaving arc ends always at the same value of $x(T)$, regardless of $x(0)$ and
$T$. Something similar happens in [18]: the author, who studies two examples
of optimal investment problems, states literally: “Without any terminal
constraints all predictions end in $x=2$”, but does not delve into this
happening. We shall see that this is a general property of turnpikes with
free-endpoint solutions.

As a matter of fact, one of us had already noticed this in the previous paper
[19]. There, a model of renewable resource exploitation in an open-access
fishery [20], more detailed and general than [17], is studied. It was noticed
that, without constraints on the terminal state (which force the trajectory to
leave the turnpike), the solution spontaneously leaves the turnpike in order
to reduce the cost of the overall trajectory, and the leaving arc always ends
at the same value of $x(T)$, for all $T$.

In this note, we intend to settle the question of the entry and leaving arcs
of the turnpike in the generic hyperbolic situation for _variational
problems_. The key point was suggested in [19] but not led to its natural
consequence there. In short, and loosely speaking, our statement can be
summarized as follows (for autonomous problems in $\mathbb{R}$): assume
$P=(x_{P},\dot{x}_{P})$ is a turnpike for a problem with initial condition
$x_{0}$ and free terminal condition, and let $T(x,\dot{x})=0$ denote the
equation giving the transversality condition. Then:

Statement: There is a function $C(x,\dot{x})$ such that:

  * •

The entry arc of the turnpike starts at

$Q_{e}=\left\\{x=x_{0}\right\\}\cap\left\\{C(x,\dot{x})=C(x_{P},\dot{x}_{P})\right\\}.$

  * •

The leaving arc of the turnpike ends at

$Q_{l}=\left\\{T(x,\dot{x})=0\right\\}\cap\left\\{C(x,\dot{x})=C(x_{P},\dot{x}_{P})\right\\}.$

The function $C(x,\dot{x})$ is well-known to any practitioner: it is the
function whose level sets are the extremals [21]. Certainly, the statement
needs to be properly formalized, but its spirit should be clear to anyone
familiar with the turnpike property. It is also more general (the problem may
have both endpoints fixed, or none).

The main tool in our argument is to study the phase space of the plane vector
field equivalent to the Euler equation associated to the variational problem.
This vector field has very nice properties (among other things, its
trajectories are both the extremals of the problem and the level sets of
$C(x,\dot{x})$) and a direct application of the classical results on ordinary
differential equations is enough to prove the statement.

The consequences of that result are straightforward: in order to determine the
entry and leaving arcs, one only needs to know the intersection points between
$C(x,\dot{x})$, the transversality condition and/or the initial and final
conditions (if any). Once these points are known, the entry arc can be
computed by forward integration, and the leaving arc by backwards integration,
as the question has become an initial value problem at this point.

We hope this work provides a useful support for the study of long-term
autonomous variational (and possibly control) problems near a steady state.

Our results are all straightforward consequences of the standard results on
continuous dependence on parameters of solutions of ordinary differential
equations, as well as the local structure of hyperbolic singularities. Despite
this fact, we dedicate Section 3 to a thorough description of the geometric
setting of the problem, with the aim of helping the reader understand the
situation. We hope this is clearer, briefer and simpler than a technical proof
which would provide no insight and would be no more informative than what we
provide.

After the formal statements in Section 4 and a suggestion for an approximate
algorithm, we dedicate Section 5 to a hopefully illustrative example, Section
6 to numerical computations in it. A final section provides some remarks on
the $n$-dimensional case.

##  2\. Statement of the problem

Consider the autonomous variational problem in one dimension:

(1)
$\mathcal{P}\equiv\left\\{\begin{array}[]{l}\displaystyle\min\int_{0}^{T}F(x(t),\dot{x}(t))\,dt\\\\[10.00002pt]
x(0)=x_{0}\end{array}\right.$

where $F$ is a $\mathcal{C}^{2}(\mathbb{R}^{2})$ function, and $T$ is large
enough. It is well known since Samuelson [2] that many of these problems have
a _turnpike_ : a value $x_{P}$ such that “most” solutions of (1) pass near it
during a long period (i.s. $x(t)\simeq x_{P}$ and $\dot{x}(t)\simeq 0$ for a
“large” inner subinterval of $[0,T]$), for $T\rightarrow\infty$. Moreover, as
Zaslavsky has proved [22], there are also initial and final curves (the
_entry_ and _leaving_ arcs) $\gamma_{e}$ and $\gamma_{l}$ such that, as
$T\rightarrow\infty$, any solution of that problem is very near $\gamma_{e}$
at the beginning, then near $x_{P}$, and finally, it is near $\gamma_{l}$ in
the end. Of course, all the terms between quotation marks can be properly
defined [10].

However, despite all the results around turnpikes, and as we have remarked in
the Introduction, there is still no programmatic way to find their entry and
exit arcs. The aim of this work is to explicitly show which curves these arcs
are and how to compute them in the generic case.

##  3\. Extremal curves, level sets and hyperbolic saddles

Given problem (1), Euler’s equation

(2) $\frac{\partial F(x(t),\dot{x}(t))}{\partial
x}-\frac{d}{dt}\left(\frac{\partial
F(x(t),\dot{x}(t))}{\partial\dot{x}}\right)=0$

is best rewritten, after simplifying a common factor $u$, for our purposes, as
a vector field, using $x$ and $u$ as subindices to indicate partial
differentiation with respect to the first and second variables:

(3) $\mathcal{E}\equiv\left\\{\begin{array}[]{l}\dot{x}=u\\\\[3.0pt]
\displaystyle\dot{u}=\frac{F_{x}-uF_{xu}}{F_{uu}}\end{array}\right.$

This vector field might have singularities where $F_{uu}=0$ (this implies, for
instance, that if the problem is strictly convex in $u$, then there are no
such singularities). The extremal curves (solutions to Euler’s equation) then
coincide with the trajectories of $\mathcal{E}$. Moreover, as the problem is
autonomous, it is well known (see, for instance [21]) that the following
function

(4) $C(x,u)=F(x,u)-uF_{u}(x,u)$

is constant in the extremals. Thus, _extremals, as $1$-dimensional manifolds,
are the level sets of $C(x,u)$_ in $\mathbb{R}^{2}$, for the problem
$\mathcal{P}$.

Let us work away from the points where $F_{uu}=0$, that is, we limit ourselves
to an open set $W$ where $F_{uu}(x,u)\neq 0$. Consider an equilibrium point
$P$ of $\mathcal{E}$, that is: a point with $u=0,F_{x}-uF_{xu}=0$. The linear
part of $\mathcal{E}$ is always of the form

$L=\begin{pmatrix}0&\star_{1}\\\ 1&\star_{2}\end{pmatrix}$

where the stars are unknown values. Except in degenerate cases, $P$ is then
either a center-focus (when both eigenvalues of $L$ are complex), a node (both
eigenvalues of $L$ are real and have the same sign) or a hyperbolic saddle
(real eigenvalues with different sign). Obviously, the nature of either
center-foci or nodes prevents such a point from being a turnpike with entry
and leaving arcs: if $P$ is a center, trajectories turn around it, if it is a
focus, then they either converge to it (so that $P$ is not strictly speaking a
turnpike) or move away from it (again, not a turnpike). For the same reasons
as for foci, nodes cannot be turnpikes. Hence, turnpikes with entry and
leaving arcs correspond, in the non-degenerate case, to hyperbolic saddles, as
is well known.

Assume then that $P=(x_{P},u_{P})$ is a hyperbolic saddle of $\mathcal{E}$,
which by definition will have $u_{P}=0$ (this is exactly what makes $P$ a
turnpike: near $P$, the velocity of $\mathcal{E}$ tends to $0$ and extremals
spend “a long time” near $P$). On the other hand, we have
$F_{x}(x_{P},u_{P})-u_{P}F_{xu}(x_{p},u_{P})=0$, which becomes at $P$ just
$F_{x}(x_{P},0)=0$. As $P$ is a hyperbolic saddle, there are two invariant
manifolds adherent to $P$: the stable $X_{s}$ and unstable $X_{u}$ ones,
meeting transversely at $P$ (see Figure 1: the stable manifold “falls” towards
$P$ and the unstable one “goes away” from it). As these manifolds are unions
of extremal curves (they are trajectories of $\mathcal{E}$), they correspond
also to level curves of $C(x,u)$ and, as $P$ belongs to both, if we denote by
$M=X_{s}\cup X_{u}$ their union, must necessarily have:

$M\equiv C(x,u)=C(x_{P},0).$

That is, the invariant set near $P$ is given by $C(x,u)=C(x_{P},0)$.

$V$$U_{11}$$U_{21}$$U_{22}$$U_{12}$$x$$u$$P$$\gamma_{s}^{1}$$\gamma_{s}^{2}$$\gamma_{u}^{2}$$\gamma_{u}^{1}$
Figure 1. Hyperbolic saddle $P$ and the open sets $U_{ij}$.

Near $P$, the set $M$ can be divided into $4$ different trajectories of
$\mathcal{E}$: $\gamma_{s}^{1},\gamma_{s}^{2}$, which are the two components
of $X_{s}\setminus\left\\{P\right\\}$ and $\gamma_{u}^{1}$, $\gamma_{u}^{2}$
for $X_{u}\setminus\left\\{P\right\\}$. Any connected open set $V$ containing
$P$ with sufficiently smooth border is divided by these four curves into four
open subsets: $U_{11}$, $U_{12}$, $U_{21}$, $U_{22}$, each $U_{ij}$
corresponding to the “angle” defined by $\gamma_{s}^{i}$ and $\gamma_{u}^{j}$,
in that order (See Figure 1).

The solutions of the variational problem $\mathcal{P}$ are extremals (so, they
correspond to trajectories of $\mathcal{E}$) which verify the initial
condition $x(0)=x_{0}$ and also satisfy the transversality condition
$F_{u}(x(T),u(T))=0$. The equation given by the trasnversality condition
$F_{u}(x,u)=0$ defines (usually) a curve in the $(x,u)$-plane. Figure 2 shows
the “general” situation in which we find ourselves. The trajectory
$\gamma_{e}$, part of the stable manifold, and $\gamma_{l}$, part of the
unstable one, are the entry and leaving arcs, respectively.

$x$$u$$x=x_{0}$$\gamma$$P$$F_{u}(x,u)=0$$x_{l}$$x(T)$$\leftarrow$$\gamma_{e}$$\gamma_{l}$$Q_{l}$$Q_{e}$
Figure 2. Hyperbolic saddle $P$ (turnpike), extremal ($\gamma$), and entry
($\gamma_{e}$) and leaving ($\gamma_{l}$) arcs. In yellow, the “slow” zone. As
long as there are no singularities of $\mathcal{E}$ in the cyan zone, the
turnpike property holds inside it, and as $T\rightarrow\infty$, the
corresponding extremal of $\mathcal{P}$ approaches $\gamma_{e}$ at the
beginning and $\gamma_{l}$ at the end. The entry arc starts at $Q_{e}$ and the
leaving arc ends at $Q_{l}$.

The intersection points between the transversality condition $F_{u}(x,u)=0$
and $M$ are, key in our statements. These points are the solutions of the
system of equations:

$\left\\{\begin{array}[]{l}F(x,u)-uF_{u}(x,u)=C(x_{P},0)\\\
F_{u}(x,u)=0\end{array}\right.$

which, after simplifying, becomes (see [19], where this system of equations
appeared for the first time):

(5) $\left\\{\begin{array}[]{l}F(x,u)=C(x_{P},0)\\\
F_{u}(x,u)=0\end{array}\right.$

In the problem $\mathcal{P}$, the initial value $x(0)=x_{0}$ is set. Assume
$Q_{e}=(x_{0},u_{e})$ belongs to $x=x_{0}\cap\\{C(x,u)=C(x_{P},0)\\}$ and to
the stable manifold of $P$, and let $Q=(x_{l},u_{l})$ be the solution of (5)
in the unstable manifold (as in Figure 2) which is nearest to $P$. Under these
assumptions, the Turnpike property happens relative to $P$ (as in Figure 2)
and extremals with $x(0)=x_{0}$ start near $Q_{e}=(x_{0},u_{e})$, so that
$u(0)\rightarrow u_{e}$, and finish near $Q_{l}$, so that and $x(T)\rightarrow
x_{l}$ as $T\rightarrow\infty$.

If there existed a solution $R=(x_{e},u_{e})$ of (5), belonging to the stable
manifold and satisfying the transversality condition (this case is not plotted
in Figure 2), a dual argument using $f(x,-\dot{x})$ shows that there are
extremals of the variational problem with no initial or terminal condition:

(6) $\mathcal{P}^{\prime}\equiv\min\int_{0}^{T}F(x(t),\dot{x}(t))\,dt$

with starting points $x(0)\rightarrow x_{e}$ (and endpoints ending at $Q_{l}$
as above).

This theoretical description which is just a qualitative expression of the
well-known results on the continuous dependence of solutions of ODEs on the
parameters, and of the local structure of hyperbolic singularities (see [24],
for example) is enough to prove our results, so that instead of proofs, we
just refer to this section.

Our statements have two versions: one in which a solution $\gamma$ of
$\mathcal{P}$ is already known, and one in which all depends just on the
solutions of (5).

##  4\. Statements of the results

As explained in the introduction, $F(x,u)$ is of class $\mathcal{C}^{2}$ in
$\mathbb{R}^{2}$, and all our statements are in an open set
$W\subset\mathbb{R}^{2}$ where $F_{uu}(x,u)\neq 0$. We shall make frequent
reference to the vector field $\mathcal{E}$ defined in (3).

Our first result assumes the existence of an extremal “sufficiently” near a
hyperbolic turnpike:

######  Theorem 1.

Let $P=(x_{P},u_{P})$ be a hyperbolic saddle of $\mathcal{E}$ and $\gamma$ an
extremal of $\mathcal{P}$ included in an open set $U\subset W$ containing $P$
which admits a subdivision $U_{ij}$ for $i,j\in\left\\{1,2\right\\}$ as above.
We assume the following conditions:

  1. (1)

The curve $F_{u}(x,u)=0$ meets $\gamma_{u}^{1}$ and $\gamma$ transversely at
the points $Q=(x_{l},u_{l})$ and $(x(T),u(T))$.

  2. (2)

That curve $F_{u}(x,u)=0$ admits an injective parametrization near $Q$,
$\eta:[-1,1]\rightarrow\mathbb{R}^{2}$ with $\eta(0)=Q$, $\eta(1)=(x(T),u(T))$
such that $\eta$ is transverse to any extremal meeting it.

  3. (3)

The extremals $\gamma_{s}^{1}$ and $\gamma$ meet the manifold $x=x_{0}$
transversely at $(x_{0},u_{e})$, $(x_{0},u_{0})$ respectively.

  4. (4)

The open set $V$ (delimited by $\gamma^{1}_{s}$ and $\gamma_{u}^{1}$, $\eta$
and the line $x=x_{0}$ contains no more singularities of $\mathcal{E}$.

Then:

  1. (1)

For any $\overline{T}>T$, the problem

$\mathcal{S}_{\overline{T}}\
\left\\{\begin{array}[]{l}\displaystyle\min\int_{0}^{\overline{T}}F(x(t),\dot{x}(t))\,dt\\\\[10.00002pt]
x(0)=x_{0}\end{array}\right.$

has an extremal $\gamma$ which is totally included in $V$.

  2. (2)

For any $\epsilon>0$ there is $T_{\epsilon}>T$, $T_{\epsilon,{e}}>0$ and
$T_{\epsilon,\infty}<T$, such that, for $\overline{T}>T_{\epsilon}$, the
corresponding extremal $\gamma_{\overline{T}}$ satisfies:

     * •

The metric distance between
$\gamma_{\overline{T}}:[0,T_{e}]\rightarrow\mathbb{R}^{2}$ and $\gamma_{e}$ is
less than $\epsilon$.

     * •

The metric distance between
$\gamma_{\overline{T}}:[T_{e},T_{l}]\rightarrow\mathbb{R}^{2}$ and $P$ is less
than $\epsilon$.

     * •

The metric distance between
$\gamma_{\overline{T}}:[T_{l},\overline{T}]\rightarrow\mathbb{R}^{2}$ and
$\gamma_{l}$ is less than $\epsilon$.

Moreover, one can also choose $T_{e}$ and $T_{l}$ such that
$T_{l}-T_{e}\rightarrow\infty$ as $\epsilon\rightarrow 0$ and $T_{l},T_{e}<K$
for some $K<\infty$.

###### Proof.

The first conclusion follows from the continuous dependence of solutions of an
ordinary differential equation on the parameters (and from all the qualitative
descriptions in Section 3). The second one follows also from the local
structure of hyperbolic saddle singularities, see for instance [24]. ∎

######  Definition 1.

The curve $\gamma_{e}$ from $x=x_{0}\cap\gamma_{e}$ to $P$ is called the
_entry arc_ to the turnpike $P$. The curve $\gamma_{l}$ from $P$ to $Q$ is
called the _leaving arc_ of the turnpike $P$.

The previous statement seems to require a lot from the equation. As it
happens, most of the hypotheses are just technical and will hold in generic
situations. On the other hand, we can also say a lot (locally) if we just know
that the transversality condition meets $\gamma_{l}$ transversely. The
following result is again a straightforward consequence of the local structure
of hyperbolic saddles and the continuous dependence on the parameters of
solutions of ODEs. All the statements are inside $W\subset F_{uu}(x,u)\neq 0$.

######  Theorem 2.

Let $P$ be a hyperbolic saddle and
$\gamma_{l}\subset\gamma_{u}^{1}\cup\gamma_{u}^{2}$ one of the components of
the unstable manifold. Assume that the transversality condition $Tr\equiv
F_{u}(x,u)=0$ meets $\gamma_{l}$ transversely at $Q_{l}=(x_{l},u_{l})$ and
that there are no more intersection points between $Q_{l}$ and $P$ belonging
to $\gamma_{l}$. Then there is $\epsilon>0$ and a parametrization
$\tau:[-\epsilon,\epsilon]\rightarrow T_{c}$ with $\tau(0)=Q_{l}$ such that
the two extremals containing $\tau(-\epsilon)$ and $\tau(\epsilon)$ satisfy
all the properties of Theorem 1. As a consequence, $P$ is a turnpike for
$\mathcal{P}$. Moreover, assume $\gamma_{s}^{1}$ is to the right of $P$ and
$\gamma_{s}^{2}$ is to its left. Assume, for simplicity, that $Q_{l}$ is to
the left of $P$ (i.e. $x_{l}<x_{P}$). Then:

  1. (1)

The point $P$ is a turnpike for $\mathcal{P}$ for
$x_{0}\in(x_{P},x_{P}+\epsilon)$, for some $\epsilon>0$, $\gamma_{l}$ (from
$P$ to $Q$) is the leaving arc and $\gamma_{s}^{1}$ (from $x=x_{0}$ to $P$) is
the entry arc.

  2. (2)

At the same time, $P$ is a turnpike for $\mathcal{P}$ for
$x_{0}\in(x_{P}-\epsilon,x_{P})$ for some $\epsilon>0$, $\gamma_{l}$ (from $P$
to $Q$) is the leaving arc and $\gamma_{s}^{2}$ (from $x=x_{0}$ to $P$) is the
entry arc.

Notice how there is a switch of entry arcs when the initial condition $x_{0}$
changes from being “greater than $x_{P}$” to “less than $x_{P}$”. This is easy
to see in Figure 2: if $x_{0}$ is less than $x_{P}$, the extremals ending near
$Q$ must approach, at their beginning, the top-left separatrix for
$T\rightarrow\infty$, instead of $\gamma_{e}$. Obviously, for the dual problem
(final condition set but initial condition free), it is the leaving arc that
changes.

Consider the problem (6) with no initial or final condition. Near a hyperbolic
singularity $P$ of $\mathcal{E}$ we _may_ have a turnpike result if the system
of equations (5) has two solutions near $P$. Again, everything is restricted
to some open set $W\subset F_{uu}(x,u)\neq 0$.

######  Theorem 3.

With the notations above, assume $P$ is a hyperbolic singularity of
$\mathcal{E}$. If $Q_{e}\in\gamma_{s}^{1}$ and $Q_{l}\in\gamma_{u}^{1}$ are
two solutions of (5) and there are no more solutions of (5) between $Q_{e}$
and $P$ and $P$ and $Q_{l}$, then $P$ is a turnpike for the problem (6). That
is, for $T\rightarrow\infty$, there are extremals $\gamma=(x(t),u(t))$ of (6)
satisfying:

  1. (1)

The origin tends to $Q_{e}$: $(x(0),u(0))\rightarrow Q_{e}$,

  2. (2)

The end tends to $Q_{l}$: $(x(T),u(T))\rightarrow Q_{l}$,

  3. (3)

The curve $\gamma$ approaches the part of $\gamma_{s}^{1}$ between $Q_{e}$ and
$P$ at the beginning (the entry arc),

  4. (4)

The curve $\gamma$ approaches the part of $\gamma_{u}^{1}$ between $P$ and
$Q_{l}$ at the end (the leaving arc).

###### Proof.

As previously, the proof is a straightforward application of the description
in 3, the structure of hyperbolic singularities and the continuous dependence
on parameters of solutions of ODEs. ∎

Finally, consider the problem with fixed endpoints:

(7)
$\overline{\mathcal{P}}\equiv\left\\{\begin{array}[]{l}\displaystyle\min\int_{0}^{T}F(x(t),\dot{x}(t))\,dt\\\\[10.00002pt]
x(0)=x_{0},\ x(T)=x_{T}\end{array}\right.$

In this case the statements holds regardless of the transversality condition.

######  Theorem 4.

Assume $P$ is a hyperbolic singularity of $\mathcal{E}$. Assume $x=x_{0}$
meets $\gamma_{1}^{s}$ at $Q_{e}$ and $x=x_{T}$ meets $\gamma_{1}^{u}$ at
$Q_{l}$. Then $P$ is a turnpike for the problem (7), and there is an open
neighborhood $V$ of $P$ such that, for $T\rightarrow\infty$ any extremal
$\gamma$ of (7) included in $V$ satisfies statements (1)–(4) of Theorem 3.

Notice that $Q_{e}$ and $Q_{l}$ in previous results can be easily computed
using the fact that $\gamma_{i}^{s}$ and $\gamma_{i}^{u}$ are level sets of
$C(x,u)$. Thus, if they exist, then

$Q_{e}\in\left\\{C(x,u)=C(P)\land
x=x_{0}\right\\},\,\,\,Q_{l}\in\left\\{C(x,u)=C(P)\land x=x_{T}\right\\}$

However, those solution sets might have more than one point, and one needs to
verify which (if any) can be a candidate.

###  4.1. Suggestion for an approximate algorithm

From the discussions above, the following method is suggested to use the
turnpike and the entry and leaving arcs as approximate extremals for $T\gg 0$.
Specifically, for Problem (1) with $x(0)$ fixed and $x(T)$ free:

  1. i)

State a tolerance $\epsilon>0$.

  2. ii)

Find the possible turnpike $P=(x_{P},0)$. This requires studying the phase
space of (3), its singularities and the separatrices $\gamma_{s}^{i}$ and
$\gamma_{u}^{i}$, for $i=1,2$. For this one can just use the level set
$C(x,u)=C(x_{P},0)$.

  3. iii)

Find the adequate $Q_{e}$ and $Q_{l}$. As explained above, $Q_{e}$ belongs to
$x=x_{0}$ and $C(x,u)=C(x_{P},0)$, whereas $Q_{l}$ is found using system (5).

  4. iv)

From $Q_{e}$ compute the trajectory $\gamma_{e}(t)$ of $\mathcal{E}$ with
$\gamma_{e}(0)=Q_{e}$ and ending at $|\gamma_{e}(T_{e})-x_{P}|<\epsilon$. This
is an IVP integrated until some condition is met.

  5. v)

From $Q_{l}$ compute the trajectory $\gamma_{l}(t)$ of $\mathcal{E}$ with
$\gamma_{l}(T_{l})=Q_{l}$ and $|\gamma_{l}(0)-x_{P}|<\epsilon$. This is a
backwards IVP integrated until some condition is met.

After those computations, if $T>T_{e}+T_{l}$, then any extremal $\gamma_{T}$
of (1) can be approximated by the turnpike as:

(8) $\gamma_{T}(t)\simeq\left\\{\begin{array}[]{l}\gamma_{e}(t)\ \mathrm{if}\
t\in[0,T_{e})\\\ x_{P}\ \mathrm{if}\ t\in[T_{e},T-T_{l}]\\\ \gamma_{l}(t)\
\mathrm{if}\ t\in(T-T_{l},T]\end{array}\right.$

##  5\. An example: shallow lakes

In this section we showcase the well-known shallow lakes model without
discount (see for instance [23] for the details), with a modified cost
function to prevent, in our example, the issues with the logarithm. The
problem to be solved is, initially, the Optimal Control problem with control
variable $v$:

(9)
$\mathcal{P}\equiv\left\\{\begin{array}[]{l}\displaystyle\max\int_{0}^{T}\left(v^{2}-cx^{2}\right)\,dt\\\\[5.0pt]
\dot{x}=v-bx+r\displaystyle\frac{x^{2}}{x^{2}+1}\\\
x(0)=x_{0}\end{array}\right.$

with $c,b,r$ positive constants. This is, in fact, a variational problem, as
$v$ can be expressed as a function of $x,\dot{x}$ and there are no
restrictions. Thus, we shall in fact study the variational problem

(10)
$\mathcal{P}\equiv\left\\{\begin{array}[]{l}\displaystyle\max\int_{0}^{T}F(x,\dot{x})\,dt\\\\[8.00003pt]
x(0)=x_{0}\end{array}\right.$

with

$F(x,\dot{x})=b^{2}x^{2}-\frac{2brx^{3}}{x^{2}+1}+2bx\dot{x}-cx^{2}+\frac{r^{2}x^{4}}{\left(x^{2}+1\right)^{2}}-\frac{2rx^{2}\dot{x}}{x^{2}+1}+\dot{x}^{2}$

and we shall set the value of the constants to $r=1,c=0.1$, and $b=0.7$. The
Euler equation for this problem is, once divided by $\dot{x}$:

(11)
$\frac{1}{(1+x^{2})^{3}}\bigg{(}x^{6}(-2\ddot{x}-1.4)+x^{4}(-6\ddot{x}-5.6)\bigg{.}+x^{2}(-6\ddot{x}-4.2)-\\\
\bigg{.}(2\ddot{x}+0.78x^{7}+2.34x^{5}+6.34x^{3}+0.78x\bigg{)}=0$

And the vector field associated to this second order equation is, in the
$(x,u)$ plane corresponding to $(x,\dot{x})$:

(12) $\mathcal{E}\equiv\left\\{\begin{array}[]{l}\dot{x}=u\\\
\dot{u}=\displaystyle\frac{0.39x\left(x^{6}-1.79487x^{5}+3.x^{4}-7.17949x^{3}+8.12821x^{2}-5.38462x+1\right)}{\left(x^{2}+1\right)^{3}}\end{array}\right.$

The denominator in $\dot{u}$ is never $0$, so that $\mathcal{E}$ is well-
defined in all $\mathbb{R}^{2}$. The vector field $\mathcal{E}$ has three
singular points: two hyperbolic saddles $P_{1}=(0,0)$ and
$P_{2}\simeq(1.5062,0)$ and a center/focus, $O=(0.2747,0)$. Figure 3 shows the
structure of $\mathcal{E}$ near its singularities (in red).

The function whose level sets are the extremals (the trajectories of
$\mathcal{E}$) is

$C(x,u)=x^{2}\left(\frac{x^{2}}{\left(x^{2}+1\right)^{2}}-\frac{1.4x}{x^{2}+1}+0.39\right)-u^{2}$

so that we need to focus our attention on the level sets

$L_{1}\equiv C(x,u)=C(P_{1})=0$

and

$L_{2}\equiv C(x,u)=C(P_{2})=-0.097$

Finally, the transversality condition in this case is given by

$Tr\equiv 2u+1.4x-\frac{2x^{2}}{1+x^{2}}=0.$

In Figure 4 we have plotted the sets $L_{1}$ (cyan), $L_{2}$ (yellow) and $Tr$
(black). Notice how $Tr\cap L_{1}$ is just the hyperbolic point $P_{1}$
whereas $Tr\cap L_{2}$ has two points, one above $u=0$ and the other one below
(both in green).

Figure 3. Stream lines of $\mathcal{E}$ in the example. The red dots are its
singulaties, at $u=0$, $x\in\left\\{0,0.2747,1.5062\right\\}$.

Surprisingly enough, the transversality condition (in black in Figure 4) only
meets the curve $M_{1}\equiv C(x,u)=C(P_{1})$ (in cyan) at $(0,0)$ so that our
results only apply to $P_{1}$ in the fixed-endpoints versions (because $Tr$
never meets $M_{1}$ transversely).

Consider the hyperbolic saddle $P_{2}$. We are going to showcase the four
turnpike possibilities for it under problem (10).

The transversality condition meets the (yellow) curve $M_{2}=C(x,u)=C(P_{2})$
at the (green) points $Q_{1}\simeq(-0.9852,1.1822)$ and
$Q_{2}\simeq(0.9852,-1.971)$. Clearly, the top left and bottom right parts of
$M$ are the stable manifolds, call them $\gamma_{s}^{1}$ and $\gamma_{s}^{2}$,
respectively, whereas $\gamma_{u}^{1}$ is the bottom-left part.

$Q_{2}$$Q_{1}$ Figure 4. Hyperbolic structure of the example. The leftmost
singularity is hyperbolic but its level set (blue) meets the transversality
condition only at the singularity. The level set of the rightmost singularity
(yellow) meets the transversality condition twice (at the green dots).

###  5.1. Initial condition fixed. Change of entry arc and of turnpike

Recall that we have called: $\gamma_{s}^{1}$ the top-left branch of $M_{2}$
(in yellow in Figure 4) and $\gamma_{s}^{2}$ the bottom-right branch (these
are the stable trajectories and will give rise to the entry arcs). Also,
$\gamma_{u}^{1}$ is the bottom-left branch, and $\gamma_{u}^{2}$ the top-right
one, which will give rise to the leaving arcs.

In this subsection we are going to study the problem (9) (i.e. with initial
condition but no end condition).

If $x_{0}>1.5062$, the extremals meet the transversality condition near
$Q_{2}\simeq(0.9852,-1.971)$ for $T\rightarrow\infty$, whatever the value of
$x_{0}$. The entry arc to the turnpike $P_{2}$ in this case is
$\gamma_{2}^{s}$ from $x=x_{0}$ to $P_{2}$: this happens for any
$x_{0}>1.5062$ because the transversality condition does not meet $M_{2}$ for
$x>1.5026$.

However, the moment $x_{0}$ is to the left of $P_{2}$, that is $x_{0}<1.5062$,
the entry arc to turnpike $P$ changes from $\gamma_{s}^{2}$ to
$\gamma_{s}^{1}$ (which is above $u=0$). As $x_{0}\rightarrow-0.9852$ (the
$x$-coordinate of $Q_{1}$), the extremals approach $\gamma_{s}^{1}$. The
problem with $x_{0}=-0.9852$ has no solution because $Q_{1}$ is the only
intersection point between an extremal which meets the transversality
condition (this is easily seen in the Figure 4).

Finally, for $x_{0}<-0.9852$, the candidate extremals for problem
$\mathcal{P}_{f}$ for $T\rightarrow\infty$ approach $P_{1}=(0,0)$, the
intersection point of $Tr$ and $M_{1}$ (the black and cyan lines in Figure 4).
Thus, there is an entry arc, from $x=x_{0}\cap Tr$ to $P_{1}$ but the turnpike
is never left, in this case.

###  5.2. Initial and final conditions fixed

When $x(0)=x_{0}$ and $x(T)=x_{T}$ are both fixed, the transversality
condition plays no role and one needs only study the relation between these
conditions and the hyperbolic singularities $P_{1}$ and $P_{2}$. For the sake
of simplicity, we are only going to show some cases. Let $x_{P_{1}}=0$ denote
the $x$-coordinate of $P_{1}$ and $x_{P_{2}}\simeq 1.5062$ the one of $P_{2}$.
Of course, in order to have a turnpike behavior, there must be at least one
singularity between $x_{0}$ and $x_{T}$.

  * •

When $x_{0}>x_{P_{2}}>x_{T}$, then $P_{2}$ is a turnpike and the entry arc is
$\gamma_{s}^{2}\cap\\{x=x_{0}\\}$, and the leaving arc is
$\gamma_{u}^{1}\cap\\{x=x_{T}\\}$.

  * •

On the other hand, if $x_{0}<x_{P_{2}}<x_{T}$ then the situation reverses at
$P_{2}$ (we are “above $u=0$” and the arcs are now: $\gamma_{s}^{1}$ the entry
one from $x=x_{0}$ to $P_{2}$ and $\gamma_{u}^{2}$ from $P_{2}$ to $x=x_{T}$.

  * •

If $x_{0},x_{T}\in(x_{P_{1}},x_{P_{2}})$ (that is, both endpoints are between
$P_{1}$ and $P_{2}$) it is easy to realize that $P_{1}$ is still a turnpike
and the entry and leaving paths correspond to $\gamma_{s}^{1}$ and
$\gamma_{u}^{2}$, respectively (starting at $x=x_{0}$ and ending at $x=x_{T}$,
also).

  * •

When, say $x_{0}<x_{P_{1}}$ and $x_{T}<x_{P_{2}}$, there are two candidate
extremal curves for $T\rightarrow\infty$: one having a turnpike at $P_{1}$,
and the other one at $P_{2}$; it is necessary here to discern the optimality
by other methods (which we shall not do, as this is out of our aim).
Obviously, each turnpike has his respective entry and leaving arcs (in this
case, $P_{1}$ arcs are the cyan unbounded curves to its left).

###  5.3. The free endpoint problem.

Finally, the free endpoint problem requires the extremals to meet the
transversality condition at $x(0)$ and $x(T)$. In our case, only $M_{2}$ meets
$Tr$ twice away from a singularity, whereas $M_{1}\cap Tr=\\{P_{1}\\}$. As far
as extremals go, the “constant curve” $(x(t),u(t))=P_{1}$ for all $t\in[0,T]$
is always a candidate trajectory (as it is an extremal which satisfies the
transversality conditions). These have obviously constant cost $F(P_{1})\times
T$.

There is, however, a second possibility giving rise to a true turnpike: the
solutions starting near $Q_{1}$ below $\gamma^{1}_{s}$, approaching $P_{2}$
and ending near $Q_{2}$ above $\gamma_{u}^{1}$. In this case, the entry arc is
$\gamma_{s}^{1}$ from $Q_{1}$ to $P$ and the leaving arc is $\gamma_{u}^{1}$
from $P_{1}$ to $Q_{2}$.

##  6\. Simulations

In this section we plot the simulations corresponding to some of the cases in
Section 5. We have used a budget computer (Intel Core i5 with 16Gb RAM) and
Mathematica, with no excessive time used (the simulations can be run in
several hours, the longest time taken by the very precise computation of the
entry and leaving arcs and, unfortunately, the plotting commands, as the
numerical solutions are interpolating functions and their evaluation is quite
slow). We restrict ourselves, for the sake of brevity, to the initial problem
(1) with $x(0)=x_{0}=0.5$ and $T\gg 0$.

The above requires computing the turnpike entry arc starting at the point
$P_{e}=(0.5,u_{e})$, which is the solution of the first equation in (5) with
$x=0.5$, that is, $u_{e}$ is the solution of:

(13) $F(0.5,u)=C(P_{2})=C(1.5062,0),$

giving $u_{e}\simeq 0.30751221580$. However, _one needs to compute $u_{e}$
with a huge precision in order to really obtain a fine approximation to the
turnpike._ In our computations, we used $30$ values of precision when
computing the solution of (13) (so that $P_{2}$ was also computed with that
precision).

We also need to compute the leaving arc of that turnpike, which requires
knowing the point $P_{l}=Q_{2}$, solution of (5):

(14) $\left\\{\begin{array}[]{l}F(x_{l},u_{l})=C(P_{2})\\\
Tr(x_{l},u_{l})=0\end{array}\right.$

which gives, as indicated above, $P_{l}\simeq(0.9852,-1.971)$ (with the same
caveat regarding the precision).

Figure 5 contains the plot of the extremal $x(t)$ (in blue) and its derivative
$u(t)=\dot{x}(t)$ (orange), corresponding to (1) with $x_{0}=0.5$ and $T=63$.
Overlain (in dashed liens) we have plotted the entry arc from $t=0$ to $t=24$,
and the leaving arc, from $t=41$ to $t=63$. There is no noticeable difference.

Figure 5. Turnpike entry and leaving arcs compared to solution for $T=63$.

Figure 6 shows, on the left, the difference between $x(t)$ and the entry arc
for the same $T=63$, and on the right, the difference between
$u(t)=\dot{x}(t)$ and the corresponding value on the turnpike, for $t=T-22$ to
$T$ (where $22$ is taken as a value where the $x-value$ of the leaving arc is
less than $10^{-5}$ from the true turnpike $P_{2}$). Notice that the time is
inverted in the latter plot because we have computed the leaving arc
“backwards”. The errors are, as can be seen, irrelevant to all purposes.

Figure 6. Absolute differences between entry (left) and leaving (right) arcs
and the corresponding part of the solution for $T=63$. On the right, the time
is reversed (from $Q_{l}$ to $P$).

Finally, Figure 7 contains the plots of the different solutions $x(t)$ for
times $T$ between $51$ and $56$ and for time $T=63$. The structure of the
entry arc is essentially the same for all and all are, obviously,
indistinguishable, whereas the leaving arc is also essentially equal but
starts at different times. Figure 8 shows the difference between the
corresponding entry and leaving arcs and the ones of the turnpike (where the
cutting point is set as above).

Figure 7. Solutions for times between 51 and 56, and for $T=63$.

Figure 8. Absolute difference between the solutions in Figure 7 and the entry
(left) and leaving (right) arcs (time is reversed on the right).

##  7\. Final remarks

Our aim in this paper is just to show, in the case of dimension $1$, which is
the most graphical one, how to compute the entry and leaving arcs of the
turnpike of an autonomous variational problem in order to settle this
question. Of course, the generalization to variational problems in which the
functional $F(x_{1},\dot{x}_{1},\ldots,x_{k},\dot{x}_{k})$ has “separated
variables”, that is problems with

$\frac{\partial^{2}F}{\partial u\partial v}=0$

whenever $u,v$ correspond to variables with different indices (i.e.
$u\in\left\\{x_{i},\dot{x}_{i}\right\\}$ and
$v\in\left\\{x_{j},\dot{x}_{j}\right\\}$ with $i\neq j$) is straightforward,
as the associated vector fields are defined by independent equations.

The most general autonomous case is, for the time being, inaccessible to us
but we hope the technique presented in this work may be useful to elucidate
their solution.

## References

  * [1] 7\. Trélat, E., Zuazua, E.: The turnpike property in finite-dimensional nonlinear optimal control. J. Differ. Equ. 258(1), 81–114 (2015)
  * [2] Dorfman, R., Samuelson, P. A., & Solow, R. M. Linear Programming and Economic Analysis. New York: McGraw-Hill, 1958.
  * [3] McKenzie, L. W. (1976). Turnpike theory. Econometrica: Journal of the Econometric Society, 841-865.
  * [4] Faulwasser, T., Korda, M., Jones, C.N., Bonvin, D.: Turnpike and dissipativity properties in dynamic real-time optimization and economic MPC. In: Proceedings of the 53rd IEEE Conference on Decision and Control-CDC 2014, pp. 2734–2739. Los Angeles, CA, USA (2014)
  * [5] Grüne, L., Müller, M.A.: On the relation between strict dissipativity and the turnpike property. Syst. Control Lett. 90, 45–53 (2016)
  * [6] 10\. Kalboussi, N., Rapaport, A., Bayen, T., Amar, N., Ellouze, F. & Harmand, J. (2018), Optimal control of membrane- ltration systems, IEEE Transactions on Automatic Control 64(5), 2128-2134.
  * [7] Faulwasser, T. & Pannocchia, G. (2019), Towards a unifying framework blending realtime optimization and economic model predictive control, Industrial and Chemical Engineering Research 58(30), 13583-13598.
  * [8] Lance, G., Trelat, E. & Zuazua, E. (2020), Shape turnpike for linear parabolic PDE models, Systems & Control Letters 142, 104733, 9.
  * [9] Clarke, F. (2013). Functional analysis, calculus of variations and optimal control. Springer.
  * [10] Zaslavski, A.J.: Turnpike Phenomenon and Infinite Horizon Optimal Control. Springer, Berlin (2014)
  * [11] Gurman, V. I., & Ukhin, M. Y. (2004). Turnpike solutions in optimization of regional development strategies. Automation and Remote Control. 65(4), 603–611.
  * [12] Damm, T., Grüne, L., Stieler, M., & Worthmann, K. (2014). An exponential turnpike theorem for dissipative optimal control problems. SIAM Journal on Control and Optimization. 52(3), 1935–1957.
  * [13] Zaslavski, A.J.: Turnpike Properties in the Calculus of Variations and Optimal Control. Springer, New York (2006)
  * [14] Zaslavski, A. J. (2013). Structure of solutions of variational problems. Springer Science & Business Media.
  * [15] B.D.O. Anderson, P.V. Kokotovic, Optimal control problems over large time intervals, Automatica J. IFAC 23(3) (1987) 355–363.
  * [16] Faulwasser, T., & Grüne, L. (2020). Turnpike properties in optimal control: An overview of discrete-time and continuous-time results. arXiv preprint arXiv:2011.13670.
  * [17] Cliff, E. & Vincent, T. (1973), An optimal policy for a fish harvest, Journal of Optimization Theory and Applications 12(5), 485-496.
  * [18] Grüne, L. (2021). Dissipativity and optimal control. arXiv preprint arXiv:2101.12606.
  * [19] L. Bayón, P.J. Garcia-Nieto, R. Garcia-Rubio, J.A. Otero, P.M. Suarez, C. Tasis, An algorithm for quasi-linear control problems in the economics of renewable resources: The steady state and end state for the infinite and long-term horizon. Journal of Computational and Applied Mathematics, Vol. 309, pp. 456-472, 2017.
  * [20] Agnarsson, S., Arnason, R., Johannsdottir, K., Ravn-Jonsen, L., Sandal, L. K., Steinshamn, S. I., & Vestergaard, N. (2008). Comparative evaluation of the fisheries policies in Denmark, Iceland and Norway: multispecies and stochastic issues.
  * [21] Kielhöfer, H. Calculus of Variations: an introduction to the one-dimensional theory with examples and exercises, Springer, Texts in Applied Mathematics, 67, 2018.
  * [22] Zaslavski, A.J.: Turnpike properties of approximate solutions of autonomous variational problems. Control Cybern. 37(2), 491–512 (2008)
  * [23] Wagener, F.O.O. (2003) , Skiba points and heteroclinic bifurcations, with applications to the shallow lake system. Journal of Economic Dynamics & Control 27, 1533-1561.
  * [24] Ilyashenko, Y. and Yakovenko, S., Lectures on Analytic Differential Equations, Springer/AMS, 2008.