# Optimal Agnostic Control of Unknown Linear Dynamics in a Bounded Parameter
Range

J. Carruth, M. F. Eggl, C. Fefferman, C. W. Rowley

###### Abstract

Here and in the follow-on paper [6], we consider a simple control problem in
which the underlying dynamics depend on a parameter $a$ that is unknown and
must be learned. In this paper, we assume that $a$ is bounded, i.e., that
$|a|\leq a_{\text{MAX}}$, and we study two variants of the control problem. In
the first variant, Bayesian control, we are given a prior probability
distribution for $a$ and we seek a strategy that minimizes the expected value
of a given cost function. Assuming that we can solve a certain PDE (the
Hamilton-Jacobi-Bellman equation), we produce optimal strategies for Bayesian
control. In the second variant, agnostic control, we assume nothing about $a$
and we seek a strategy that minimizes a quantity called the regret. We produce
a prior probability distribution $d\text{Prior}(a)$ supported on a finite
subset of $[-a_{\text{MAX}},a_{\text{MAX}}]$ so that the agnostic control
problem reduces to the Bayesian control problem for the prior
$d\text{Prior}(a)$.

###### Contents

  1. 1 Introduction
  2. 1 The Game
  3. 2 Tame Strategies Associated to Partitions
    1. 1 Setup
    2. 2 Estimates for probabilities of outliers
    3. 3 The Probability Density
    4. 4 Analytic Continuation
    5. 5 Moments of Increments
    6. 6 Stability Under Change of Assumption
    7. 7 Disasters Due to Undercontrol
    8. 8 Costing by Integrals
    9. 9 Continuous vs Discrete
    10. 10 Refining a Partition
  4. 3 Bayesian Strategies Associated to Partitions
    1. 1 Setup
    2. 2 Posterior Probabilities and Expectations
    3. 3 The PDE Assumption
    4. 4 The Allegedly Optimal Strategy
    5. 5 Performance of Competing Strategies
    6. 6 Stability of the Allegedly Optimal Strategy
    7. 7 The Second Bayesian Main Lemma
    8. 8 Allowing for Dependence on Coin Flips
    9. 9 Reformulating the Main Bayesian Results
    10. 10 Comparing the Allegedly Optimal Strategies for Two Partitions
  5. 4 Decisions in Continuous Time
    1. 1 Tame Strategies with Decisions in Continuous Time
    2. 2 Not-Necessarily-Tame Strategies
  6. 5 Agnostic Control
    1. 1 Mixed Strategies
    2. 2 Efficient Strategies are Bayesian
    3. 3 Regret
    4. 4 The Main Lemma on Agnostic Control
    5. 5 An Interval of Allowed Parameters

###  1 Introduction

Here and in [7, 16], we explore a new flavor of adaptive control theory, which
we call “agnostic control.”00footnotetext: This work was supported by AFOSR
grant FA9550-19-1-0005 and by the Joachim Herz Foundation. Our introduction
borrows heavily from that of [6].

Many works in adaptive control theory attempt to control a system whose
underlying dynamics are initially unknown and must be learned from
observation. The goal is then to bound Regret, a quantity defined by comparing
our expected cost with that incurred by an opponent who knows the underlying
dynamics. Typically one tries to achieve a regret whose order of magnitude is
as small as possible after a long time. Adaptive control theory has extensive
practical applications; see, e.g., [4, 8, 21, 29].

In some applications, we don’t have the luxury of waiting for a long time.
This is the case, e.g., for a pilot attempting to land an airplane following
the sudden loss of a wing, as in [5]. Our goal, here and in [6], is to achieve
the absolute minimum possible regret over a fixed, finite time horizon. This
poses formidable mathematical challenges, even for simple model systems.

We will study a one-dimensional, linear model system whose dynamics depend on
a single unknown parameter $a$. When $a$ is large positive, the system is
highly unstable. (There is no “stabilizing gain” for all $a$.) Here, we
suppose that the unknown $a$ is confined to a known interval
$[-a_{\text{MAX}},a_{\text{MAX}}]$ and we don’t assume that we are given a
Bayesian prior probability distribution for it. In [6], we extend our results
to deal with the case in which $a$ may be any real number.

Modulo an arbitrarily small increase in regret, we reduce the problem, here
and in [6], to a Bayesian variant in which the unknown $a$ is confined to a
finite set and governed by a prior probability distribution.

For the Bayesian problem, our task is to find a strategy that minimizes the
expected cost. This leads naturally to a PDE, the Bellman equation. We prove
here that the optimal strategy for Bayesian control is indeed given in terms
of the solution of the Bellman equation, and that any strategy significantly
different from that optimum incurs a significantly higher cost. We proceed
modulo assumptions about existence and regularity of the relevant PDE
solutions, for which we lack rigorous proofs. (However, we have obtained
numerical solutions, which seem to behave as expected.)

Let us now explain the above in more detail.

#### The Model System

Our system consists of a particle moving in one dimension, influenced by our
control and buffeted by noise. The position of our particle at time $t$ is
denoted by $q(t)\in\mathbb{R}$. At each time $t$, we may specify a “control”
$u(t)\in\mathbb{R}$, determined by history up to time $t$, i.e., by
$(q(s))_{s\in[0,t]}$. A “strategy” (aka “policy”) is a rule for specifying
$u(t)$ in terms of $(q(s))_{s\in[0,t]}$ for each $t$. We write
$\sigma,\sigma^{\prime},\sigma^{*},\text{etc.}$ to denote strategies. The
noise is provided by a standard Brownian motion $(W(t))_{t\geq 0}$.

The particle moves according to the stochastic ODE

$dq(t)=\big{(}aq(t)+u(t)\big{)}dt+dW(t),\qquad q(0)=q_{0},$ (1.1)

where $a$ and $q_{0}$ are real parameters. Due to the noise in (1.1), $q(t)$
and $u(t)$ are random variables; these random variables depend on our strategy
$\sigma$, and we often write $q^{\sigma}(t)$, $u^{\sigma}(t)$ to make that
dependence explicit.

Over a time horizon $T>0$, we incur a Cost, given222By rescaling, we can
consider seemingly different cost functions of the form
$\int_{0}^{T}(q^{2}+\lambda u^{2})$ for $\lambda>0$. by

$\textsc{Cost}=\int_{0}^{T}\big{\\{}(q(t))^{2}+(u(t))^{2}\big{\\}}dt.$ (1.2)

This quantity is a random variable determined by $a,q_{0},T$ and our strategy
$\sigma$. Here, the starting position $q_{0}$ and time horizon $T$ are fixed
and known, but we don’t know the parameter $a$.

We would like to keep our cost as low as possible. We write
$\textsc{ECost}(\sigma,a)$ to denote the expected value of the Cost (1.2) for
the given $a$ in (1.1).

In this paper we study two variants of the above control problem, which we
call _Bayesian control_ and _agnostic control_.

For Bayesian control, we are given a prior probability distribution
$d\text{Prior}(a)$ for the unknown $a$ in (1.1). We assume that
$d\text{Prior}$ is supported in an interval
$[-a_{\text{MAX}},a_{\text{MAX}}]$. Our task is to pick the strategy $\sigma$
to minimize

$\textsc{ECost}(\sigma,d\text{Prior})=\int_{-a_{\text{MAX}}}^{a_{\text{MAX}}}\textsc{ECost}(\sigma,a)d\text{Prior}(a).$
(1.3)

For agnostic control, we are given that $a$ belongs to a known interval
$[-a_{\text{MAX}},a_{\text{MAX}}]$, but we aren’t given a prior probability
distribution $d\text{Prior}(a)$, so we can’t define an expected cost by (1.3).
Instead, our goal will be to minimize _worst-case regret_ , defined by
comparing the performance of our strategy with that of an opponent who knows
the value of $a$ and plays optimally. Let $\sigma_{\text{opt}}(a)$ be the
optimal strategy for known $a$. Thus $\textsc{ECost}(\sigma,a)$ is minimized
over all $\sigma$ by taking $\sigma=\sigma_{\text{opt}}(a)$.111See standard
textbooks (e.g., [3]) for the computation of $\sigma_{\text{opt}}(a)$ and its
expected cost. We will introduce several variants of the notion of regret.

To a given strategy $\sigma$ we associate the following functions on
$[-a_{\text{MAX}},a_{\text{MAX}}]$:

  * •

_Additive Regret_ , defined as

$\text{AReg}(\sigma,a)=\textsc{ECost}(\sigma,a)-\textsc{ECost}(\sigma_{\text{opt}}(a),a)\geq
0.$

  * •

_Multiplicative Regret_ (aka “competitive ratio”), defined as

$\text{MReg}(\sigma,a)=\frac{\textsc{ECost}(\sigma,a)}{\textsc{ECost}(\sigma_{\text{opt}}(a),a)}\geq
1.$

  * •

_Hybrid Regret_ , defined in terms of a parameter $\gamma>0$ by setting

$\text{HReg}_{\gamma}(\sigma,a)=\frac{\textsc{ECost}(\sigma,a)}{\textsc{ECost}(\sigma_{\text{opt}}(a),a)+\gamma}$

See [6] for a discussion of the regimes in which these three notions provide
useful information.

Writing $\textsc{Regret}(\sigma,a)$ to denote any one of the above three
functions on $[-a_{\text{MAX}},+a_{\text{MAX}}]$, we define the _worst-case
regret_ :

$\textsc{Regret}^{*}(\sigma)=\sup\\{\textsc{Regret}(\sigma,a):a\in[-a_{\text{MAX}},a_{\text{MAX}}]\\}.$
(1.4)

We seek a strategy $\sigma$ having the least possible worst-case regret.

Thus, we have posed two problems: For Bayesian control, find the strategy that
minimizes our expected cost; and for agnostic control, find a strategy that
minimizes worst-case regret.

To prepare to present our results, we next discuss a relevant PDE, the
_Bellman equation_.

We will see that the problem of Bayesian control is intimately connected to
the following PDE for an unknown function $S(q,t,\zeta_{1},\zeta_{2})$ of four
variables:

$\begin{split}0=&\partial_{t}S+(\bar{a}(\zeta_{1},\zeta_{2})q+u_{\text{opt}})\partial_{q}S+\bar{a}(\zeta_{1},\zeta_{2})q^{2}\partial_{\zeta_{1}}S+q^{2}\partial_{\zeta_{2}}S+\frac{1}{2}\partial_{q}^{2}S\\\
&+q\partial_{q\zeta_{1}}S+\frac{1}{2}q^{2}\partial_{\zeta_{1}}^{2}S+(q^{2}+u_{\text{opt}}^{2}),\end{split}$
(1.5)

where

$u_{\text{opt}}=-\frac{1}{2}\partial_{q}S,$ (1.6)

with terminal condition

$S|_{t=T}=0.$ (1.7)

Here, $\bar{a}(\zeta_{1},\zeta_{2})$ is a known, smooth function of two
variables.

We have succeeded in finding numerical solutions of (1.5)–(1.7), but we lack
rigorous proofs of existence and smoothness of solutions. Accordingly, we
impose a _PDE Assumption_ to the effect that (1.5)–(1.7) admit a solution $S$
satisfying plausible estimates (see Section 3). Our numerics suggest that the
PDE Assumption is correct. Our results below are conditional on the PDE
Assumption.

We are ready to state our main results. We begin with Bayesian control. For a
function $\bar{a}(\zeta_{1},\zeta_{2})$ given in terms of $d\text{Prior}$ by
an elementary formula, we define a function
$u_{\text{opt}}(q,t,\zeta_{1},\zeta_{2})$ as in (1.5)–(1.7), and then specify
a strategy $\sigma=\sigma_{\text{Bayes}}(d\text{Prior})$ by setting

$\displaystyle
u^{\sigma}(t)=u_{\text{opt}}(q^{\sigma}(t),t,\zeta_{1}(t),\zeta_{2}(t)),\;\text{with}$
(1.8)
$\displaystyle\zeta_{1}(t)=\int_{0}^{t}q^{\sigma}(s)[dq^{\sigma}(s)-u^{\sigma}(s)ds]\qquad\text{and}\qquad\zeta_{2}(t)=\int_{0}^{t}(q^{\sigma}(s))^{2}\
ds.$ (1.9)

Note that $\zeta_{1}(t)$, $\zeta_{2}(t)$ are determined by past history up
through time $t$, hence so is $u^{\sigma}(t)$ in (1.8). As explained in [6],
heuristic reasoning suggests that $\sigma_{\text{Bayes}}(d\text{Prior})$ is
the optimal strategy for Bayesian control with prior belief $d\text{Prior}$.
Our rigorous result confirms this intuition. Recall that $q_{0}$ is our
starting position.

######  Theorem 1.

Fix a probability distribution $d\emph{Prior}$ on
$[-a_{\emph{MAX}},a_{\emph{MAX}}]$, and let $S$, $u_{\emph{opt}}$,
$\sigma=\sigma_{\emph{Bayes}}(d\emph{Prior})$ be as above. Then the following
hold.

  1. _(A)_

$\textsc{ECost}(\sigma,d\emph{Prior})=S(q_{0},0,0,0)$.

  2. _(B)_

Let $\sigma^{\prime}$ be any other strategy. Then

$\textsc{ECost}(\sigma^{\prime},d\emph{Prior})\geq\textsc{ECost}(\sigma,d\emph{Prior}).$

For a class of “tame strategies” $\sigma^{\prime}$, we can sharpen (B) above
to a quantitative result. A _tame strategy_ $\sigma^{\prime}$ satisfies the
estimate

$|u^{\sigma^{\prime}}(t)|\leq\hat{C}[|q^{\sigma^{\prime}}(t)|+1]\;\text{(all
}t\in[0,T])$

with probability 1, for a constant $\hat{C}$, called a _tame constant for
$\sigma^{\prime}$_. The quantitative version of (B) is as follows.

######  Theorem 2 (Quantitative Uniqueness).

Let $d\emph{Prior}$, $\sigma=\sigma_{\emph{Bayes}}(d\emph{Prior})$ be as in
Theorem 1. Given $\varepsilon>0$ and given a constant $\hat{C}$, there exists
$\delta>0$ for which the following holds:

Let $\sigma^{\prime}$ be a tame strategy with tame constant $\hat{C}$.

If
$\textsc{ECost}(\sigma^{\prime},d\emph{Prior})\leq\textsc{ECost}(\sigma,d\emph{Prior})+\delta$,
then the expected value of

$\int_{0}^{T}\\{|q^{\sigma}(t)-q^{\sigma^{\prime}}(t)|^{2}+|u^{\sigma}(t)-u^{\sigma^{\prime}}(t)|^{2}\\}\
dt$

is less than $\varepsilon$.

Quantitative Uniqueness will play a crucial rôle in our analysis of agnostic
control.

Our main result for agnostic control is as follows.

######  Theorem 3.

Fix $[-a_{\emph{MAX}},a_{\emph{MAX}}]$, $q_{0}$, $T$ (and $\gamma$ if we use
hybrid regret). Then there exist a probability measure $d\emph{Prior}$, a
finite subset $E\subset[-a_{\emph{MAX}},a_{\emph{MAX}}]$, and a strategy
$\sigma$, for which the following hold.

  * _(I)_

$\sigma$ is the optimal Bayesian strategy for the prior probability
distribution $d\emph{Prior}$.

  * _(II)_

$d\emph{Prior}$ is supported in the finite set $E$.

  * _(III)_

$E$ is precisely the set of points $a\in[-a_{\emph{MAX}},a_{\emph{MAX}}]$ at
which the function $[-a_{\emph{MAX}},a_{\emph{MAX}}]\ni
a\mapsto\textsc{Regret}(\sigma,a)$ achieves its maximum.

  * _(IV)_

$\textsc{Regret}^{*}(\sigma)\leq\textsc{Regret}^{*}(\sigma^{\prime})$ for any
other strategy $\sigma^{\prime}$.

So, for optimal agnostic control, we should pretend to believe that the
unknown $a$ is confined to a finite set $E$ and governed by the probability
distribution $d\text{Prior}$, even though in fact we know nothing about $a$
except that it lies in $[-a_{\text{MAX}},a_{\text{MAX}}]$.

Let $d\text{Prior}$, $\sigma$, $E$ be as in (I), (II), (III) of Theorem 3.
Since $\sigma$ is the optimal Bayesian strategy for $d\text{Prior}$ (by (I)),
and since $d\text{Prior}$ is supported on the finite set $E$ (by (II)), we
have for any other strategy $\sigma^{\prime}$ that

$\textsc{ECost}(\sigma,a_{0})\leq\textsc{ECost}(\sigma^{\prime},a_{0})\;\text{for
some}\;a_{0}\in E.$

In particular, we have

$\textsc{Regret}(\sigma,a_{0})\leq\textsc{Regret}(\sigma^{\prime},a_{0})\;\text{for
some}\;a_{0}\in E.$

Combining this with (III), we see that for any
$a\in[-a_{\text{MAX}},a_{\text{MAX}}]$ we have

$\textsc{Regret}(\sigma,a)\leq\textsc{Regret}(\sigma^{\prime},a_{0}).$

Therefore (I), (II), (III) of Theorem 3 easily imply (IV). The hard part of
Theorem 3 is the assertion that there exist $d\text{Prior}$, $E$, $\sigma$,
satisfying (I), (II), (III).

Theorem 3 lets us search for optimal agnostic strategies: We first guess a
finite set $E$ and a probability measure $d\text{Prior}$ concentrated on $E$.
By solving the Bellman equation, we produce the optimal Bayesian strategy
$\sigma=\sigma_{\text{Bayes}}(d\text{Prior})$, which allows us to compute the
function $[-a_{\text{MAX}},a_{\text{MAX}}]\ni
a\mapsto\textsc{Regret}(\sigma,a)$. If the maximum of that function occurs
precisely at the points of $E$, then $\sigma$ is the desired optimal agnostic
strategy. Otherwise, we modify our guess $(E,d\text{Prior})$. We have carried
this out numerically for several $[-a_{\text{MAX}},a_{\text{MAX}}],q_{0},T$.

This concludes our introductory discussion of agnostic control for bounded $a$
(i.e. $a\in[-a_{\text{MAX}},a_{\text{MAX}}]$).

We briefly touch on another variant of the control problem (1.1): _agnostic
control for unbounded $a$_.

Suppose we assume absolutely nothing about our unknown $a$; it might be any
real number. For any strategy $\sigma$ we define $\textsc{Regret}^{*}(\sigma)$
as in (1.4), except that now the $\sup$ is taken over all $a\in\mathbb{R}$.
Our task is to pick $\sigma$ to minimize $\textsc{Regret}^{*}(\sigma)$.

Our companion paper [6] analyzes this problem by comparing optimal agnostic
control for arbitrary $a$ with the case in which $a$ is confined to a large
interval $[-a_{\text{MAX}}(\varepsilon),+a_{\text{MAX}}(\varepsilon)]$,
depending on a small parameter $\varepsilon>0$. Roughly speaking, [6] shows
that any strategy for $a$ confined to
$[-a_{\text{MAX}}(\varepsilon),+a_{\text{MAX}}(\varepsilon)]$ may be modified
to produce a strategy for arbitrary $a\in\mathbb{R}$, with an increase in
worst-case hybrid regret of at most $\varepsilon$. (See [6] for precise
statements.)

#### Recap

Let us summarize what we have achieved. Suppose our goal is to minimize worst-
case hybrid regret in the setting in which $a$ may be any real number. Modulo
an arbitrarily small increase in regret, we may reduce matters to the case in
which $a$ is confined to a bounded interval
$[-a_{\text{MAX}},a_{\text{MAX}}]$. We then look for a probability measure
$d\text{Prior}$ living on a finite set
$E\subset[-a_{\text{MAX}},a_{\text{MAX}}]$, such that the regret of the
optimal Bayesian strategy for $d\text{Prior}$ is maximized precisely on $E$.
We can calculate the optimal Bayesian strategy for a given prior probability
measure by solving a Bellman equation. However, our results are conditional;
we have to make an assumption on the existence, smoothness, and size of
solutions to the Bellman equation. In numerical simulations, we have produced
evidence for our PDE Assumptions, and we have produced optimal agnostic
strategies for cases in which the unknown $a$ is confined to an interval.

#### Ideas from the Proofs

We mention one significant technical point regarding the proofs of Theorems 1,
2, 3: We need a rigorous definition of a strategy. Certainly the phrase “a
rule for determining $u(t)$ from past history” isn’t precise.

We want to allow $u(t)$ to depend discontinuously on past history
$(q(s))_{s\in[0,t]}$. For instance, we should be allowed to set

$u(t)=\begin{cases}-q(t)&\text{if}\;|q(t)|>1,\\\
0&\text{otherwise}.\end{cases}$

On the other hand, we had better make sure that we can produce solutions of
our stochastic ODE

$dq=(aq+u)dt+dW.$

We proceed as follows.

At first we fix a partition

$0=t_{0}<t_{1}<\dots<t_{N}=T$ (1.10)

of the time interval $[0,T]$. We restrict ourselves to strategies $\sigma$ in
which the control $u(t)$ is constant in each interval $[t_{\nu},t_{\nu+1})$,
and in which, for each $\nu$, $u(t_{\nu})$ is determined by
$q(t_{1}),\dots,q(t_{\nu})$, together with “coin flips”
$\vec{\xi}=(\xi_{1},\xi_{2},\cdots)\in\\{0,1\\}^{\mathbb{N}}$. For all $\nu$,
we assume that $u(t_{\nu})$ is a Borel measurable function of
$(q(t_{1}),\dots,q(t_{\nu}),\vec{\xi})$, and that

$|u(t_{\nu})|\leq C_{\text{TAME}}[|q(t_{\nu})|+1].$

A strategy as above is called a _tame strategy associated to the partition_
(1.10), with a _tame constant_ $C_{\text{TAME}}$. For such strategies, it is
easy to define the solutions $q^{\sigma}(t)$, $u^{\sigma}(t)$ of our
stochastic ODE (1.1).

Most of our work lies in controlling and optimizing tame strategies associated
to a sufficiently fine partition. In particular, we will prove approximate
versions of Theorems 1, 2 in the setting of such strategies.

We will then define a tame strategy (not associated to any partition) by
considering a sequence $\pi_{1},\pi_{2},\dots$ of ever-finer partitions of
$[0,T]$. To each partition $\pi_{n}$ we associate a tame strategy $\sigma_{n}$
with a tame constant $C_{\text{TAME}}$ independent of $n$. If the resulting
$q^{\sigma_{n}}(t)$ and $u^{\sigma_{n}}(t)$ tend to limits, in an appropriate
sense, as $n\rightarrow\infty$, then we declare these limits $q(t)$, $u(t)$ to
arise from a _tame strategy_ $\sigma$ with a _tame constant_
$C_{\text{TAME}}$.

Finally, we drop the restriction to tame strategies and consider general
strategies. To do so, we consider a sequence $(\sigma_{n})_{n=0,1,2,\dots}$ of
tame strategies, NOT assumed to have a tame constant independent of $n$. If
the relevant $q^{\sigma_{n}}(t)$ and $u^{\sigma_{n}}(t)$ converge, in a
suitable sense, as $n\rightarrow\infty$, then we say that the limits $q(t)$,
$u(t)$ arise from a strategy $\sigma$.

It isn’t hard to pass from tame strategies associated to partitions of $[0,T]$
to general tame strategies, and then to pass from such tame strategies to
general strategies. The work in proving Theorems 1 and 2 lies in our close
study of tame strategies associated to fine partitions.

We provide only a few comments on the proof of Theorem 3. The main work lies
in proving an analogue of Theorem 3 in which the unknown $a$ is confined to a
finite set $A\subset[-a_{\text{MAX}},a_{\text{MAX}}]$, rather than to the
whole of $[-a_{\text{MAX}},a_{\text{MAX}}]$. We apply that analogue to a
sequence $A_{1},A_{2},\dots$ of fine nets in
$[-a_{\text{MAX}},a_{\text{MAX}}]$, e.g.,
$A_{n}=[-a_{\text{MAX}},a_{\text{MAX}}]\cap 2^{-n}\mathbbm{Z}$, and pass to
the limit as $n\rightarrow\infty$ using a weak compactness argument. To
establish the result for finite $A$, we proceed by induction on the number of
elements of $A$. Details may be found in Chapter 5.

#### Future Directions

Our work suggests several unsolved problems, among which we mention:

  * •

Prove (or disprove) the PDE Assumption.

  * •

Consider problems in which the particle lives in $\mathbb{R}^{N}$, not just in
$\mathbb{R}^{1}$; and in which the dynamics of the particle depend on more
than one unknown parameter. Can that be done without rendering the relevant
numerics hopelessly impractical?

  * •

Even for the model problem considered in this paper, improve the numerics to
let us produce optimal agnostic strategies for a larger range of
$a_{\text{MAX}}$ and $T$ than we can deal with today.

We speculate briefly on a particular model problem in which we don’t know a
priori what our control does.

Consider a particle governed by the stochastic ODE

$dq(t)=au(t)dt+dW(t),\qquad q(0)=0.$ (1.11)

As usual, $q(t)$ denotes position, $u(t)$ is our control, $W(t)$ is Brownian
motion, and we incur a cost

$\int_{0}^{T}\\{(q(t))^{2}+(u(t))^{2}\\}\ dt.$

In the simplest case, suppose we know a priori that $a=1$ or $a=-1$, each with
probability $1/2$. We write $\textsc{ECost}(\sigma)$ to denote the expected
cost incurred by executing a strategy $\sigma$, and we set

$\textsc{ECost}^{*}=\inf\\{\textsc{ECost}(\sigma):\text{All strategies
}\sigma\\}.$ (1.12)

For this simple model problem we conjecture that the inf in (1.12) is not
achieved by any strategy $\sigma$, because heuristic reasoning suggests that
there is a regime in which we would like to set $u=\pm\infty$ to gain instant
information about $a$.

Clearly there is much to be done before we can claim to understand agnostic
control theory.

#### Survey of Prior Literature

Literature that considers adaptive control of a simple linear system similar
to the one considered in this paper commonly consists of one or more of the
following features: (_i_) unknown governing dynamics, (_ii_) unknown cost
function and (_iii_) adversarial noise. Examples of such work include [24, 26,
33, 18, 11, 25, 14] as well as our own prior work [16], [7].

Initial work in obtaining regret bounds in the infinite time horizon for the
related LQR (linear-quadratic regulator) problem was undertaken in [1], which
proved that under certain assumptions, the expected additive regret of the
adaptive controller is bounded by $\tilde{O}(\sqrt{T})$. Further progress was
made on this problem in [9]. Assuming controllability of the system, the
authors gave the first efficient algorithm capable of attaining sublinear
additive regret in a single trajectory in the setting of online nonstochastic
control. See also the related [28], which obtained sublinear adaptive regret
bounds, a stronger metric than standard regret and more suitable for time-
varying systems. Additional adaptive control approaches include [12, 13] using
the system level synthesis. This expands on ideas in [31], which showed that
the ordinary least-squares estimator learns a linear system nearly optimally
in one shot. Other work uses Thompson Sampling [2, 23] or deep learning [10].
Perhaps most related to the work performed in this study is [22], which
designed an online learning algorithm with sublinear expected regret that
moves away from episodic estimates of the state dynamics (meaning that no
boundedness or initially stabilizing control needed to be assumed).

In [16], the third and fourth authors of the present paper, along with B.
Guillén Pegueroles and M. Weber, found regret minimizing strategies for a
problem with simple unknown dynamics (a particle moving in one-dimension at a
constant, unknown velocity subject to Brownian motion). In [20], along with D.
Goswami and D. Gurevich, they generalized these results to an analogous,
higher-dimensional system with the addition of sensor noise. In [16], they
also posed the problem of finding regret minimizing strategies for the more
complicated dynamics (1.1). In [7], the authors of the present paper, along
with M. Weber, took the first steps toward resolving this problem.
Specifically, we exhibited a strategy for the dynamics (1.1) with bounded
multiplicative regret.

Historically, significant work has been undertaken in the closely related
“multi-armed bandit” problem; see, for instance, the classic papers [30, 32].
Recent work considering this paradigm includes [34], which used reinforcement
learning to obtain dynamic regret whose order of magnitude is optimal, and
[15], which studied the more general Generalized Linear Bandits (GLBs) and
obtained similar regret bounds.

We finally want to point out the parallel field of adversarial control, where
the noise profile is chosen by an adversary instead of randomly. This includes
[27], which attained minimum dynamic regret and guaranteed compliance with
hard safety constraints in the face of uncertain disturbance realizations
using the system level synthesis framework, and [19], which studied the
problem of competitive control.

As this list of references is by no means exhaustive and does not do justice
to the wealth of studies in the literature, we point the reader to the book
[21] and the references therein for a more thorough overview of online
control.

We emphasize that our approach in [6], [7], [16], and the present paper
differs from the other work cited above in that:

  * •

We seek strategies that minimize the worst-case regret for a fixed time
horizon $T$, whereas the literature is mainly concerned with
$T\rightarrow\infty$.

  * •

Typically in the literature one assumes either that the dynamics are bounded
or that one is given a stabilizing control. We make no such assumptions in [6]
and so we must control a system that is arbitrarily unstable.

  * •

However, we achieve the above ambitious goals only for a simple model system.

We thank Amir Ali Ahmadi, Brittany Hamfeldt, Elad Hazan, Robert Kohn, Sam
Otto, Allan Sly, and Melanie Weber for helpful conversations. We are grateful
to the Air Force Office of Scientific Research, especially Frederick Leve, for
their generous support. The second named author thanks the Joachim Herz
foundation for support.

##  Chapter 1 The Game

We will deal with random variables

$\displaystyle a_{\text{TRUE}}\in[-a_{\text{MAX}},+a_{\text{MAX}}],$ (0.1)
$\displaystyle\text{``Coin flips'' }\xi_{1},\xi_{2},\dots\in\\{0,1\\}\text{
(we write }\vec{\xi}\text{ for }(\xi_{1},\xi_{2},\dots)),\text{and}$ (0.2)
$\displaystyle\text{Brownian motion }W(t),\text{ starting at }W(0)=0.$ (0.3)

The random variable $a_{\text{TRUE}}$ is deterministic and known in Chapter 2,
unknown but subject to a known prior in Chapters 3 and 4, and unknown without
a known prior in Chapter 5.

The $\xi_{\nu}$, the real number $a_{\text{TRUE}}$, and the Brownian motion
are mutually independent. The variable $a_{\text{TRUE}}$ has a prior
probability distribution given by the measure $d\text{Prior}(a)$ in Chapters 3
and 4; and each $\xi_{\nu}$ is equal to 0 with probability $1/2$, and to 1
with probability $1/2$.

When $a_{\text{TRUE}}$ has a prior probability distribution, we write
$\text{Prob}[\mathcal{E}]$ to denote the probability of an event $\mathcal{E}$
with respect to the above probability space, and we write $\text{E}[X]$ for
the expected value of $X$ with respect to that probability space.

Given $a\in[-a_{\text{MAX}},+a_{\text{MAX}}]$, we write
$\text{Prob}_{a}[\mathcal{E}]$ for the probability of event $\mathcal{E}$
conditioned on the event $a_{\text{TRUE}}=a$, and we write $\text{E}_{a}[X]$
for the expectation of $X$ conditioned on $a_{\text{TRUE}}=a$.
$\text{Prob}_{a}[\cdots]$ and $\text{E}_{a}[\cdots]$ make sense without an
assumed prior for $a$.

Similarly, given $a\in[-a_{\text{MAX}},+a_{\text{MAX}}]$ and
$\vec{\eta}=(\eta_{1},\eta_{2},\dots)\in\\{0,1\\}^{\mathbb{N}}$, we write
$\text{Prob}_{a,\vec{\eta}}[\mathcal{E}]$ and $\text{E}_{a,\vec{\eta}}[X]$ to
denote the probability and expectation, respectively, conditioned on the event
$a_{\text{TRUE}}=a$ and $\xi_{\nu}=\eta_{\nu}$ for all $\nu$.

Also, we write $\text{E}_{\vec{\eta}}[X]$ and
$\text{Prob}_{\vec{\eta}}[\mathcal{E}]$ to denote expectation and probability,
respectively, conditioned on $\vec{\xi}=\vec{\eta}$.

The above conditional expectations make sense even if, e.g., $a$ isn’t in the
support of $d\text{Prior}$.

Fix a _terminal time_ $T>0$ and a partition

$0=t_{0}<t_{1}<\dots<t_{N}=T\;\text{of}\;[0,T].$

Fix a _starting position_ $q_{0}\in\mathbb{R}$. A _tame rule_ at time
$t_{\nu}$ is a Borel measurable function
$\sigma_{t_{\nu}}:\mathbb{R}^{\nu}\times\\{0,1\\}^{\mathbb{N}}\rightarrow\mathbb{R}$,
satisfying the estimate

$|\sigma_{t_{\nu}}(q_{1},\dots,q_{\nu},\vec{\xi})|\leq
C_{\text{TAME}}[|q_{\nu}|+1]$ (0.4)

for all
$(q_{1},\dots,q_{\nu},\vec{\xi})\in\mathbb{R}^{\nu}\times\\{0,1\\}^{\mathbb{N}}.$
If $\nu=0$, then $\sigma_{t_{\nu}}$ is simply a function on
$\\{0,1\\}^{\mathbb{N}}$. (We use the product topology on
$\mathbb{R}^{\nu}\times\\{0,1\\}^{\mathbb{N}}$ to define Borel measurability.
We require Borel measurability to avoid technicalities. In particular, the
composition of Borel measurable functions is Borel measurable, whereas the
composition of Lebesgue measurable functions needn’t be Lebesgue measurable.)

A _tame strategy_ is an array $\sigma=(\sigma_{t_{\nu}})_{\nu=0,1,\dots,N-1}$,
where, for each $\nu$, $\sigma_{t_{\nu}}$ is a tame rule at time $t_{\nu}$
with the same $C_{\text{TAME}}$ serving in (0.4) for all the $t_{\nu}$. We
call $C_{\text{TAME}}$ a _tame constant_ for the strategy $\sigma$. Until
further notice, we say simply “strategy” in place of “tame strategy.” If the
$\sigma_{t_{\nu}}$ don’t depend on the coin flips $\vec{\xi}$, we call
$\sigma$ a _deterministic strategy_. We will often write $\sigma_{\nu}$ in
place of $\sigma_{t_{\nu}}$.

Given a strategy $\sigma=(\sigma_{t_{\nu}})_{\nu=0,1,\dots,N-1}$, we define
random variables $q^{\sigma}(t)$ for $t\in[0,T]$ and $u^{\sigma}(t)$ for
$t\in[0,T)$, as follows.

By induction on $\nu$, we define $q^{\sigma}(t)$ for $t\in[0,t_{\nu}]$ and
$u^{\sigma}(t)$ for $t\in[0,t_{\nu})$.

In the base case $\nu=0$, we set $q^{\sigma}(t)=q_{0}$ for
$t\in[0,t_{\nu}]=\\{0\\}$. Since $[0,t_{\nu})=[0,0)$ is empty, there is no
need to define $u^{\sigma}$ in the base case.

For the induction step, we fix $\nu\geq 0$, and assume that we have defined
$q^{\sigma}(t)$ for $t\in[0,t_{\nu}]$ and $u^{\sigma}(t)$ for
$t\in[0,t_{\nu})$. We extend the definition of $q^{\sigma}(t)$ to
$t\in[0,t_{\nu+1}]$, and that of $u^{\sigma}(t)$ to $t\in[0,t_{\nu+1})$, as
follows.

  * •

For $t\in[t_{\nu},t_{\nu+1})$, we set

$u^{\sigma}(t)=\sigma_{t_{\nu}}(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\nu}),\vec{\xi}).$

  * •

For $t\in[t_{\nu},t_{\nu+1}]$, we define $q^{\sigma}(t)$ as the solution of
the stochastic ODE

$dq^{\sigma}(t)=(a_{\text{TRUE}}q^{\sigma}(t)+u^{\sigma}(t))dt+dW(t),$

with the initial value $q^{\sigma}(t_{\nu})$ already given by our induction
hypothesis.

This completes our induction on $\nu$, so we have defined the random variables
$q^{\sigma}(t),u^{\sigma}(t)$.

In addition to $q^{\sigma}(t),u^{\sigma}(t)$, we define random variables
$\zeta_{1}^{\sigma}(t_{\nu})$, $\zeta_{2}^{\sigma}(t_{\nu})$ $(0\leq\nu<N)$ by
the following induction.

$\displaystyle\zeta_{1}^{\sigma}(t_{0})=\zeta_{2}^{\sigma}(t_{0})=0.\quad\text{(Recall,
$t_{0}=0$.)}$
$\displaystyle\zeta_{1}^{\sigma}(t_{\nu+1})=\zeta_{1}^{\sigma}(t_{\nu})+q^{\sigma}(t_{\nu})\cdot[\Delta
q^{\sigma}_{\nu}-u^{\sigma}(t_{\nu})\Delta t_{\nu}],\;\text{where}$
$\displaystyle\qquad\Delta
q^{\sigma}_{\nu}=q^{\sigma}(t_{\nu+1})-q^{\sigma}(t_{\nu})\;\text{and}\;\Delta
t_{\nu}=t_{\nu+1}-t_{\nu}.$
$\displaystyle\zeta_{2}^{\sigma}(t_{\nu+1})=\zeta_{2}^{\sigma}(t_{\nu})+(q^{\sigma}(t_{\nu}))^{2}\Delta
t_{\nu}.$

Thus,

$\zeta_{1}^{\sigma}(t_{\nu})=\sum_{0\leq\mu<\nu}q^{\sigma}(t_{\mu})(\Delta
q^{\sigma}_{\mu}-u^{\sigma}(t_{\mu})\Delta t_{\mu})$

and

$\zeta_{2}^{\sigma}(t_{\nu})=\sum_{0\leq\mu<\nu}(q^{\sigma}(t_{\mu}))^{2}\Delta
t_{\mu}.$

We will try to pick our strategy $\sigma$ to make the expected value of

$\int_{0}^{T}[(q^{\sigma}(t))^{2}+(u^{\sigma}(t))^{2}]\ dt$

as small as possible.

##  Chapter 2 Tame Strategies Associated to Partitions

###  1 Setup

In this chapter, we take $a_{\text{TRUE}}$ to be fixed, $a_{\text{TRUE}}=a$,
and we suppose that our strategy makes no use of coin flips.

We fix a partition

$0=t_{0}<t_{1}<\dots<t_{N}=T$ (1.1)

of a time interval $[0,T]$.

We fix a (deterministic) strategy $\sigma$ for the game with starting position
$q_{0}$. We assume that our strategy is tame, i.e.,

$|u^{\sigma}(t_{\nu})|\leq C_{\text{TAME}}[|q^{\sigma}(t_{\nu})|+1]$ (1.2)

for a constant $C_{\text{TAME}}$.

We write $c$, $C$, $C^{\prime}$, etc. to denote constants determined by

  * •

$C_{\text{TAME}}$ in (1.2)

  * •

An upper bound for the time horizon $T$

  * •

An upper bound for $a_{\text{TRUE}}$

  * •

An upper bound for $|q_{0}|$.

These symbols may denote different constants in different occurrences.

We define

$\Delta t_{\nu}:=t_{\nu+1}-t_{\nu}\;\text{for all}\;\nu\;(0\leq\nu<N)$

and we assume that

$(\Delta t_{\text{MAX}}):=\max_{\nu}\Delta t_{\nu}\;\text{is less than a small
enough constant }c.$ (1.3)

We write $X=O(Y)$ to denote the estimate $|X|\leq CY$.

We write $q_{\nu}$ to denote $q^{\sigma}(t_{\nu})$, $u_{\nu}$ to denote
$u^{\sigma}(t_{\nu})$, and $\Delta q_{\nu}$ to denote $q_{\nu+1}-q_{\nu}$.

Note that

$u_{\nu}=\sigma_{\nu}(q_{1},\dots,q_{\nu}),$ (1.4)

where $\sigma_{\nu}$ is given by the strategy $\sigma$ for decisions at time
$t_{\nu}$.

Thanks to (1.2), we have

$|u_{\nu}|\leq C[|q_{\nu}|+1].$ (1.5)

Recall that the $q_{\nu}$ evolve as follows.

$q_{0}$ is given, and $u_{0}$ is specified by the strategy $\sigma$.
$q_{\nu+1}$ and $u_{\nu+1}$ are then determined from $q_{\nu}$ and $u_{\nu}$
as follows.

  * •

We solve the stochastic ODE

$dq(t)=(aq(t)+u_{\nu})dt+dW(t)\qquad\text{for }t\in[t_{\nu},t_{\nu+1}],$ (1.6)

with initial condition $q(t_{\nu})=q_{\nu}$. Here, $a$ is the (given) value of
$a_{\text{TRUE}}$, and $W(t)$ denotes Brownian motion at time $t$.

  * •

We set $q_{\nu+1}=q(t_{\nu+1})$.

  * •

We set $u_{\nu+1}=\sigma_{\nu+1}(q_{1},\dots,q_{\nu+1})$ (compare with (1.4)).

Thus, the $q_{\nu}$, $u_{\nu}$ are random variables defined by induction on
$\nu$.

Solving the ODE (1.6) using an integrating factor, we find that

$q(t)-q(t_{\nu})=(aq_{\nu}+u_{\nu})\bigg{[}\frac{e^{a(t-t_{\nu})}-1}{a}\bigg{]}+\int_{t_{\nu}}^{t}e^{a(t-s)}\
dW(s)$ (1.7)

for $t\in[t_{\nu},t_{\nu+1}]$.

In particular,

$\Delta q_{\nu}=q_{\nu+1}-q_{\nu}=(aq_{\nu}+u_{\nu})\Delta t_{\nu}^{*}+\Delta
W_{\nu},$ (1.8)

where

$\Delta t_{\nu}^{*}=\bigg{[}\frac{e^{a\Delta t_{\nu}}-1}{a}\bigg{]}$ (1.9)

and

$\Delta W_{\nu}=\int_{t_{\nu}}^{t_{\nu+1}}e^{a(t_{\nu+1}-s)}\ dW(s).$ (1.10)

(If $a=0$, we interpret the above fractions in square brackets as
$(t-t_{\nu})$ in (1.7), and $\Delta t_{\nu}$ in (1.9).) We warn the reader
that $\Delta W_{\nu}\neq W(t_{\nu+1})-W(t_{\nu})$.

Note that $\Delta W_{\nu}$ is a normal random variable with mean 0 and
variance

$\Delta\tilde{t}_{\nu}=\bigg{[}\frac{e^{2a\Delta t_{\nu}}-1}{2a}\bigg{]}$
(1.11)

(again, equal to $\Delta t_{\nu}$ if $a=0$).

Note that

$\Delta\tilde{t}_{\nu},\Delta t_{\nu}^{*}=\Delta t_{\nu}+O((\Delta
t_{\nu})^{2}).$ (1.12)

Note also that $\Delta\tilde{t}_{\nu}$, $\Delta t_{\nu}^{*}$, $\Delta W_{\nu}$
depend on $a$.

We introduce the sigma algebras $\mathcal{F}_{\nu}$, defined as the algebra of
events determined by the $\Delta W_{\mu}$ for $0\leq\mu<\nu$. Note that
$q_{\nu}$, $u_{\nu}$ and $\Delta W_{\mu}$ ($\mu<\nu$) are
$\mathcal{F}_{\nu}$-measurable (i.e., they are deterministic once we condition
on $\mathcal{F}_{\nu}$), while the $\Delta W_{\mu}$ for $\mu\geq\nu$ are
independent of $\mathcal{F}_{\nu}$.

######  Remark 1.

Thanks to equation (1.8), the sigma algebra $\mathcal{F}_{\nu}$ may be
equivalently defined to consist of all events determined by by
$q_{1},\dots,q_{\nu}$. This equivalence holds because $a_{\emph{TRUE}}$ has
been fixed $(a_{\emph{TRUE}}=a)$.

###  2 Estimates for probabilities of outliers

We suppose

$Q>C\;\text{for a large enough}\;C,$ (2.1)

and we estimate the probability that $\max_{\nu}|q_{\nu}|>Q$. To do so, we set

$\displaystyle
u_{\nu}^{1}=u_{\nu}/q_{\nu}\;\text{and}\;u_{\nu}^{0}=0\;\text{if}\;|q_{\nu}|>1;$
$\displaystyle
u_{\nu}^{1}=0\;\text{and}\;u_{\nu}^{0}=u_{\nu}\;\text{otherwise}.$

Thus,

$u_{\nu}=u_{\nu}^{1}q_{\nu}+u_{\nu}^{0}$ (2.2)

and

$|u_{\nu}^{1}|,|u_{\nu}^{0}|\leq C.$ (2.3)

Also, $u_{\nu}^{1}$ and $u_{\nu}^{0}$ are $\mathcal{F}_{\nu}$-measurable.

Thanks to (2.2), we can rewrite (1.8) in the form

$\Delta q_{\nu}=(a+u_{\nu}^{1})q_{\nu}(\Delta t_{\nu}^{*})+u_{\nu}^{0}(\Delta
t_{\nu}^{*})+\Delta W_{\nu},$

or equivalently,

$[e^{-at_{\nu+1}}q_{\nu+1}]=(1+e^{-a\Delta t_{\nu}}\Delta
t_{\nu}^{*}u_{\nu}^{1})[e^{-at_{\nu}}q_{\nu}]+e^{-at_{\nu+1}}\Delta
t_{\nu}^{*}u_{\nu}^{0}+e^{-at_{\nu+1}}\Delta W_{\nu}.$

(See (1.9).)

Setting

$\displaystyle M_{\nu}=\prod_{0\leq\mu<\nu}(1+e^{-a\Delta t_{\mu}}\Delta
t_{\mu}^{*}u_{\mu}^{1})^{-1},$ (2.4) $\displaystyle
m_{\nu}=e^{-at_{\nu+1}}\Delta t_{\nu}^{*}u_{\nu}^{0},\;\text{and}$ (2.5)
$\displaystyle q_{\nu}^{*}=M_{\nu}e^{-at_{\nu}}q_{\nu},$ (2.6)

we see that

$\displaystyle
q_{\nu+1}^{*}=q_{\nu}^{*}+M_{\nu+1}m_{\nu}+M_{\nu+1}e^{-at_{\nu+1}}\Delta
W_{\nu},$ (2.7)

and that

$\displaystyle q_{0}^{*}=q_{0}.$ (2.8)

Since $|e^{-a\Delta t_{\mu}}u_{\mu}^{i}|\leq C\;(i=0,1)$ and

$\sum_{\mu}(\Delta t_{\mu}^{*})\leq C,$

we have

$c<M_{\nu}<C\;\text{and}\;|m_{\nu}|<C\Delta t_{\nu}.$ (2.9)

Moreover, $M_{\nu+1}$ is $\mathcal{F}_{\nu}$-measurable.

Let $\lambda\in\mathbb{R}$, to be fixed below. From (2.7), (2.9), we have

$\exp(\lambda q_{\nu+1}^{*})\leq\\{\exp(\lambda
q_{\nu}^{*})\cdot\exp(C|\lambda|\Delta
t_{\nu})\\}\exp(\\{M_{\nu+1}e^{-at_{\nu+1}}\\}\lambda\Delta W_{\nu}).$ (2.10)

Here, the quantities in curly brackets are $\mathcal{F}_{\nu}$-measurable,
while $\Delta W_{\nu}$ is independent of $\mathcal{F}_{\nu}$.

Recalling that $\Delta W_{\nu}$ is normal, with mean 0 and variance $O(\Delta
t_{\nu})$, we deduce from (2.9), (2.10) that

$\text{E}[\exp(\lambda q_{\nu+1}^{*})|\mathcal{F}_{\nu}]\leq\exp(\lambda
q_{\nu}^{*})\exp(C|\lambda|\Delta t_{\nu})\exp(C\lambda^{2}\Delta t_{\nu}).$

Thus, the random variables

$Z_{\nu}=\exp(-C[|\lambda|+\lambda^{2}]t_{\nu})\exp(\lambda
q_{\nu}^{*})\quad(0\leq\nu\leq N)$ (2.11)

form a supermartingale, with

$Z_{0}=\exp(\lambda q_{0})\leq\exp(C|\lambda|).$

Consequently, for any $Q>0$ we have

$\text{Prob}[\max_{\nu}Z_{\nu}>\exp(|\lambda|Q)]\leq\exp(|\lambda|(C-Q)).$

By definition (2.11), this means that

$\text{Prob}[\lambda
q_{\nu}^{*}-C[|\lambda|+\lambda^{2}]t_{\nu}>|\lambda|Q\;\text{for
some}\;\nu]\leq\exp(|\lambda|(C-Q)).$ (2.12)

Taking $Q$ greater that $2C$ in (2.12) (see (2.1)), and picking $\lambda=\pm
Q$, we learn from (2.12) that

$\text{Prob}[|q_{\nu}^{*}|>CQ\;\text{for some}\;\nu]\leq C\exp(-cQ^{2}).$

Recalling (2.6) and (2.9), we conclude that

$\text{Prob}[\max_{\nu}|q_{\nu}|>Q]\leq C\exp(-cQ^{2})$ (2.13)

if $Q$ satisfies (2.1).

Thus, we have succeeded in estimating the probability that
$\max_{\nu}|q_{\nu}|$ is large.

Immediately from (1.5) and (2.13) we have also

$\text{Prob}[\max_{\nu}|u_{\nu}|>Q]\leq C\exp(-cQ^{2})$ (2.14)

if $Q$ satisfies (2.1).

We now turn our attention to

$\zeta_{1}(t_{\nu})=\sum_{0\leq\mu<\nu}q_{\mu}[\Delta q_{\mu}-u_{\mu}\Delta
t_{\mu}]$ (2.15)

and

$\zeta_{2}(t_{\nu})=\sum_{0\leq\mu<\nu}q_{\mu}^{2}\Delta t_{\mu}.$ (2.16)

Note that $\zeta_{1}(t_{\nu})$ can be rewritten as

$\begin{split}\zeta_{1}(t_{\nu})=&\sum_{0\leq\mu<\nu}\Big{\\{}\frac{1}{2}(q_{\mu+1}^{2}-q_{\mu}^{2})-\frac{1}{2}(\Delta
q_{\mu})^{2}\Big{\\}}-\sum_{0\leq\mu<\nu}u_{\mu}q_{\mu}\Delta t_{\mu}\\\
=&\frac{1}{2}q_{\nu}^{2}-\frac{1}{2}q_{0}^{2}-\frac{1}{2}\sum_{\mu=0}^{\nu-1}(\Delta
q_{\mu})^{2}-\sum_{0\leq\mu<\nu}u_{\mu}q_{\mu}\Delta t_{\mu}.\end{split}$
(2.17)

Now suppose that

$\max_{\nu}|q_{\nu}|,\max_{\nu}|u_{\nu}|\leq CQ\;\text{with}\;Q\;\text{as in
\eqref{eq: 1.12}}.$ (2.18)

Then from (2.16), (2.17), we have

$|\zeta_{2}(t_{\nu})|\leq CQ^{2}\quad\text{(all }\nu),$ (2.19)

and

$|\zeta_{1}(t_{\nu})|\leq CQ^{2}+\Big{|}\sum_{0\leq\mu<\nu}\\{(\Delta
q_{\mu})^{2}-\Delta\tilde{t}_{\mu}\\}\Big{|}\quad\text{(all }\nu),$ (2.20)

since also

$\sum_{\mu=0}^{N}(\Delta\tilde{t}_{\mu})\leq C\sum_{\mu=0}^{N}\Delta
t_{\mu}\leq C^{\prime}.$

We will show that

$\text{Prob}\Big{[}\max_{\nu}\Big{|}\sum_{0\leq\mu<\nu}\big{\\{}(\Delta
q_{\mu})^{2}-\Delta\tilde{t}_{\mu}\big{\\}}\Big{|}>CQ^{2}(\Delta
t_{\text{MAX}})^{1/2}\Big{]}\\\ \leq C\exp(-cQ^{2}).$ (2.21)

In view of (2.13), (2.14), and (2.21), estimates (2.19), (2.20) imply the
inequalities

$\text{Prob}[\max_{\nu}|\zeta_{1}(t_{\nu})|>CQ^{2}]\leq C\exp(-cQ^{2})$ (2.22)

and

$\text{Prob}[\max_{\nu}|\zeta_{2}(t_{\nu})|>CQ^{2}]\leq C\exp(-cQ^{2})$ (2.23)

for $Q$ as in (2.1). Thus, to prove (2.22) and (2.23), it remains only to
prove (2.21). Estimate (2.21) will have further applications in a later
section.

We now prove (2.21).

From (1.8), (1.12) we have

$\begin{split}\sum_{\mu<\nu}\\{(\Delta
q_{\mu})^{2}-\Delta\tilde{t}_{\mu}\\}=&\sum_{\mu<\nu}(aq_{\mu}+u_{\mu})^{2}(\Delta
t_{\mu}^{*})^{2}\\\ &+2\sum_{\mu<\nu}(aq_{\mu}+u_{\mu})(\Delta
t_{\mu}^{*})\Delta W_{\mu}\\\ &+\sum_{\mu<\nu}\\{(\Delta
W_{\mu})^{2}-\Delta\tilde{t}_{\mu}\\}\\\ \equiv&\text{TERM 1}(\nu)+\text{TERM
2}(\nu)+\text{TERM 3}(\nu),\end{split}$ (2.24)

with

$0\leq\text{TERM 1}(\nu)\leq
C\max_{\mu}\\{|q_{\mu}|+|u_{\mu}|\\}^{2}\cdot(\Delta
t_{\text{MAX}}),\;\text{all }\nu.$ (2.25)

To estimate $\text{TERM 2}(\nu)$, we fix $Q>C$ as in (2.1) and study the
random variables

$Y_{\nu}=\exp(-\hat{C}\lambda^{2}Q^{2}(\Delta t_{\text{MAX}})^{2}t_{\nu})\\\
\cdot\exp\Big{(}\lambda\sum_{\mu<\nu}(aq_{\mu}+u_{\mu})\cdot\mathbbm{1}_{|aq_{\mu}+u_{\mu}|<CQ}(\Delta
t_{\mu}^{*})\Delta W_{\mu}\Big{)}$ (2.26)

for a large enough constant $\hat{C}$, and for $\lambda\in\mathbb{R}$ to be
picked below. Since
$(aq_{\mu}+u_{\mu})\cdot\mathbbm{1}_{|aq_{\mu}+u_{\mu}|<CQ}$ is
$\mathcal{F}_{\nu}$-measurable for $\mu\leq\nu$, we have

$\begin{split}\text{E}[Y_{\nu+1}|\mathcal{F}_{\nu}]=&Y_{\nu}\cdot\exp(-\hat{C}\lambda^{2}Q^{2}(\Delta
t_{\text{MAX}})^{2}\Delta t_{\nu})\cdot\\\
&\text{E}[\exp([\lambda(aq_{\nu}+u_{\nu})\cdot\mathbbm{1}_{|aq_{\nu}+u_{\nu}|<CQ}\cdot(\Delta
t_{\nu}^{*})]\Delta W_{\nu}|\mathcal{F}_{\nu}]\\\
\leq&Y_{\nu}\cdot\exp(-\hat{C}\lambda^{2}Q^{2}(\Delta
t_{\text{MAX}})^{2}\Delta t_{\nu})\\\
&\cdot\exp(C[\lambda(aq_{\nu}+u_{\nu})\cdot\mathbbm{1}_{|aq_{\nu}+u_{\nu}|<CQ}\cdot(\Delta
t_{\nu}^{*})]^{2}\Delta\tilde{t}_{\nu}).\end{split}$ (2.27)

(Here, we use the fact that $\Delta W_{\nu}$ is independent of
$\mathcal{F}_{\nu}$, and normal with mean 0 and variance
$\Delta\tilde{t}_{\nu}$.)

If we take $\hat{C}$ large enough, then the product of the exponentials on the
right in (2.27) is less than 1. Thus,

$\text{E}[Y_{\nu+1}|\mathcal{F}_{\nu}]\leq Y_{\nu},\;\text{with}\;Y_{0}\equiv
1.$

So the $Y_{\nu}$ form a supermartingale. Consequently,

$\text{Prob}(\exists\nu\;\text{s.t.}\;Y_{\nu}>\exp(cQ^{2}))\leq\exp(-cQ^{2}).$
(2.28)

We now pick $\lambda=[\text{sgn}]\tilde{c}[(\Delta t_{\text{MAX}})]^{-1}$ with
$\text{sgn}=\pm 1$ and $\tilde{c}>0$ a small enough constant. Combining (2.26)
and (2.28) then yields the estimate

$\text{Prob}\Big{(}\exists\nu\;\text{s.t}\;[(\Delta
t_{\text{MAX}})]^{-1}\Big{|}\sum_{\mu<\nu}(aq_{\mu}+u_{\mu})\cdot\mathbbm{1}_{|aq_{\mu}+u_{\mu}|<CQ}(\Delta
t_{\mu}^{*})\Delta W_{\mu}\Big{|}>CQ^{2}\Big{)}\\\ \leq\exp(-cQ^{2}).$

Consequently,

$\text{Prob}\Big{(}\exists\nu\;\text{s.t.}\;\Big{|}\sum_{\mu<\nu}(aq_{\mu}+u_{\mu})(\Delta
t_{\mu}^{*})\Delta W_{\mu}\Big{|}>CQ^{2}(\Delta t_{\text{MAX}})\Big{)}\\\
\leq\exp(-cQ^{2})+\text{Prob}(\exists\mu\;\text{s.t.}\;|aq_{\mu}+u_{\mu}|>CQ).$

Recalling (2.24), we have

$\text{Prob}(\max_{\nu}|\text{TERM 2}(\nu)|>CQ^{2}(\Delta t_{\text{MAX}}))\\\
\leq\exp(-cQ^{2})+\text{Prob}(\max_{\mu}\\{|q_{\mu}|+|u_{\mu}|\\}>cQ).$ (2.29)

We turn our attention to $\text{TERM 3}(\nu)$. Let

$\mathcal{Z}_{\nu}=\exp(-\hat{C}\lambda^{2}(\Delta
t_{\text{MAX}})t_{\nu})\cdot\exp\bigg{(}\lambda\sum_{\mu<\nu}\\{(\Delta
W_{\mu})^{2}-\Delta\tilde{t}_{\mu}\\}\bigg{)}$ (2.30)

for a large enough constant $\hat{C}$, and for $\lambda\in\mathbb{R}$ to be
picked below.

Then $\mathcal{Z}_{0}\equiv 1$, and

$\text{E}[\mathcal{Z}_{\nu+1}|\mathcal{F}_{\nu}]=\mathcal{Z}_{\nu}\exp(-\hat{C}\lambda^{2}(\Delta
t_{\text{MAX}})\Delta t_{\nu})\cdot\text{E}[\exp(\lambda\\{(\Delta
W_{\nu})^{2}-\Delta\tilde{t}_{\nu}\\})].$ (2.31)

For $|\lambda|<c(\Delta t_{\text{MAX}})^{-1}$, we have

$\begin{split}\text{E}[\exp(\lambda\\{(\Delta
W_{\nu})^{2}-&\Delta\tilde{t}_{\nu}\\})]\\\
&=\exp(-\lambda\Delta\tilde{t}_{\nu})\cdot\frac{1}{\sqrt{2\pi\Delta\tilde{t}_{\nu}}}\int_{-\infty}^{\infty}e^{\lambda
x^{2}}e^{-x^{2}/2\Delta\tilde{t}_{\nu}}\ dx\\\
&=\exp(-\lambda\Delta\tilde{t}_{\nu})\cdot(1-2\lambda(\Delta\tilde{t}_{\nu}))^{-1/2}\\\
&\leq\exp(C\lambda^{2}(\Delta t_{\nu})^{2})\leq\exp(C\lambda^{2}(\Delta
t_{\text{MAX}})\Delta t_{\nu}).\end{split}$ (2.32)

Substituting (2.32) into (2.31), and taking $\hat{C}$ large enough, we find
that

$\text{E}[\mathcal{Z}_{\nu+1}|\mathcal{F}_{\nu}]\leq\mathcal{Z}_{\nu},$

i.e., the $\mathcal{Z}_{\nu}$ form a supermartingale. This holds for
$|\lambda|<c(\Delta t_{\text{MAX}})^{-1}$. Since $\mathcal{Z}_{0}\equiv 1$, it
follows that

$\text{Prob}(\exists\nu\;\text{s.t.}\;\mathcal{Z}_{\nu}>\exp(cQ^{2}))\leq\exp(-cQ^{2}).$

Taking $\lambda=(\Delta t_{\text{MAX}})^{-1/2}\cdot[\text{sgn}]$ with
$\text{sgn}=\pm 1$, we conclude that

$\text{Prob}\Big{(}\exists\nu\;\text{s.t.}\;\exp\Big{(}-\hat{C}t_{\nu}+(\Delta
t_{\text{MAX}})^{-1/2}\Big{|}\sum_{\mu<\nu}\\{(\Delta
W_{\mu})^{2}-\Delta\tilde{t}_{\mu}\\}\Big{|}\Big{)}>\exp(cQ^{2})\Big{)}\\\
\leq\exp(-cQ^{2}),$

so that

$\text{Prob}\Big{(}\max_{\nu}\Big{|}\sum_{\mu<\nu}\\{(\Delta
W_{\mu})^{2}-\Delta\tilde{t}_{\mu}\\}\Big{|}>CQ^{2}(\Delta
t_{\text{MAX}})^{1/2}\Big{)}\leq\exp(-cQ^{2}).$

Recalling (2.24), we conclude that

$\text{Prob}(\max_{\nu}|\text{TERM 3}(\nu)|>CQ^{2}(\Delta
t_{\text{MAX}})^{1/2}))\leq\exp(-cQ^{2}).$ (2.33)

Estimates (2.25), (2.29), (2.33) control TERM 1($\nu$), TERM 2$(\nu)$, TERM
3($\nu$). Substituting these estimates into (2.24), we learn that

$\text{Prob}\Big{(}\max_{\nu}\Big{|}\sum_{\mu<\nu}\\{(\Delta
q_{\mu})^{2}-\Delta\tilde{t}_{\mu}\\}\Big{|}>C^{\prime}Q^{2}(\Delta
t_{\text{MAX}})^{1/2}\Big{)}\\\ \leq
C\exp(-cQ^{2})+C\text{Prob}(\max_{\mu}\\{|q_{\mu}|+|u_{\mu}|\\}>cQ).$

Finally, recalling (2.13) and (2.14), we see that

$\text{Prob}\Big{(}\max_{\nu}\Big{|}\sum_{\mu<\nu}\\{(\Delta
q_{\mu})^{2}-\Delta\tilde{t}_{\mu}\\}\Big{|}>C^{\prime\prime}Q^{2}(\Delta
t_{\text{MAX}})^{1/2}\Big{)}\leq C\exp(-cQ^{2}),$

completing the proof of (2.21).

Next, we estimate

$\displaystyle\Delta
q^{\sigma}_{\nu}=q^{\sigma}(t_{\nu+1})-q^{\sigma}(t_{\nu}),$
$\displaystyle\Delta\zeta_{1,\nu}^{\sigma}=\zeta_{1}^{\sigma}(t_{\nu+1})-\zeta_{1}^{\sigma}(t_{\nu}),$
$\displaystyle\Delta\zeta_{2,\nu}^{\sigma}=\zeta_{2}^{\sigma}(t_{\nu+1})-\zeta_{2}^{\sigma}(t_{\nu}).$

Recall that

$\displaystyle\Delta
q^{\sigma}_{\nu}=(aq^{\sigma}_{\nu}+u^{\sigma}_{\nu})\Delta t_{\nu}^{*}+\Delta
W_{\nu},$ $\displaystyle\Delta\zeta_{1,\nu}^{\sigma}=q^{\sigma}_{\nu}(\Delta
q^{\sigma}_{\nu}-u^{\sigma}_{\nu}\Delta t_{\nu}),$
$\displaystyle\Delta\zeta_{2,\nu}^{\sigma}=(q^{\sigma}_{\nu})^{2}\Delta
t_{\nu}.$

Let $Q\geq C$ for large enough $C$, and suppose $|q^{\sigma}_{\nu}|\leq Q.$
Then also $|u^{\sigma}_{\nu}|\leq C[|q^{\sigma}_{\nu}|+1]\leq C^{\prime}Q$, so

$\displaystyle|\Delta q^{\sigma}_{\nu}|\leq CQ(\Delta t_{\nu})+|\Delta
W_{\nu}|,$ $\displaystyle|\Delta\zeta_{1,\nu}^{\sigma}|\leq Q(|\Delta
q^{\sigma}_{\nu}|+CQ\Delta t_{\nu})\leq C^{\prime}Q^{2}(\Delta
t_{\nu})+Q|\Delta W_{\nu}|,\;\text{and}$
$\displaystyle|\Delta\zeta_{2,\nu}^{\sigma}|\leq Q^{2}(\Delta t_{\nu}),$

hence for $p\geq 1$, we have

$(|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|)^{p}\leq
C_{p}Q^{2p}(\Delta t_{\nu})^{p}+C_{p}Q^{p}|\Delta W_{\nu}|^{p}.$

Recall that $\Delta W_{\nu}$ is independent of $\mathcal{F}_{\nu}$ and normal,
with mean 0 and variance at most $C\Delta t_{\nu}$. it follows that, for any
$p\geq 1$, we have

$\text{E}[(|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|)^{p}|\mathcal{F}_{\nu}]\leq
C_{p}Q^{2p}(\Delta t_{\nu})^{p}+C_{p}Q^{p}(\Delta t_{\nu})^{p/2}$ (2.34)

whenever $|q^{\sigma}_{\nu}|\leq Q$. (Recall, $q^{\sigma}_{\nu}$ is
deterministic once we condition on $\mathcal{F}_{\nu}$.) In particular, (2.34)
implies that

$\text{Prob}[|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|>(\Delta
t_{\nu})^{2/5}|\mathcal{F}_{\nu}]\leq C(\Delta t_{\nu})^{1000}$ (2.35)

if $|q^{\sigma}_{\nu}|\leq Q$ and $C\leq Q\leq(\Delta t_{\nu})^{-1/1000}$.

Together with (2.34) for $p=2,4$ and Cauchy-Schwartz, (2.35) implies the
estimate

$\text{E}[(|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|)^{p}\cdot\mathbbm{1}_{|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|>(\Delta
t_{\nu})^{2/5}}|\mathcal{F}_{\nu}]\\\ \leq C(\Delta
t_{\nu})^{100}\;\text{for}\;p=1,2,$

provided $|q^{\sigma}_{\nu}|\leq Q$ and $C^{\prime}\leq Q\leq(\Delta
t_{\nu})^{-1/1000}$ for large enough $C^{\prime}$.

Next, we estimate
$|q^{\sigma}(t)-q^{\sigma}_{\nu}|=|q^{\sigma}(t)-q^{\sigma}(t_{\nu})|$ for
$t\in[t_{\nu},t_{\nu+1}]$.

Recall that

$q^{\sigma}(t)-q^{\sigma}_{\nu}=(aq^{\sigma}_{\nu}+u^{\sigma}_{\nu})\cdot\bigg{[}\frac{e^{a(t-t_{\nu})}-1}{a}\bigg{]}+\int_{t_{\nu}}^{t}e^{a(t-s)}\
dW(s)$

for $t\in[t_{\nu},t_{\nu+1}]$.

If $|q^{\sigma}_{\nu}|\leq Q$ with $Q\geq C$ (for large enough $C$), then also
$|u^{\sigma}_{\nu}|\leq CQ$, hence

$|q^{\sigma}(t)-q^{\sigma}_{\nu}|\leq CQ(\Delta
t_{\nu})+\bigg{|}\int_{t_{\nu}}^{t}e^{a(t-s)}\ dW(s)\bigg{|}$

for $t\in[t_{\nu},t_{\nu+1}]$.

Applying the reflection principle [17] to the Gaussian process

$W^{\\#}(t)=\int_{t_{\nu}}^{t}e^{-as}\ dW(s)\qquad(t\geq t_{\nu}),$

we see that

$\text{Prob}\Big{[}\max_{t\in[t_{\nu},t_{\nu+1}]}\Big{|}\int_{t_{\nu}}^{t}e^{a(t-s)}\
dW(s)\Big{|}>CQ(\Delta t_{\nu})^{1/2}\Big{]}\leq C\exp(-cQ^{2}).$

Since $\int_{t_{\nu}}^{t}e^{a(t-s)}\ dW(s)$ $(t\geq t_{\nu})$ is independent
of $\mathcal{F}_{\nu}$, it now follows that

$\text{Prob}\Big{[}\max_{t\in[t_{\nu},t_{\nu+1}]}|q^{\sigma}(t)-q^{\sigma}_{\nu}|>C^{\prime}Q(\Delta
t_{\nu})^{1/2}\Big{|}\mathcal{F}_{\nu}\Big{]}\leq C\exp(-cQ^{2})$

provided $|q^{\sigma}_{\nu}|\leq Q$ and $C\leq Q$ for large enough $C$. Taking

$Q=(\Delta t_{\nu})^{2/5}/[C^{\prime}(\Delta t_{\nu})^{1/2}],$

we find that

$\text{Prob}\Big{[}\max_{t\in[t_{\nu},t_{\nu+1}]}|q^{\sigma}(t)-q^{\sigma}_{\nu}|>(\Delta
t_{\nu})^{2/5}\Big{|}\mathcal{F}_{\nu}\Big{]}\leq C(\Delta t_{\nu})^{1000}$
(2.36)

provided $|q^{\sigma}_{\nu}|\leq c\cdot(\Delta t_{\nu})^{-1/10}$.

Let us summarize the results of the above discussion.

######  Lemma 1 (Lemma on Rare Events).

We condition on $a_{\emph{TRUE}}=a$, $\vec{\xi}=\vec{\eta}$. Fix a strategy
$\sigma$. For constants $c,C$ depending only on upper bounds for $|q_{0}|$,
$a_{\emph{MAX}}$, $C_{\emph{TAME}}$, and $T$, the following holds.

Suppose $\Delta t_{\emph{MAX}}\equiv\max_{\nu}(t_{\nu+1}-t_{\nu})<c$.

Then, for $Q>C$, the following hold with probability $>1-\exp(-cQ^{2})$:

  * •

$|q^{\sigma}(t_{\nu})|,|u^{\sigma}(t_{\nu})|\leq Q$ for all $\nu$

  * •

$|\zeta_{1}^{\sigma}(t_{\nu})|,|\zeta_{2}^{\sigma}(t_{\nu})|\leq Q^{2}$ for
all $\nu$

  * •

$\big{|}\sum_{0\leq\mu<\nu}(q^{\sigma}(t_{\mu+1})-q^{\sigma}(t_{\mu}))^{2}-t_{\nu}\big{|}\leq
Q^{2}(\Delta t_{\emph{MAX}})^{1/2}$ for all $\nu$.

Moreover, suppose we fix $\nu$ and condition on $\mathcal{F}_{\nu}$, the sigma
algebra of events determined by the $q^{\sigma}(t_{\mu})$ $(0\leq\mu\leq\nu)$.
Suppose that $|q_{\nu}|<(\Delta t_{\nu})^{-1/1000}$. Then

$\emph{E}[(|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|)^{p}\cdot\mathbbm{1}_{|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|>(\Delta
t_{\nu})^{2/5}}|\mathcal{F}_{\nu}]\leq C(\Delta t_{\nu})^{100}$

for $p=1,2$, and

$\emph{Prob}[|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|>(\Delta
t_{\nu})^{2/5}|\mathcal{F}_{\nu}]\leq C(\Delta t_{\nu})^{1000}.$

Also, we have

$\emph{Prob}\Big{[}\max_{t\in[t_{\nu},t_{\nu+1}]}|q^{\sigma}(t)-q^{\sigma}_{\nu}|>(\Delta
t_{\nu})^{2/5}|\mathcal{F}_{\nu}\Big{]}\leq C(\Delta t_{\nu})^{1000}$

provided $|q^{\sigma}_{\nu}|\leq c\cdot(\Delta t_{\nu})^{-1/10}$. Finally, for
$Q\geq C$, we have

$\emph{Prob}\Big{[}\max_{t\in[t_{\nu},t_{\nu+1}]}|q^{\sigma}(t)-q^{\sigma}_{\nu}|>C^{\prime}Q(\Delta
t_{\nu})^{1/2}|\mathcal{F}_{\nu}\Big{]}\leq
C\exp(-cQ^{2})\;\text{if}\;|q_{\nu}|\leq Q.$

###### Proof.

To deduce the third bullet point from (2.21), we note that

$\sum_{0\leq\mu<\nu}(\Delta\tilde{t}_{\mu})=\sum_{0\leq\mu<\nu}[(\Delta
t_{\mu})+O((\Delta t_{\mu})^{2})]=t_{\nu}\cdot(1+O(\Delta t_{\text{MAX}})).$

The remaining assertions of the lemma have already been proved as stated. ∎

###  3 The Probability Density

We continue to adopt the assumptions and notation of Section 2. Our goal is to
derive an approximate formula for the joint probability density of
$(q_{1},\dots,q_{\bar{N}})=(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\bar{N}}))$
for fixed ${\bar{N}}\leq N$. Let us denote this joint probability density by
$\Phi(\bar{q}_{1},\dots,\bar{q}_{\bar{N}}).$ Thus,

$\text{Prob}((q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\bar{N}}))\in
E)=\int_{E}\Phi(\bar{q}_{1},\dots,\bar{q}_{\bar{N}})\ d\bar{q}_{1}\cdots
d\bar{q}_{\bar{N}}$ (3.1)

for measurable sets $E\subset\mathbb{R}^{\bar{N}}$.

By formula (1.8),

$\Delta q_{\nu}=(aq_{\nu}+u_{\nu})\Delta t_{\nu}^{*}+\Delta W_{\nu}$

with $\Delta W_{\nu}$ mutually independent and normal, with mean 0 and
variance $\Delta\tilde{t}_{\nu}$; consequently, the joint probability $\Phi$
is given by

$\Phi(\bar{q}_{1},\dots,\bar{q}_{\bar{N}})=\prod_{\nu=0}^{{\bar{N}}-1}\phi_{\nu}$
(3.2)

with

$\phi_{\nu}=\frac{1}{\sqrt{2\pi\Delta\tilde{t}_{\nu}}}\exp\bigg{(}-\frac{1}{2\Delta\tilde{t}_{\nu}}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}^{*}]^{2}\bigg{)}.$ (3.3)

Here, $\Delta\bar{q}_{\nu}\equiv\bar{q}_{\nu+1}-\bar{q}_{\nu}$,
$\bar{q}_{0}\equiv q_{0}$, and $\bar{u}_{\nu}$ denotes the control exercised
by the strategy $\sigma$ at time $t_{\nu}$ given that
$q^{\sigma}(t_{\mu})=\bar{q}_{\mu}$ for $0\leq\mu\leq\nu$ and
$\vec{\xi}=\vec{\eta}$. Note that $\bar{u}_{\nu}$ is determined by
$\bar{q}_{1},\dots,\bar{q}_{\nu}$ (and $q_{0})$.

We make the following assumptions on $(\bar{q}_{1},\dots,\bar{q}_{\bar{N}})$:

$\displaystyle\max_{\nu}(|\bar{q}_{\nu}|+|\bar{u}_{\nu}|)\leq Q,$ (3.4)
$\displaystyle\Big{|}\sum_{\nu}(\Delta\bar{q}_{\nu})^{2}-t_{\bar{N}}\Big{|}\leq
Q^{2}(\Delta t_{\text{MAX}})^{1/4},$ (3.5)

with $Q\geq C$ given.

Thanks to Lemma 1, (3.4) and (3.5) are very likely true for
$(\bar{q}_{1},\dots,\bar{q}_{\bar{N}})=(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\bar{N}})).$
Under assumptions (3.4) and (3.5) we will simplify (3.2), (3.3).

First of all, since $\Delta t_{\nu}^{*}=\Delta t_{\nu}+O((\Delta
t_{\nu})^{2})$ we have

$[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}^{*}]=[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]+\text{ERR}_{\nu},$

with

$\text{ERR}_{\nu}=O(Q(\Delta t_{\nu})^{2})$

thanks to (3.4).

Hence,

$\displaystyle[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}^{*}]^{2}=$
$\displaystyle[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})^{2}\Delta
t_{\nu}]^{2}+\text{ERR}_{\nu}^{2}$
$\displaystyle+2\text{ERR}_{\nu}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]$ $\displaystyle=$
$\displaystyle[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}+O(Q^{2}(\Delta t_{\nu})^{3})$
$\displaystyle+2\text{ERR}_{\nu}(\Delta\bar{q}_{\nu}).$

Therefore,

$\begin{split}\sum_{\nu}\frac{1}{2\Delta\tilde{t}_{\nu}}[\Delta\bar{q}_{\nu}-&(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}^{*}]^{2}\\\
=&\sum_{\nu}\frac{1}{2\Delta\tilde{t}_{\nu}}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}+\sum_{\nu}O(Q^{2}(\Delta t_{\nu})^{2})\\\
&+\sum_{\nu}\frac{\text{ERR}_{\nu}}{\Delta\tilde{t}_{\nu}}(\Delta\bar{q}_{\nu}).\end{split}$
(3.6)

The last sum has absolute value at most

$C\sum_{\nu}(\Delta
t_{\nu})^{-1/2}\bigg{\\{}\frac{\text{ERR}_{\nu}}{\Delta\tilde{t}_{\nu}}\bigg{\\}}^{2}+C\sum_{\nu}(\Delta
t_{\nu})^{1/2}(\Delta\bar{q}_{\nu})^{2}.$ (3.7)

The expression (3.7) is, in turn, at most

$\displaystyle\sum_{\nu}O(Q^{2}(\Delta t_{\nu})^{3/2})+$ $\displaystyle
C(\Delta t_{\text{MAX}})^{1/2}\sum_{\nu}(\Delta\bar{q}_{\nu})^{2}$
$\displaystyle=$ $\displaystyle O(Q^{2}(\Delta t_{\text{MAX}})^{1/2})+C(\Delta
t_{\text{MAX}})^{1/2}\Big{|}\sum_{\nu}(\Delta\bar{q}_{\nu})^{2}-t_{\bar{N}}\Big{|}$
$\displaystyle+Ct_{\bar{N}}(\Delta t_{\text{MAX}})^{1/2}$ $\displaystyle=$
$\displaystyle O(Q^{2}(\Delta t_{\text{MAX}})^{1/2}),$

thanks to (3.5). Therefore, (3.6) implies that

$\sum_{\nu}\frac{1}{2\Delta\tilde{t}_{\nu}}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}^{*}]^{2}\\\
=\sum_{\nu}\frac{1}{2\Delta\tilde{t}_{\nu}}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}+O(Q^{2}(\Delta t_{\text{MAX}})^{1/2}).$ (3.8)

We want to replace $\Delta\tilde{t}_{\nu}$ by $\Delta t_{\nu}$ on the right in
(3.8). To do so, note that

$\frac{1}{2\Delta\tilde{t}_{\nu}}-\frac{1}{2\Delta
t_{\nu}}=-\frac{a}{2}+O(\Delta t_{\nu}),$

thanks to (1.11). Consequently,

$\begin{split}\sum_{\nu}\Big{(}\frac{1}{2\Delta\tilde{t}_{\nu}}-\frac{1}{2\Delta
t_{\nu}}\Big{)}[\Delta\bar{q}_{\nu}&-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}\\\ =&\sum_{\nu}[a+O(\Delta
t_{\nu})](a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta t_{\nu}(\Delta\bar{q}_{\nu})\\\
&+\sum_{\nu}\Big{[}-\frac{a}{2}+O(\Delta
t_{\nu})\Big{]}(\Delta\bar{q}_{\nu})^{2}\\\
&+\sum_{\nu}\Big{[}-\frac{a}{2}+O(\Delta
t_{\nu})\Big{]}(a\bar{q}_{\nu}+\bar{u}_{\nu})^{2}(\Delta t_{\nu})^{2}\\\
\equiv&\text{Term}\;\alpha+\text{Term}\;\beta+\text{Term}\;\gamma\end{split}$
(3.9)

Now

$\displaystyle\text{Term}\;\beta$
$\displaystyle=-\frac{a}{2}\sum_{\nu}(\Delta\bar{q}_{\nu})^{2}+O(\Delta
t_{\text{MAX}})\sum_{\nu}(\Delta\bar{q}_{\nu})^{2}$
$\displaystyle=-\frac{a}{2}t_{\bar{N}}+O(Q^{2}(\Delta t_{\text{MAX}})^{1/4})$

by (3.5), while

$\text{Term}\;\gamma=\sum_{\nu}O(Q^{2}(\Delta t_{\nu})^{2})=O(Q^{2}(\Delta
t_{\text{MAX}}))$

by (3.4). To estimate $\text{Term}\;\alpha$, we apply (3.4) and (3.5) to
write:

$\displaystyle|\text{Term}\;\alpha|\leq$ $\displaystyle
C\sum_{\nu}(a\bar{q}_{\nu}+\bar{u}_{\nu})^{2}(\Delta
t_{\nu})^{3/2}+C\sum_{\nu}(\Delta t_{\nu})^{1/2}(\Delta\bar{q}_{\nu})^{2}$
$\displaystyle\leq$ $\displaystyle O(Q^{2}(\Delta
t_{\text{MAX}})^{1/2})+C(\Delta
t_{\text{MAX}})^{1/2}\sum_{\nu}(\Delta\bar{q}_{\nu})^{2}$ $\displaystyle=$
$\displaystyle O(Q^{2}(\Delta t_{\text{MAX}})^{1/2})+C(\Delta
t_{\text{MAX}})^{1/2}\Big{[}\sum_{\nu}(\Delta\bar{q}_{\nu})^{2}-t_{\bar{N}}\Big{]}$
$\displaystyle=$ $\displaystyle O(Q^{2}(\Delta t_{\text{MAX}})^{1/2}).$

Combining our estimates for Terms $\alpha,\beta,\gamma$, and recalling (3.9),
we learn that

$\sum_{\nu}\Big{(}\frac{1}{2\Delta\tilde{t}_{\nu}}-\frac{1}{2\Delta
t_{\nu}}\Big{)}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}=-\frac{a}{2}t_{\bar{N}}+O(Q^{2}(\Delta t_{\text{MAX}})^{1/4}).$

Consequently, (3.8) implies that

$\sum_{\nu}\frac{1}{2\Delta\tilde{t}_{\nu}}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}^{*}]^{2}\\\ =-\frac{a}{2}t_{\bar{N}}+\sum_{\nu}\frac{1}{2\Delta
t_{\nu}}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}+O(Q^{2}(\Delta t_{\text{MAX}})^{1/4}).$ (3.10)

Again applying (1.11), we see that

$\frac{1}{\sqrt{2\pi\Delta\tilde{t}_{\nu}}}=\frac{1}{\sqrt{2\pi\Delta
t_{\nu}}}\cdot(1-\frac{1}{2}a\Delta t_{\nu}+O(\Delta t_{\nu})^{2})\\\
=\frac{1}{\sqrt{2\pi\Delta t_{\nu}}}\exp(-\frac{1}{2}a\Delta t_{\nu}+O(\Delta
t_{\nu})^{2})$

so that

$\begin{split}\prod_{\nu=0}^{{\bar{N}}-1}\frac{1}{\sqrt{2\pi\Delta\tilde{t}_{\nu}}}=&\bigg{(}\prod_{\nu=0}^{{\bar{N}}-1}\frac{1}{\sqrt{2\pi\Delta
t_{\nu}}}\bigg{)}\exp\Big{(}-\frac{a}{2}\sum_{\nu}\\{(\Delta t_{\nu})+O(\Delta
t_{\nu})^{2}\\}\Big{)}\\\
=&\bigg{(}\prod_{\nu=0}^{{\bar{N}}-1}\frac{1}{\sqrt{2\pi\Delta
t_{\nu}}}\bigg{)}\exp\Big{(}-\frac{a}{2}t_{\bar{N}}+O(\Delta
t_{\text{MAX}})\Big{)}.\end{split}$ (3.11)

Putting (3.10) and (3.11) into (3.2) and (3.3), we find that

$\begin{split}\Phi(\bar{q}_{1},\dots,\bar{q}_{\bar{N}})=\prod_{\nu=0}^{{\bar{N}}-1}&\Big{\\{}\frac{1}{\sqrt{2\pi\Delta
t_{\nu}}}\exp\Big{(}-\frac{1}{2\Delta
t_{\nu}}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}\Big{)}\Big{\\}}\\\ &\cdot(1+O(Q^{2}(\Delta
t_{\text{MAX}})^{1/4})).\end{split}$ (3.12)

In particular, the factors $\exp(\frac{a}{2}t_{\bar{N}})$ arising from (3.10)
and (3.11) cancel.

We record our result (3.12) as a lemma.

######  Lemma 2 (Lemma on the Probability Distribution).

We condition on $a_{\emph{TRUE}}=a$, $\vec{\xi}=\vec{\eta}$.

Then, for constants $c,C$ determined by $q_{0}$, $a_{\emph{MAX}}$,
$C_{\emph{TAME}},$ and an upper bound for $T$, the following holds.

Suppose $\Delta t_{\emph{MAX}}=\max_{\nu}(t_{\nu+1}-t_{\nu})<c$. Fix
${\bar{N}}\leq N$. Let $\Phi(\bar{q}_{1},\dots,\bar{q}_{\bar{N}})$ be the
joint probability density for
$(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\bar{N}}))$.

Let $Q>C$, and suppose $(\bar{q}_{1},\dots,\bar{q}_{\bar{N}})$ satisfies

$\displaystyle\max_{\nu}(|\bar{q}_{\nu}|+|\bar{u}_{\nu}|)\leq Q\;\text{and}$
$\displaystyle\Big{|}\sum_{\nu}(\bar{q}_{\nu+1}-\bar{q}_{\nu})^{2}-t_{\bar{N}}\Big{|}\leq
Q^{2}(\Delta t_{\emph{MAX}})^{1/4},$

where $\bar{u}_{\nu}$ is the control exercised by the strategy $\sigma$ at
time $t_{\nu}$ when

$(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\nu}))=(\bar{q}_{1},\dots,\bar{q}_{\nu})\;\text{and}\;\vec{\xi}=\vec{\eta}.$

Then

$\displaystyle\Phi(\bar{q}_{1},\dots,\bar{q}_{\bar{N}})=\prod_{\nu=0}^{{\bar{N}}-1}$
$\displaystyle\Big{\\{}\frac{1}{\sqrt{2\pi\Delta
t_{\nu}}}\exp\Big{(}-\frac{1}{2\Delta
t_{\nu}}[\Delta\bar{q}_{\nu}-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}\Big{)}\Big{\\}}$ $\displaystyle\cdot(1+O(Q^{2}(\Delta
t_{\emph{MAX}})^{1/4})).$

Here, $\Delta\bar{q}_{\nu}=\bar{q}_{\nu+1}-\bar{q}_{\nu}$ (with
$\bar{q}_{0}=q_{0}$), $\Delta t_{\nu}=t_{\nu+1}-t_{\nu}$, and $O(Q^{2}(\Delta
t_{\emph{MAX}})^{1/4})$ denotes a quantity whose absolute value is at most
$CQ^{2}(\Delta t_{\emph{MAX}})^{1/4}$.

###  4 Analytic Continuation

In this section we prepare to make an analytic continuation of the function
mapping $a\in[-a_{\text{MAX}},+a_{\text{MAX}}]$ to the expected cost incurred
by a strategy $\sigma$ assuming that $a_{\text{MAX}}=a$. We set up notation.

We fix a tame strategy $\sigma=(\sigma_{\nu})_{0\leq\nu<N}$ and coin flips
$\vec{\eta}$; we write $\bar{u}_{\nu}$ to denote the control exercised by the
strategy $\sigma$ at time $t_{\nu}$ assuming that
$q^{\sigma}(t_{\mu})=\bar{q}_{\mu}$ for $1\leq\mu\leq\nu$ and
$\vec{\xi}=\vec{\eta}$ (i.e.,
$\bar{u}_{\nu}=u^{\sigma}(t_{\nu})=\sigma_{\nu}(\bar{q}_{1},\dots,\bar{q}_{\nu},\vec{\eta})$).

We define functions

$\psi_{\nu}(\Delta\bar{q},\bar{q},\bar{u},a):=\frac{1}{\sqrt{2\pi\Delta
t_{\nu}}}\exp\Big{(}-\frac{[\Delta\bar{q}-(a\bar{q}+\bar{u})\Delta
t_{\nu}]^{2}}{2\Delta t_{\nu}}\Big{)}$ (4.1)

and

$\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a)=\prod_{\nu=0}^{N-1}\psi_{\nu}(\Delta\bar{q}_{\nu},\bar{q}_{\nu},\bar{u}_{\nu}(\bar{q}_{1},\dots,\bar{q}_{\nu}),a).$
(4.2)

In (4.1), $\Delta\bar{q}$, $\bar{q}$, $\bar{u}$ are real variables, while
$a\in\mathbb{C}$. In (4.2),
$\Delta\bar{q}_{\nu}:=\bar{q}_{\nu+1}-\bar{q}_{\nu}$, with
$\bar{q}_{0}:=q_{0}$. Again, in (4.2), $a\in\mathbb{C}$.

We introduce a set

$\begin{split}E=\\{(\bar{q}_{1},\dots,\bar{q}_{N})\in\mathbb{R}^{N}:&\max_{\nu}|\bar{q}_{\nu}|\leq(\Delta
t_{\text{MAX}})^{-1/16}\\\
&\text{and}\;|\sum_{\nu}(\bar{q}_{\nu+1}-\bar{q}_{\nu})^{2}-T|\leq(\Delta
t_{\text{MAX}})^{1/8}\\}.\end{split}$ (4.3)

We denote by

$\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a)\qquad(a\in[-a_{\text{MAX}},+a_{\text{MAX}}])$

the probability density for $(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))$
assuming that $a_{\text{TRUE}}=a$ and $\vec{\xi}=\vec{\eta}$.

According to Lemma 1 and Lemma 2, we have

$\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a)=\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a)\cdot(1+\text{ERR}(\bar{q}_{1},\dots,\bar{q}_{N},a))$
(4.4)

for $(\bar{q}_{1},\dots,\bar{q}_{N})\in E$ and
$a\in[-a_{\text{MAX}},a_{\text{MAX}}]$, with

$|\text{ERR}(\bar{q}_{1},\dots,\bar{q}_{N},a)|\leq(\Delta
t_{\text{MAX}})^{1/8};$ (4.5)

and

$\text{E}_{a,\vec{\eta}}[\mathbbm{1}_{(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))\notin
E}]\leq C\exp(-c(\Delta t_{\text{MAX}})^{-1/8}).$ (4.6)

Since $|u^{\sigma}|\leq C[|q^{\sigma}|+1]$ for a tame rule $\sigma$, we have,
for $a\in[-a_{\text{MAX}},a_{\text{MAX}}]$,

$\sum_{\nu}\\{(u^{\sigma}_{\nu})^{2}+(q^{\sigma}_{\nu})^{2}\\}\Delta
t_{\nu}\leq C\max_{\nu}|q_{\nu}|^{2}+C.$ (4.7)

By Lemma 1, we have

$\text{E}_{a,\vec{\eta}}[\\{\max_{\nu}|q^{\sigma}(t_{\nu})|^{2}+|u^{\sigma}(t_{\nu})|^{2}\\}^{2}]\leq
C.$ (4.8)

We study the function

$[-a_{\text{MAX}},+a_{\text{MAX}}]\ni
a\mapsto\text{E}_{{a,\vec{\eta}}}\Big{[}\sum_{\nu=0}^{N-1}\\{(q^{\sigma}(t_{\nu}))^{2}+(u^{\sigma}(t_{\nu}))^{2}\\}\Delta
t_{\nu}\Big{]};$

we denote this function by $\text{ECOST}(a)$.

The above definitions and estimates yield:

$\begin{split}\text{ECOST}(a)=&\text{E}_{a,\vec{\eta}}\Big{[}\sum_{\nu=0}^{N-1}\\{(q^{\sigma}(t_{\nu}))^{2}+(u^{\sigma}(t_{\nu}))^{2}\\}\Delta
t_{\nu}\cdot\mathbbm{1}_{(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))\in
E}\Big{]}\\\ &+\text{ERROR 1}(a),\end{split}$ (4.9)

with

$\begin{split}|\text{ERROR
1}(a)|\leq&\text{E}_{a,\vec{\eta}}[C\max_{\nu}\\{|q^{\sigma}(t_{\nu})|^{2}+|u^{\sigma}(t_{\nu})|^{2}\\}\cdot\mathbbm{1}_{(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\nu}))\notin
E}]\\\
\leq&\Big{(}\text{E}_{a,\vec{\eta}}[C\max\\{|q^{\sigma}(t_{\nu})|^{2}+|u^{\sigma}(t_{\nu})|^{2}\\}^{2}])^{1/2}\\\
&\cdot\Big{(}\text{Prob}_{a,\vec{\eta}}((q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))\notin
E)\Big{)}^{1/2}\\\ \leq&C^{\prime}\exp(-c^{\prime}(\Delta
t_{\text{MAX}})^{-1/8}).\end{split}$ (4.10)

Moreover,

$\displaystyle\text{E}_{a,\vec{\eta}}\Big{[}\sum_{\nu}$
$\displaystyle\\{(q^{\sigma}(t_{\nu}))^{2}+(u^{\sigma}(t_{\nu}))^{2}\\}\Delta
t_{\nu}\cdot\mathbbm{1}_{(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\nu}))\in
E}\Big{]}$ $\displaystyle=$
$\displaystyle\int_{(\bar{q}_{1},\dots,\bar{q}_{N})\in
E}\Big{(}\sum_{\nu}\\{\bar{q}_{\nu}^{2}+\bar{u}_{\nu}^{2}\\}\Delta
t_{\nu}\Big{)}\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a)\ d\bar{q}_{1}\cdots
d\bar{q}_{N}$ $\displaystyle=$
$\displaystyle\int_{(\bar{q}_{1},\dots,\bar{q}_{N})\in
E}\Big{\\{}\Big{(}\sum_{\nu}\\{\bar{q}_{\nu}^{2}+\bar{u}_{\nu}^{2}\\}\Delta
t_{\nu}\Big{)}\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a)$
$\displaystyle\quad\cdot(1+\text{ERR}(\bar{q}_{1},\dots,\bar{q}_{N},a))\
d\bar{q}_{1}\cdots d\bar{q}_{N}\Big{\\}}$ $\displaystyle=$
$\displaystyle(1+\text{ERROR
2}(a))\cdot\int_{(\bar{q}_{1},\dots,\bar{q}_{N})\in
E}\Big{\\{}\Big{(}\sum_{\nu}\\{\bar{q}_{\nu}^{2}+\bar{u}_{\nu}^{2}\\}(\Delta
t_{\nu})\Big{)}$
$\displaystyle\qquad\qquad\qquad\qquad\;\cdot\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a)\
d\bar{q}_{1}\cdots d\bar{q}_{N}\Big{\\}},$

with

$|\text{ERROR 2}(a)|\leq C(\Delta t_{\text{MAX}})^{1/8},$ (4.11)

thanks to (4.4), (4.5). Together with (4.9), (4.10), this yields

$\displaystyle\text{ECOST}(a)=$ $\displaystyle\text{ERROR 1}(a)$
$\displaystyle+(1+\text{ERROR
2}(a))\cdot\int_{E}\Big{\\{}\Big{(}\sum_{\nu}\\{\bar{q}_{\nu}^{2}+\bar{u}_{\nu}^{2}\\}\Delta
t_{\nu}\Big{)}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\;\;\;\cdot\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a)\Big{\\}}\
d\bar{q}_{1}\cdots d\bar{q}_{N}$

with $\text{ERROR 1}(a)$, $\text{ERROR 2}(a)$ controlled by (4.10), (4.11).
Since also

$0\leq\text{ECOST}(a)\leq C$

by Lemma 1, it follows that

$\begin{split}\text{ECOST}(a)=&\int_{(\bar{q}_{1},\dots,\bar{q}_{N})\in
E}\Big{\\{}\Big{(}\sum_{\nu}\\{\bar{q}_{\nu}^{2}+\bar{u}_{\nu}^{2}\\}\Delta
t_{\nu}\Big{)}\\\
&\quad\cdot\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a)\Big{\\}}\;d\bar{q}_{1}\cdots
d\bar{q}_{N}+\text{ERROR 3}(a),\end{split}$ (4.12)

with

$|\text{ERROR 3}(a)|\leq C(\Delta t_{\text{MAX}})^{1/8}.$ (4.13)

Equation (4.12) and estimate (4.13) hold for
$a\in[-a_{\text{MAX}},+a_{\text{MAX}}]$. We make an analytic continuation of
the integral in (4.12) from $a\in[-a_{\text{MAX}},+a_{\text{MAX}}]$ to
$a=a_{R}+ia_{I}$ in the rectangle

$\mathcal{R}=\\{a_{R}+ia_{I}:a_{R}\in[-a_{\text{MAX}},a_{\text{MAX}}],a_{I}\in[-\delta,\delta]\\}$

for a small $\delta>0$ to be picked below.

We write $I(a)$ to denote the integral in (4.12). Thus,

$I(a)=\int_{E}\Big{(}\sum_{\nu}\\{\bar{q}_{\nu}^{2}+\bar{u}_{\nu}^{2}\\}\Delta
t_{\nu}\Big{)}\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a)\ d\bar{q}_{1}\cdots
d\bar{q}_{N}$ (4.14)

for $a=a_{R}+ia_{I}\in\mathcal{R}$, and

$|\text{ECOST}(a)-I(a)|\leq C(\Delta
t_{\text{MAX}})^{1/8}\;\text{for}\;a\in[-a_{\text{MAX}},+a_{\text{MAX}}].$
(4.15)

A glance at (4.1), (4.2) shows that the integrand in (4.14) has the form

$B(\bar{q}_{1},\dots,\bar{q}_{N})\exp(a^{2}G_{2}(\bar{q}_{1},\dots,\bar{q}_{N})+aG_{1}(\bar{q}_{1},\dots,\bar{q}_{N})+G_{0}(\bar{q}_{1},\dots,\bar{q}_{N})),$

where $B,G_{0},G_{1},G_{2}$ are bounded measurable functions of
$(\bar{q}_{1},\dots,\bar{q}_{N})$ on $E$. Moreover, the region of integration,
$E$, is bounded; see (4.3). Therefore, $I(a)$ is an analytic function on
$\mathcal{R}$.

Next, we estimate

$\int_{E}\Big{(}\sum_{\nu}\\{\bar{q}_{\nu}^{2}+\bar{u}_{\nu}^{2}\\}\Delta
t_{\nu}\Big{)}|\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a_{R}+ia_{I})|\;d\bar{q}_{1}\cdots
d\bar{q}_{N}$ (4.16)

for $a_{R}+ia_{I}\in\mathcal{R}$. From (4.1) we have

$|\psi_{\nu}(\Delta\bar{q},\bar{q},\bar{u},a_{R}+ia_{I})|=\exp(\frac{a_{I}^{2}}{2}\bar{q}^{2}\Delta
t_{\nu})\psi_{\nu}(\Delta\bar{q},\bar{q},\bar{u},a_{R}),$

hence (4.2) implies that

$|\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a_{R}+ia_{I})|=\exp\Big{(}\sum_{\nu}\frac{a_{I}^{2}}{2}\bar{q}_{\nu}^{2}\Delta
t_{\nu}\Big{)}\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a_{R}).$

Moreover, for $(\bar{q}_{1},\dots,\bar{q}_{N})\in E$ and
$a_{R}\in[-a_{\text{MAX}},+a_{\text{MAX}}]$, (4.4) and (4.5) yield

$\Psi(\bar{q}_{1},\dots,\bar{q}_{N},a_{R})\leq
2\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a_{R}).$

Consequently, the integrand in (4.16) is at most

$2\Big{(}\sum_{\nu}\\{\bar{q}_{\nu}^{2}+\bar{u}_{\nu}^{2}\\}\Delta
t_{\nu}\Big{)}\exp\Big{(}\frac{a_{I}^{2}}{2}\sum_{\nu}\bar{q}_{\nu}^{2}\Delta
t_{\nu}\Big{)}\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a_{R})$

for $a_{R}\in[-a_{\text{MAX}},a_{\text{MAX}}]$. Since
$\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a_{R})$ is the probability density for
$(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))$ assuming $a_{\text{TRUE}}=a_{R}$
and $\vec{\xi}=\vec{\eta}$, it follows that the integral (4.16) is at most

$2\text{E}_{a_{R},\vec{\eta}}\Big{[}\mathbbm{1}_{(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))\in
E}\cdot\Big{(}\sum_{\nu}\\{q^{\sigma}(t_{\nu}))^{2}+(u^{\sigma}(t_{\nu}))^{2}\\}\Delta
t_{\nu}\Big{)}\\\
\cdot\exp\Big{(}\frac{a_{I}^{2}}{2}\sum_{\nu}(q^{\sigma}(t_{\nu}))^{2}\Delta
t_{\nu}\Big{)}\Big{]}\;\text{for}\;a\in\mathcal{R}.$

Recall that $|u^{\sigma}(t_{\nu})|\leq C[|q^{\sigma}(t_{\nu})|+1]$ and that
$|a_{I}|\leq\delta$ for $a=a_{R}+ia_{I}\in\mathcal{R}$.

Consequently, for $a_{R}+ia_{I}\in\mathcal{R}$, we have

$\Big{(}\sum_{\nu}\\{(q^{\sigma}(t_{\nu}))^{2}+(u^{\sigma}(t_{\nu}))^{2}\\}\Delta
t_{\nu}\Big{)}\exp\Big{(}\frac{a_{I}^{2}}{2}\sum_{\nu}(q^{\sigma}(t_{\nu}))^{2}\Delta
t_{\nu}\Big{)}\\\ \leq
C_{\delta}\cdot\exp\Big{(}C\delta^{2}\max_{\nu}|q^{\sigma}(t_{\nu})|^{2}\Big{)}.$

So the integral (4.16) is at most

$C_{\delta}\text{E}_{a_{R},\vec{\eta}}\Big{[}\exp\Big{(}C\delta^{2}\max_{\nu}|q^{\sigma}(t_{\nu})|^{2}\Big{)}\Big{]}\;\text{for}\;a\in\mathcal{R}.$
(4.17)

On the other hand, Lemma 1 gives

$\text{E}_{a_{R},\vec{\eta}}[\exp(c\max_{\nu}|q^{\sigma}(t_{\nu})|^{2})]\leq
C$ (4.18)

for $a_{R}\in[-a_{\text{MAX}},a_{\text{MAX}}]$ and $c>0$ small enough.

We now fix $\delta=\hat{c}$ small enough that $C\delta^{2}<c$ with $C,c$ as in
(4.17), (4.18). We conclude that the integral (4.16) is less than a large
constant $C$, independent of $a\in\mathcal{R}$.

So we have shown that $I(a)$ is analytic and bounded for

$a\in(-a_{\text{MAX}},+a_{\text{MAX}})\times(-\delta,\delta).$

Recalling that we have taken $\delta=\hat{c}$ and that (4.15) holds, we obtain
the following result.

######  Lemma 3 (Analytic Continuation Lemma).

Let $\sigma$ be a tame strategy, and let
$\vec{\eta}\in\\{0,1\\}^{\mathbb{N}}$.

Then there exists an analytic function $I_{\vec{\eta}}(a)$ on the rectangle

$\mathcal{R}=\\{a_{R}+ia_{I}:a_{R}\in(-a_{\emph{MAX}},a_{\emph{MAX}}),a_{I}\in(-\hat{c},\hat{c})\\}$

such that

$|I_{\vec{\eta}}(a)|\leq C\;\text{on}\;\mathcal{R}$

and

$\Big{|}\emph{E}_{a,\vec{\eta}}\Big{[}\sum_{\nu=0}^{N-1}\\{(q^{\sigma}(t_{\nu}))^{2}+(u^{\sigma}(t_{\nu}))^{2}\\}\Delta
t_{\nu}\Big{]}-I_{\vec{\eta}}(a)\Big{|}\leq C(\Delta t_{\emph{MAX}})^{1/8}$

for $a\in(-a_{\emph{MAX}},+a_{\emph{MAX}})$. Explicitly,

$\displaystyle I_{\vec{\eta}}(a)=$
$\displaystyle\int_{E}\Big{(}\sum_{\nu=0}^{N-1}\\{q_{\nu}^{2}+u_{\nu}^{2}\\}\Delta
t_{\nu}\Big{)}$
$\displaystyle\cdot\prod_{\nu=0}^{N-1}\Big{\\{}\frac{1}{\sqrt{2\pi\Delta
t_{\nu}}}\exp\Big{(}-\frac{[q_{\nu+1}-q_{\nu}-(aq_{\nu}+u_{\nu})\Delta
t_{\nu}]^{2}}{2\Delta t_{\nu}}\Big{)}\Big{\\}}\ dq_{1}\cdots dq_{N}$

where

$\begin{split}E=\\{(\bar{q}_{1},\dots,\bar{q}_{N})\in\mathbb{R}^{N}:&\max_{\nu}|\bar{q}_{\nu}|\leq(\Delta
t_{\emph{MAX}})^{-1/16}\\\
&\emph{and}\;|\sum_{\nu}(\bar{q}_{\nu+1}-\bar{q}_{\nu})^{2}-T|\leq(\Delta
t_{\emph{MAX}})^{1/8}\\}.\end{split}$ (4.19)

and $u_{\nu}$ denotes the value assigned by $\sigma$ to the control at time
$t_{\nu}$ assuming that $q^{\sigma}(t_{\mu})=q_{\mu}$ for $0\leq\mu\leq\nu$
and $\vec{\xi}=\vec{\eta}$.

###  5 Moments of Increments

We retain the assumptions and notation of Section 2. Recall that
$\mathcal{F}_{\nu}$ is the sigma algebra of events determined by
$q^{\sigma}(t_{\mu})$ ($0\leq\mu\leq\nu)$, and that

$\displaystyle\Delta t_{\nu}=t_{\nu+1}-t_{\nu},$ $\displaystyle\Delta
q_{\nu}=q^{\sigma}(t_{\nu+1})-q^{\sigma}(t_{\nu}),$
$\displaystyle\Delta\zeta_{1,\nu}=\zeta_{1}^{\sigma}(t_{\nu+1})-\zeta_{1}^{\sigma}(t_{\nu}),$
$\displaystyle\Delta\zeta_{2,\nu}=\zeta_{2}^{\sigma}(t_{\nu+1})-\zeta_{2}^{\sigma}(t_{\nu}).$

We suppose that $a_{\text{TRUE}}=a$ and $\vec{\xi}=\vec{\eta}$.

Recall also that

$\displaystyle\Delta q_{\nu}=(aq_{\nu}+u_{\nu})\Delta t_{\nu}^{*}+\Delta
W_{\nu},$ (5.1) $\displaystyle\Delta\zeta_{1,\nu}=q_{\nu}(\Delta
q_{\nu}-u_{\nu}\Delta t_{\nu})=aq_{\nu}^{2}\Delta
t_{\nu}^{*}+q_{\nu}u_{\nu}(\Delta t_{\nu}^{*}-\Delta t_{\nu})+q_{\nu}\Delta
W_{\nu},$ (5.2) $\displaystyle\Delta\zeta_{2,\nu}=q_{\nu}^{2}\Delta t_{\nu}.$
(5.3)

We condition on $\mathcal{F}_{\nu}$; thus, $q_{\nu}$ and $u_{\nu}$ are
deterministic, while $\Delta W_{\nu}$ is normal, with mean 0 and variance
$\Delta\tilde{t}_{\nu}$. We suppose that

$|q_{\nu}|,|u_{\nu}|\leq Q,\;\text{with}\;Q\geq C\;\text{given}.$ (5.4)

Then

$\displaystyle\Delta q_{\nu}=O(Q)\Delta t_{\nu}+\Delta W_{\nu},$
$\displaystyle\Delta\zeta_{1,\nu}=O(Q^{2})\Delta t_{\nu}+q_{\nu}\Delta
W_{\nu},$ $\displaystyle\Delta\zeta_{2,\nu}=O(Q^{2})\Delta t_{\nu},$

so that

$\displaystyle(\Delta q_{\nu})^{2}=O(Q^{2})(\Delta t_{\nu})^{2}+2O(Q)\Delta
t_{\nu}\Delta W_{\nu}+(\Delta W_{\nu})^{2},$ (5.5)
$\displaystyle(\Delta\zeta_{1,\nu})(\Delta q_{\nu})=O(Q^{3})(\Delta
t_{\nu})^{2}+O(Q^{2})\Delta t_{\nu}\Delta W_{\nu}+q_{\nu}(\Delta
W_{\nu})^{2},$ (5.6) $\displaystyle(\Delta\zeta_{2,\nu})(\Delta
q_{\nu})=O(Q^{3})(\Delta t_{\nu})^{2}+O(Q^{2})\Delta t_{\nu}\Delta W_{\nu},$
(5.7) $\displaystyle(\Delta\zeta_{1,\nu})^{2}=O(Q^{4})(\Delta
t_{\nu})^{2}+O(Q^{3})(\Delta t_{\nu})(\Delta W_{\nu})+q_{\nu}^{2}(\Delta
W_{\nu})^{2},$ (5.8)
$\displaystyle(\Delta\zeta_{2,\nu})(\Delta\zeta_{1,\nu})=O(Q^{4})(\Delta
t_{\nu})^{2}+O(Q^{3})(\Delta t_{\nu})(\Delta W_{\nu})$ (5.9)
$\displaystyle(\Delta\zeta_{2,\nu})^{2}=O(Q^{4})(\Delta t_{\nu})^{2}.$ (5.10)

Moreover, all the quantities $O(Q^{\text{power}})$ above are deterministic
once we condition on $\mathcal{F}_{\nu}$.

Therefore, (5.1)–(5.3) and (5.5)–(5.10) yield the following.

$\displaystyle\text{E}[\Delta
q_{\nu}|\mathcal{F}_{\nu}]=(aq_{\nu}+u_{\nu})\Delta t_{\nu}+O(Q(\Delta
t_{\nu})^{2}),$ (5.11)
$\displaystyle\text{E}[\Delta\zeta_{1,\nu}|\mathcal{F}_{\nu}]=aq_{\nu}^{2}(\Delta
t_{\nu})+O(Q^{2}(\Delta t_{\nu})^{2}),$ (5.12)
$\displaystyle\text{E}[\Delta\zeta_{2,\nu}|\mathcal{F}_{\nu}]=q_{\nu}^{2}\Delta
t_{\nu},$ (5.13)
$\displaystyle\text{E}[(\Delta\zeta_{1,\nu})^{2}|\mathcal{F}_{\nu}]=q_{\nu}^{2}\Delta
t_{\nu}+O(Q^{4}(\Delta t_{\nu})^{2}),$ (5.14)
$\displaystyle\text{E}[(\Delta\zeta_{1,\nu})(\Delta\zeta_{2,\nu})|\mathcal{F}_{\nu}]=O(Q^{4}(\Delta
t_{\nu})^{2}),,$ (5.15)
$\displaystyle\text{E}[(\Delta\zeta_{2,\nu})^{2}|\mathcal{F}_{\nu}]=O(Q^{4}(\Delta
t_{\nu})^{2}),$ (5.16) $\displaystyle\text{E}[(\Delta
q_{\nu})^{2}|\mathcal{F}_{\nu}]=\Delta t_{\nu}+O(Q^{2}(\Delta t_{\nu})^{2}),$
(5.17) $\displaystyle\text{E}[(\Delta
q_{\nu})(\Delta\zeta_{1,\nu})|\mathcal{F}_{\nu}]=q_{\nu}\Delta
t_{\nu}+O(Q^{3}(\Delta t_{\nu})^{2}),$ (5.18) $\displaystyle\text{E}[(\Delta
q_{\nu})(\Delta\zeta_{2,\nu})|\mathcal{F}_{\nu}]=O(Q^{3}(\Delta
t_{\nu})^{2}).$ (5.19)

(Here we have used the fact that $\Delta t_{\nu}^{*}$ and
$\Delta\tilde{t}_{\nu}$ are $\Delta t_{\nu}+O((\Delta t_{\nu})^{2}).$)

We define an event

$\text{TAME}_{\nu}=\\{|\Delta q_{\nu}|\leq 2(\Delta
t_{\nu})^{2/5},|\Delta\zeta_{1,\nu}|\leq 2(\Delta t_{\nu})^{2/5},\\\
|\Delta\zeta_{2,\nu}|\leq 2(\Delta t_{\nu})^{2/5}\\}.$ (5.20)

From Lemma 1 we obtain the estimate

$\text{E}[(\Delta
q_{\nu})^{\alpha_{0}}(\Delta\zeta_{1,\nu})^{\alpha_{1}}(\Delta\zeta_{2,\nu})^{\alpha_{2}}\cdot\mathbbm{1}_{\text{NOT
TAME}_{\nu}}|\mathcal{F}_{\nu}]=O((\Delta t_{\nu})^{100})$

for integers $\alpha_{0},\alpha_{1},\alpha_{2}\geq 0$ with
$\alpha_{0}+\alpha_{1}+\alpha_{2}\leq 2$. Also from Lemma 1, we recall that

$\text{Prob}[\text{NOT TAME}_{\nu}|\mathcal{F}_{\nu}]\leq C\cdot(\Delta
t_{\nu})^{1000}.$

Consequently, (5.11)–(5.19) imply the conclusions of the following lemma.

######  Lemma 4 (Lemma on Moments of Increments).

We fix $a_{\emph{TRUE}}=a$ and $\vec{\xi}=\vec{\eta}$.

We suppose that

$Q\geq C\text{ and }(\Delta t_{\emph{MAX}})<Q^{-1000}.$

Define the event

$\emph{TAME}(\nu)=\\{|\Delta q^{\sigma}_{\nu}|\leq 2(\Delta
t_{\nu})^{2/5},|\Delta\zeta_{1,\nu}^{\sigma}|\leq 2(\Delta
t_{\nu})^{2/5},|\Delta\zeta_{2,\nu}^{\sigma}|\leq 2(\Delta t_{\nu})^{2/5}\\}.$

Let $\mathcal{F}_{\nu}$ be the sigma algebra of events determined by
$q^{\sigma}(t_{\mu})$ ($0\leq\mu\leq\nu)$.

Fix $\nu$, and suppose that

$|q^{\sigma}(t_{\nu})|,|u^{\sigma}(t_{\nu})|\leq Q.$

Then the following hold.

$\displaystyle\emph{E}[(\Delta
q^{\sigma}_{\nu})\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=(aq^{\sigma}_{\nu}+u^{\sigma}_{\nu})(\Delta
t_{\nu})+\emph{ERR}\;1,$
$\displaystyle\emph{E}[(\Delta\zeta_{1,\nu}^{\sigma})\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=a(q^{\sigma}_{\nu})^{2}(\Delta
t_{\nu})+\emph{ERR}\;2,$
$\displaystyle\emph{E}[(\Delta\zeta_{2,\nu}^{\sigma})\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=(q^{\sigma}_{\nu})^{2}(\Delta
t_{\nu})+\emph{ERR}\;3$ $\displaystyle\emph{E}[(\Delta
q^{\sigma}_{\nu})^{2}\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=(\Delta
t_{\nu})+\emph{ERR}\;4,$ $\displaystyle\emph{E}[(\Delta
q^{\sigma}_{\nu})(\Delta\zeta_{1,\nu}^{\sigma})\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=q^{\sigma}_{\nu}(\Delta
t_{\nu})+\emph{ERR}\;5,$ $\displaystyle\emph{E}[(\Delta
q^{\sigma}_{\nu})(\Delta\zeta_{2,\nu}^{\sigma})\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=\emph{ERR}\;6,$
$\displaystyle\emph{E}[(\Delta\zeta_{1,\nu}^{\sigma})^{2}\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=(q^{\sigma}_{\nu})^{2}(\Delta
t_{\nu})+\emph{ERR}\;7,$
$\displaystyle\emph{E}[(\Delta\zeta_{1,\nu}^{\sigma})(\Delta\zeta_{2,\nu}^{\sigma})\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=\emph{ERR}\;8,$
$\displaystyle\emph{E}[(\Delta\zeta_{2,\nu}^{\sigma})^{2}\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=\emph{ERR}\;9,$

where

$|\emph{ERR}\;1|,\dots,|\emph{ERR}\;9|\leq C^{\prime}Q^{4}(\Delta
t_{\nu})^{2}.$

Also, under the above assumptions, we have

$\emph{Prob}[\emph{NOT TAME}(\nu)|\mathcal{F}_{\nu}]\leq(\Delta
t_{\nu})^{20}.$

Here, of course,

$\displaystyle q^{\sigma}_{\nu}=q^{\sigma}(t_{\nu}),$ $\displaystyle
u^{\sigma}_{\nu}=u^{\sigma}(t_{\nu}),$ $\displaystyle\Delta
t_{\nu}=t_{\nu+1}-t_{\nu},$ $\displaystyle\Delta
q^{\sigma}_{\nu}=q^{\sigma}(t_{\nu+1})-q^{\sigma}(t_{\nu}),$
$\displaystyle\Delta\zeta_{1,\nu}^{\sigma}=\zeta_{1}^{\sigma}(t_{\nu+1})-\zeta_{1}^{\sigma}(t_{\nu}),$
$\displaystyle\Delta\zeta_{2,\nu}^{\sigma}=\zeta_{2}^{\sigma}(t_{\nu+1})-\zeta_{2}^{\sigma}(t_{\nu}).$

The constants $C,C^{\prime}$ are determined by
$q_{0},a_{\emph{MAX}},C_{\emph{TAME}}$ and an upper bound for $T$.

###  6 Stability Under Change of Assumption

Let $f(\bar{q}_{1},\dots,\bar{q}_{N})$ be a nonnegative function on
$\mathbb{R}^{N}$. For $\vec{\eta}\in\\{0,1\\}^{\mathbb{N}}$ and
$a_{1},a_{2}\in[-a_{\text{MAX}},+a_{\text{MAX}}]$, we compare

$\text{E}_{a_{1},\vec{\eta}}[f(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))]$

with

$\text{E}_{a_{2},\vec{\eta}}[f(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))]$

for a tame strategy $\sigma$.

To do so, let $\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a)$ denote the probability
density of

$(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))$

assuming that $\vec{\xi}=\vec{\eta}$ and $a_{\text{TRUE}}=a$.

According to Lemma 2, the following holds for $a=a_{1},a_{2}$:

Let $Q>C$ and $Q\leq(\Delta t_{\text{MAX}})^{-1/1000}$, and suppose that

$\begin{split}&\max_{\nu}(|\bar{q}_{\nu}|+|\bar{u}_{\nu}|)\leq Q,\\\
&\Big{|}\sum_{0\leq\nu<N}(\bar{q}_{\nu+1}-\bar{q}_{\nu})^{2}-T\Big{|}\leq
Q^{2}(\Delta t_{\text{MAX}})^{1/4},\end{split}$ (6.1)

where $\bar{u}_{\nu}$ denotes the value of $u^{\sigma}(t_{\nu})$ assuming
$q^{\sigma}(t_{\mu})=\bar{q}_{\mu}$ for $\mu\leq\nu$. (Recall that the
$\bar{u}_{\nu}$ don’t depend on $a$.) Then

$\begin{split}\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a&)\\\
=\prod_{0\leq\nu<N}\Big{\\{}&\frac{1}{\sqrt{2\pi\Delta
t_{\nu}}}\exp\Big{(}-\frac{1}{2\Delta
t_{\nu}}[(\bar{q}_{\nu+1}-\bar{q}_{\nu})-(a\bar{q}_{\nu}+\bar{u}_{\nu})\Delta
t_{\nu}]^{2}\Big{)}\Big{\\}}\\\
&\cdot(1+\text{ERR}(\bar{q}_{1},\dots,\bar{q}_{N},a)\end{split}$ (6.2)

with

$|\text{ERR}(\bar{q}_{1},\dots,\bar{q}_{N},a)|\leq CQ^{2}(\Delta
t_{\text{MAX}})^{1/4}$ (6.3)

and $\bar{q}_{0}\equiv q_{0}$.

Applying (6.2), (6.3) with $a=a_{1}$ and with $a=a_{2}$, we see that if (6.1)
holds, then

$\begin{split}\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a_{2})=&\Phi(\bar{q}_{1},\dots,\bar{q}_{N},a_{1})\\\
&\cdot\exp\Big{(}-\frac{1}{2}[a_{2}^{2}-a_{1}^{2}]\bar{\zeta}_{2}^{\sigma}(T)+[a_{2}-a_{1}]\bar{\zeta}_{1}^{\sigma}(T)\Big{)}\\\
&\cdot(1+\text{ERR}(\bar{q}_{1},\dots,\bar{q}_{N},a_{1},a_{2}))\end{split}$
(6.4)

with

$\bar{\zeta}_{1}^{\sigma}(T):=\sum_{0\leq\nu<N}\bar{q}_{\nu}([\bar{q}_{\nu+1}-\bar{q}_{\nu}]-\bar{u}_{\nu}\Delta
t_{\nu}),\qquad\bar{\zeta}_{2}^{\sigma}(T):=\sum_{0\leq\nu<N}\bar{q}_{\nu}^{2}\Delta
t_{\nu},$ (6.5)

and

$|\text{ERR}(\bar{q}_{1},\dots,\bar{q}_{N},a_{1},a_{2})|\leq CQ^{2}(\Delta
t_{\text{MAX}})^{1/4}.$ (6.6)

If (6.1) holds, then

$|\bar{\zeta}_{1}^{\sigma}(T)|,|\bar{\zeta}_{2}^{\sigma}(T)|\leq CQ^{2}$ (6.7)

(see (2.17)).

Letting $\mathcal{E}$ denote the event that
$(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))$ satisfies (6.1), we conclude
from (6.4), (6.6), (6.7) that

$\begin{split}\text{E}_{a_{2},\vec{\eta}}[f(q^{\sigma}(t_{1}),&\dots,q^{\sigma}(t_{N}))\cdot\mathbbm{1}_{\mathcal{E}}]\\\
\leq&\text{E}_{a_{1},\vec{\eta}}[f(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))\cdot\mathbbm{1}_{\mathcal{E}}]\cdot\exp(CQ^{2}|a_{2}-a_{1}|)\\\
&\cdot(1+\text{ERR}_{f}(a_{1},a_{2}))\end{split}$ (6.8)

with

$|\text{ERR}_{f}(a_{1},a_{2})|\leq CQ^{2}(\Delta t_{\text{MAX}})^{1/4}.$ (6.9)

On the other hand, Lemma 1 shows that the complement of $\mathcal{E}$, denoted
$\prescript{c}{}{\mathcal{E}}$, satisfies

$\text{Prob}_{a_{2},\vec{\eta}}[\prescript{c}{}{\mathcal{E}}]\leq\exp(-cQ^{2}),$

and therefore, by Cauchy-Schwarz, we have

$\begin{split}\text{E}_{a_{2},\vec{\eta}}[f(q^{\sigma}(t_{1}),\dots,&q^{\sigma}(t_{N}))\cdot\mathbbm{1}_{\prescript{c}{}{\mathcal{E}}}]\\\
&\leq\exp(-c^{\prime}Q^{2})\Big{(}\text{E}_{a_{2},\vec{\eta}}[f^{2}(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))]\Big{)}^{1/2}.\end{split}$
(6.10)

Combining (6.9) and (6.10), we obtain the following result.

######  Lemma 5 (Lemma on Change of Assumption).

Suppose $Q\geq C$ for a large enough constant $C$.

Let $a_{1},a_{2}\in[-a_{\emph{MAX}},+a_{\emph{MAX}}]$, let
$\vec{\eta}\in\\{0,1\\}^{\mathbb{N}}$, and let
$f(\bar{q}_{1},\dots,\bar{q}_{N})$ be a nonnegative function on
$\mathbb{R}^{\mathbb{N}}$. Let $\sigma$ be a tame strategy. Assume that
$Q\leq(\Delta t_{\emph{MAX}})^{-1/1000}$. Then

$\displaystyle\emph{E}_{a_{2},\vec{\eta}}[$ $\displaystyle
f(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))]$ $\displaystyle\leq$
$\displaystyle\exp(CQ^{2}|a_{2}-a_{1}|)(1+CQ^{2}(\Delta
t_{\emph{MAX}})^{1/4})\emph{E}_{a_{1},\vec{\eta}}[f(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))]$
$\displaystyle+\exp(-cQ^{2})\Big{(}\emph{E}_{a_{2},\vec{\eta}}[f^{2}(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{N}))]\Big{)}^{1/2}.$

###  7 Disasters Due to Undercontrol

Our tame strategies $\sigma$ are defined to guarantee that

$|u^{\sigma}(t_{\nu})|\leq C[|q^{\sigma}(t_{\nu})|+1],$ (7.1)

which is reasonable, given our assumption that
$a_{\text{TRUE}}\in[-a_{\text{MAX}},a_{\text{MAX}}]$. However, if
$a_{\text{TRUE}}\gg a_{\text{MAX}}$, then we expect (7.1) to undercontrol,
leading to exponentially large expected cost.

The following lemma confirms that intuition.

######  Lemma 6 (Lemma on Undercontrol).

Let $\sigma$ be a deterministic strategy satisfying (7.1), and suppose
$a_{\emph{TRUE}}=a$, where $a$ exceeds a large enough constant $C_{*}$. Write
$\emph{E}_{a}[\dots]$ for the corresponding expectation. Assume that $\Delta
t_{\emph{MAX}}$ is less than a small enough positive number determined by $a$
and $T$. Then

$\emph{E}_{a}\Big{[}\sum_{0\leq\nu<N}\\{q^{\sigma}(t_{\nu}))^{2}+(u^{\sigma}(t_{\nu}))^{2}\\}\Delta
t_{\nu}\Big{]}\geq cT^{2}\exp(caT).$

###### Proof.

We write $q_{\nu}$ for $q^{\sigma}(t_{\nu})$, $\Delta q_{\nu}$ for
$q_{\nu+1}-q_{\nu}$, $u_{\nu}$ for $u^{\sigma}(t_{\nu})$, and $\Delta t_{\nu}$
for $t_{\nu+1}-t_{\nu}$. We let $\mathcal{F}_{\nu}$ denote the sigma algebra
of events determined by $q_{1},\dots,q_{\nu}$. Thus, $q_{\nu}$ and $u_{\nu}$
are deterministic once we condition on $\mathcal{F}_{\nu}$.

Recall that

$\Delta q_{\nu}=(aq_{\nu}+u_{\nu})(\Delta t_{\nu}^{*})+\Delta W_{\nu}$ (7.2)

where $\Delta W_{\nu}$ is normal with mean 0 and variance

$\Delta\tilde{t}_{\nu}=\frac{\exp(2a\Delta t_{\nu})-1}{2a};$

moreover $\Delta W_{\nu}$ is independent of $\mathcal{F}_{\nu}$. Here,

$\Delta t_{\nu}^{*}=\frac{\exp(a\Delta t_{\nu})-1}{a}.$

Since $\Delta t_{\nu}<\Delta t_{\text{MAX}}$ is less than a small enough
positive number determined by $a$ and $T$, we have

$|\Delta\tilde{t}_{\nu}-\Delta t_{\nu}|,|\Delta t_{\nu}^{*}-\Delta
t_{\nu}|<10^{-3}\Delta t_{\nu}.$ (7.3)

From (7.2) we have

$\displaystyle q_{\nu+1}^{2}=$ $\displaystyle
q_{\nu}^{2}+2q_{\nu}[(aq_{\nu}+u_{\nu})(\Delta t_{\nu}^{*})+\Delta
W_{\nu}]+(aq_{\nu}+u_{\nu})^{2}(\Delta t_{\nu}^{*})^{2}$
$\displaystyle+2(aq_{\nu}+u_{\nu})(\Delta t_{\nu}^{*})\Delta W_{\nu}+(\Delta
W_{\nu})^{2}$ $\displaystyle\geq$ $\displaystyle q_{\nu}^{2}(1+2a\Delta
t_{\nu}^{*})+2q_{\nu}u_{\nu}(\Delta
t_{\nu}^{*})+[2q_{\nu}+2(aq_{\nu}+u_{\nu})(\Delta t_{\nu}^{*})]\Delta W_{\nu}$
$\displaystyle+(\Delta W_{\nu})^{2}.$

Consequently,

$\text{E}_{a}[q_{\nu+1}^{2}|\mathcal{F}_{\nu}]\geq q_{\nu}^{2}(1+2a\Delta
t_{\nu}^{*})+2q_{\nu}u_{\nu}(\Delta t_{\nu}^{*})+(\Delta\tilde{t}_{\nu}).$
(7.4)

For $\delta>0$ to be picked in a moment, we have

$|2q_{\nu}u_{\nu}|\leq\delta^{-2}q_{\nu}^{2}+\delta^{2}u_{\nu}^{2}\leq
C\delta^{-2}q_{\nu}^{2}+C\delta^{2},$

thanks to (7.1).

Putting this inequality into (7.4), we find that

$\text{E}_{a}[q_{\nu+1}^{2}|\mathcal{F}_{\nu}]\geq
q_{\nu}^{2}(1+[2a-C\delta^{-2}]\Delta
t_{\nu}^{*})+(\Delta\tilde{t}_{\nu}-C\delta^{2}\Delta t_{\nu}^{*}).$ (7.5)

We take $\delta$ to be a small enough constant $c$ such that

$\Delta\tilde{t}_{\nu}-C\delta^{2}\Delta t_{\nu}^{*}\geq\frac{1}{2}\Delta
t_{\nu};$

see (7.3). Since $a$ exceeds a large enough constant $C$, we then have

$[2a-C\delta^{-2}]>a,$

so from (7.5) we obtain

$\text{E}_{a}[q_{\nu+1}^{2}|\mathcal{F}_{\nu}]\geq q_{\nu}^{2}(1+a\Delta
t_{\nu}^{*})+\frac{1}{2}\Delta t_{\nu}.$

Again applying (7.3), and recalling that $\Delta t_{\nu}<\Delta
t_{\text{MAX}}$ is less than a small enough positive nuumber determined by $a$
and $T$, we conclude that

$\text{E}_{a}[q_{\nu+1}^{2}|\mathcal{F}_{\nu}]\geq\exp\Big{(}\frac{1}{2}a\Delta
t_{\nu}\Big{)}q_{\nu}^{2}+\frac{1}{2}\Delta t_{\nu},$

and therefore

$\text{E}_{a}[q_{\nu+1}^{2}]\geq\exp\Big{(}\frac{1}{2}a\Delta
t_{\nu}\Big{)}\text{E}_{a}[q_{\nu}^{2}]+\frac{1}{2}\Delta t_{\nu}.$ (7.6)

Since (7.6) implies that

$\text{E}_{a}[q_{\nu+1}^{2}]\geq\text{E}_{a}[q_{\nu}^{2}]+\frac{1}{2}\Delta
t_{\nu}\;\text{for each}\;\nu,$

we conclude that

$\text{E}_{a}[q_{\nu}^{2}]\geq\frac{1}{2}t_{\nu}\;\text{for each}\;\nu.$

We pick $\nu_{0}$ so that

$\frac{1}{2}T<t_{\nu_{0}}<\frac{2}{3}T.$

(Our smallness assumption on $\Delta t_{\text{MAX}}$ implies that such a
$\nu_{0}$ exists.) Then

$\text{E}_{a}[q_{\nu_{0}}^{2}]\geq\frac{1}{2}t_{\nu_{0}}>\frac{1}{4}T.$

Returning to (7.6), we have

$\text{E}_{a}[q_{\nu+1}^{2}]\geq\exp\Big{(}\frac{1}{2}a\Delta
t_{\nu}\Big{)}\text{E}_{a}[q_{\nu}^{2}]\;\text{for each}\;\nu,$

hence for $\nu\geq\nu_{0}$ we have

$\text{E}_{a}[q_{\nu}^{2}]\geq\exp\Big{(}\frac{1}{2}a[t_{\nu}-t_{\nu_{0}}]\Big{)}\text{E}[q_{\nu_{0}}^{2}]\geq\frac{1}{4}T\exp\Big{(}\frac{1}{2}a[t_{\nu}-t_{\nu_{0}}]\Big{)}.$

In particular,

$\text{E}_{a}[q_{\nu}^{2}]\geq\frac{1}{4}T\exp(caT)\;\text{for}\;t_{\nu}\in\Big{[}\frac{3}{4}T,T\Big{]}.$

Consequently,

$\displaystyle\text{E}_{a}\Big{[}\sum_{0\leq\nu<N}\\{q_{\nu}^{2}+u_{\nu}^{2}\\}\Delta
t_{\nu}\Big{]}$
$\displaystyle\geq\text{E}_{a}\Big{[}\sum_{t_{\nu}\in[\frac{3}{4}T,T]}q_{\nu}^{2}\Delta
t_{\nu}\Big{]}$
$\displaystyle\geq\frac{1}{4}T\exp(caT)\cdot\sum_{t_{\nu}\in[\frac{3}{4}T,T]}\Delta
t_{\nu}$ $\displaystyle\geq cT^{2}\exp(caT),$

since each $\Delta t_{\nu}<\Delta t_{\text{MAX}}$ is less than a small enough
positive number determined by $a$ and $T$. The proof of the lemma is complete.
∎

###  8 Costing by Integrals

Let $\sigma$ be a deterministic tame strategy. We condition on

$a_{\text{TRUE}}=a\in[-a_{\text{MAX}},+a_{\text{MAX}}],$

and write $\text{Prob}_{a}[\cdot]$ and $E_{a}[\cdot]$ to denote the
corresponding probability and expectation.

We want to compare

$\textsc{Cost}(\sigma)=\int_{0}^{T}\\{(q^{\sigma}(t))^{2}+(u^{\sigma}(t))^{2}\\}\
dt$

with

$\textsc{Cost}_{D}(\sigma)=\sum_{0\leq\nu<N}\\{(q^{\sigma}(t_{\nu}))^{2}+(u^{\sigma}(t_{\nu}))^{2}\\}\
\Delta t_{\nu}$

(“$D$” for “discrete”).

######  Lemma 7 (Lemma on Costing by Integrals).

For any $m\geq 1$, we have

$\emph{E}_{a}[|\textsc{Cost}(\sigma)-\textsc{Cost}_{D}(\sigma)|^{m}]\leq
C_{m}(\Delta t_{\emph{MAX}})^{m/2}.$

###### Proof.

Recall that $u^{\sigma}(t)=u^{\sigma}(t_{\nu})$ for $t\in[t_{\nu},t_{\nu+1}]$.
Hence,

$\displaystyle\textsc{Cost}(\sigma)-$
$\displaystyle\textsc{Cost}_{D}(\sigma)=\sum_{0\leq\nu<N}\int_{t_{\nu}}^{t_{\nu+1}}\\{(q^{\sigma}(t))^{2}-(q^{\sigma}(t_{\nu}))^{2}\\}\
dt$
$\displaystyle=\sum_{0\leq\nu<N}\int_{t_{\nu}}^{t_{\nu+1}}\\{[q^{\sigma}(t)-q^{\sigma}(t_{\nu})]^{2}+2q^{\sigma}(t_{\nu})[q^{\sigma}(t)-q^{\sigma}(t_{\nu})]\\}\
dt.$

Setting

$\textsc{Osc}(\nu)=\max_{t\in[t_{\nu},t_{\nu+1}]}|q^{\sigma}(t)-q^{\sigma}(t_{\nu})|,$

we therefore have

$|\textsc{Cost}(\sigma)-\textsc{Cost}_{D}(\sigma)|\leq\sum_{0\leq\nu<N}\\{(\textsc{Osc}(\nu))^{2}+2|q^{\sigma}(t_{\nu})|(\textsc{Osc}(\nu))\\}\Delta
t_{\nu},$

so that by Hölder’s inequality,

$|\textsc{Cost}(\sigma)-\textsc{Cost}_{D}(\sigma)|^{m}\\\ \leq
C_{m}\sum_{0\leq\nu<N}\\{(\textsc{Osc}(\nu))^{2m}+|q^{\sigma}(t_{\nu})|^{m}(\textsc{Osc}(\nu))^{m}\\}\Delta
t_{\nu}.$

Consequently,

$\begin{split}\text{E}_{a}[|\textsc{Cost}(\sigma)-&\textsc{Cost}_{D}(\sigma)|^{m}]\\\
\leq&C_{m}\sum_{0\leq\nu<N}\text{E}_{a}[(\textsc{Osc}(\nu)^{2m}](\Delta
t_{\nu})\\\
&+C_{m}\sum_{0\leq\nu<N}(\text{E}_{a}[|q^{\sigma}(t_{\nu})|^{2m}])^{1/2}(\text{E}_{a}[(\textsc{Osc}(\nu))^{2m}])^{1/2}\Delta
t_{\nu}.\end{split}$ (8.1)

We now estimate the right-hand side of (8.1).

Fix a large enough constant $C_{*}$, and let $\mathcal{F}_{\nu}$ denote the
sigma algebra of events determined by $q^{\sigma}(t_{\mu})$ for
$\mu=1,\dots,\nu$.

Lemma 1 shows that

$\text{E}_{a}[|q^{\sigma}(t_{\nu})|^{2m}]\leq C_{m}$ (8.2)

and that, given $Q_{2}\geq Q_{1}\geq C_{*}$, we have

$\text{Prob}_{a}[\textsc{Osc}(\nu)>Q_{2}(\Delta
t_{\nu})^{1/2}|\mathcal{F}_{\nu}]\leq
C\exp(-cQ_{2}^{2})\;\text{if}\;|q^{\sigma}(t_{\nu})|\leq Q_{1}.$ (8.3)

From (8.3) we see that

$\text{E}_{a}[(\textsc{Osc}(\nu))^{2m}|\mathcal{F}_{\nu}]\leq
C_{m}Q_{1}^{2m}(\Delta t_{\nu})^{m}\;\text{if}\;|q^{\sigma}(t_{\nu})|\leq
Q_{1}.$

In particular,

$\text{E}_{a}[(\textsc{Osc}(\nu))^{2m}\cdot\mathbbm{1}_{|q^{\sigma}(t_{\nu})|\leq
C_{*}}]\leq C_{m}(\Delta t_{\nu})^{m}$ (8.4)

and, for $k\geq 0$,

$\text{E}_{a}[(\textsc{Osc}(\nu))^{2m}\cdot\mathbbm{1}_{|q^{\sigma}(t_{\nu})|\in[C_{*}2^{k},C_{*}2^{(k+1)})}]\\\
\leq C_{m}\cdot(C_{*}2^{(k+1)})^{2m}(\Delta
t_{\nu})^{m}\cdot\text{Prob}_{a}[|q^{\sigma}(t_{\nu})|>C_{*}2^{k}].$ (8.5)

Another application of Lemma 1 gives

$\text{Prob}_{a}[|q^{\sigma}(t_{\nu})|>C_{*}2^{k}]\leq C\exp(-c2^{2k}),$

so that (8.5) implies that

$\text{E}_{a}[(\textsc{Osc}(\nu))^{2m}\cdot\mathbbm{1}_{|q^{\sigma}(t_{\nu})|\in[C_{*}2^{k},C_{*}2^{(k+1)})}]\leq
C_{m}\cdot(2^{2mk})(\Delta t_{\nu})^{m}\cdot\exp(-c2^{2k}).$

Summing over $k\geq 0$, and combining the result with (8.4), we learn that

$\text{E}_{a}[(\textsc{Osc}(\nu))^{2m}]\leq C_{m}(\Delta t_{\nu})^{m}.$ (8.6)

From (8.1), (8.2), and (8.6) we conclude that

$\text{E}_{a}[|\textsc{Cost}(\sigma)-\textsc{Cost}_{D}(\sigma)|^{m}]\leq
C_{m}\sum_{0\leq\nu<N}(\Delta t_{\nu})^{m/2+1}\leq C_{m}^{\prime}(\Delta
t_{\text{MAX}})^{m/2},$

which is the conclusion of the lemma. ∎

###  9 Continuous vs Discrete

Let $\sigma$ be a tame strategy, possibly depending on coin flips $\vec{\xi}$.
In this section we compare the following random functions of time.

$\displaystyle q^{\sigma}_{C}(t)=q^{\sigma}(t)\;\text{for}\;t\in[0,T]$
$\displaystyle
q^{\sigma}_{D}(t)=q^{\sigma}(t_{\nu})\;\text{for}\;t\in[t_{\nu},t_{\nu+1}),\;\text{each}\;\nu<N.$
$\displaystyle\zeta_{1,C}^{\sigma}(t)=\frac{1}{2}(q^{\sigma}(t))^{2}-\frac{1}{2}q_{0}^{2}-\frac{1}{2}t-\int_{0}^{t}u^{\sigma}(s)q^{\sigma}(s)\
ds\;\text{for}\;t\in[0,T].$
$\displaystyle\zeta_{1,D}^{\sigma}(t)=\frac{1}{2}(q^{\sigma}(t_{\nu}))^{2}-\frac{1}{2}q_{0}^{2}-\frac{1}{2}\sum_{0\leq\mu<\nu}(\Delta
q^{\sigma}(t_{\mu}))^{2}-\sum_{0\leq\mu<\nu}u^{\sigma}(t_{\mu})q^{\sigma}(t_{\mu})\
\Delta t_{\mu}$
$\displaystyle\qquad\qquad\text{for}\;t\in[t_{\nu},t_{\nu+1}),\;\text{each}\;\nu<N.$
$\displaystyle\zeta_{2,C}^{\sigma}(t)=\int_{0}^{t}(q^{\sigma}(s))^{2}\
ds\;\text{for}\;t\in[0,T].$
$\displaystyle\zeta_{2,D}^{\sigma}(t)=\sum_{0\leq\mu<\nu}(q^{\sigma}(t_{\mu}))^{2}\Delta
t_{\mu}\;\text{for}\;t\in[t_{\nu},t_{\nu+1}),\;\text{each}\;\nu<N.$

We recall that $u^{\sigma}(t)$ is constant on $[t_{\nu},t_{\nu+1})$ for each
$\nu$, so there is no need to introduce analogous quantities for $u^{\sigma}$.

We establish the following result.

######  Lemma 8 (Lemma on Continuous Variants).

Let $Q$ be greater than a large enough constant $C$. Then there exists an
event $\textsc{bad}(\sigma,Q)$ with the following properties.

  * •

For each $a\in[-a_{\emph{MAX}},+a_{\emph{MAX}}]$ and
$\vec{\eta}\in\\{0,1\\}^{\mathbb{N}}$, we have

$\emph{Prob}_{a,\vec{\eta}}[\textsc{bad}(\sigma,Q)]\leq C\exp(-cQ^{2}).$

  * •

If $\textsc{bad}(\sigma,Q)$ does not occur, then

$\displaystyle\max_{t\in[0,T)}|q^{\sigma}_{C}(t)-q^{\sigma}_{D}(t)|\leq
CQ(\Delta t_{\emph{MAX}})^{1/4}$
$\displaystyle\max_{t\in[0,T)}|\zeta_{1,C}^{\sigma}(t)-\zeta_{1,D}^{\sigma}(t)|\leq
CQ^{2}(\Delta t_{\emph{MAX}})^{1/4},$
$\displaystyle\max_{t\in[0,T)}|\zeta_{2,C}^{\sigma}(t)-\zeta_{2,D}^{\sigma}(t)|\leq
CQ^{2}(\Delta t_{\emph{MAX}})^{1/4}.$

###### Proof.

From Lemma 1, the following have probability at most $C\exp(-cQ^{2})$ when
conditioned on $a_{\text{TRUE}}=a$, $\vec{\xi}=\vec{\eta}$ (any
$a\in[-a_{\text{MAX}},a_{\text{MAX}}]$,
$\vec{\eta}\in\\{0,1\\}^{\mathbb{N}}):$

$\displaystyle\max_{\nu}\Big{|}\sum_{0\leq\mu<\nu}(\Delta
q^{\sigma}_{\mu})^{2}-t_{\nu}\Big{|}>Q^{2}(\Delta t_{\text{MAX}})^{1/2}$ (9.1)
$\displaystyle\max_{\mu}|q^{\sigma}_{\mu}|>Q$ (9.2)
$\displaystyle\max_{\mu}|u^{\sigma}_{\mu}|>Q.$ (9.3)

Moreover, with

$\textsc{Osc}(\nu):=\max_{t\in[t_{\nu},t_{\nu+1}]}|q^{\sigma}(t)-q^{\sigma}(t_{\nu})|$

and $\mathcal{F}_{\nu}^{\sigma}$ defined as the sigma algebra of events
determined by the $q^{\sigma}(t_{\mu})$ for $\mu\leq\nu$, Lemma 1 gives also
the following, for each fixed $\nu$.

$\text{Prob}_{a,\vec{\eta}}[\textsc{Osc}(\nu)>CQ(\Delta
t_{\nu})^{1/2}|\mathcal{F}_{\nu}]\leq C\exp(-cQ^{2})\;\text{if}\;|q_{\nu}|\leq
Q.$

This implies that

$\text{Prob}_{a,\vec{\eta}}[\textsc{Osc}(\nu)>CQ(\Delta
t_{\nu})^{1/2}\;\text{and}\;|q_{\nu}|\leq Q]\leq C\exp(-cQ^{2}).$

Since also

$\text{Prob}_{a,\vec{\eta}}[|q_{\nu}|>Q]\leq C\exp(-cQ^{2}),$

it follows that

$\text{Prob}_{a,\vec{\eta}}[\textsc{Osc}(\nu)>CQ(\Delta t_{\nu})^{1/2}]\leq
C\exp(-cQ^{2})$

for each fixed $\nu$, and for all $Q\geq C$.

Taking $Q(\Delta t_{\nu})^{-1/4}$ in place of $Q$ here, we find that

$\text{Prob}_{a,\vec{\eta}}[\textsc{Osc}(\nu)>CQ(\Delta t_{\nu})^{1/4}]\\\
\leq C\exp(-cQ^{2}(\Delta t_{\nu})^{-1/2})\leq C^{\prime}\exp(-cQ^{2})(\Delta
t_{\nu}).$

Summing over $\nu$, we find that the event

$\textsc{Osc}(\nu)>CQ(\Delta t_{\nu})^{1/4}\;\text{for some}\;\nu,$ (9.4)

conditioned on $a_{\text{TRUE}}=a$ and $\vec{\xi}=\vec{\eta}$, has probability
at most $C\exp(-cQ^{2})$.

We now define $\textsc{bad}(Q,\sigma)$ to be the event that at least one of
the conditions (9.1)–(9.4) holds. Then, as claimed,

$\text{Prob}_{a,\vec{\eta}}[\textsc{bad}(Q,\sigma)]\leq C\exp(-cQ^{2})$

for any $a\in[-a_{\text{MAX}},+a_{\text{MAX}}]$ and any
$\vec{\eta}\in\\{0,1\\}^{\mathbb{N}}$.

We now suppose that $\textsc{bad}(Q,\sigma)$ does not occur, and compare
$q^{\sigma}_{C}(t)$ with $q^{\sigma}_{D}(t)$, $\zeta_{1,C}^{\sigma}(t)$ with
$\zeta_{1,D}^{\sigma}(t)$, and $\zeta_{2,C}^{\sigma}(t)$ with
$\zeta_{2,D}^{\sigma}(t)$.

Since $\textsc{bad}(Q,\sigma)$ does not occur, we have

$\displaystyle\max_{\nu}|q^{\sigma}(t_{\nu})|,\max_{\nu}|u^{\sigma}(t_{\nu})|\leq
Q,$ (9.5) $\displaystyle\max_{\nu}\Big{|}\sum_{0\leq\mu<\nu}(\Delta
q^{\sigma}_{\nu})^{2}-t_{\nu}\Big{|}\leq Q^{2}(\Delta t_{\text{MAX}})^{1/2},$
(9.6) $\displaystyle\max_{\nu}\textsc{Osc}(\nu)\leq CQ(\Delta
t_{\text{MAX}})^{1/4}.$ (9.7)

For any $\nu$, and any $t\in[t_{\nu},t_{\nu+1})$, we have

$|q^{\sigma}_{C}(t)-q^{\sigma}_{D}(t)|=|q^{\sigma}(t)-q^{\sigma}(t_{\nu})|\leq\textsc{Osc}(\nu)\leq
CQ(\Delta t_{\text{MAX}})^{1/4}.$

Thus,

$\max_{t\in[0,T]}|q^{\sigma}_{C}(t)-q^{\sigma}_{D}(t)|\leq CQ(\Delta
t_{\text{MAX}})^{1/4},$

as claimed. Next, for any $\nu$ and any $t\in[t_{\nu},t_{\nu+1})$, we have

$\displaystyle\zeta_{1,C}^{\sigma}(t)-\zeta_{1,D}^{\sigma}(t)=$
$\displaystyle\frac{1}{2}[(q^{\sigma}(t))^{2}-(q^{\sigma}(t_{\nu}))^{2}]+\frac{1}{2}\Big{[}\sum_{0\leq\mu<\nu}(\Delta
q^{\sigma}_{\mu})^{2}-t_{\nu}\Big{]}$
$\displaystyle-\frac{1}{2}(t-t_{\nu})-\sum_{0\leq\mu<\nu}\int_{t_{\mu}}^{t_{\mu+1}}\\{u^{\sigma}(t_{\mu})[q^{\sigma}(s)-q^{\sigma}(t_{\mu})]\\}\
ds$ $\displaystyle-\int_{t_{\nu}}^{t}u^{\sigma}(t_{\nu})q^{\sigma}(s)\ ds$
$\displaystyle\equiv$
$\displaystyle\textsc{Term}\;1+\textsc{Term}\;2-\textsc{Term}\;3-\textsc{Term}\;4-\textsc{Term}\;5.$

(Here, we have used the fact that $u^{\sigma}(s)=u^{\sigma}(t_{\mu})$ for
$s\in[t_{\mu},t_{\mu+1})$.) We note that

$\max_{t\in[0,T]}|q^{\sigma}(t)|\leq\max_{0\leq\nu<N}\\{|q^{\sigma}(t_{\nu})|+\textsc{Osc}(\nu)\\}\leq
CQ,$ (9.8)

by (9.5) and (9.7). Hence, for $t\in[t_{\nu},t_{\nu+1})$, we have

$\displaystyle|\textsc{Term}\;1|$
$\displaystyle\leq\max_{\tilde{t}\in[0,T]}|q^{\sigma}(\tilde{t})|\cdot|q^{\sigma}(t)-q^{\sigma}(t_{\nu})|$
$\displaystyle\leq CQ\textsc{Osc}(\nu)\leq CQ^{2}(\Delta
t_{\text{MAX}})^{1/4}.$

So

$|\textsc{Term}\;1|\leq CQ^{2}(\Delta t_{\text{MAX}})^{1/4}\;\text{for
all}\;t\in[0,T).$

Next, (9.6) tells us that

$|\textsc{Term}\;2|\leq Q^{2}(\Delta t_{\text{MAX}})^{1/2}\;\text{for
all}\;t\in[0,T).$

Clearly

$|\textsc{Term}\;3|\leq(\Delta t_{\text{MAX}}).$

Furthermore,

$\displaystyle|\textsc{Term}\;4|$
$\displaystyle\leq\sum_{0\leq\mu<N}|u^{\sigma}(t_{\mu})|\int_{t_{\mu}}^{t_{\mu+1}}|q^{\sigma}(s)-q^{\sigma}(t_{\mu})|\
ds$ $\displaystyle\leq CQ\sum_{0\leq\mu<N}\textsc{Osc}(\mu)\Delta t_{\mu}\leq
CQ\cdot\max_{\mu}\textsc{Osc}(\mu)\leq CQ^{2}(\Delta t_{\text{MAX}})^{1/4}$

thanks to (9.5) and (9.7). Finally,

$\displaystyle|\textsc{Term}\;5|$
$\displaystyle\leq\max_{\nu}|u^{\sigma}(t_{\nu})|\cdot\max_{\tilde{t}\in[0,T]}|q^{\sigma}(\tilde{t})|\cdot(\Delta
t_{\text{MAX}})$ $\displaystyle\leq CQ^{2}(\Delta t_{\text{MAX}}),$

by (9.5) and (9.8). Combining our estimates for Terms 1–5, we find that

$|\zeta_{1,C}^{\sigma}(t)-\zeta_{1,D}^{\sigma}(t)|\leq CQ^{2}(\Delta
t_{\text{MAX}})^{1/4},$

as claimed. We pass to $\zeta_{2,C}^{\sigma}$, $\zeta_{2,D}^{\sigma}$. For
$t\in[t_{\nu},t_{\nu+1})$, we have

$\displaystyle|\zeta_{2,C}^{\sigma}(t)-\zeta_{2,D}^{\sigma}(t)|=$
$\displaystyle\Big{|}\sum_{0\leq\mu<\nu}\int_{t_{\mu}}^{t_{\mu+1}}\\{(q^{\sigma}(s))^{2}-(q^{\sigma}(t_{\mu}))^{2}\\}\
ds+\int_{t_{\nu}}^{t}(q^{\sigma}(s))^{2}\ ds\Big{|}$ $\displaystyle\leq$
$\displaystyle\sum_{0\leq\mu<\nu}C\Big{(}\max_{\tilde{t}\in[0,T]}|q^{\sigma}(\tilde{t})|\Big{)}\int_{t_{\mu}}^{t_{\mu+1}}|q^{\sigma}(s)-q^{\sigma}(t_{\mu})|\
ds$
$\displaystyle+\max_{\tilde{t}\in[0,T]}|q^{\sigma}(\tilde{t})|^{2}\cdot\Delta
t_{\text{MAX}}$ $\displaystyle\leq$ $\displaystyle
CQ\sum_{0\leq\mu<\nu}\textsc{Osc}(\mu)\Delta t_{\mu}+CQ^{2}\Delta
t_{\text{MAX}}$ $\displaystyle\leq$ $\displaystyle
C^{\prime}Q\max_{0\leq\mu<N}\textsc{Osc}(\mu)+CQ^{2}\Delta t_{\text{MAX}}$
$\displaystyle\leq$ $\displaystyle C^{\prime\prime}Q^{2}(\Delta
t_{\text{MAX}})^{1/4},\;\text{thanks to \eqref{eq: cvd 7} and \eqref{eq: cvd
8}}.$

The proof of the lemma is complete. ∎

###  10 Refining a Partition

Let $\sigma$ be a tame strategy $\sigma=(\sigma_{t_{\nu}})_{0\leq\nu<N}$
associated to a partition

$0=t_{0}<t_{1}<\dots<t_{N}=T.$ (10.1)

Suppose

$0=\hat{t}_{0}<\hat{t}_{1}<\dots<\hat{t}_{\hat{N}}=T$ (10.2)

is a refinement of the partition (10.1).

We would like to associate a tame strategy
$\hat{\sigma}=(\hat{\sigma}_{\hat{t}_{\mu}})_{0\leq\mu<{\hat{N}}}$ to the
partition (10.2) in such a way that

$\begin{split}&q^{\hat{\sigma}}(t)=q^{\sigma}(t)\;\text{for
all}\;t\in[0,T],\;\text{and}\\\ &u^{\hat{\sigma}}(t)=u^{\sigma}(t)\;\text{for
all}\;t\in[0,T).\end{split}$ (10.3)

That would tell us that refining the partition (10.1) allows additional tame
strategies, but doesn’t rule out any tame strategies $\sigma$.

Unfortunately, no such $\hat{\sigma}$ exists. The problem is that, in order to
be a tame strategy, $\hat{\sigma}$ must satisfy

$|u^{\hat{\sigma}}(\hat{t}_{\mu})|\leq
C_{\text{TAME}}[|q^{\hat{\sigma}}(\hat{t}_{\mu})|+1]\;\text{with
probability}\;1.$ (10.4)

It may happen that

$|u^{\sigma}(\hat{t}_{\mu})|=|u^{\sigma}(t_{\nu})|\gg
C_{\text{TAME}}\cdot[|q^{\sigma}(\hat{t}_{\mu})|+1]$

for some $\hat{t}_{\mu}\in(t_{\nu},t_{\nu+1})$, in which case (10.4)
contradicts (10.3). Accordingly, we modify (10.3), as follows.

For each $\mu$ ($0\leq\mu<{\hat{N}}$), define $\bar{\nu}(\mu)$ to be the index
$\nu$ for which $t_{\nu}\leq\hat{t}_{\mu}<t_{\nu+1}$.

Define a stopping time $\tau$ by setting

$\tau=\begin{cases}\text{Least }\hat{t}_{\mu}\;\text{s.t.
}|u^{\sigma}(t_{\bar{\nu}(\mu)})|>2C_{\text{TAME}}[|q^{\sigma}(\hat{t}_{\mu})|+1],&\text{if
such a }\hat{t}_{\mu}\;\text{exists},\\\ T&\text{otherwise}.\end{cases}$

Then define random processes $\hat{q}(t)$, $\hat{u}(t)$ ($t\in[0,T]$) by
setting

$\begin{cases}\hat{q}(t)=q^{\sigma}(t)\;\text{and}\;\hat{u}(t)=u^{\sigma}(t)&\text{for}\;0\leq
t<\tau.\\\ \hat{u}(t)=0&\text{for}\;\tau\leq t\leq T,\end{cases}$

and on $[\tau,T]$ defining $\hat{q}$ by

$\begin{cases}d\hat{q}(t)=\big{(}a_{\text{TRUE}}\cdot\hat{q}(t)\big{)}dt+dW(t)\\\
\text{with initial condition}\;\hat{q}(\tau)=q^{\sigma}(\tau).\end{cases}$

Note that

$|\hat{u}(\hat{t}_{\mu})|\leq
2C_{\text{TAME}}[|\hat{q}(\hat{t}_{\mu})|+1]\;\text{for all}\;\hat{t}_{\mu}.$
(10.5)

It is a tedious exercise (left to the reader) to exhibit a tame strategy
$\hat{\sigma}=(\hat{\sigma}_{\hat{t}_{\mu}})_{0\leq\mu<{\hat{N}}}$ associated
to the partition (10.2), such that $\hat{q}(t)=q^{\hat{\sigma}}(t)$ for all
$t\in[0,T]$ and $\hat{u}(t)=u^{\hat{\sigma}}(t)$ for all $t\in[0,T)$.

Whereas our original tame strategy $\sigma$ satisfies

$|\sigma_{t_{\nu}}(q_{1},\dots,q_{\nu},\vec{\xi})|\leq
C_{\text{TAME}}[|q_{\nu}|+1],$ (10.6)

the strategy $\hat{\sigma}$ satisfies instead

$|\hat{\sigma}_{\hat{t}_{\mu}}(q_{1},\dots,q_{\mu},\vec{\xi})|\leq
2C_{\text{TAME}}[|q_{\mu}|+1];$ (10.7)

compare with (10.5).

In place of (10.3), we will show that $q^{\hat{\sigma}}(t)$ and
$u^{\hat{\sigma}}(t)$ are likely very close to $q^{\sigma}(t)$,
$u^{\sigma}(t)$, respectively. To see this, we fix
$a\in[-a_{\text{MAX}},a_{\text{MAX}}]$ and
$\vec{\eta}\in\\{0,1\\}^{\mathbb{N}}$, and condition on $a_{\text{TRUE}}=a$,
$\vec{\xi}=\vec{\eta}$.

We define random variables

$\textsc{Osc}(\nu)=\max_{t\in[t_{\nu},t_{\nu+1}]}|q^{\sigma}(t)-q^{\sigma}(t_{\nu})|$
(10.8)

and an event

$\textsc{disaster}:\;\textsc{Osc}(\nu)\geq 1\;\text{for some}\;\nu.$ (10.9)

In Section 8, we proved that

$\text{E}_{a,\vec{\eta}}[(\textsc{Osc}(\nu))^{m}]\leq C_{m}(\Delta
t_{\nu})^{m/2}\;\text{for all}\;m\geq 1.$ (10.10)

Hence,

$\text{Prob}_{a,\vec{\eta}}[\textsc{Osc}(\nu)\geq 1]\leq C_{\bar{m}}(\Delta
t_{\nu})^{\bar{m}}\;\text{for all}\;\bar{m}\geq 1,$

and consequently

$\text{Prob}_{a,\vec{\eta}}[\textsc{disaster}]\leq\sum_{0\leq\nu<N}C_{\bar{m}}(\Delta
t_{\nu})^{\bar{m}}\leq C_{\bar{m}-1}^{\prime}(\Delta
t_{\text{MAX}})^{\bar{m}-1}$ (10.11)

for any $\bar{m}\geq 1$.

Next, we prepare to estimate

$\int_{0}^{T}\\{|q^{\hat{\sigma}}(t)-q^{\sigma}(t)|^{m}+|u^{\hat{\sigma}}(t)-u^{\sigma}(t)|^{m}\\}\
dt.$

Let

$q^{\sigma}_{0}(t)=q^{\sigma}(t_{\nu})\;\text{for
}t\in[t_{\nu},t_{\nu+1}),\;0\leq\nu<N.$

Then

$\int_{t_{\nu}}^{t_{\nu+1}}|q^{\sigma}_{0}(t)-q^{\sigma}(t)|^{2m}\;dt\leq(\textsc{Osc}(\nu))^{2m}\Delta
t_{\nu},$

hence

$\begin{split}\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}|q^{\sigma}_{0}(t)-q^{\sigma}(t)|^{2m}\
dt\Big{]}&\leq\sum_{\nu}\text{E}_{a,\vec{\eta}}[(\textsc{Osc}(\nu))^{2m}]\Delta
t_{\nu}\\\ &\leq C_{m}(\Delta t_{\text{MAX}})^{m},\end{split}$ (10.12)

thanks to (10.10). Similarly,

$\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}|q^{\hat{\sigma}}_{0}(t)-q^{\hat{\sigma}}(t)|^{2m}\
dt\Big{]}\leq C_{m}(\Delta t_{\text{MAX}})^{m},$ (10.13)

where $q^{\hat{\sigma}}_{0}(t)=q^{\hat{\sigma}}(\hat{t}_{\mu})$ for
$t\in[\hat{t}_{\mu},\hat{t}_{\mu+1})$, $0\leq\mu<{\hat{N}}$. We have also

$\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}|q^{\sigma}_{0}(t)|^{2m}\
dt\Big{]}=\text{E}_{a,\vec{\eta}}\Big{[}\sum_{0\leq\nu<N}|q^{\sigma}(t_{\nu})|^{2m}\Delta
t_{\nu}\Big{]}\leq C_{m},$ (10.14)

thanks to Lemma 1. Similarly,

$\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}|q^{\hat{\sigma}}_{0}(t)|^{2m}\
dt\Big{]}\leq C_{m}.$ (10.15)

From (10.12) and (10.14) we obtain

$\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}|q^{\sigma}(t)|^{2m}\ dt\Big{]}\leq
C_{m}\;\text{for}\;m\geq 1.$ (10.16)

Similarly, from (10.13) and (10.15), we have

$\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}|q^{\hat{\sigma}}(t)|^{2m}\
dt\Big{]}\leq C_{m}\;\text{for }m\geq 1.$ (10.17)

Turning to $u^{\sigma}$ and $u^{\hat{\sigma}}$, we recall that
$u^{\sigma}(t)=u^{\sigma}(t_{\nu})$ for $t\in[t_{\nu},t_{\nu+1})$,
$0\leq\nu<N$; hence,

$\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}|u^{\sigma}(t)|^{2m}\
dt\Big{]}=\text{E}_{a,\vec{\eta}}\Big{[}\sum_{0\leq\nu<N}|u^{\sigma}(t_{\nu})|^{2m}\Delta
t_{\nu}\Big{]}\leq C_{m}\;(m\geq 1),$ (10.18)

by Lemma 1. Similarly

$\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}|u^{\hat{\sigma}}(t)|^{2m}\
dt\Big{]}\leq C_{m}\;(m\geq 1).$ (10.19)

From (10.16)–(10.18) we see that

$\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}\\{|q^{\hat{\sigma}}(t)-q^{\sigma}(t)|+|u^{\hat{\sigma}}(t)-u^{\sigma}(t)|\\}^{2m}\
dt\Big{]}\leq C_{m}\;(m\geq 1).$ (10.20)

Moreover, unless disaster occurs, we have $\tau=T$, hence

$q^{\hat{\sigma}}(t)=q^{\sigma}(t)\;\text{and}\;u^{\hat{\sigma}}(t)=u^{\sigma}(t)\;\text{for
all}\;t\in[0,T].$ (10.21)

Indeed, if disaster doesn’t occur, then for $0\leq\nu<N$ and
$t\in[t_{\nu},t_{\nu+1})$ we have

$|q^{\sigma}(t)-q^{\sigma}(t_{\nu})|\leq 1,$

hence $[|q^{\sigma}(t)|+1]$ and $[|q^{\sigma}(t_{\nu})|+1]$ differ by at most
a factor of 2. Since $|u^{\sigma}(t_{\nu})|\leq
C_{\text{TAME}}[|q^{\sigma}(t_{\nu})|+1]$, it follows that
$|u^{\sigma}(t_{\nu})|\leq 2C_{\text{TAME}}[|q^{\sigma}(t)|+1]$ for
$t\in[t_{\nu},t_{\nu+1})$, $0\leq\nu<N$. In particular,

$|u^{\sigma}(t_{\bar{\nu}(\mu)})|\leq
2C_{\text{TAME}}[|q^{\sigma}(t_{\mu})|+1]\;\text{for
all}\;\mu\;(0\leq\mu<{\hat{N}}).$ (10.22)

Comparing (10.22) with the definition of $\tau$, we see that, as claimed,
$\tau=T$ unless disaster occurs.

Thus (10.21) holds unless disaster occurs.

From (10.20), (10.21), (10.11), we now have

$\begin{split}\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}&\\{|q^{\hat{\sigma}}(t)-q^{\sigma}(t)|+|u^{\hat{\sigma}}(t)-u^{\sigma}(t)|\\}^{m}\
dt\Big{]}\\\
=&\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}\\{|q^{\hat{\sigma}}(t)-q^{\sigma}(t)|+|u^{\hat{\sigma}}(t)-u^{\sigma}(t)|\\}^{m}\mathbbm{1}_{\textsc{disaster}}\
dt\Big{]}\\\
\leq&\Big{(}\text{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}\\{|q^{\hat{\sigma}}(t)-q^{\sigma}(t)|+|u^{\hat{\sigma}}(t)-u^{\sigma}(t)|\\}^{2m}\
dt\Big{]}\Big{)}^{1/2}\\\
&\cdot(\text{Prob}_{a,\vec{\eta}}[\textsc{disaster}])^{1/2}\\\
\leq&C_{m,\bar{m}}\cdot(\Delta t_{\text{MAX}})^{(\bar{m}-1)/2}\;\text{for
any}\;m,\bar{m}\geq 1.\end{split}$ (10.23)

We record this result as a lemma.

######  Lemma 9 (Refinement Lemma).

Let $\sigma$ be a tame strategy associated to a partition

$0=t_{0}<t_{1}<\dots<t_{N}=T,$ (A)

and let

$0=\hat{t}_{0}<\hat{t}_{1}<\dots<\hat{t}_{\hat{N}}=T$ (B)

be a refinement of the partition _(A)_.

Then there exists a tame strategy $\hat{\sigma}$ associated to the partition
_(B)_ , such that

$\emph{E}_{a,\vec{\eta}}\Big{[}\int_{0}^{T}\\{|q^{\hat{\sigma}}(t)-q^{\sigma}(t)|+|u^{\hat{\sigma}}(t)-u^{\sigma}(t)|\\}^{m}\
dt\Big{]}\leq C_{m,\bar{m}}(\Delta t_{\emph{MAX}})^{\bar{m}}$

for all $m,\bar{m}\geq 1$ and all $a\in[-a_{\emph{MAX}},+a_{\emph{MAX}}]$,
$\vec{\eta}\in\\{0,1\\}^{\mathbb{N}}$.

The strategy $\hat{\sigma}$ satisfies the same estimates as we assumed for
$\sigma$ (see 7.1), except that $C_{\emph{TAME}}$ is replaced by
$2C_{\emph{TAME}}$.

##  Chapter 3 Bayesian Strategies Associated to Partitions

###  1 Setup

In this chapter, we take $a_{\text{TRUE}}$ to be governed by a known prior
probability distribution $d\text{Prior}(a)$, concentrated on an interval
$[-a_{\text{MAX}},+a_{\text{MAX}}]$.

We fix a partition

$0=t_{0}<t_{1}<\dots<t_{N}=T$ (1.1)

of the time interval $[0,T]$.

We fix a deterministic strategy $\sigma$ for the game starting at position
$q_{0}$.

We assume that our strategy $\sigma$ is tame, i.e.,

$|u^{\sigma}(t_{\nu})|\leq C_{\text{TAME}}^{\sigma}[|q^{\sigma}(t_{\nu})|+1]$
(1.2)

for a constant $C_{\text{TAME}}^{\sigma}$. We call $C_{\text{TAME}}^{\sigma}$
the _tame constant_ for $\sigma$.

We write

$\displaystyle q_{\nu}^{\sigma}=q^{\sigma}(t_{\nu}),$ $\displaystyle\Delta
q_{\nu}^{\sigma}=q_{\nu+1}^{\sigma}-q_{\nu}^{\sigma},$
$\displaystyle\zeta_{1,\nu}^{\sigma}=\zeta_{1}^{\sigma}(t_{\nu}),$
$\displaystyle\Delta\zeta_{1,\nu}^{\sigma}=\zeta_{1,\nu+1}^{\sigma}-\zeta_{1,\nu}^{\sigma},$
$\displaystyle\zeta_{2,\nu}^{\sigma}=\zeta_{2}^{\sigma}(t_{\nu}),$
$\displaystyle\Delta\zeta_{2,\nu}^{\sigma}=\zeta_{2,\nu+1}^{\sigma}-\zeta_{2,\nu}^{\sigma},$
$\displaystyle u^{\sigma}_{\nu}=u^{\sigma}(t_{\nu}).$

Until further notice, $c,C,C^{\prime},$ etc. will denote constants determined
by $C_{\text{TAME}}^{\sigma}$ in (1.2) together with $a_{\text{MAX}}$ and
upper bounds for $T$ and $|q_{0}|$. The symbols $c,C,C^{\prime},$ etc. may
denote different constants in different occurrences. We assume that

$\Delta t_{\text{MAX}}\equiv\max_{0\leq\nu<N}(t_{\nu+1}-t_{\nu})<c\;\text{for
a small enough constant }c.$ (1.3)

We write $X=O(Y)$ to indicate that $|X|\leq CY$. We write
$\mathcal{F}_{\nu}^{\sigma}$ to denote the sigma algebra of events determined
by $q^{\sigma}(t_{\mu})$ for $\mu=0,1,\dots,\nu$. Note that
$\mathcal{F}_{\nu}^{\sigma}$ depends on $\sigma$, because $a_{\text{TRUE}}$
isn’t deterministic.

We write $\text{Prob}[\dots]$ to denote probability, and we write
$\text{E}[\dots]$ to denote expectation.

If we condition on $a_{\text{TRUE}}=a$, then we write $\text{Prob}_{a}[\dots]$
and $\text{E}_{a}[\dots]$ to denote the corresponding probability and
expectation. Thus, for instance, $\text{E}_{a}[X|\mathcal{F}_{\nu}^{\sigma}]$
dentoes the expected value of $X$ conditioned on $\mathcal{F}_{\nu}^{\sigma}$,
given that $a_{\text{TRUE}}=a$.

For any event $\mathcal{E}$ we have

$\text{Prob}[\mathcal{E}]=\int_{-a_{\text{MAX}}}^{a_{\text{MAX}}}\text{Prob}_{a}[\mathcal{E}]\
d\text{Prior}(a)$ (1.4)

and

$\text{Prob}[\mathcal{E}|\mathcal{F}_{\nu}^{\sigma}]=\int_{-a_{\text{MAX}}}^{a_{\text{MAX}}}\text{Prob}_{a}[\mathcal{E}|\mathcal{F}_{\nu}^{\sigma}]\
d\text{Post}(a|\mathcal{F}_{\nu}^{\sigma}),$ (1.5)

where $d\text{Post}(a|\mathcal{F}_{\nu}^{\sigma})$ is the posterior
probability distribution for $a_{\text{TRUE}}$ conditioned on
$\mathcal{F}_{\nu}^{\sigma}$.

Similarly, for any random variable $X$, we have

$\text{E}[X]=\int_{-a_{\text{MAX}}}^{a_{\text{MAX}}}\text{E}_{a}[X]\
d\text{Prior}(a)$ (1.6)

and

$\text{E}[X|\mathcal{F}_{\nu}^{\sigma}]=\int_{-a_{\text{MAX}}}^{a_{\text{MAX}}}\text{E}_{a}[X|\mathcal{F}_{\nu}^{\sigma}]\
d\text{Post}(a|\mathcal{F}_{\nu}^{\sigma}).$ (1.7)

Thanks to (1.4)–(1.7), the following results are immediate from Lemma 1.

######  Lemma 10.

[Bayesian Lemma on Rare Events] Suppose $Q>C$ for a large enough $C$. Then the
following hold with probability $>1-\exp(-cQ^{2})$:

  * •

$|q^{\sigma}(t_{\nu})|,|u^{\sigma}(t_{\nu})|\leq Q$, for all $\nu$.

  * •

$|\zeta_{1}^{\sigma}(t_{\nu})|,|\zeta_{2}^{\sigma}(t_{\nu})|\leq Q^{2}$, for
all $\nu$.

  * •

$|\sum_{0\leq\mu<\nu}(q^{\sigma}(t_{\mu+1})-q^{\sigma}(t_{\mu}))^{2}-t_{\nu}|\leq
Q^{2}(\Delta t_{\emph{MAX}})^{1/2}$ for all $\nu$.

Moreover, suppose we fix $\nu$ and condition on $\mathcal{F}_{\nu}^{\sigma}$,
the sigma algebra of events determined by $q^{\sigma}(t_{\mu})$ for
$0\leq\mu\leq\nu$. Suppose $|q^{\sigma}(t_{\nu})|\leq Q$, where $C\leq
Q\leq(\Delta t_{\nu})^{-1/1000}$ (for large enough $C$).

Then for $p=1,2$, we have

$\emph{E}[(|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|)^{p}\cdot\mathbbm{1}_{|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|>(\Delta
t_{\nu})^{2/5}}|\mathcal{F}_{\nu}^{\sigma}]\\\ \leq C\cdot(\Delta
t_{\nu})^{100};$

also

$\emph{Prob}\Big{[}\max_{t\in[t_{\nu},t_{\nu+1}]}|q^{\sigma}(t)-q^{\sigma}_{\nu}|>(\Delta
t_{\nu})^{2/5}|\mathcal{F}_{\nu}^{\sigma}\Big{]}\leq C\cdot(\Delta
t_{\nu})^{1000}$

and

$\emph{Prob}[|\Delta
q^{\sigma}_{\nu}|+|\Delta\zeta_{1,\nu}^{\sigma}|+|\Delta\zeta_{2,\nu}^{\sigma}|>(\Delta
t_{\nu})^{2/5}|\mathcal{F}_{\nu}^{\sigma}]\leq C\cdot(\Delta t_{\nu})^{1000}.$

###  2 Posterior Probabilities and Expectations

Fix $Q>C$ for large enough $C$.

For each $\nu$, let $\text{OK}(\nu)$ denote the set of all
$(\bar{q}_{1},\dots,\bar{q}_{\nu})\in\mathbb{R}^{\nu}$ s.t.

$\max_{\mu\leq\nu}|\bar{q}_{\mu}|+|\bar{u}_{\mu}|\leq Q$ (2.1)

and

$\Big{|}\sum_{\mu<\nu}(\Delta\bar{q}_{\mu})^{2}-t_{\nu}\Big{|}\leq
Q^{2}(\Delta t_{\text{MAX}})^{1/4},$ (2.2)

where $\Delta\bar{q}_{\mu}=\bar{q}_{\mu+1}-\bar{q}_{\mu}$, $\bar{q}_{0}\equiv
q_{0}$, and $\bar{u}_{\mu}$ is defined to be the value assigned to
$u^{\sigma}(t_{\mu})$ provided $q^{\sigma}(t_{\gamma})=\bar{q}_{\gamma}$ for
$\gamma\leq\mu$.

According to the Bayesian Lemma on Rare Events, we have

$\text{Prob}[\text{NOT }\text{OK}(\nu)]\leq\exp(-cQ^{2}).$ (2.3)

Thanks to (1.3) and Lemma 2, the following holds for
$a\in[-a_{\text{MAX}},a_{\text{MAX}}]$ and
$(\bar{q}_{1},\dots,\bar{q}_{\nu})\in\text{OK}(\nu)$:

$\text{Prob}[a_{\text{TRUE}}\in[a,a+da]\;\text{and}\;q^{\sigma}(t_{\mu})\in[\bar{q}_{\mu},\bar{q}_{\mu}+d\bar{q}_{\mu}]\;\text{for}\;\mu\leq\nu]\\\
=(1+O(Q^{2}(\Delta t_{\text{MAX}})^{1/4}))\cdot(*)\cdot d\bar{q}_{1}\cdots
d\bar{q}_{\nu},$

where

$(*)=\Big{[}d\text{Prior}(a)\cdot\prod_{0\leq\mu<\nu}\Big{\\{}\frac{1}{\sqrt{2\pi\Delta
t_{\mu}}}\exp\Big{(}-\frac{1}{2\Delta
t_{\mu}}(\Delta\bar{q}_{\mu}-(a\bar{q}_{\mu}+\bar{u}_{\mu})\Delta
t_{\mu})^{2}\Big{)}\Big{\\}}\Big{]}.$

Note that

$(*)=d\text{Prior}(a)\cdot\exp\Big{(}-\frac{1}{2}\bar{\zeta}_{2,\nu}a^{2}+\bar{\zeta}_{1,\nu}a\Big{)}\cdot\\{\text{Factor
independent of }a\\},$

where

$\bar{\zeta}_{2,\nu}=\bar{\zeta}_{2,\nu}(\bar{q}_{1},\dots,\bar{q}_{\nu})=\sum_{0\leq\mu<\nu}\bar{q}_{\mu}^{2}\Delta
t_{\mu}$ (2.4)

and

$\bar{\zeta}_{1,\nu}=\bar{\zeta}_{1,\nu}(\bar{q}_{1},\dots,\bar{q}_{\nu})=\sum_{0\leq\mu<\nu}\bar{q}_{\mu}(\Delta\bar{q}_{\mu}-\bar{u}_{\mu}\Delta
t_{\mu}).$ (2.5)

Hence, for

$(\bar{q}_{1},\dots,\bar{q}_{\nu})\in\text{OK}(\nu),$

the posterior probability distribution for $a_{\text{TRUE}},$ given that
$q^{\sigma}(t_{\mu})=\bar{q}_{\mu}$ for $\mu=1,\dots,\nu,$ is given by

$d\text{Post}(a|\bar{q}_{1},\dots,\bar{q}_{\nu})\\\ =(1+O(Q^{2}(\Delta
t_{\text{MAX}})^{1/4}))\cdot\frac{d\text{Prior}(a)\cdot\exp\Big{(}-\frac{1}{2}\bar{\zeta}_{2,\nu}a^{2}+\bar{\zeta}_{1,\nu}a\Big{)}}{Z}$
(2.6)

for a normalizing constant $Z$. Since

$\int_{-a_{\text{MAX}}}^{a_{\text{MAX}}}d\text{Post}(a|\bar{q}_{1},\dots,\bar{q}_{\nu})=1,$

equation (2.6) holds with

$Z=\int_{-a_{\text{MAX}}}^{a_{\text{MAX}}}d\text{Prior}(a)\cdot\exp\Big{(}-\frac{1}{2}\bar{\zeta}_{2,\nu}a^{2}+\bar{\zeta}_{1,\nu}a\Big{)}.$
(2.7)

We have thus proven the following result.

######  Lemma 11 (Lemma on Posterior Probabilities).

Suppose $Q\geq C$ for a large enough $C$. Fix $\nu$, and suppose
$(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\nu}))$ satisfy

$\displaystyle\max_{\mu\leq\nu}|q^{\sigma}(t_{\mu})|+|u^{\sigma}(t_{\mu})|\leq
Q,$ (2.8) $\displaystyle\Big{|}\sum_{\mu<\nu}(\Delta
q^{\sigma}(t_{\mu}))^{2}-t_{\nu}\Big{|}\leq Q^{2}(\Delta
t_{\emph{MAX}})^{1/4}.$ (2.9)

Then the posterior probability distribution for $a_{\emph{TRUE}}$ conditioned
on $\mathcal{F}_{\nu}^{\sigma}$ is given by

$\begin{split}d\emph{Post}(a|\mathcal{F}_{\nu}^{\sigma})=&(1+O(Q^{2}(\Delta
t_{\emph{MAX}})^{1/4}))\\\
&\cdot\frac{\exp\Big{(}-\frac{1}{2}\zeta_{2}^{\sigma}(t_{\nu})a^{2}+\zeta_{1}^{\sigma}(t_{\nu})a\Big{)}\cdot
d\emph{Prior}(a)}{Z}\end{split}$ (2.10)

with

$Z=\int_{-a_{\emph{MAX}}}^{a_{\emph{MAX}}}\exp\Big{(}-\frac{1}{2}\zeta_{2}^{\sigma}(t_{\nu})b^{2}+\zeta_{1}^{\sigma}(t_{\nu})b\Big{)}\
d\emph{Prior}(b).$ (2.11)

######  Corollary 1.

Under the assumption of the above Lemma, we have

$\emph{E}[a_{\emph{TRUE}}|\mathcal{F}_{\nu}^{\sigma}]=O(Q^{2}(\Delta
t_{\emph{MAX}})^{1/4})+\bar{a}(\zeta_{1}^{\sigma}(t_{\nu}),\zeta_{2}^{\sigma}(t_{\nu})),$
(2.12)

where

$\bar{a}(\bar{\zeta_{1}},\bar{\zeta_{2}})\equiv\frac{\int_{-a_{\emph{MAX}}}^{a_{\emph{MAX}}}a\exp\Big{(}-\frac{a^{2}}{2}\bar{\zeta_{2}}+a\bar{\zeta_{1}}\Big{)}\
d\emph{Prior}(a)}{\int_{-a_{\emph{MAX}}}^{a_{\emph{MAX}}}\exp\Big{(}-\frac{a^{2}}{2}\bar{\zeta_{2}}+a\bar{\zeta_{1}}\Big{)}\
d\emph{Prior}(a)}$ (2.13)

for $(\bar{\zeta_{1}},\bar{\zeta_{2}})\in\mathbb{R}^{2}$.

Thanks to the above Corollary, formula (1.7), and Lemma 4, we now have the
following results.

######  Lemma 12 (Lemma on Posterior Expectations).

Suppose $Q\geq C$ for large enough $C$, and assume that $\Delta
t_{\emph{MAX}}\leq Q^{-1000}$.

Define the event

$\emph{TAME}(\nu)=\\{|\Delta q^{\sigma}_{\nu}|\leq 2(\Delta
t_{\nu})^{2/5},|\Delta\zeta_{1,\nu}^{\sigma}|\leq 2(\Delta
t_{\nu})^{2/5},|\Delta\zeta_{2,\nu}^{\sigma}|\leq 2(\Delta t_{\nu})^{2/5}\\}.$

Fix $\nu$, and suppose we have

$\displaystyle\max_{\mu\leq\nu}|q^{\sigma}(t_{\mu})|+|u^{\sigma}(t_{\mu})|\leq
Q,\;\text{and}$ $\displaystyle\Big{|}\sum_{0\leq\mu<\nu}(\Delta
q^{\sigma}(t_{\mu}))^{2}-t_{\nu}\Big{|}\leq Q^{2}(\Delta
t_{\emph{MAX}})^{1/4}.$

Then the following hold.

$\displaystyle\emph{E}[(\Delta
q^{\sigma}_{\nu})\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}^{\sigma}]=[\bar{a}(\zeta_{1}^{\sigma}(t_{\nu}),\zeta_{2}^{\sigma}(t_{\nu}))q^{\sigma}_{\nu}+u^{\sigma}_{\nu}](\Delta
t_{\nu})+\emph{ERR}\;1,$
$\displaystyle\emph{E}[(\Delta\zeta_{1,\nu}^{\sigma})\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}^{\sigma}]=\bar{a}(\zeta_{1,\nu}^{\sigma}(t_{\nu}),\zeta_{2,\nu}^{\sigma}(t_{\nu}))\cdot(q^{\sigma}_{\nu})^{2}(\Delta
t_{\nu})+\emph{ERR}\;2,$
$\displaystyle\emph{E}[(\Delta\zeta_{2,\nu}^{\sigma})\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}^{\sigma}]=(q^{\sigma}_{\nu})^{2}(\Delta
t_{\nu})+\emph{ERR}\;3,$ $\displaystyle\emph{E}[(\Delta
q^{\sigma}_{\nu})^{2}\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}^{\sigma}]=(\Delta
t_{\nu})+\emph{ERR}\;4,$ $\displaystyle\emph{E}[(\Delta
q^{\sigma}_{\nu})\cdot(\Delta\zeta_{1,\nu}^{\sigma})\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}^{\sigma}]=q^{\sigma}_{\nu}(\Delta
t_{\nu})+\emph{ERR}\;5,$ $\displaystyle\emph{E}[(\Delta
q^{\sigma}_{\nu})\cdot(\Delta\zeta_{2,\nu}^{\sigma})\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}^{\sigma}]=\emph{ERR}\;6,$
$\displaystyle\emph{E}[(\Delta\zeta_{1,\nu}^{\sigma})^{2}\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}^{\sigma}]=(q^{\sigma}_{\nu})^{2}(\Delta
t_{\nu})+\emph{ERR}\;7,$
$\displaystyle\emph{E}[(\Delta\zeta_{1,\nu}^{\sigma})(\Delta\zeta_{2,\nu}^{\sigma})\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}]=\emph{ERR}\;8,$
$\displaystyle\emph{E}[(\Delta\zeta_{2,\nu}^{\sigma})^{2}\cdot\mathbbm{1}_{\emph{TAME}(\nu)}|\mathcal{F}_{\nu}^{\sigma}]=\emph{ERR}\;9,$

where

$|\emph{ERR}\;1|,\dots,|\emph{ERR}\;9|\leq C^{\prime}Q^{4}(\Delta
t_{\emph{MAX}})^{1/4}\Delta t_{\nu}.$

Moreover, under the above assumptions on
$(q^{\sigma}(t_{1}),\dots,q^{\sigma}(t_{\nu}))$, we have

$\emph{Prob}[\emph{NOT TAME}(\nu)|\mathcal{F}_{\nu}^{\sigma}]\leq(\Delta
t_{\nu})^{20}.$

Here,

$\bar{a}(\zeta_{1},\zeta_{2}):=\frac{\int_{-a_{\emph{MAX}}}^{a_{\emph{MAX}}}a\exp\Big{(}-\frac{a^{2}}{2}\zeta_{2}+a\zeta_{1}\Big{)}\
d\emph{Prior}(a)}{\int_{-a_{\emph{MAX}}}^{a_{\emph{MAX}}}\exp\Big{(}-\frac{a^{2}}{2}\zeta_{2}+a\zeta_{1}\Big{)}\
d\emph{Prior}(a)}.$

###  3 The PDE Assumption

For our given probability distribution $d\text{Prior}(a)$, we fix the function
$\bar{a}(\zeta_{1},\zeta_{2})$ given in the preceding section, and we
introduce the following PDE for an unknown function
$S(q,t,\zeta_{1},\zeta_{2})$ defined on
$\mathbb{R}\times[0,T]\times\mathbb{R}\times[0,\infty)$:

$\begin{split}0=&\partial_{t}S+(\bar{a}(\zeta_{1},\zeta_{2})q+u_{\text{opt}})\partial_{q}S+\bar{a}(\zeta_{1},\zeta_{2})q^{2}\partial_{\zeta_{1}}S+q^{2}\partial_{\zeta_{2}}S+\frac{1}{2}\partial_{q}^{2}S\\\
&+q\partial_{q\zeta_{1}}S+\frac{1}{2}q^{2}\partial_{\zeta_{1}}^{2}S+(q^{2}+u_{\text{opt}}^{2}),\end{split}$
(3.1)

where

$u_{\text{opt}}=-\frac{1}{2}\partial_{q}S.$ (3.2)

We _assume_ the existence of a solution of the above PDE, satisfying the
following additional conditions.

The terminal condition:

$S(q,t,\zeta_{1},\zeta_{2})=0\;\text{at}\;t=T.$ (3.3)

Positivity:

$S(q,t,\zeta_{1},\zeta_{2})\geq 0.$ (3.4)

Estimates: We assume $S\in C^{1,2}$. For $|\alpha|\leq 3$, we have almost
everywhere that

$|\partial^{\alpha}S(q,t,\zeta_{1},\zeta_{2})|\leq
K\cdot[1+|q|+|\zeta_{1}|+|\zeta_{2}|]^{m_{0}}$ (3.5)

for constants $K$, $m_{0}\geq 1$. Moreover,

$|u_{\text{opt}}(q,t,\zeta_{1},\zeta_{2})|\leq
C_{\text{TAME}}^{\text{opt}}[|q|+1].$ (3.6)

Note that (3.5) holds everywhere, not just almost everywhere, when
$|\alpha|\leq 2$.

We call $K$, $m_{0}$, $C_{\text{TAME}}^{\text{opt}}$, $a_{\text{MAX}}$, and
our upper bounds for $T$ and $|q_{0}|$ the boilerplate constants. We now
broaden our definition of constants $c$, $C$, $C^{\prime}$, etc., to allow
them to depend on the boilerplate constants. As usual, these symbols may
denote different constants in different occurrences.

We strengthen our large $Q$ assumption. More precisely, we assume from now on
that $Q\geq C$ for a large enough constant $C$. Since the meaning of the
constant $C$ has changed, the above is stronger than our previous large $Q$
assumption.

We assume that

$\Delta t_{\text{MAX}}\leq Q^{-1000}.$

###  4 The Allegedly Optimal Strategy

Let $u_{\text{opt}}=u_{\text{opt}}(q,t,\zeta_{1},\zeta_{2})$ be as in Section
3.

We define a strategy $\tilde{\sigma}$ based on the function $u_{\text{opt}}$.

Given $\nu$ ($1\leq\nu\leq N)$ and given real numbers $q_{1},\dots,q_{\nu}$,
we define numbers $u_{\mu}$, $\zeta_{1,\mu}$, $\zeta_{2,\mu}$ by induction on
$\mu=0,1,\dots,\nu$ so that $\zeta_{1,0}=\zeta_{2,0}=0$, and

$\displaystyle
u_{\mu}=u_{\text{opt}}(q_{\mu},t_{\mu},\zeta_{1,\mu},\zeta_{2,\mu})$
$\displaystyle\zeta_{1,\mu}=\sum_{0\leq\gamma<\mu}q_{\gamma}([q_{\gamma+1}-q_{\gamma}]-u_{\gamma}\Delta
t_{\gamma})$
$\displaystyle\zeta_{2,\mu}=\sum_{0\leq\gamma<\mu}q_{\gamma}^{2}\Delta
t_{\gamma}$

for each $\mu$.

We then set $\tilde{\sigma}_{\nu}(q_{1},\dots,q_{\nu})$ equal to the above
$u_{\mu}$ for $\mu=\nu$.

We define our allegedly optimal strategy $\tilde{\sigma}$ to be the collection
of tame rules

$\tilde{\sigma}=(\tilde{\sigma}_{\nu})_{\nu=0,1,\dots,N-1}.$

Thanks to our PDE Assumption (see (3.6)), each $\tilde{\sigma}_{\nu}$ is
indeed a tame rule with tame constant $C_{\text{TAME}}^{\text{opt}}$, hence
$\tilde{\sigma}$ is a strategy.

###  5 Performance of Competing Strategies

We have just defined the allegedly optimal strategy $\tilde{\sigma}$, based on
the functions $u_{\text{opt}}(q,t,\zeta_{1},\zeta_{2})$,
$S(q,t,\zeta_{1},\zeta_{2})$, and $\bar{a}(\zeta_{1},\zeta_{2})$.