# Stochastic Linear-Quadratic Optimal Control with Partial Observation

Jingrui Sun and Jie Xiong Department of Mathematics, Southern University of
Science and Technology, Shenzhen, 518055, China (Email: sunjr@sustech.edu.cn).
This author is supported by NSFC grant 11901280 and Guangdong Basic and
Applied Basic Research Foundation 2021A1515010031.Department of Mathematics
and SUSTech International center for Mathematics, Southern University of
Science and Technology, Shenzhen, 518055, China (Email:
xiongj@sustech.edu.cn). This author is supported by NSFC Grants 61873325 and
11831010.

Abstract. The paper studies a class of quadratic optimal control problems for
partially observable linear dynamical systems. In contrast to the full
information case, the control is required to be adapted to the filtration
generated by the observation system, which in turn is influenced by the
control. The variation method fails in this case due to the fact that the
filtration is not fixed. To overcome the difficulty, we use the orthogonal
decomposition of the state process to write the cost functional as the sum of
two parts: one is a functional of the control and the filtering process and
the other part is independent of the choice of the control. The first part
possesses a mathematical structure similar to the full information problem. By
completing the square, it is shown that the optimal control is given by a
feedback representation via the filtering process. The optimal value is also
obtained explicitly.

Key words. optimal control, linear-quadratic, observation process, filtering,
Riccati equation.

AMS 2020 Mathematics Subject Classification. 49N10, 49N30, 93E11, 93E20.

##  1 Introduction

Let $(\varOmega,\mathscr{F},\mathbb{P})$ be a complete probability space on
which two standard independent Brownian motions
$W=\\{W(t)=(W_{1}(t),\ldots,W_{d}(t))^{\top};\,t\geq 0\\}$ and
$W^{\prime}=\\{W^{\prime}(t)=(W^{\prime}_{1}(t),\ldots,W^{\prime}_{k}(t))^{\top};\,t\geq
0\\}$, with values in $\mathbb{R}^{d}$ and $\mathbb{R}^{k}$, respectively, are
defined. The superscript $\top$ denotes the transpose of a vector or a matrix,
so both $W(t)$ and $W^{\prime}(t)$ are column vectors. Let
$\mathbb{F}=\\{\mathscr{F}_{t}\\}$ be the usual augmentation of the natural
filtration generated by $(W,W^{\prime})$, and let
$\mathbb{G}=\\{\mathscr{G}_{t}\\}$ be a sub-filtration of $\mathbb{F}$. For a
random variable $\xi$, we write $\xi\in\mathscr{G}_{t}$ if $\xi$ is
$\mathscr{G}_{t}$-measurable; and for a stochastic process $\varphi$, we write
$\varphi\in\mathbb{G}$ if it is progressively measurable with respect to the
filtration $\mathbb{G}$.

Consider the following linear stochastic differential equation (SDE) over a
finite horizon $[0,T]$:

$\left\\{\begin{aligned}
dX(t)&=[A(t)X(t)+B(t)u(t)+a(t)]dt+C(t)dW(t)+D(t)dW^{\prime}(t),\\\
X(0)&=x,\end{aligned}\right.$ (1.1)

where the initial state $x$ is a constant vector in $\mathbb{R}^{n}$, and the
coefficients $A$, $B$, $a$, $C$, and $D$ are deterministic, bounded functions
on $[0,T]$, with values in $\mathbb{R}^{n\times n}$, $\mathbb{R}^{n\times m}$,
$\mathbb{R}^{n}$, $\mathbb{R}^{n\times d}$, and $\mathbb{R}^{n\times k}$,
respectively. For a control $u$ that belongs to the space

$\displaystyle\textstyle{\cal
U}=\Big{\\{}u:[0,T]\times\varOmega\to\mathbb{R}^{m}\bigm{|}u\in\mathbb{F}~{}\text{and}~{}\mathbb{E}\int_{0}^{T}|u(t)|^{2}dt<\infty\Big{\\}},$
(1.2)

the state equation (1.1) admits a unique strong solution $X=\\{X(t);\,0\leq
t\leq T\\}$, which is a square-integrable
$\\{\mathscr{F}_{t}\\}$-semimartingale satisfying

$\mathbb{E}\left[\sup_{0\leq t\leq T}|X(t)|^{2}\right]<\infty.$ (1.3)

The classical linear-quadratic (LQ) optimal control problem is to find a
control $u^{*}\in{\cal U}$ that minimizes the quadratic cost functional

$\displaystyle J(x;u)$ $\displaystyle=\mathbb{E}\Big{\\{}\langle
GX(T),X(T)\rangle+2\langle g,X(T)\rangle$ $\displaystyle\hphantom{=\
}+\int_{0}^{T}\bigg{[}\Big{\langle}\\!\\!\begin{pmatrix}[l]Q(t)&\\!S(t)^{\top}\\\
S(t)&\\!R(t)\end{pmatrix}\\!\begin{pmatrix}X(t)\\\
u(t)\end{pmatrix}\\!,\begin{pmatrix}X(t)\\\
u(t)\end{pmatrix}\\!\\!\Big{\rangle}$ $\displaystyle\hphantom{=\
}+2\Big{\langle}\\!\\!\begin{pmatrix}q(t)\\\
r(t)\end{pmatrix}\\!,\begin{pmatrix}X(t)\\\
u(t)\end{pmatrix}\\!\\!\Big{\rangle}\bigg{]}dt\bigg{\\}}$ (1.4)

over ${\cal U}$, subject to the state equation (1.1). In (1),
$\langle\cdot\,,\cdot\rangle$ is the Frobenius inner product; $G$ is a
symmetric $n\times n$ constant matrix; $g\in\mathbb{R}^{n}$ is a constant
vector; $Q,S,R$ are bounded deterministic matrix-valued functions of proper
dimensions over $[0,T]$ such that the blocked matrix in the Lebesgue integral
is symmetric; $q,r$ are bounded deterministic functions over $[0,T]$, with
values in $\mathbb{R}^{n}$ and $\mathbb{R}^{m}$, respectively.

The above LQ problem can be elegantly solved by a Riccati equation approach,
whose optimal control turns out to be a linear feedback of the current state
(see, for example, [15, 8]). Thus, the optimal control could be constructed if
the controller has access to the exact value of the state. However, in many
practical situations it often happens that some components of the state may be
inaccessible for observation, and there could be noise existing in the
observation systems. In this case only partial information is available to the
controller, and the control has to be selected according to the information
provided by the observation systems.

In this paper we consider the case that the observation process evolves
according to the following SDE:

$\left\\{\begin{aligned} dY(t)&=[H(t)X(t)+h(t)]dt+K(t)dW(t),\quad 0\leq t\leq
T,\\\ Y(0)&=0,\end{aligned}\right.$ (1.5)

where $H$, $h$, and $K$ are deterministic, bounded functions on $[0,T]$, with
values in $\mathbb{R}^{d\times n}$, $\mathbb{R}^{d}$, and $\mathbb{R}^{d\times
d}$, respectively. For each control $u\in{\cal U}$, the observation process
$Y=\\{Y(t);\,0\leq t\leq T\\}$ is a square-integrable
$\\{\mathscr{F}_{t}\\}$-semimartingale. Let
$\mathscr{Y}^{u}=\\{\mathscr{Y}^{u}_{t}\\}_{0\leq t\leq T}$ be the usual
augmentation of the filtration generated by $Y$. Clearly, the filtration
$\mathscr{Y}^{u}$ depends on the choice of $u$. We say that a control
$u\in{\cal U}$ is admissible if it is progressively measurable with respect to
$\mathscr{Y}^{u}$; that is, the set of admissible controls is

$\displaystyle\textstyle{\cal
U}_{ad}=\Big{\\{}u:[0,T]\times\varOmega\to\mathbb{R}^{m}\bigm{|}u\in\mathscr{Y}^{u}~{}\hbox{and}~{}\mathbb{E}\int_{0}^{T}|u(t)|^{2}dt<\infty\Big{\\}}.$
(1.6)

At each time $t$, the controller first observes $Y(t)$, then applies $u(t)$ to
the state system (1.1) immediately. The objective of our control problem is to
choose an admissible control such that the cost functional (1) is minimized.

###### Problem (O).

For a given initial state $x\in\mathbb{R}^{n}$, find a control
$v\in\mathscr{Y}^{v}$ such that

$\displaystyle J(x;v)=\inf_{u\in\mathscr{Y}^{u}}J(x;u)\equiv V(x).$

The process $v$, if exists, is called an optimal control for the initial state
$x$, and the function $V$ is called the value function of Problem (O).

In contrast to the completely observable case, the essential difficulty is
that the filtration $\mathscr{Y}^{u}$ is not fixed (depending on the control
$u$) and the linear structure of the admissible control set is thereby
corrupted. For this reason it is hard to derive the optimality system by the
variational method. In fact, for a given control $u\in{\cal U}$, it is not
even easy to decide whether it is admissible or not. To overcome this
difficulty, one way is to apply the separation principle to decouple the
problems of optimal control and state estimation. This method is based on an
additional requirement on the admissible control, i.e., $u\in{\cal U}$ is said
to be admissible if, in addition to $u\in\mathscr{Y}^{u}$, $u$ is adapted to
the smaller filtration $\mathscr{Y}^{0}$; see, for example, Wonham [13] and
Bensoussan [1]. Along this line, many research on LQ optimal control of
partially observable systems were carried out in recent years, among which we
would like to mention the works [2, 5, 9] on backward stochastic control
systems under partial information, the works [6, 7, 11] on optimal control
problems of forward-backward stochastic differential equations (FBSDEs) with
partial information, the works [12, 14] on differential games with partially
observable systems. Another way of analyzing the problem is to convert it into
a completely observable stochastic optimal control problem by Girsanov’s
transformation (see, for example, Wang, Wu, and Xiong [10], Yong and Zhou [15,
Chapter 2], or the references therein). However, this approach turns the
problem into an infinite-dimensional one that is also difficult to solve and
requires higher integrability of the state process which cannot be fulfilled
in the LQ case in general.

In this paper we introduce a new method of solving Problem (O) without
imposing additional requirements on the admissible control. The idea is to
first derive the filtering equation for a fixed admissible control and then
use the orthogonal decomposition of the state process to write the cost
functional as the sum of two independent parts. One part is a functional of
the admissible control and the filtering process, and the other part is a
functional of the estimate error that is independent of the choice of the
admissible control. The first part possesses a mathematical structure similar
to the full information problem, which we solve by completing the square. The
second part is simplified using integration by parts. Finally, we show by
verification that the optimal control is given by a feedback representation
via the filtering process. We also obtain the optimal value explicitly.

The rest of the paper is organized as follows. In Section 2 we collect some
notation and preliminary results that we need later. Then we present the main
results of the paper in Section 3. The subsequent two sections are devoted to
the proofs of the main results. We derive the filtering equation for a fixed
admissible control in Section 4 and construct the optimal control in Section
5. Finally, we give some concluding remarks in Section 6.

##  2 Preliminaries

We begin by introducing some notation. All vectors in the paper are column
vectors, and the transpose is denoted by the superscript $\top$. The Euclidean
space $\mathbb{R}^{n\times m}$ of $n\times m$ real matrices is equipped with
the Frobenius inner product

$\langle M,N\rangle=\hbox{tr$\,$}(M^{\top}N),\quad M,N\in\mathbb{R}^{n\times
m},$

where $\hbox{tr$\,$}(M^{\top}N)$ is the trace of $M^{\top}N$. The norm induced
by the Frobenius inner product is denoted by $|\cdot|$. The identity matrix of
size $n$ is denoted by $I_{n}$. For a subset $\mathbb{H}$ of
$\mathbb{R}^{n\times m}$, we denote by $C([0,T];\mathbb{H})$ the space of
continuous functions from $[0,T]$ into $\mathbb{H}$, and by
$L^{\infty}(0,T;\mathbb{H})$ the space of Lebesgue measurable, essentially
bounded functions from $[0,T]$ into $\mathbb{H}$. Let $\mathbb{S}^{n}$ be the
subspace of $\mathbb{R}^{n\times n}$ consisting of symmetric matrices. For
$\mathbb{S}^{n}$-valued functions $M$ and $N$, we write $M\geq N$
(respectively, $M>N$) if $M-N$ is positive semidefinite (respectively,
positive definite) almost everywhere with respect to the Lebesgue measure.

Throughout this paper we impose the following assumptions.

###### (A1)

The deterministic functions

$\displaystyle A:[0,T]\to\mathbb{R}^{n\times n},\quad
B:[0,T]\to\mathbb{R}^{n\times m},\quad a:[0,T]\to\mathbb{R}^{n},\quad
C:[0,T]\to\mathbb{R}^{n\times d},$ $\displaystyle
D:[0,T]\to\mathbb{R}^{n\times k},\quad H:[0,T]\to\mathbb{R}^{d\times n},\quad
h:[0,T]\to\mathbb{R}^{d},\quad K:[0,T]\to\mathbb{R}^{d\times d}$

are Lebesgue measurable and bounded on $[0,T]$.

###### (A2)

$K(t)$ is invertible for a.e. $t\in[0,T]$, and $K^{-1}$ is bounded on $[0,T]$.

###### (A3)

$G\in\mathbb{S}^{n}$, $g\in\mathbb{R}^{n}$, and the deterministic functions

$Q:[0,T]\to\mathbb{S}^{n},~{}S:[0,T]\to\mathbb{R}^{m\times
n},~{}R:[0,T]\to\mathbb{S}^{m},~{}q:[0,T]\to\mathbb{R}^{n},~{}r:[0,T]\to\mathbb{R}^{m}$

are Lebesgue measurable and bounded on $[0,T]$, satisfying

$G\geq 0,\quad R(t)\geq\delta I_{m},\quad Q(t)-S(t)^{\top}R(t)^{-1}S(t)\geq
0,\quad\text{a.s.}~{}t\in[0,T],$

where $\delta>0$ is some constant.

The following result is standard in the LQ theory. For a quick proof, see Yong
and Zhou [15, pp. 297–298].

######  Lemma 2.1.

Let (A1) and (A3) hold. Then the Riccati equation

$\left\\{\begin{aligned} &\dot{P}(t)+P(t)A(t)+A(t)^{\top}P(t)+Q(t)\\\
&\hphantom{\dot{P}(t)}-[P(t)B(t)+S(t)^{\top}]R(t)^{-1}[B(t)^{\top}P(t)+S(t)]=0,\\\
&P(T)=G\end{aligned}\right.$ (2.1)

admits a unique solution $P\in C([0,T];\mathbb{S}^{n})$, which is positive
semidefinite everywhere on $[0,T]$.

For a process $Z=\\{Z(t);\,0\leq t\leq T\\}$, let

$\displaystyle\mathscr{Z}_{t}^{\circ}\triangleq\sigma(Z(s);\,0\leq s\leq
t),\quad 0\leq t\leq T,$
$\displaystyle\mathscr{N}^{{}_{Z}}\triangleq\\{N\subseteq\varOmega;\,\exists\,G\in\mathscr{Z}_{T}^{\circ}\text{
with }N\subseteq G\text{ and }\mathbb{P}(G)=0\\},$

and define

$\mathscr{Z}_{T}\triangleq\sigma\big{(}\mathscr{Z}_{T}^{\circ},\mathscr{N}^{{}_{Z}}\big{)},\quad\mathscr{Z}_{t}\triangleq\bigcap_{s>t}\sigma\big{(}\mathscr{Z}_{s}^{\circ},\mathscr{N}^{{}_{Z}}\big{)}=\sigma\big{(}\mathscr{Z}_{t+}^{\circ},\mathscr{N}^{{}_{Z}}\big{)},\quad
0\leq t<T.$

The filtered space
$(\varOmega,\\{\mathscr{Z}_{t}\\},\mathscr{Z}_{T},\mathbb{P})$ satisfies the
usual conditions. We call $\\{\mathscr{Z}_{t}\\}$ the usual augmentation of
$\\{\mathscr{Z}_{t}^{\circ}\\}$, the filtration generated by $Z$.

Let $\mathbb{G}=\\{\mathscr{G}_{t}\\}_{t\geq 0}$ be a sub-filtration of
$\mathbb{F}=\\{\mathscr{F}_{t}\\}$, which also satisfies the usual conditions.
Consider an integrable process $Z=\\{Z(t);\,0\leq t\leq T\\}$. The conditional
expectation

$\hat{Z}(t)=\mathbb{E}[Z(t)\,|\,\mathscr{G}_{t}]$

defines $\hat{Z}(t)$ up to a zero probability set but does not give us the
paths of $\hat{Z}$, which requires specifying its values simultaneously at the
uncountable set of times in $[0,T]$. We need a good version for $\hat{Z}$ that
is at least progressively measurable with respect to $\\{\mathscr{G}_{t}\\}$.
Such a version is referred to as the $\\{\mathscr{G}_{t}\\}$-optional
projection of $Z$, which exists under various conditions. For our purpose, we
shall focus on the case that $Z=\\{Z(t);\,0\leq t\leq T\\}$ is a continuous,
square-integrable $\\{\mathscr{F}_{t}\\}$-semimartingale. We have the
following result.

######  Proposition 2.2.

Let $Z=\\{Z(t);\,0\leq t\leq T\\}$ be a continuous, square-integrable
$\\{\mathscr{F}_{t}\\}$-semimartingale. Then there exists a
$\\{\mathscr{G}_{t}\\}$-adapted process $\hat{Z}=\\{\hat{Z}(t);\,0\leq t\leq
T\\}$ whose sample paths are right-continuous with finite left-hand limits
(RCLL), such that

$\hat{Z}(t)=\mathbb{E}[Z(t)\,|\,\mathscr{G}_{t}],\quad\text{a.s.}~{}\forall\,0\leq
t\leq T.$

###### Proof.

Since $Z$ is a continuous, square-integrable
$\\{\mathscr{F}_{t}\\}$-semimartingale, it admits the decomposition

$Z(t)=\theta_{1}(t)-\theta_{2}(t)+\eta(t),$

where $\theta_{i}=\\{\theta_{i}(t);\,0\leq t\leq T\\}$, $i=1,2$, are
continuous, $\\{\mathscr{F}_{t}\\}$-adapted increasing processes, and
$\eta=\\{\eta(t);\,0\leq t\leq T\\}$ is a continuous
$\\{\mathscr{F}_{t}\\}$-martingale. Let

$\hat{\theta}_{i}(t)=\mathbb{E}[\theta_{i}(t)\,|\,\mathscr{G}_{t}],\quad\hat{\eta}(t)=\mathbb{E}[\eta(t)\,|\,\mathscr{G}_{t}].$

For $0\leq s<t\leq T$,

$\displaystyle\mathbb{E}[\hat{\theta}_{i}(t)\,|\,\mathscr{G}_{s}]=\mathbb{E}[\theta_{i}(t)\,|\,\mathscr{G}_{s}]\geq\mathbb{E}[\theta_{i}(s)\,|\,\mathscr{G}_{s}]=\hat{\theta}_{i}(s),$
$\displaystyle\mathbb{E}[\hat{\eta}(t)\,|\,\mathscr{G}_{s}]=\mathbb{E}[\eta(t)\,|\,\mathscr{G}_{s}]=\mathbb{E}\big{\\{}\mathbb{E}[\eta(t)\,|\,\mathscr{F}_{s}]\,|\,\mathscr{G}_{s}\big{\\}}=\mathbb{E}[\eta(s)\,|\,\mathscr{G}_{s}]=\hat{\eta}(s).$

Thus, both $\\{\hat{\theta}_{i}(t);\,0\leq t\leq T\\}$ and
$\\{\hat{\eta}(t);\,0\leq t\leq T\\}$ are
$\\{\mathscr{G}_{t}\\}$-submartingales. Moreover, the functions

$t\mapsto\mathbb{E}[\hat{\theta}_{i}(t)]=\mathbb{E}[\theta_{i}(t)],\quad
t\mapsto\mathbb{E}[\hat{\eta}(t)]=\mathbb{E}[\eta(t)]$

are continuous. Because the filtration $\\{\mathscr{G}_{t}\\}$ satisfies the
usual conditions, the processes $\hat{\theta}_{i}$ ($i=1,2$) and $\hat{\eta}$
have $\\{\mathscr{G}_{t}\\}$-adapted, RCLL modifications (see Karatzas and
Shreve [3], Theorem 1.3.13), and therefore so is
$\mathbb{E}[Z(t)\,|\,\mathscr{G}_{t}]$. ∎

######  Notation 2.3.

Hereafter, we shall use the notation $\hat{Z}=\\{\hat{Z}(t);\,0\leq t\leq
T\\}$ for the optional projection of a process $Z=\\{Z(t);\,0\leq t\leq T\\}$,
and use $\tilde{Z}=\\{\tilde{Z}(t);\,0\leq t\leq T\\}$ for the difference
$Z-\hat{Z}$.

When $Z$ is square-integrable, for each $t\in[0,T]$,
$\mathbb{E}[Z(t)\,|\,\mathscr{G}_{t}]$ is the orthogonal projection of $Z(t)$
onto $L^{2}(\varOmega,\mathscr{G}_{t})$, the Hilbert space of
$\mathscr{G}_{t}$-measurable, square-integrable random variables. In this
case, $\tilde{Z}(t)=Z(t)-\hat{Z}(t)$ is independent of $\mathscr{G}_{t}$, and

$\mathbb{E}|Z(t)|^{2}=\mathbb{E}|\hat{Z}(t)|^{2}+\mathbb{E}|\tilde{Z}(t)|^{2}.$

##  3 Main results

In this section, we present the main results of the paper and briefly
illustrate the idea. The rigorous proofs are deferred to the subsequent
sections.

As mentioned previously, the main difficulty of solving Problem (O) is that
the filtration generated by the observation process depends on the control, in
which case the traditional variation method fails. To deal with this
difficulty, we shall first derive, for a fixed admissible control $u\in{\cal
U}_{ad}$, the filtering equation for

$\hat{X}(t)=\mathbb{E}[X(t)\,|\,\mathscr{Y}_{t}^{u}],$

where $\\{\mathscr{Y}_{t}^{u}\\}$ is the usual augmentation of the filtration
generated by the observation process $Y$ (depending on $u$). The following
ordinary differential equation (ODE) will be involved:

$\left\\{\begin{aligned}
\dot{\varSigma}(t)&=[A(t)-C(t)K(t)^{-1}H(t)]\varSigma(t)+\varSigma(t)[A(t)-C(t)K(t)^{-1}H(t)]^{\top}\\\
&\hphantom{=\ }-\varSigma(t)H(t)^{\top}N(t)^{-1}H(t)\varSigma(t)+M(t),\quad
t\in[0,T],\\\ \varSigma(0)&=0,\end{aligned}\right.$ (3.1)

where $x\in\mathbb{R}^{n}$ is the initial state of the state equation (1.1),
and

$M(t)\triangleq D(t)D(t)^{\top},\quad N(t)\triangleq K(t)K(t)^{\top},\quad
t\in[0,T].$ (3.2)

By reversing time,

$\tau=T-t,\quad t\in[0,T],$

one sees that (3.1) can be converted into a Riccati equation of the form (2.1)
and hence admits a unique positive semidefinite solution $\varSigma\in
C([0,T];\mathbb{S}^{n})$ under the assumptions (A1) and (A2).

######  Theorem 3.1.

Let (A1)–(A2) hold. For a fixed control $u\in{\cal U}$, let $X$ be the
corresponding state process with initial state $x$ and $Y$ be the observation
process. If $u$ is admissible, then $\hat{X}$ evolves according to the
following SDE:

$\left\\{\begin{aligned}
d\hat{X}(t)&=[A(t)\hat{X}(t)+B(t)u(t)+a(t)]dt+[\varSigma(t)H(t)^{\top}\\!+C(t)K(t)^{\top}]N(t)^{-1}dV(t),\\\
\hat{X}(0)&=x,\end{aligned}\right.$ (3.3)

where $\varSigma$ is the solution of (3.1), and $V=\\{V(t);\,0\leq t\leq T\\}$
is the innovation process defined by

$V(t)\triangleq Y(t)-\int_{0}^{t}\big{[}H(s)\hat{X}(s)+h(s)\big{]}ds,\quad
0\leq t\leq T.$ (3.4)

######  Corollary 3.2.

Let (A1)–(A2) hold. If $u\in{\cal U}_{ad}$, then the difference
$\tilde{X}\triangleq X-\hat{X}$ satisfies the following SDE:

$\left\\{\begin{aligned} d\tilde{X}(t)&=[A-(\varSigma
H^{\top}\\!+CK^{\top})N^{-1}H]\tilde{X}dt-\varSigma(K^{-1}H)^{\top}dW+DdW^{\prime},\\\
\tilde{X}(0)&=0.\end{aligned}\right.$ (3.5)

Further, $\varSigma(t)=\mathbb{E}[\tilde{X}(t)\tilde{X}(t)^{\top}]$.

We observe from 3.2 that the process $\tilde{X}$ is independent of the control
$u$ and the initial state $x$. Moreover, by the discussion at the end of
Section 2, $\tilde{X}$ is orthogonal to $\hat{X}$, i.e.,

$\mathbb{E}\langle\tilde{X}(t),\hat{X}(t)\rangle=0,\quad\mathbb{E}|X(t)|^{2}=\mathbb{E}|\hat{X}(t)|^{2}+\mathbb{E}|\tilde{X}(t)|^{2},\quad
t\in[0,T].$

This suggests the following decomposition of the cost functional (1):

$J(x;u)=\hat{J}(x;u)+\tilde{J},$

where $\hat{J}(x;u)$ and $\tilde{J}$ are given by

$\displaystyle\hat{J}(x;u)$ $\displaystyle=\mathbb{E}\Big{\\{}\langle
G\hat{X}(T),\hat{X}(T)\rangle+2\langle g,\hat{X}(T)\rangle$
$\displaystyle\hphantom{=\
}+\int_{0}^{T}\bigg{[}\Big{\langle}\\!\\!\begin{pmatrix}[l]Q(t)&\\!S(t)^{\top}\\\
S(t)&\\!R(t)\end{pmatrix}\\!\begin{pmatrix}\hat{X}(t)\\\
u(t)\end{pmatrix}\\!,\begin{pmatrix}\hat{X}(t)\\\
u(t)\end{pmatrix}\\!\\!\Big{\rangle}$ $\displaystyle\hphantom{=\
}+2\Big{\langle}\\!\\!\begin{pmatrix}q(t)\\\
r(t)\end{pmatrix}\\!,\begin{pmatrix}\hat{X}(t)\\\
u(t)\end{pmatrix}\\!\\!\Big{\rangle}\bigg{]}dt\bigg{\\}},$ (3.6)

and

$\displaystyle\tilde{J}$ $\displaystyle=\mathbb{E}\bigg{\\{}\langle
G\tilde{X}(T),\tilde{X}(T)\rangle+2\langle g,\tilde{X}(T)\rangle$
$\displaystyle\hphantom{=\ }+\int_{0}^{T}\Big{[}\langle
Q(t)\tilde{X}(t),\tilde{X}(t)\rangle+2\langle
q(t),\tilde{X}(t)\rangle\Big{]}dt\bigg{\\}},$ (3.7)

respectively. Note that $\tilde{J}$ does not depend on the control $u$.

To deal with $\hat{J}(x;u)$, we recall from 2.1 that under our assumptions
(A1)–(A3), the Riccati equation (2.1) admits a unique positive semidefinite
solution $P\in C([0,T];\mathbb{S}^{n})$. Let

$\Theta(t)\triangleq-R(t)^{-1}[B(t)^{\top}P(t)+S(t)],\quad t\in[0,T].$ (3.8)

The following ODE obviously has a unique solution $\varphi$:

$\left\\{\begin{aligned}
&\dot{\varphi}(t)+[A(t)+B(t)\Theta(t)]^{\top}\varphi(t)+\Theta(t)^{\top}r(t)+P(t)a(t)+q(t)=0,\quad
t\in[0,T],\\\ &\varphi(T)=g.\end{aligned}\right.$ (3.9)

Further, we write

$\left\\{\begin{aligned} C(t)&=(C_{1}(t),\cdots,C_{d}(t)),\\\
D(t)&=(D_{1}(t),\cdots,D_{k}(t)),\\\
\varDelta(t)&=(\varDelta_{1}(t),\cdots,\varDelta_{d}(t))\triangleq\varSigma(t)[K(t)^{-1}H(t)]^{\top}.\end{aligned}\right.$
(3.10)

######  Proposition 3.3.

Let (A1)–(A3) hold. For each $u\in{\cal U}_{ad}$,

$\displaystyle\hat{J}(x;u)$ $\displaystyle=\langle
P(0)x,x\rangle+2\langle\varphi(0),x\rangle$ $\displaystyle\hphantom{=\
}+\mathbb{E}\int_{0}^{T}\bigg{\\{}\langle
R[\Theta\hat{X}-R^{-1}(B^{\top}\varphi+r)-u],[\Theta\hat{X}-R^{-1}(B^{\top}\varphi+r)-u]\rangle$
$\displaystyle\hphantom{=\ }-\langle
R^{-1}(B^{\top}\varphi+r),B^{\top}\varphi+r\rangle+2\langle\varphi,a\rangle+\sum_{i=1}^{d}\langle
P(\varDelta_{i}+C_{i}),\varDelta_{i}+C_{i}\rangle\bigg{\\}}dt.$ (3.11)

To deal with $\tilde{J}$, we let

${\cal A}(t)\triangleq
A(t)-[\varSigma(t)H(t)^{\top}\\!+C(t)K(t)^{\top}]N(t)^{-1}H(t),\quad
t\in[0,T],$ (3.12)

and further introduce the linear ODEs

$\left\\{\begin{aligned} &\dot{\varPi}(t)+\varPi(t){\cal A}(t)+{\cal
A}(t)^{\top}\varPi(t)+Q(t)=0,\\\ &\varPi(T)=G\end{aligned}\right.$ (3.13)

and

$\left\\{\begin{aligned} &\dot{\pi}(t)+{\cal A}(t)^{\top}\pi(t)+q(t)=0,\\\
&\pi(T)=g.\end{aligned}\right.$ (3.14)

######  Proposition 3.4.

Let (A1)–(A3) hold. Then

$\displaystyle\tilde{J}$
$\displaystyle=\int_{0}^{T}\Big{[}\sum_{i=1}^{k}\langle\varPi(t)D_{i}(t),D_{i}(t)\rangle+\sum_{i=1}^{d}\langle\varPi(t)\varDelta_{i}(t),\varDelta_{i}(t)\rangle\Big{]}dt.$
(3.15)

Finally, combining the previous results, we can obtain the following theorem,
which gives the optimal control and the value function of Problem (O).

######  Theorem 3.5.

Let (A1)–(A3) hold. For each initial state $x\in\mathbb{R}^{n}$, Problem (O)
admits a unique optimal control, given by the following observation-feedback
form:

$\displaystyle u^{*}(t)$
$\displaystyle=\Theta(t)\hat{X}(t)-R(t)^{-1}[B(t)^{\top}\varphi(t)+r(t)].$
(3.16)

The optional projection $\hat{X}$ evolves according to the following SDE:

$\left\\{\begin{aligned}
d\hat{X}(t)&=\big{\\{}[A(t)+B(t)\Theta(t)]\hat{X}(t)-B(t)R(t)^{-1}[B(t)^{\top}\varphi(t)+r(t)]+a(t)\big{\\}}dt\\\
&\hphantom{=\
}+[\varSigma(t)H(t)^{\top}\\!+C(t)K(t)^{\top}]N(t)^{-1}\big{\\{}dY(t)-[H(t)\hat{X}(t)+h(t)]dt\big{\\}},\\\
\hat{X}(0)&=x.\end{aligned}\right.$ (3.17)

Moreover, the optimal value is given by

$\displaystyle J(x;u^{*})$ $\displaystyle=\langle
P(0)x,x\rangle+2\langle\varphi(0),x\rangle+\int_{0}^{T}\bigg{\\{}\sum_{i=1}^{k}\langle\varPi(t)D_{i}(t),D_{i}(t)\rangle$
$\displaystyle\hphantom{=\
}+\sum_{i=1}^{d}\langle\varPi(t)\varDelta_{i}(t),\varDelta_{i}(t)\rangle+\sum_{i=1}^{d}\langle
P(t)[\varDelta_{i}(t)+C_{i}(t)],\varDelta_{i}(t)+C_{i}(t)\rangle$
$\displaystyle\hphantom{=\ }-\langle
R(t)^{-1}[B(t)^{\top}\varphi(t)+r(t)],B(t)^{\top}\varphi(t)+r(t)\rangle+2\langle\varphi(t),a(t)\rangle\bigg{\\}}dt.$
(3.18)

##  4 The filtering equation

Suppose that $u\in{\cal U}$ is an admissible control. Let $X$ be the
corresponding state process with initial state $x$ and Y the observation
process. In this section we establish the filtering equation for

$\hat{X}(t)=\mathbb{E}[X(t)\,|\,\mathscr{Y}_{t}^{u}],$

as well as 3.2. For natational simplicity, we let

$\tilde{X}(t)\triangleq
X(t)-\hat{X}(t),\quad\varSigma(t)\triangleq\mathbb{E}[\tilde{X}(t)\tilde{X}(t)^{\top}].$

It will be shown later that the function $\varSigma$ defined above is the
solution of the ODE (3.1).

Let $V=\\{V(t);\,0\leq t\leq T\\}$ be the innovation process defined by (3.4):

$V(t)\triangleq Y(t)-\int_{0}^{t}\big{[}H(s)\hat{X}(s)+h(s)\big{]}ds.$

First, we have the following result.

######  Lemma 4.1.

Let (A1)–(A2) hold. For a fixed admissible control $u\in{\cal U}_{ad}$, the
process $\check{V}=\\{\check{V}(t);\,0\leq t\leq T\\}$ defined by

$\check{V}(t)\triangleq\int_{0}^{t}K(s)^{-1}dV(s)$

is a standard $\\{\mathscr{Y}_{t}^{u}\\}$-Brownian motion in $\mathbb{R}^{d}$.

###### Proof.

Clearly, $\check{V}$ is a continuous, $\\{\mathscr{Y}_{t}^{u}\\}$-adapted,
integrable process, and

$\displaystyle d\check{V}(t)$
$\displaystyle=K(t)^{-1}dV(t)=K(t)^{-1}H(t)[X(t)-\hat{X}(t)]dt+dW(t)$
$\displaystyle=K(t)^{-1}H(t)\tilde{X}(t)dt+dW(t).$

Thus, for $0\leq s<t\leq T$,

$\displaystyle\check{V}(t)-\check{V}(s)$
$\displaystyle=W(t)-W(s)+\int_{s}^{t}K(\tau)^{-1}H(\tau)\tilde{X}(\tau)d\tau.$

Since for $\tau\geq s$, $\tilde{X}(\tau)$ is independent of
$\mathscr{Y}_{s}^{u}$,

$\displaystyle\mathbb{E}[\check{V}(t)-\check{V}(s)\,|\,\mathscr{Y}_{s}^{u}]$
$\displaystyle=\mathbb{E}\Big{[}\mathbb{E}[W(t)-W(s)\,|\,\mathscr{F}_{s}]\,|\,\mathscr{Y}_{s}^{u}\Big{]}+\mathbb{E}\left[\int_{s}^{t}K(\tau)^{-1}H(\tau)\tilde{X}(\tau)d\tau\,\Big{|}\,\mathscr{Y}_{s}^{u}\right]$
$\displaystyle=\int_{s}^{t}K(\tau)^{-1}H(\tau)\mathbb{E}[\tilde{X}(\tau)]d\tau=0.$

This shows that $\check{V}$ is a $\\{\mathscr{Y}_{t}^{u}\\}$-martingale.
Further,

$\displaystyle d[\check{V}(t)\check{V}(t)^{\top}]$
$\displaystyle=K(t)^{-1}H(t)\tilde{X}(t)\check{V}(t)^{\top}dt+[dW(t)]\check{V}(t)^{\top}$
$\displaystyle\hphantom{=\
}+\check{V}(t)[K(t)^{-1}H(t)\tilde{X}(t)]^{\top}dt+\check{V}(t)dW(t)^{\top}+I_{d}\,dt,$

and hence

$\displaystyle\check{V}(t)\check{V}(t)^{\top}-\check{V}(s)\check{V}(s)^{\top}=\int_{s}^{t}K(\tau)^{-1}H(\tau)\tilde{X}(\tau)\check{V}(\tau)^{\top}d\tau+\left[\int_{s}^{t}\check{V}(\tau)dW(\tau)^{\top}\right]^{\top}$
$\displaystyle\qquad\quad+\left[\int_{s}^{t}K(\tau)^{-1}H(\tau)\tilde{X}(\tau)\check{V}(\tau)^{\top}d\tau\right]^{\top}+\int_{s}^{t}\check{V}(\tau)dW(\tau)^{\top}+(t-s)I_{d}.$
(4.1)

Again, since $\tilde{X}(\tau)$ is independent of $\mathscr{Y}_{s}^{u}$ for
$\tau\geq s$, we have

$\displaystyle\mathbb{E}\left[\int_{s}^{t}K(\tau)^{-1}H(\tau)\tilde{X}(\tau)\check{V}(\tau)^{\top}d\tau\,\Big{|}\,\mathscr{Y}_{s}^{u}\right]$
$\displaystyle\quad=\int_{s}^{t}K(\tau)^{-1}H(\tau)\mathbb{E}\Big{[}\tilde{X}(\tau)\check{V}(\tau)^{\top}\,\big{|}\,\mathscr{Y}_{s}^{u}\Big{]}d\tau$
$\displaystyle\quad=\int_{s}^{t}K(\tau)^{-1}H(\tau)\mathbb{E}\Big{\\{}\mathbb{E}\big{[}\tilde{X}(\tau)\check{V}(\tau)^{\top}\,\big{|}\,\mathscr{Y}_{\tau}^{u}\big{]}\,\big{|}\,\mathscr{Y}_{s}^{u}\Big{\\}}d\tau$
$\displaystyle\quad=\int_{s}^{t}K(\tau)^{-1}H(\tau)\mathbb{E}\Big{\\{}\mathbb{E}[\tilde{X}(\tau)]\check{V}(\tau)^{\top}\,\big{|}\,\mathscr{Y}_{s}^{u}\Big{\\}}d\tau$
$\displaystyle\quad=0.$

For the stochastic integral in (4), we have

$\mathbb{E}\left[\int_{s}^{t}\check{V}(\tau)dW(\tau)^{\top}\,\Big{|}\,\mathscr{Y}_{s}^{u}\right]=\mathbb{E}\left\\{\mathbb{E}\left[\int_{s}^{t}\check{V}(\tau)dW(\tau)^{\top}\,\Big{|}\,\mathscr{F}_{s}\right]\bigg{|}\mathscr{Y}_{s}^{u}\right\\}=0.$

So taking conditional expectations with respect to $\mathscr{Y}_{s}^{u}$ on
both sides of (4) yields

$\mathbb{E}\Big{[}\check{V}(t)\check{V}(t)^{\top}-\check{V}(s)\check{V}(s)^{\top}\,\big{|}\,\mathscr{Y}_{s}^{u}\Big{]}=(t-s)I_{d},$

which implies that the cross-variations are given by

$\langle\check{V}_{i},\check{V}_{j}\rangle_{t}=\delta_{ij}t;\quad 1\leq
i,j\leq d.$

It then follows from the martingale characterization of Brownian motion that
$\check{V}$ is a standard $d$-dimensional Brownian motion. ∎

Now we consider the process $\varLambda=\\{\varLambda(t);\,0\leq t\leq T\\}$
defined by

$\varLambda(t)\triangleq\hat{X}(t)-x-\int_{0}^{t}\big{[}A(s)\hat{X}(s)+B(s)u(s)+a(s)\big{]}ds.$
(4.2)

Clearly, it is $\\{\mathscr{Y}_{t}^{u}\\}$-adapted and RCLL, with
$\varLambda(0)=0$ almost surely. Further, by (1.3) it is also square-
integrable. The following result shows that $\varLambda$ is actually a
$\\{\mathscr{Y}_{t}^{u}\\}$-martingale.

######  Lemma 4.2.

The process $\varLambda=\\{\varLambda(t);\,0\leq t\leq T\\}$ defined by (4.2)
is an RCLL, square-integrable $\\{\mathscr{Y}_{t}^{u}\\}$-martingale with
$\varLambda(0)=0$ a.s..

###### Proof.

For fixed $0\leq s<t\leq T$, we have

$\displaystyle\mathbb{E}\Big{[}\varLambda(t)-\varLambda(s)\,|\,\mathscr{Y}_{s}^{u}\Big{]}$
$\displaystyle\quad=\mathbb{E}\Big{[}\hat{X}(t)-\hat{X}(s)\,|\,\mathscr{Y}_{s}^{u}\Big{]}-\mathbb{E}\left\\{\int_{s}^{t}\big{[}A(\tau)\hat{X}(\tau)+B(\tau)u(\tau)+a(\tau)\big{]}d\tau\,\Big{|}\,\mathscr{Y}_{s}^{u}\right\\}$
$\displaystyle\quad=\mathbb{E}\Big{[}X(t)-X(s)\,|\,\mathscr{Y}_{s}^{u}\Big{]}-\int_{s}^{t}\mathbb{E}\Big{[}A(\tau)\hat{X}(\tau)+B(\tau)u(\tau)+a(\tau)\,\big{|}\,\mathscr{Y}_{s}^{u}\Big{]}d\tau$
$\displaystyle\quad=\mathbb{E}\Big{[}X(t)-X(s)\,|\,\mathscr{Y}_{s}^{u}\Big{]}-\int_{s}^{t}\mathbb{E}\Big{[}A(\tau)X(\tau)+B(\tau)u(\tau)+a(\tau)\,\big{|}\,\mathscr{Y}_{s}^{u}\Big{]}d\tau$
$\displaystyle\quad=\mathbb{E}\Big{[}X(t)-X(s)\,|\,\mathscr{Y}_{s}^{u}\Big{]}-\mathbb{E}\left\\{\int_{s}^{t}\big{[}A(\tau)X(\tau)+B(\tau)u(\tau)+a(\tau)\big{]}d\tau\,\Big{|}\,\mathscr{Y}_{s}^{u}\right\\}$
$\displaystyle\quad=\mathbb{E}\left[\int_{s}^{t}C(\tau)dW(\tau)+\int_{s}^{t}D(\tau)dW^{\prime}(\tau)\,\Big{|}\,\mathscr{Y}_{s}^{u}\right]$
$\displaystyle\quad=\mathbb{E}\left\\{\mathbb{E}\left[\int_{s}^{t}C(\tau)dW(\tau)+\int_{s}^{t}D(\tau)dW^{\prime}(\tau)\,\Big{|}\,\mathscr{F}_{s}\right]\bigg{|}\mathscr{Y}_{s}^{u}\right\\}$
$\displaystyle\quad=0.$

This shows that $\varLambda$ is a $\\{\mathscr{Y}_{t}^{u}\\}$-martingale. ∎

It is not clear whether the process $\varLambda$ is adapted to the smaller
filtration generated by the Brownian motion $\check{V}$. So we cannot conclude
directly by the martingale representation theorem that $L$ can be expressed as
a stochastic integral with respect to $\check{V}$. Fortunately, thanks to the
theorem of Fujisaki, Kallianpur and Kunita (see Rogers and Williams [4],
VI.8), there exists an $\mathbb{R}^{n\times d}$-valued, square-integrable,
$\\{\mathscr{Y}_{t}^{u}\\}$-progressively measurable process
$\lambda=\\{\lambda(t);\,0\leq t\leq T\\}$ such that

$\varLambda(t)=\int_{0}^{t}\lambda(s)N(s)^{-1}dV(s),\quad 0\leq t\leq T.$
(4.3)

Next we determine the process $\lambda$.

######  Proposition 4.3.

The process $\lambda$ in (4.3) is given by

$\lambda(t)=\varSigma(t)H(t)^{\top}\\!+C(t)K(t)^{\top},\quad 0\leq t\leq T,$
(4.4)

where $\varSigma(t)\triangleq\mathbb{E}[\tilde{X}(t)\tilde{X}(t)^{\top}]$.

###### Proof.

Let $\zeta=\\{\zeta(t);\,0\leq t\leq T\\}$ be a fixed but arbitrary
$\mathbb{R}^{n\times d}$-valued, square-integrable,
$\\{\mathscr{Y}_{t}^{u}\\}$-progressively measurable processes. Consider the
$\\{\mathscr{Y}_{t}^{u}\\}$-martingale

$\eta(t)=\int_{0}^{t}\zeta(s)N(s)^{-1}dV(s),\quad 0\leq t\leq T.$

Using (4.3) we have

$\displaystyle\mathbb{E}\Big{[}\varLambda(t)\eta(t)^{\top}\Big{]}$
$\displaystyle=\mathbb{E}\int_{0}^{t}\lambda(s)N(s)^{-1}\zeta(s)^{\top}ds,\quad\forall
0\leq t\leq T.$ (4.5)

On the other hand, using (4.2) we have

$\displaystyle\mathbb{E}\Big{[}\varLambda(t)\eta(t)^{\top}\Big{]}$
$\displaystyle=\mathbb{E}\Big{[}\hat{X}(t)\eta(t)^{\top}\Big{]}-\int_{0}^{t}\mathbb{E}\Big{\\{}\big{[}A(s)\hat{X}(s)+B(s)u(s)+a(s)\big{]}\eta(t)^{\top}\Big{\\}}ds.$

Since $\\{\eta(t);\,0\leq t\leq T\\}$ is a
$\\{\mathscr{Y}_{t}^{u}\\}$-martingale, we have for $0\leq s\leq t\leq T$,

$\displaystyle\mathbb{E}\Big{[}\hat{X}(s)\eta(t)^{\top}\Big{]}$
$\displaystyle=\mathbb{E}\Big{[}\mathbb{E}\big{[}\hat{X}(s)\eta(t)^{\top}\,\big{|}\,\mathscr{Y}_{s}^{u}\big{]}\Big{]}=\mathbb{E}\Big{[}\hat{X}(s)\mathbb{E}\big{[}\eta(t)^{\top}\,|\,\mathscr{Y}_{s}^{u}\big{]}\Big{]}$
$\displaystyle=\mathbb{E}\Big{[}\hat{X}(s)\eta(s)^{\top}\Big{]}=\mathbb{E}\Big{[}\mathbb{E}[X(s)\,|\,\mathscr{Y}_{s}^{u}]\eta(s)^{\top}\Big{]}$
$\displaystyle=\mathbb{E}\Big{[}X(s)\eta(s)^{\top}\Big{]}.$

Similarly, for $0\leq s\leq t\leq T$,

$\displaystyle\mathbb{E}\Big{\\{}[B(s)u(s)+a(s)]\eta(t)^{\top}\Big{\\}}=\mathbb{E}\Big{\\{}[B(s)u(s)+a(s)]\eta(s)^{\top}\Big{\\}}.$

It follows that

$\displaystyle\mathbb{E}\Big{[}\varLambda(t)\eta(t)^{\top}\Big{]}$
$\displaystyle=\mathbb{E}\Big{[}X(t)\eta(t)^{\top}\Big{]}-\mathbb{E}\int_{0}^{t}\big{[}A(s)X(s)+B(s)u(s)+a(s)\big{]}\eta(s)^{\top}ds.$
(4.6)

We observe that

$\displaystyle d\eta(t)$
$\displaystyle=\zeta(t)N(t)^{-1}dV(t)=\zeta(t)N(t)^{-1}\Big{\\{}dY(t)-[H(t)\hat{X}(t)+h(t)]dt\Big{\\}}$
$\displaystyle=\zeta(t)N(t)^{-1}\Big{[}K(t)dW(t)+H(t)\tilde{X}(t)dt\Big{]}$
$\displaystyle=\zeta(t)N(t)^{-1}H(t)\tilde{X}(t)dt+\zeta(t)N(t)^{-1}K(t)dW(t).$

Thus, integration by parts yields

$\displaystyle\mathbb{E}[X(t)\eta(t)^{\top}]$
$\displaystyle=\mathbb{E}\int_{0}^{t}[A(s)X(s)+B(s)u(s)+a(s)]\eta(s)^{\top}ds$
$\displaystyle\hphantom{=\
}+\mathbb{E}\int_{0}^{t}X(s)\tilde{X}(s)^{\top}H(s)^{\top}N(s)^{-1}\zeta(s)^{\top}ds$
$\displaystyle\hphantom{=\
}+\mathbb{E}\int_{0}^{t}C(s)K(s)^{\top}N(s)^{-1}\zeta(s)^{\top}ds.$ (4.7)

Using the facts

$\zeta(s)\in\mathscr{Y}_{s}^{u};\quad\tilde{X}(s)~{}\text{is independent
of}~{}\mathscr{Y}_{s}^{u};\quad\mathbb{E}[\tilde{X}(s)]=0,$

we see that

$\displaystyle\mathbb{E}\int_{0}^{t}X(s)\tilde{X}(s)^{\top}H(s)^{\top}N(s)^{-1}\zeta(s)^{\top}ds$
$\displaystyle~{}=\mathbb{E}\\!\int_{0}^{t}\tilde{X}(s)\tilde{X}(s)^{\top}H(s)^{\top}N(s)^{-1}\zeta(s)^{\top}ds+\mathbb{E}\\!\int_{0}^{t}\hat{X}(s)\tilde{X}(s)^{\top}H(s)^{\top}N(s)^{-1}\zeta(s)^{\top}ds$
$\displaystyle~{}=\mathbb{E}\\!\int_{0}^{t}\mathbb{E}[\tilde{X}(s)\tilde{X}(s)^{\top}]H(s)^{\top}N(s)^{-1}\zeta(s)^{\top}ds+\mathbb{E}\\!\int_{0}^{t}\hat{X}(s)\mathbb{E}[\tilde{X}(s)^{\top}]H(s)^{\top}N(s)^{-1}\zeta(s)^{\top}ds$
$\displaystyle~{}=\mathbb{E}\\!\int_{0}^{t}\varSigma(s)H(s)^{\top}N(s)^{-1}\zeta(s)^{\top}ds.$

Now we can obtain from (4) that

$\displaystyle\mathbb{E}[X(t)\eta(t)^{\top}]$
$\displaystyle=\mathbb{E}\int_{0}^{t}\big{[}A(s)X(s)+B(s)u(s)+a(s)\big{]}\eta(s)^{\top}ds$
$\displaystyle\hphantom{=\
}+\mathbb{E}\int_{0}^{t}\Big{[}\varSigma(s)H(s)^{\top}+C(s)K(s)^{\top}\Big{]}N(s)^{-1}\zeta(s)^{\top}ds.$

The above, together with (4.6), gives

$\displaystyle\mathbb{E}\Big{[}\varLambda(t)\eta(t)^{\top}\Big{]}$
$\displaystyle=\mathbb{E}\int_{0}^{t}\Big{[}\varSigma(s)H(s)^{\top}+C(s)K(s)^{\top}\Big{]}N(s)^{-1}\zeta(s)^{\top}ds.$
(4.8)

Comparing (4.5) and (4.8) and noting that $\zeta$ is arbitrary, we get (4.4).
∎

We are ready now for the proofs of Theorem 3.1 and 3.2.

###### Proofs of Theorem 3.1 and 3.2.

From the definition (4.2) of $\varLambda$, the representation (4.3) of
$\varLambda$, and 4.3, it follows that

$\displaystyle d\hat{X}(t)$
$\displaystyle=[A(t)\hat{X}(t)+B(t)u(t)+a(t)]dt+[\varSigma(t)H(t)^{\top}\\!+C(t)K(t)^{\top}]N(t)^{-1}dV(t).$
(4.9)

By the definition of $V$ and $Y$,

$dV(t)=dY(t)-[H(t)\hat{X}(t)+h(t)]dt=H(t)\tilde{X}(t)dt+K(t)dW(t),$

which, substituted in (4.9), yields

$\displaystyle d\hat{X}(t)$ $\displaystyle=(A\hat{X}+Bu+a)dt+(\varSigma
H^{\top}\\!+CK^{\top})N^{-1}H\tilde{X}dt$ $\displaystyle\hphantom{=\
}+(\varSigma H^{\top}\\!+CK^{\top})N^{-1}KdW(t).$ (4.10)

Noting that $N(t)=K(t)K(t)^{\top}$, we obtain by subtracting (4) from the SDE
of $X$ that

$\displaystyle d\tilde{X}(t)$ $\displaystyle=[A-(\varSigma
H^{\top}\\!+CK^{\top})N^{-1}H]\tilde{X}dt-\varSigma(K^{-1}H)^{\top}dW+DdW^{\prime}.$

Integration by parts then gives

$\displaystyle\varSigma(t)-\varSigma(0)$
$\displaystyle=\mathbb{E}[\tilde{X}(t)\tilde{X}(t)^{\top}]-\mathbb{E}[\tilde{X}(0)\tilde{X}(0)^{\top}]$
$\displaystyle=\mathbb{E}\int_{0}^{t}[A(s)-(\varSigma(s)H(s)^{\top}\\!+C(s)K(s)^{\top})N(s)^{-1}H(s)]\tilde{X}(s)\tilde{X}(s)^{\top}ds$
$\displaystyle\hphantom{=\
}+\mathbb{E}\int_{0}^{t}\tilde{X}(s)\tilde{X}(s)^{\top}[A(s)-(\varSigma(s)H(s)^{\top}\\!+C(s)K(s)^{\top})N(s)^{-1}H(s)]^{\top}ds$
$\displaystyle\hphantom{=\
}+\mathbb{E}\int_{0}^{t}\varSigma(s)[K(s)^{-1}H(s)]^{\top}K(s)^{-1}H(s)\varSigma(s)ds+\mathbb{E}\int_{0}^{t}D(s)D(s)^{\top}ds$
$\displaystyle=\int_{0}^{t}[A(s)-(\varSigma(s)H(s)^{\top}\\!+C(s)K(s)^{\top})N(s)^{-1}H(s)]\varSigma(s)ds$
$\displaystyle\hphantom{=\
}+\int_{0}^{t}\varSigma(s)[A(s)-(\varSigma(s)H(s)^{\top}\\!+C(s)K(s)^{\top})N(s)^{-1}H(s)]^{\top}ds$
$\displaystyle\hphantom{=\
}+\int_{0}^{t}[\varSigma(s)H(s)^{\top}N(s)^{-1}H(s)\varSigma(s)+M(s)]ds$
$\displaystyle=\int_{0}^{t}\Big{\\{}[A(s)-C(s)K(s)^{-1}H(s)]\varSigma(s)+\varSigma(s)[A(s)-C(s)K(s)^{-1}H(s)]^{\top}$
$\displaystyle\hphantom{=\
}-\varSigma(s)H(s)^{\top}N(s)^{-1}H(s)\varSigma(s)+M(s)\Big{\\}}ds,$

which is exactly the integral version of (3.1). ∎

##  5 The optimal control

This section is devoted to the proofs of 3.3, 3.4 and Theorem 3.5. Recall that
if the control $u\in{\cal U}$ is admissible, then $\tilde{X}$ is orthogonal to
$\hat{X}$ and thereby we can write the cost functional as

$J(x;u)=\hat{J}(x;u)+\tilde{J},$

where $\hat{J}(x;u)$ and $\tilde{J}$ are given by (3) and (3), respectively.
With the notation (3.10), we see that

$[\varSigma(t)H(t)^{\top}\\!+C(t)K(t)^{\top}][K(t)^{\top}]^{-1}=(\varDelta_{1}(t)+C_{1}(t),\cdots,\varDelta_{d}(t)+C_{d}(t)).$
(5.1)

We prove 3.3 first.

###### Proof of 3.3..

For an admissible control $u\in{\cal U}_{ad}$, we know from Theorem 3.1 that
$\hat{X}$, the $\\{\mathscr{Y}_{t}^{u}\\}$-optional projection of the state
process $X$, evolves according to the SDE (3.3). Let $\check{V}_{i}(t)$ be the
$i$th component of $\check{V}(t)$ so that

$d\hat{X}(t)=[A(t)\hat{X}(t)+B(t)u(t)+a(t)]dt+\sum_{i=1}^{d}[\varDelta_{i}(t)+C_{i}(t)]d\check{V}_{i}(t).$

Recall from 4.1 that the processes $\check{V}_{i}$; $1\leq i\leq d$ are
independent Brownian motions. We obtain by applying Itô’s formula to
$t\mapsto\langle P(t)\hat{X}(t),\hat{X}(t)\rangle$ that

$\displaystyle\mathbb{E}\langle G\hat{X}(T),\hat{X}(T)\rangle-\langle
P(0)x,x\rangle$
$\displaystyle\quad=\mathbb{E}\int_{0}^{T}\Big{[}\langle\dot{P}\hat{X},\hat{X}\rangle+2\langle
P\hat{X},A\hat{X}+Bu+a\rangle+\sum_{i=1}^{d}\langle
P(\varDelta_{i}+C_{i}),\varDelta_{i}+C_{i}\rangle\Big{]}dt.$

Furthermore, applying Itô’s formula to
$t\mapsto\langle\varphi(t),\hat{X}(t)\rangle$ yields

$\displaystyle\mathbb{E}\langle
g,\hat{X}(T)\rangle-\langle\varphi(0),x\rangle=\mathbb{E}\int_{0}^{T}\Big{[}\langle\dot{\varphi},\hat{X}\rangle+\langle\varphi,A\hat{X}+Bu+a\rangle\Big{]}dt.$

Substituting for $\mathbb{E}\langle G\hat{X}(T),\hat{X}(T)\rangle$ and
$\mathbb{E}\langle g,\hat{X}(T)\rangle$ in the cost functional $\hat{J}(x;u)$,
then using (2.1) and (3.9) to simplify the computation, we obtain (3.3). ∎

Next we prove 3.4.

###### Proof of 3.4..

Recall (3.10) and (3.12)–(3.14), and note that the SDE (3.5) can be written as

$\left\\{\begin{aligned} d\tilde{X}(t)&={\cal
A}(t)\tilde{X}(t)dt-\sum_{i=1}^{d}\varDelta_{i}(t)dW_{i}(t)+\sum_{i=1}^{k}D_{i}(t)dW^{\prime}_{i}(t),\\\
\tilde{X}(0)&=0.\end{aligned}\right.$ (5.2)

Applying Itô’s formula to
$t\mapsto\langle\varPi(t)\tilde{X}(t),\tilde{X}(t)\rangle$ yields

$\displaystyle\mathbb{E}\langle G\tilde{X}(T),\tilde{X}(T)\rangle$
$\displaystyle\quad=\mathbb{E}\int_{0}^{T}\Big{[}\langle\dot{\varPi}\tilde{X},\tilde{X}\rangle+2\langle\varPi\tilde{X},{\cal
A}\tilde{X}\rangle+\sum_{i=1}^{d}\langle\varPi\varDelta_{i},\varDelta_{i}\rangle+\sum_{i=1}^{k}\langle\varPi
D_{i},D_{i}\rangle\Big{]}dt$
$\displaystyle\quad=\mathbb{E}\int_{0}^{T}\Big{[}\sum_{i=1}^{d}\langle\varPi\varDelta_{i},\varDelta_{i}\rangle+\sum_{i=1}^{k}\langle\varPi
D_{i},D_{i}\rangle-\langle Q\tilde{X},\tilde{X}\rangle\Big{]}dt.$ (5.3)

Applying Itô’s formula to $t\mapsto\langle\pi(t),\tilde{X}(t)\rangle$ yields

$\displaystyle\mathbb{E}\langle g,\tilde{X}(T)\rangle$
$\displaystyle=\mathbb{E}\int_{0}^{T}\Big{[}\langle\dot{\pi},\tilde{X}\rangle+\langle\pi,{\cal
A}\tilde{X}\rangle\Big{]}dt=-\mathbb{E}\int_{0}^{T}\langle q,\tilde{X}\rangle
dt.$ (5.4)

Substitution of (5) and (5.4) into (3) results in (3.15). ∎

Finally, we present the proof of Theorem 3.5.

###### Proof of Theorem 3.5..

For a fixed admissible control $u\in{\cal U}_{ad}$, let $X$ be the
corresponding state process with initial state $x$ and $Y$ the observation
process. By Theorem 3.1, the $\\{\mathscr{Y}_{t}^{u}\\}$-optional projection
$\hat{X}$ of $X$ evolves according to the SDE (3.3), and by 3.2, the
difference process $\tilde{X}$ evolves according to the SDE (3.5). The cost
functional can be written as the sum of $\hat{J}(x;u)$ and $\tilde{J}$, given
by (3) and (3), respectively. Observe that the process $\tilde{X}$ does not
depend on the choice of $u\in{\cal U}_{ad}$, and hence neither does
$\tilde{J}$. So Problem (O) is equivalent to finding a control
$v\in\mathscr{Y}^{v}$ such that $\hat{J}(x;u)$ is minimized over $u\in{\cal
U}_{ad}$. On the other hand, according to 3.3,

$\displaystyle\hat{J}(x;u)$ $\displaystyle\geq\langle
P(0)x,x\rangle+2\langle\varphi(0),x\rangle+\mathbb{E}\int_{0}^{T}\bigg{[}\sum_{i=1}^{d}\langle
P(\varDelta_{i}+C_{i}),\varDelta_{i}+C_{i}\rangle$ $\displaystyle\hphantom{=\
}-\langle
R^{-1}(B^{\top}\varphi+r),B^{\top}\varphi+r\rangle+2\langle\varphi,a\rangle\bigg{]}dt,$

with equality if and only if

$u(t)=\Theta(t)\hat{X}(t)-R(t)^{-1}[B(t)^{\top}\varphi(t)+r(t)].$

This means that the admissible control given by (3.16) is optimal.
Substituting (3.16) into the filtering equation (3.3) yields (3.17). Finally,
adding $\hat{J}(x;u^{*})$ and $\tilde{J}$, we get the optimal value (3.5). ∎

##  6 Conclusion remarks

We have studied a class of linear-quadratic optimal control problems for
partially observable dynamical systems. Without imposing additional
requirements on the admissible control, we showed that the optimal control is
given by a feedback representation via the filtering process and obtained the
optimal value explicitly. Our method is based on the orthogonal decomposition
of the state process, which allows us to write the cost functional as the sum
of two independent parts: one depends only on the control and the filtering
process and the other is a functional of the estimate error independent of the
choice of the control. An important feature making our approach work is that
the diffusion of the state equation does not involve the state and the
control. With this feature, our idea could apply to more general models, for
example, the mean-field model, the backward optimal control problem for
partially observable dynamical systems, and the optimal control problem of
FBSDEs with partial information. We hope to report some results relevant to
these problems in our future publications.

## References

  * [1] A. Bensoussan, Estimation and control of dynamical systems, Springer, Heidelberg, 2018.
  * [2] J. Huang, G. Wang, and J. Xiong, A maximum principle for partial information backward stochastic control problems with applications, SIAM J. Control Optim., 48 (2009), pp. 2106–2117.
  * [3] I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus, 2nd ed., Springer-Verlag, New York, 1991.
  * [4] L. C. G. Rogers and D. Williams, Diffusions, Markov Processes, and Martingales, Volume 2: Ito Calculus, 2nd ed., Cambridge University Press, 2000.
  * [5] P. Huang, G. Wang, and H. Zhang, A partial information linear-quadratic optimal control problem of backward stochastic differential equation with its applications, Sci. China Inf. Sci., (63) 2020, pp. 1–13.
  * [6] J. Shi and Z. Wu, The maximum principle for partially observed optimal control of fully coupled forward-backward stochastic system, J. Optim. Theory Appl., 145 (2010), pp. 543–578.
  * [7] Y. Shi and Q. Zhu, Partially observed optimal controls of forward-backward doubly stochastic systems, ESAIM Control Optim. Calc. Var., (19) 2013, pp. 828–843.
  * [8] J. Sun and J. Yong, Stochastic Linear-Quadratic Optimal Control Theory: Open-Loop and Closed-Loop Solutions, SpringerBriefs in Mathematics, Springer, Cham, 2020.
  * [9] G. Wang, W. Wang, and Z. Yan, Linear quadratic control of backward stochastic differential equation with partial information, Appl. Math. Comput., 403 (2021), pp. 126–164.
  * [10] G. Wang, Z. Wu, and J. Xiong, Maximum principles for forward-backward stochastic control systems with correlated state and obervation noises, SIAM J. Control Optim., 51 (2013), pp. 491–524.
  * [11] G. Wang, Z. Wu, and J. Xiong, A linear-quadratic optimal control problem of forward-backward stochastic differential equations with partial information, IEEE Trans. Automat. Control, 60 (2015), pp. 2904–2916.
  * [12] G. Wang and Z. Yu, A partial information non-zero sum differential game of backward stochastic differential equations with applications, Automatica, (48) 2012, pp. 342–352.
  * [13] W. M. Wonham, On the separation theorem of stochastic control, SIAM J. Control, 6 (1968), pp. 312–326.
  * [14] Z. Wu and Y. Zhuang, Linear-quadratic partially observed forward-backward stochastic differential games and its application in finance, Appl. Math. Comput., 321 (2018), pp. 577–592.
  * [15] J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999.


# Derivation and Extensions of the Linear Feedback Particle Filter
based on Duality Formalisms

Jin-Won Kim, Amirhossein Taghvaei and Prashant G. Mehta Financial support from
the NSF CMMI grant 1462773 is gratefully acknowledged. J-W. Kim, A. Taghvaei
and P. G. Mehta are with the Coordinated Science Laboratory and the Department
of Mechanical Science and Engineering at the University of Illinois at Urbana-
Champaign (UIUC); Corresponding email<EMAIL_ADDRESS>
###### Abstract

This paper is concerned with a duality-based approach to derive the linear
feedback particle filter (FPF). The FPF is a controlled interacting particle
system where the control law is designed to provide an exact solution for the
nonlinear filtering problem. For the linear Gaussian special case, certain
simplifications arise whereby the linear FPF is identical to the square-root
form of the ensemble Kalman filter. For this and for the more general
nonlinear non-Gaussian case, it has been an open problem to derive/interpret
the FPF control law as a solution of an optimal control problem. In this
paper, certain duality-based arguments are employed to transform the filtering
problem into an optimal control problem. Its solution is shown to yield the
deterministic form of the linear FPF. An extension is described to incorporate
stochastic effects due to noise leading to a novel homotopy of exact ensemble
Kalman filters. All the derivations are based on duality formalisms.

##  I Introduction

Amongst the many derivations of the Kalman filter, a particularly appealing
one (to control theorists) is based on duality arguments; cf., [1, Chapter 7].
In this derivation, the linear Gaussian filtering (estimation) problem is
modeled as an optimization problem involving minimization of the mean-squared
error (variance) of the estimate. By designing a suitable dual process
(evolving in backward time), the optimization problem is transformed into a
finite-horizon deterministic linear quadratic (LQ) optimal control problem.
The solution to this LQ problem yields the Kalman filter.

The duality theorem originally appeared in Kalman’s celebrated paper [2]. The
duality relationship between optimal control and linear filtering has proved
to be very useful: (i) it provides an interpretation of the Kalman filter as
the minimum variance estimator; (ii) it helps explain why the filtering
equation for the error covariance is same as the dynamic Ricatti equation
(DRE) of the optimal control; and (iii) quantitative results on the asymptotic
properties of the solution of the DRE have been used to derive results on
asymptotic stability of the linear filter [3].

Given the historical significance of these problems, several extensions have
been considered. Early on, the linear duality was related to the duality in
nonlinear programming [4]. Extensions of the duality principle to constrained
linear estimation are described in [5].

Extensions of the duality theory to nonlinear non-Gaussian settings is a
closely related subject of historical importance beginning with Mortensen’s
maximum-likelihood estimator [6]. Over the years, there have been a number of
important contributions in this area, e.g., the use of logarithmic (Hopf-Cole)
transformation to convert the filtering equation into the Hamilton-Jacobi-
Bellman (HJB) equation of optimal control [7]. An information theoretic
interpretation for the same appears in [8]. More recently, these
considerations have led to the development and application of the path
integral approaches to filtering and smoothing problems [9]. For a recent
review with a comprehensive reference list, cf., [10].

The goal of this paper is to generalize and apply Kalman’s duality principle
for the purposes of deriving particle filters, in the tractable linear
Gaussian settings. Specifically, the linear feedback particle filter (also
referred to as the square-root form of the ensemble Kalman filter) is derived
by extending results from the classical duality theory.

As some pertinent background, the feedback particle filter (FPF) is an example
of a controlled interacting particle system to approximate the solution of the
continuous-time nonlinear filtering problem. In FPF, the importance sampling
step of the conventional particle filter is replaced with feedback control.
Other steps such as resampling, reproduction, death or birth of particles are
altogether avoided. A salient feature of the FPF is that it is an exact filter
even in nonlinear-non-Gaussian settings. An expository review of the
continuous-time filters including the progression from the Kalman filter
(1960s) to the ensemble Kalman filter (1990s) to the feedback particle filter
(2010s) appears in [11].

The contributions of this paper are as follows: This paper is the first to
present a derivation of linear FPF/ensemble Kalman filters based on duality
considerations. The adaptation of the duality formalisms to the particle
filter, as proposed here, is also original. These considerations are used to
obtain a novel homotopy of stochastic and deterministic (linear) particle
filters all of which are exact.

Although it is not an explicit focus of this paper, the derivation potentially
provides new tools to analyze and interpret the optimality properties of the
filter, in terms of the simulation variance of its estimates. These properties
are important in applications of ensemble Kalman filters, in particular, for
high-dimension problems.

The outline of the remainder of this paper is as follows: Sec. II provides the
background on FPF. In Sec. III, the duality formalism for the particle filter
is introduced to construct an optimization problem. The solution to this
problem leading to the deterministic form of the linear FPF is presented in
Sec. IV. The generalization to the stochastic case appears on Sec. V. All the
proofs appear in the Appendix.

##  II Preliminaries and background

In this paper, we consider the linear Gaussian filtering problem:

$\displaystyle\,\mathrm{d}X_{t}$
$\displaystyle=A_{t}X_{t}\,\mathrm{d}t+\,\mathrm{d}B_{t}$ (1a)
$\displaystyle\,\mathrm{d}Z_{t}$
$\displaystyle=C_{t}X_{t}\,\mathrm{d}t+\,\mathrm{d}W_{t}$ (1b)

where $X_{t}\in\mathbb{R}^{d}$ is the (hidden) state at time $t$,
$Z_{t}\in\mathbb{R}^{m}$ is the observation; $A_{t}$, $C_{t}$ are matrices of
appropriate dimension whose elements are continuous in $t$; and $\\{B_{t}\\}$,
$\\{W_{t}\\}$ are mutually independent Wiener processes taking values in
$\mathbb{R}^{d}$ and $\mathbb{R}^{m}$, respectively. The covariance associated
with $\\{B_{t}\\}$ and $\\{W_{t}\\}$ are denoted by $Q_{t}$ and $R_{t}$,
respectively. The initial condition $X_{0}$ is drawn from a Gaussian
distribution $\mathcal{N}(m_{0},\Sigma_{0})$, independent of $\\{B_{t}\\}$ or
$\\{W_{t}\\}$. It is assumed that the covariance matrices $Q_{t}$, $R_{t}$ and
$\Sigma_{0}$ are strictly positive definite for all $t$. The filtering problem
is to compute the posterior distribution ${\sf P}(X_{t}|{\cal Z}_{t})$ where
${\cal Z}_{t}:=\sigma(Z_{s};s\in[0,t])$ denotes the filtration (the time-
history of observations).

For the linear Gaussian problem (1a)-(1b), the posterior distribution ${\sf
P}(X_{t}|{\cal Z}_{t})$ is Gaussian $\mathcal{N}(m_{t},\Sigma_{t})$, whose
mean and covariance are given by the Kalman-Bucy filter [12]:

$\displaystyle\,\mathrm{d}m_{t}$
$\displaystyle=A_{t}m_{t}\,\mathrm{d}t+\mathsf{K}_{t}(\,\mathrm{d}Z_{t}-C_{t}m_{t}\,\mathrm{d}t)$
(2a) $\displaystyle\frac{\,\mathrm{d}\Sigma_{t}}{\,\mathrm{d}t}$
$\displaystyle=A_{t}\Sigma_{t}+\Sigma_{t}A_{t}^{\top}+Q_{t}-\Sigma_{t}C^{\top}R_{t}^{-1}C\Sigma_{t}$
(2b)

where ${\sf K}_{t}:=\Sigma_{t}C^{\top}R_{t}^{-1}$ is the Kalman gain and the
filter is initialized with the Gaussian prior $(m_{0},\Sigma_{0})$.

Feedback particle filter (FPF) is a controlled interacting particle system to
approximate the Kalman filter111Although the considerations of this paper are
limited to the linear Gaussian problem (1a)-(1b), the FPF algorithm is more
broadly applicable to nonlinear non-Gaussian filtering problems [13, 14].. In
the following, the McKean-Vlasov stochastic differential equation (sde) are
presented for the linear FPF algorithm. For these models, the state at time
$t$ is denoted as $\bar{X}_{t}$. Two types of FPF algorithm have been
described in the literature [14, 15]:

(A) Stochastic linear FPF: The state $\bar{X}_{t}$ evolves according to the
McKean-Vlasov sde:

$\displaystyle\,\mathrm{d}\bar{X}_{t}$
$\displaystyle=A_{t}\bar{X}_{t}\,\mathrm{d}t+\,\mathrm{d}\bar{B}_{t}+\bar{{\sf
K}}_{t}(\,\mathrm{d}Z_{t}-\frac{C_{t}\bar{X}_{t}+C_{t}\bar{m}_{t}}{2}\,\mathrm{d}t)$
(3)

where $\bar{{\sf K}}_{t}=\bar{\Sigma}_{t}C_{t}^{\top}R_{t}^{-1}$ is the Kalman
gain; the mean-field terms are the mean $\bar{m}_{t}={\sf E}[\bar{X}_{t}|{\cal
Z}_{t}]$ and the covariance $\bar{\Sigma}_{t}={\sf
E}[(\bar{X}_{t}-\bar{m}_{t})(\bar{X}_{t}-\bar{m}_{t})^{\top}|{\cal Z}_{t}]$;
$\\{\bar{B}_{t}\\}$ is an independent copy of the process noise $\\{B_{t}\\}$;
and the initial condition $\bar{X}_{0}\sim\mathcal{N}(m_{0},\Sigma_{0})$.

(B) Deterministic linear FPF: The McKean-Vlasov sde is:

$\displaystyle\,\mathrm{d}\bar{X}_{t}=$ $\displaystyle
A_{t}\bar{X}_{t}\,\mathrm{d}t+\frac{1}{2}Q_{t}\bar{\Sigma}_{t}^{-1}(\bar{X}_{t}-\bar{m}_{t})\,\mathrm{d}t+\bar{{\sf
K}}_{t}(\,\mathrm{d}Z_{t}-\frac{C_{t}\bar{X}_{t}+C_{t}\bar{m}_{t}}{2}\,\mathrm{d}t)$
(4)

where (as before) $\bar{{\sf K}}_{t}=\bar{\Sigma}_{t}C_{t}^{\top}R_{t}^{-1}$
is the Kalman gain; the mean $\bar{m}_{t}={\sf E}[\bar{X}_{t}|{\cal Z}_{t}]$
and the covariance $\bar{\Sigma}_{t}={\sf
E}[(\bar{X}_{t}-\bar{m}_{t})(\bar{X}_{t}-\bar{m}_{t})^{\top}|{\cal Z}_{t}]$;
the initial condition $\bar{X}_{0}\sim\mathcal{N}(m_{0},\Sigma_{0})$.

The difference between the two algorithms is that the process noise term
$\,\mathrm{d}\bar{B}_{t}$ in the stochastic FPF (3) is replaced by
$\frac{1}{2}Q_{t}\bar{\Sigma}_{t}^{-1}(\bar{X}_{t}-\bar{m}_{t})$ in the
deterministic FPF (4).

The following Proposition, borrowed from [15], shows that both the filters are
exact:

###### Proposition 1

(Theorem 1 in [15]) Consider the linear Gaussian filtering problem (1a)-(1b),
and the linear FPF (Eq. (3) or Eq. (4)). If ${\sf P}(X_{0})={\sf
P}(\bar{X}_{0})$ then

${\sf P}(\bar{X}_{t}|{\cal Z}_{t})={\sf P}(X_{t}|{\cal Z}_{t}),\quad\forall
t>0$

Therefore, $m_{t}=\bar{m}_{t}$ and $\Sigma_{t}=\bar{\Sigma}_{t}$.

In a numerical implementation, the filter is simulated with $N$ interacting
particles where $N$ is typically large. The filter state is
$\\{X_{t}^{i}:1\leq i\leq N\\}$, where $X_{t}^{i}$ is the state of the
$i^{\text{th}}$-particle at time $t$. The evolution of $X_{t}^{i}$ is obtained
upon empirically approximating the mean-field terms. For example, the
finite-$N$ algorithm for the stochastic FPF is as follows:

(A) Finite-$N$ stochastic FPF: The evolution of $X_{t}^{i}$ is given by the
sde:

$\displaystyle\,\mathrm{d}X^{i}_{t}$
$\displaystyle=A_{t}X^{i}_{t}\,\mathrm{d}t+\,\mathrm{d}B_{t}^{i}+{\sf
K}^{(N)}_{t}(\,\mathrm{d}Z_{t}-\frac{C_{t}X^{i}_{t}+C_{t}m^{(N)}_{t}}{2}\,\mathrm{d}t)$
(5)

where ${\sf K}^{(N)}_{t}:=\Sigma^{(N)}_{t}C_{t}^{\top}R_{t}^{-1}$;
$\\{B^{i}_{t}\\}_{i=1}^{N}$ are independent copies of $B_{t}$;
$X^{i}_{0}\stackrel{{\scriptstyle\text{i.i.d}}}{{\sim}}\mathcal{N}(m_{0},\Sigma_{0})$
for $i=1,2,\ldots,N$; and the empirical approximations of the two mean-field
terms are as follows:

$\displaystyle m^{(N)}_{t}$
$\displaystyle:=\frac{1}{N}\sum_{j=1}^{N}X^{i}_{t},~{}\Sigma^{(N)}_{t}:=\frac{1}{N-1}\sum_{j=1}^{N}(X^{i}_{t}-m^{(N)}_{t})(X^{i}_{t}-m^{(N)}_{t})^{\top}$

The McKean-Vlasov sde (3) represents the mean-field limit (as
$N\rightarrow\infty$) of the finite-$N$ system (5).

##  III Duality

###  III-A Classical duality

Classical duality is concerned with the problem of constructing a minimum
variance estimator for the random variable $a^{\top}X_{T}$, where $X_{T}$ is
the hidden state (defined according to the model (1a)) at (some fixed) time
$T>0$ and $a\in\mathbb{R}^{d}$ is an arbitrary (but known) vector.

Given observations $Z_{t}$ for $t\in[0,T]$ (defined according to the model
(1b)), the assumed linear structure for the causal estimator is as follows:

$S_{T}=b_{T}^{\top}m_{0}+\int_{0}^{T}u_{t}^{\top}\,\mathrm{d}Z_{t}$ (6)

The filter is thus parametrized by the vector $b_{T}\in\mathbb{R}^{d}$ and the
(possibly) time-dependent control input $u:[0,T]\to\mathbb{R}^{m}$. The latter
is denoted as $u_{t}$.

The optimal filter parameters are obtained by solving the following mean-
squared optimization problem:

$\min_{b_{T},u_{t}}\;\;{\sf E}[|S_{T}-a^{\top}X_{T}|^{2}]$

subject to the following constraints: (1a)-(1b) for the processes $X_{t}$ and
$Z_{t}$, and (6) for the random variable $S_{T}$.

In the duality based derivation of Kalman filter, the optimization problem is
converted into a deterministic optimal control problem by introducing a
suitable dual process; cf., [1, Chapter 7]. The solution of the optimal
control problem yields the optimal control input $u_{t}$ and vector $b_{T}$.
By expressing the estimator (6) in its differential form, one obtains the
Kalman filter.

###  III-B Duality for the particle filter

A particle filter empirically approximates the posterior distribution. By the
$L^{2}$ optimality property of the posterior, the particle filter can be used
to obtain a mean-squared estimate of any arbitrary function of the hidden
state.

The particle analogue of (6) is as follows: Define $N$ random variables
$\\{S_{T}^{i}:1\leq i\leq N\\}$ according to

$S_{T}^{i}=\sum_{j=1}^{N}(b_{T}^{ij})^{\top}X_{0}^{j}+\int_{0}^{T}(u_{t}^{i})^{\top}\,\mathrm{d}Z_{t}$
(7)

where the filter parameters now are $b_{T}^{ij}\in\mathbb{R}^{d}$ for
$i,j=1,\ldots,N$ and the control input $u^{i}:[0,T]\to\mathbb{R}^{m}$ for
$i=1,\ldots,N$. As before, the filter state depends linearly on the known data
– observations up to time $T$ and the initial condition $X_{0}^{j}$ for
$j=1,\ldots,N$. The initial condition
$X^{j}_{0}\stackrel{{\scriptstyle\text{i.i.d}}}{{\sim}}\mathcal{N}(m_{0},\Sigma_{0})$.
Such a sampling ensures that the filter provides consistent estimates (as
$N\rightarrow\infty$) at time $T=0$.

The optimal filter parameters are chosen by solving the following mean-squared
optimization problem:

$\min_{b_{T}^{ij},u_{t}^{i}}\;\;{\sf
E}\Big{[}\big{|}\frac{1}{N}\sum_{i=1}^{N}f(S_{T}^{i})-f(a^{\top}X_{T})\big{|}^{2}\Big{]}$

subject to the constraints: (1a)-(1b) for the processes $X_{t}$ and $Z_{t}$,
and (7) for the random variables $S_{T}^{i}$. This is a multi-objective
optimization problem (as the function $f$ is arbitrary). With the choice of
$f(x)=x$, one obtains the optimization problem considered in classical
duality.

There is a modeling trade-off here: Ideally, one would want to consider a
large enough class of functions $f$, and the filter parameters $b$ and $u$,
that can represent and approximate the posterior distribution. However, the
class may not be too large because one would still want to be able to solve
the multi-objective optimization problem.

In order to make the analysis tractable, the following simplifications are
proposed:

Step 1: Consider the following simplifications for the filter parameters:

$\displaystyle u_{t}^{i}$ $\displaystyle:=u_{t}$
$\displaystyle\sum_{j=1}^{N}(b_{T}^{ij})^{\top}X_{0}^{j}$
$\displaystyle:=c_{T}^{\top}(X_{0}^{i}-X_{0}^{(N)})+b_{T}^{\top}X_{0}^{(N)}$

where $X_{0}^{(N)}:=\frac{1}{N}\sum_{i=1}^{N}X_{0}^{i}$ is the empirical mean.
Using this new parametrization, the filter (7) becomes

$S_{T}^{i}=c_{T}^{\top}(X_{0}^{i}-X_{0}^{(N)})+b_{T}^{\top}X_{0}^{(N)}+\int_{0}^{T}u_{t}^{\top}\,\mathrm{d}Z_{t}$
(8)

Step 2: The mean-field limit of the estimator is obtained by letting
$N\rightarrow\infty$. In the limit, the random variable is denoted as
$\bar{S}_{T}$. The mean-field counterpart of the filter (8) is

$\bar{S}_{T}=c_{T}^{\top}(\bar{X}_{0}-m_{0})+b_{T}^{\top}m_{0}+\int_{0}^{T}u_{t}^{\top}\,\mathrm{d}Z_{t}$
(9)

The optimization problem is to chose the filter parameters $b_{T}$, $c_{T}$
and $u_{t}$ to minimize the following mean-squared cost

$\min_{b_{T},c_{T},u_{t}}\;\;{\sf E}[|{\sf E}[f(\bar{S}_{T})|{\cal
Z}_{T}]-f(a^{\top}X_{T})|^{2}]$

Step 3: In the final step, we restrict the class of functions to $f(x)=x$ and
$f(x)=x^{2}$. The justification for considering only linear and quadratic
functions is that the density of the random variable $\bar{S}_{T}$ is Gaussian
(see also the Remark 1 at the end of this section).

###  III-C Optimization problem

In summary, the mathematical problem is a bi-objective optimization problem:

$\min_{b_{T},c_{T},u_{t}}\left({\sf E}[|{\sf E}[\bar{S}_{T}|{\cal
Z}_{T}]-a^{\top}X_{T}|^{2}],\;{\sf E}[|{\sf E}[\bar{S}_{T}^{\,2}|{\cal
Z}_{T}]-(a^{\top}X_{T})^{2}|^{2}]\right)$ (10)

subject to the respective constraints for the state, observation, and the
estimator:

$\displaystyle\,\mathrm{d}X_{t}$
$\displaystyle=A_{t}X_{t}\,\mathrm{d}t+\,\mathrm{d}B_{t},\quad X_{0}\sim{\cal
N}(m_{0},\Sigma_{0})$ (11a) $\displaystyle\,\mathrm{d}Z_{t}$
$\displaystyle=C_{t}X_{t}\,\mathrm{d}t+\,\mathrm{d}W_{t}$ (11b)
$\displaystyle\bar{S}_{T}$
$\displaystyle=b_{T}^{\top}m_{0}+c_{T}^{\top}(\bar{X}_{0}-m_{0})+\int_{0}^{T}u_{t}^{\top}\,\mathrm{d}Z_{t},\quad\bar{X}_{0}\sim{\cal
N}(m_{0},\Sigma_{0})$ (11c)

where the noise processes $W_{t}$ and $B_{t}$, and the initial conditions
$X_{0}$ and $\bar{X}_{0}$, are all assumed to be mutually independent; recall
that ${\cal Z}_{t}:=\sigma(Z_{s};s\in[0,t])$ denotes the time-history of
observations up to time $t$ (filtration).

A solution to this problem appears in the following section.

##  IV Deterministic particle filter via duality

Denote $\Sigma_{t}$ to be the solution of the DRE (2b) with the initial
condition $\Sigma_{0}$, and ${\sf K}_{t}:=\Sigma_{t}C^{\top}R_{t}^{-1}$ is the
Kalman gain.

Define the following state transition matrices:

$\dfrac{\,\mathrm{d}\Phi}{\,\mathrm{d}t}(t;\tau)=\big{(}-A_{t}^{\top}+C_{t}^{\top}{\sf
K}_{t}^{\top}\big{)}\Phi(t;\tau)$ (12)
$\dfrac{\,\mathrm{d}\Psi}{\,\mathrm{d}t}(t;\tau)=\big{(}-A_{t}^{\top}+\dfrac{1}{2}C_{t}^{\top}{\sf
K}_{t}^{\top}-\dfrac{1}{2}\Sigma_{t}^{-1}Q_{t}\big{)}\Psi(t;\tau)$ (13)

with $\Phi(\tau;\tau)=\Psi(\tau;\tau)=I$, the identity matrix.

These definitions are useful to describe the solution to the optimization
problem (10), as presented in the following Theorem. Its proof appears in the
Appendix (Sec. -A).

###### Theorem 1

Consider the optimization problem (10) subject to the constraints (11). Its
solution is as follows:

$\displaystyle b_{T}$ $\displaystyle=\Phi(0;T)a$ (14a) $\displaystyle c_{T}$
$\displaystyle=\Psi(0;T)a$ (14b) $\displaystyle u_{t}$ $\displaystyle={\sf
K}_{t}^{\top}\Phi(t;T)a$ (14c)

The solution for $c_{T}$ is unique up to a sign for the scalar ($d=1$) case
but it is not unique for the vector ($d>1$) case.

Using the optimal parameters (14), the estimator is

$\begin{split}\bar{S}_{T}=&a^{\top}\Big{(}\Phi^{\top}(0;T)m_{0}+\Psi^{\top}(0;T)(\bar{X}_{0}-m_{0})\\\
&+\int_{0}^{T}\Phi^{\top}(t;T){\sf
K}_{t}\,\mathrm{d}Z_{t}\Big{)},\quad\bar{X}_{0}\sim{\cal
N}(m_{0},\Sigma_{0})\end{split}$ (15)

This estimator is exact. That is,

${\sf E}(g(\bar{S}_{T})|{\cal Z}_{T})={\sf E}(g(a^{\top}X_{T})|{\cal Z}_{T})$

for all smooth test functions $g$ and vectors $a\in\mathbb{R}^{d}$.

The following Proposition provides the differential (recursive) form of the
filter which also reveals the connection to the FPF algorithm (compare with
formula (4) for the deterministic FPF). The proof appears in the Appendix
(Sec. -B).

###### Proposition 2

For any given $a\in\mathbb{R}^{d}$ and $T\geq 0$,
$\bar{S}_{T}=a^{\top}\bar{X}_{T}$ where $\bar{X}_{T}$ is the strong solution
of the following mean-field sde:

$\displaystyle\,\mathrm{d}$
$\displaystyle\bar{X}_{t}=A_{t}\bar{X}_{t}\,\mathrm{d}t+{\sf
K}_{t}\Big{(}\,\mathrm{d}Z_{t}-\frac{C_{t}\bar{X}_{t}+C_{t}\bar{m}_{t}}{2}\,\mathrm{d}t\Big{)}+\frac{1}{2}Q_{t}\Sigma_{t}^{-1}(\bar{X}_{t}-\bar{m}_{t})\,\mathrm{d}t$

where $\bar{m}_{t}={\sf E}[\bar{X}_{t}|{\cal Z}_{t}]$ and the initial
condition $\bar{X}_{0}\sim{\cal N}(m_{0},\Sigma_{0})$.

###### Remark 1

The derivation of the deterministic FPF offers an a posteriori justification
of our simplifying choices for the filter parameters (in step 1) and the
functions $f$ (in step 3). That these choices were sufficient is primarily due
to the linear Gaussian nature of the problem. More generally, if the
distribution is sub-Gaussian, the method of moments suggests considering
$\\{f(x)=x^{k}\;|\;k=1,2,\ldots\\}$ as a class of functions [16, Section 30].

##  V Stochastic particle filter via duality

In this section, we extend the deterministic filter structure (8) to now
include noise terms:

$\begin{split}{S}_{T}^{i}=&b_{T}^{\top}X_{0}^{(N)}+c_{T}^{\top}(X_{0}^{i}-X_{0}^{(N)})\\\
&+\int_{0}^{T}u_{t}^{\top}\,\mathrm{d}Z_{t}+\int_{0}^{T}v_{t}^{\top}\,\mathrm{d}B_{t}^{i}+\int_{0}^{T}w_{t}^{\top}\,\mathrm{d}W_{t}^{i}\end{split}$

where the $\\{B_{t}^{i}\\}$ and $\\{W_{t}^{i}\\}$ are independent copies of
the process noise (with covariance $Q_{t}$) and the measurement error (with
covariance $R_{t}$), respectively. These are also independent of
$B_{t},W_{t},X_{0},X_{0}^{i}$. The design problem is to chose the filter
parameters $\\{b_{T},c_{T},u_{t},v_{t},w_{t}\\}$.

The motivation of considering this type of filter structure is two-fold: (i)
${S}_{T}^{i}$ now depends linearly on all of the data—observation, initial
condition as well as the copies of the process noise and the measurement
noise; and (ii) particle filters with noise terms are widely used in practice
[11, 17]. For example, the stochastic FPF (3) includes noise term to simulate
the effect of process noise. The ensemble Kalman filter include noise terms to
simulate the effect of both process noise and measurement noise [17].

On repeating the steps 1-3, the stochastic counterpart of the optimization
problem (10)-(11) is obtained as follows:

$\min_{\begin{subarray}{c}b_{T},c_{T},\\\
u_{t},v_{t},w_{t}\end{subarray}}\left({\sf E}[|{\sf E}[\bar{S}_{T}|{\cal
Z}_{T}]-a^{\top}X_{T}|^{2}],\;{\sf E}[|{\sf E}[\bar{S}_{T}^{\,2}|{\cal
Z}_{T}]-(a^{\top}X_{T})^{2}|^{2}]\right)$ (16)

subject to

$\displaystyle\,\mathrm{d}X_{t}=$ $\displaystyle
A_{t}X_{t}\,\mathrm{d}t+\,\mathrm{d}B_{t},\quad X_{0}\sim{\cal
N}(m_{0},\Sigma_{0})$ (17a) $\displaystyle\,\mathrm{d}Z_{t}=$ $\displaystyle
C_{t}X_{t}\,\mathrm{d}t+\,\mathrm{d}W_{t}$ (17b) $\displaystyle\bar{S}_{T}=$
$\displaystyle
b_{T}^{\top}m_{0}+c_{T}^{\top}(\bar{X}_{0}-m_{0})+\int_{0}^{T}u_{t}^{\top}\,\mathrm{d}Z_{t}$
$\displaystyle+\int_{0}^{T}v_{t}^{\top}\,\mathrm{d}\bar{B}_{t}+\int_{0}^{T}w_{t}^{\top}\,\mathrm{d}\bar{W}_{t},\quad\bar{X}_{0}\sim{\cal
N}(m_{0},\Sigma_{0})$ (17c)

A solution of the optimization problem is given in the following theorem whose
proof appears in the Appendix (Sec. -C). As in the deterministic case, the
solution is given in terms of certain state transition matrices. The state-
transition matrix $\Phi$ is as defined in (12). The state transition matrix
$\Psi$ is modified to the following:

$\dfrac{\,\mathrm{d}\Psi}{\,\mathrm{d}t}(t;\tau)=\Big{(}-A_{t}^{\top}+\frac{1+\gamma_{2}^{2}}{2}C_{t}^{\top}{\sf
K}_{t}^{\top}-\frac{1-\gamma_{1}^{2}}{2}\Sigma_{t}^{-1}Q_{t}\Big{)}\Psi(t;\tau)$

where $\gamma_{1}$ and $\gamma_{2}$ are real-valued parameters. With
$\gamma_{1}=\gamma_{2}=0$, one obtains the original definition (13) for
$\Psi$.

###### Theorem 2

Consider the optimization problem (16) subject to the dynamic constraints
(17). One solution of the optimization problem is as follows:

$\displaystyle b_{T}$ $\displaystyle=\Phi(0;T)a$ $\displaystyle c_{T}$
$\displaystyle=\Psi(0;T)a$ $\displaystyle u_{t}$ $\displaystyle={\sf
K}_{t}^{\top}\Phi(t;T)a$ $\displaystyle v_{t}$
$\displaystyle=\gamma_{1}\Psi(t;T)a$ $\displaystyle w_{t}$
$\displaystyle=\gamma_{2}{\sf K}_{t}^{\top}\Psi(t;T)a$

where the values of the parameters $\gamma_{1}$ and $\gamma_{2}$ can be
arbitrarily chosen. The resulting optimal estimator is:

$\begin{split}\bar{S}_{T}=&a^{\top}\Big{(}\Phi^{\top}(0;T)m_{0}+\Psi^{\top}(0;T)(\bar{X}_{0}-m_{0})\\\
&+\int_{0}^{T}\Phi^{\top}(t;T){\sf
K}_{t}\,\mathrm{d}Z_{t}+\gamma_{1}\int_{0}^{T}\Psi^{\top}(t;T)\,\mathrm{d}\bar{B}_{t}\\\
&+\gamma_{2}\int_{0}^{T}\Psi^{\top}(t;T){\sf
K}_{t}\,\mathrm{d}\bar{W}_{t}\Big{)}\end{split}$ (18)

The estimator is exact for every choice of the parameter values $\gamma_{1}$
and $\gamma_{2}$. That is,

${\sf E}(g(\bar{S}_{T})|{\cal Z}_{T})={\sf E}(g(a^{\top}X_{T})|{\cal Z}_{T})$

for all smooth test functions $g$ and vectors $a\in\mathbb{R}^{d}$.

The differential form of the filter is given in the following Proposition
whose proof appears in the Appendix.

###### Proposition 3

For any given $a\in\mathbb{R}^{n}$ and $T\geq 0$,
$\bar{S}_{T}=a^{\top}\bar{X}_{T}$ where $\bar{X}_{T}$ is the strong solution
of the following mean-field sde:

$\begin{split}\,\mathrm{d}\bar{X}_{t}=&A_{t}\bar{X}_{t}\,\mathrm{d}t+\gamma_{1}\,\mathrm{d}\bar{B}_{t}+\frac{1-\gamma_{1}^{2}}{2}Q_{t}\Sigma_{t}^{-1}(\bar{X}_{t}-\bar{m}_{t})\,\mathrm{d}t\\\
&+{\sf
K}_{t}\Big{(}\,\mathrm{d}Z_{t}-C_{t}\Big{(}\frac{(1+\gamma_{2}^{2})\bar{X}_{t}+(1-\gamma_{2}^{2})\bar{m}_{t}}{2}\Big{)}\,\mathrm{d}t+\gamma_{2}\,\mathrm{d}\bar{W}_{t}\Big{)}\end{split}$

where $\bar{m}_{t}={\sf E}[\bar{X}_{t}|{\cal Z}_{t}]$ and the initial
condition $\bar{X}_{0}\sim{\cal N}(m_{0},\Sigma_{0})$.

###### Remark 2

The parameters $\gamma_{1}$ and $\gamma_{2}$ parametrize a homotopy of
ensemble Kalman filters/linear FPFs, all of which are exact in the linear
Gaussian settings.

  1. 1.

For $\gamma_{1}=\gamma_{2}=0$, one obtains the deterministic form of the
linear FPF (Eq. (4)).

  2. 2.

For $\gamma_{1}=1$ and $\gamma_{2}=0$, one obtains the stochastic linear FPF
(Eq. (3)).

  3. 3.

For $\gamma_{1}=\gamma_{2}=1$, one obtains the original form of the ensemble
Kalman filter [11, Eq. (7)] where a copy of measurement noise is introduced in
the error.

By varying the parameters $\gamma_{1}$ and $\gamma_{2}$ in the range $[0,1]$,
one goes from the two stochastic filters to the deterministic filter. The
filters are exact for arbitrary (even time-varying) values of parameters
$\gamma_{1}$ and $\gamma_{2}$.

##  VI Conclusion and directions for future work

In this paper, a novel homotopy of exact linear Gaussian particle filters is
derived, based on a certain extension of the classical duality. The filter is
also related to the linear FPF and the ensemble Kalman filter.

There are several possible directions of future work. The first direction is
to extend the proposed duality framework to now incorporate the finite-$N$
effects. Although it was not an explicit focus of this paper, the relationship
to optimal control potentially provides new tools to analyze and interpret the
optimality properties of the particle filter, in terms of the simulation
variance of its estimates. The other direction is to extend the duality
framework to nonlinear and non-Gaussian settings. Note that FPF algorithm is
known to be exact in these settings. If possible, it will certainly be of
interest to derive the general FPF using the duality framework.

## References

  * [1] K. J. Åström, _Introduction to Stochastic Control Theory_. Academic Press, 1970.
  * [2] R. E. Kalman, “A new approach to linear filtering and prediction problems,” _Journal of basic Engineering_ , vol. 82, no. 1, pp. 35–45, 1960.
  * [3] D. Ocone and E. Pardoux, “Asymptotic stability of the optimal filter with respect to its initial condition,” _SIAM Journal on Control and Optimization_ , vol. 34, no. 1, pp. 226–243, 1996.
  * [4] K. W. Simon and A. R. Stubberud, “Duality of linear estimation and control,” _Journal of Optimization Theory and Applications_ , vol. 6, no. 1, pp. 55–67, 1970.
  * [5] G. C. Goodwin, J. A. de Doná, M. M. Seron, and X. W. Zhuo, “Lagrangian duality between constrained estimation and control,” _Automatica_ , vol. 41, no. 6, pp. 935–944, 2005.
  * [6] R. E. Mortensen, “Maximum-likelihood recursive nonlinear filtering,” _Journal of Optimization Theory and Applications_ , vol. 2, no. 6, pp. 386–394, 1968.
  * [7] W. Fleming and S. Mitter, “Optimal control and nonlinear filtering for nondegenerate diffusion processes,” _Stochastics_ , vol. 8, pp. 63–77, 1982\.
  * [8] S. K. Mitter and N. J. Newton, “A variational approach to nonlinear estimation,” _SIAM journal on control and optimization_ , vol. 42, no. 5, pp. 1813–1833, 2003.
  * [9] H. Ruiz and H. J. Kappen, “Particle smoothing for hidden diffusion processes: Adaptive path integral smoother,” _IEEE Transactions on Signal Processing_ , vol. 65, no. 12, pp. 3191–3203, 2017.
  * [10] R. Chetrite and H. Touchette, “Variational and optimal control representations of conditioned and driven processes,” _Journal of Statistical Mechanics: Theory and Experiment_ , vol. 2015, no. 12, p. P12001, 2015.
  * [11] A. Taghvaei, J. de Wiljes, P. G. Mehta, and S. Reich, “Kalman filter and its modern extensions for the continuous-time nonlinear filtering problem,” _Journal of Dynamic Systems, Measurement, and Control_ , vol. 140, no. 3, p. 030904, 2018.
  * [12] R. E. Kalman and R. S. Bucy, “New results in linear filtering and prediction theory,” _Journal of basic engineering_ , vol. 83, no. 1, pp. 95–108, 1961\.
  * [13] T. Yang, P. G. Mehta, and S. P. Meyn, “Feedback particle filter,” _IEEE Transactions on Automatic Control_ , vol. 58, no. 10, pp. 2465–2480, October 2013\.
  * [14] T. Yang, R. S. Laugesen, P. G. Mehta, and S. P. Meyn, “Multivariable feedback particle filter,” _Automatica_ , vol. 71, pp. 10–23, 2016.
  * [15] A. Taghvaei and P. G. Mehta, “An optimal transport formulation of the linear feedback particle filter,” in _American Control Conference (ACC), 2016_. IEEE, 2016, pp. 3614–3619.
  * [16] P. Billingsley, _Probability and measure_ , ser. Wiley series in probability and mathematical statistics. Wiley, 1986.
  * [17] K. Bergemann and S. Reich, “An ensemble kalman-bucy filter for continuous data assimilation,” _Meteorologische Zeitschrift_ , vol. 21, no. 3, pp. 213–219, 2012.

###  -A Proof of Theorem 1

Express the filter (9) as

$\bar{S}_{T}=c_{T}^{\top}(\bar{X}_{0}-m_{0})+M_{T}(b_{T},u_{t})$

where

$M_{T}(b_{T},u_{t}):=b_{T}^{\top}m_{0}+\int_{0}^{T}u_{t}\,\mathrm{d}Z_{t}={\sf
E}[\bar{S}_{T}|{\cal Z}_{T}]$ (19)

is the conditional mean of $\bar{S}_{T}$.

The first objective involves minimization of the mean-squared error (filter
variance):

${\sf E}[|{\sf E}[\bar{S}_{T}|{\cal
Z}_{T}]-a^{\top}X_{T}|^{2}]\;\;=\;\;\underbrace{{\sf
E}[|M_{T}-a^{\top}X_{T}|^{2}]}_{\text{term (i)}}$

The second objective function in (10) is expressed as a sum of two terms:

$\displaystyle{\sf E}\Big{[}\big{|}{\sf E}\big{[}$
$\displaystyle\bar{S}_{T}^{2}|{\cal
Z}_{T}\big{]}-(a^{\top}X_{T})^{2}\big{|}^{2}\Big{]}={\sf
E}\Big{[}\big{|}c_{T}^{\top}\Sigma_{0}c_{T}+M_{T}^{2}-(a^{\top}X_{T})^{2}\big{|}^{2}\Big{]}$
$\displaystyle=$ $\displaystyle\;\;\underbrace{{\sf
var}\big{[}(a^{\top}X_{T})^{2}-M_{T}^{2}\big{]}}_{\text{term
(ii)}}\;\;+\;\;\underbrace{\big{(}c_{T}^{\top}\Sigma_{0}c_{T}-{\sf
E}\big{[}(a^{\top}X_{T})^{2}-M_{T}^{2}\big{]}\big{)}^{2}}_{\text{term (iii)}}$
(20)

The optimization of the three terms (i)-(iii) is the subject of the three
steps in this proof.

  1. 1.

In step 1, term (i) is minimized by choosing the vector $b_{T}$ and the
control $u_{t}$. The solution of this problem is given by classical duality.

  2. 2.

In step 2, it is shown that solution thus obtained also minimizes term (ii).

  3. 3.

In step 3, the vector $c_{T}$ is chosen to minimize the square term (iii). Its
minimum value is $0$.

The details of the three steps appear next.

Step 1: The solution to the term (i) minimization problem is given by classic
duality [1]. A dual process is introduced:

$\,\mathrm{d}y_{t}=-A_{t}^{\top}y_{t}\,\mathrm{d}t+C_{t}^{\top}u_{t}\,\mathrm{d}t,\quad
y_{T}=a$ (21)

Since
$\,\mathrm{d}(y_{t}^{\top}X_{t})=u_{t}^{\top}C_{t}X_{t}\,\mathrm{d}t+y_{t}^{\top}\,\mathrm{d}B_{t}$,

$a^{\top}X_{T}=y_{0}^{\top}X_{0}+\int_{0}^{T}u_{t}^{\top}C_{t}X_{t}\,\mathrm{d}t+\int_{0}^{T}y_{t}^{\top}\,\mathrm{d}B_{t}$

Using (19) and $\,\mathrm{d}Z_{t}=C_{t}X_{t}\,\mathrm{d}t+\,\mathrm{d}W_{t}$,
one obtains

$M_{T}-a^{\top}X_{T}=b_{T}^{\top}m_{0}-y_{0}^{\top}X_{0}+\int_{0}^{T}u_{t}^{\top}\,\mathrm{d}W_{t}-\int_{0}^{T}y_{t}^{\top}\,\mathrm{d}B_{t}$

Squaring and taking expectations, the term (i) is expressed as

$\big{(}(b_{T}^{\top}-y_{0}^{\top})m_{0}\big{)}^{2}+y_{0}^{\top}\Sigma_{0}y_{0}+\int_{0}^{T}y_{t}^{\top}Q_{t}y_{t}+u_{t}^{\top}R_{t}u_{t}\,\mathrm{d}t$

With the minimizing choice of $b_{T}=y_{0}=:b_{T}^{*}$, the term (i)
minimization problem is transformed into a linear quadratic (LQ) optimal
control problem

$\min_{u_{t}}\quad
y_{0}^{\top}\Sigma_{0}y_{0}+\int_{0}^{T}y_{t}^{\top}Q_{t}y_{t}+u_{t}^{\top}R_{t}u_{t}\,\mathrm{d}t$

subject to the dynamic constraints given by the dual system (21).

The optimal control law is easily obtained in the feedback form

$u_{t}={\sf K}_{t}^{\top}y_{t}$

where the gain ${\sf K}_{t}:=\Sigma_{t}C^{\top}R_{t}^{-1}$ is obtained by
solving the DRE (2b). Upon using the defintion (12) of the state transition
matrix $\Phi(t,\tau)$,

$u_{t}={\sf K}_{t}^{\top}y_{t}=\Phi(t;T)y_{T}=\Phi(t;T)a=:u_{T}^{*}$

We use the notation $\hat{S}_{T}$ to denote $M_{T}$ with the optimal choice of
the parameters $b_{T}^{*}$ and $u_{t}^{*}$. Explicitly,

$\hat{S}_{T}:=M_{T}(b_{T}^{*},u_{t}^{*})=a^{\top}\Phi^{\top}(0;T)m_{0}+\int_{0}^{T}a^{\top}\Phi^{\top}(t;T){\sf
K}_{t}\,\mathrm{d}Z_{t}$ (22)

$\hat{S}_{T}$ has the following properties which are useful in the remainder
of the proof:

  1. 1.

$\hat{S}_{T}$ is the conditional mean, i.e.,
$\hat{S}_{T}=E[a^{\top}X_{T}|{\cal Z}_{T}]$.

  2. 2.

The error $(a^{\top}X_{T}-\hat{S}_{T})$ satisfies the orthogonal property
whereby ${\sf E}\big{[}(a^{\top}X_{T}-\hat{S}_{T})\hat{S}_{T}\big{]}=0$.

  3. 3.

The optimal value ${\sf
E}\big{[}|\hat{S}_{T}-a^{\top}X_{T}|^{2}\big{]}=a^{\top}\Sigma_{T}a$.

Step 2: The optimal parameters, $b_{T}^{*}$ and $u_{t}^{*}$, obtained in step
1 also minimize the term (ii). The reasoning is as follows: Note that the term
(ii) involves minimization of the variance again by choosing $b_{T}$ and
$u_{t}$. Since the resulting random variable $M_{T}^{2}$ is ${\cal
Z}_{T}$-measurable, any minimizer of term (ii) is of the general form

$M_{T}^{2}={\sf E}[(a^{\top}X_{T})^{2}|{\cal Z}_{T}]+\text{(const.)}$ (23)

Now, the conditional variance of $a^{\top}X_{T}$ is given by

$\displaystyle{\sf var}[a^{\top}X_{T}|{\cal Z}_{T}]$ $\displaystyle={\sf
E}[(a^{\top}X_{T}-{\sf E}[a^{\top}X_{T}|{\cal Z}_{T}])^{2}|{\cal Z}_{T}]$
$\displaystyle={\sf E}[(a^{\top}X_{T}-{\sf E}[a^{\top}X_{T}|{\cal
Z}_{T}])^{2}]$ $\displaystyle={\sf
E}[(a^{\top}X_{T}-\hat{S}_{T})^{2}]=a^{\top}\Sigma_{T}a$

The second equality comes from the fact that orthogonality implies
independence for a Gaussian random variable. Since

$\displaystyle{\sf var}[a^{\top}X_{T}|{\cal Z}_{T}]$ $\displaystyle={\sf
E}[(a^{\top}X_{T})^{2}|{\cal Z}_{T}]-({\sf E}[a^{\top}X_{T}|{\cal
Z}_{T}])^{2}$ $\displaystyle={\sf E}[(a^{\top}X_{T})^{2}|{\cal
Z}_{T}]-\hat{S}_{T}^{2}$

we have

$\hat{S}_{T}^{2}={\sf E}[(a^{\top}X_{T})^{2}|{\cal
Z}_{T}]-a^{\top}\Sigma_{T}a$

Comparing with (23), it follows that $M_{T}=\hat{S}_{T}$ is a minimizer of the
term (ii). Consequently, $b_{T}^{*}$ and $u_{t}^{*}$ are optimal choices for
minimizing term (ii) as well. It is noted that $\hat{S}_{T}$ (defined in (22))
is the only solution that simultaneously minimizes both terms (i) and (ii).

Step 3: Now that the vector $b_{T}$ and the control $u_{t}$ have been
obtained, the vector $c_{T}$ is chosen to make the square term (iii) zero. We
have

$\displaystyle c_{T}^{\top}\Sigma_{0}c_{T}$ $\displaystyle={\sf
E}\big{[}(a^{\top}X_{T})^{2}-(\hat{S}_{T})^{2}\big{]}$ $\displaystyle={\sf
E}\big{[}(a^{\top}X_{T})^{2}-(\hat{S}_{T})^{2}\big{]}-2{\sf
E}\big{[}(a^{\top}X_{T}-\hat{S}_{T})\hat{S}_{T}\big{]}$ $\displaystyle={\sf
E}\big{[}|a^{\top}X_{T}-\hat{S}_{T}|^{2}\big{]}=a^{\top}\Sigma_{T}a$

Therefore, the minimizing choice of $c_{T}$ is obtained by solving the scalar
equation

$c_{T}^{\top}\Sigma_{0}c_{T}=a^{\top}\Sigma_{T}a$

It is straightforward to verify that $c_{T}=\Psi(0;T)a$ is a solution of this
equation. A more constructive proof follows from introducing a backward-time
process

$\frac{\,\mathrm{d}\xi_{t}}{\,\mathrm{d}t}=\bigg{(}-A_{t}^{\top}+\frac{1}{2}C_{t}^{\top}{\sf
K}_{t}^{\top}-\frac{1}{2}\Sigma_{t}^{-1}Q_{t}\bigg{)}\xi_{t},\quad\xi_{T}=a$

Since $\Sigma_{t}$ is a solution of the DRE (2b), it is easy to then verify
that $\,\mathrm{d}(\xi_{t}^{\top}\Sigma_{t}\xi_{t})=0$, and so
$\xi_{0}^{\top}\Sigma_{0}\xi_{0}=a^{\top}\Sigma_{T}a$. Thus,
$c_{T}=\xi_{0}=\Psi(0;T)a$ is a solution.

The proof for exactness is deferred to the following Sections.

###  -B Proof of Proposition 2

The equation (15) for the optimal estimator has the following form:
$\bar{S}_{T}=a^{\top}\bar{X}_{T}$. Upon denoting the (arbitrary) final time
$T$ simply as $t$, one writes

$\bar{X}_{t}=\Psi^{\top}(0;t)(\bar{X}_{0}-m_{0})+\bar{m}_{t}$ (24)

where $\bar{m}_{t}=\Phi^{\top}(0;t)m_{0}+\int_{0}^{t}\Phi^{\top}(s;t){\sf
K}_{s}\,\mathrm{d}Z_{s}={\sf E}[\bar{X}_{t}|{\cal Z}_{t}]$.

Upon differentiating (24), and using the formulae (12) and (13) for the state
transition matrices,

$\displaystyle\,\mathrm{d}\bar{X}_{t}=$ $\displaystyle(A_{t}-{\sf
K}_{t}C_{t})\Phi^{\top}(0;t)m_{0}\,\mathrm{d}t$
$\displaystyle+\Big{(}A_{t}-\frac{1}{2}{\sf
K}_{t}C_{t}+\frac{1}{2}Q_{t}\Sigma_{t}^{-1}\Big{)}\Psi^{\top}(0;t)(\bar{X}_{0}-m_{0})\,\mathrm{d}t$
$\displaystyle+\Big{(}\int_{0}^{t}(A_{t}-{\sf K}_{t}C_{t})\Phi^{\top}(s;t){\sf
K}_{s}\,\mathrm{d}Z_{s}\Big{)}\,\mathrm{d}t+{\sf K}_{t}\,\mathrm{d}Z_{t}$
$\displaystyle=$ $\displaystyle(A_{t}-{\sf
K}_{t}C_{t})\bar{X}_{t}\,\mathrm{d}t+\Big{(}\frac{1}{2}{\sf
K}_{t}C_{t}+\frac{1}{2}Q_{t}\Sigma_{t}^{-1}\Big{)}(\bar{X}_{t}-\bar{m}_{t})\,\mathrm{d}t$

This yields the recursive formula for the filter. The proof of exactness of
this filter has already appeared in [15] (see also Prop. 1). It is also a
special case of the more general stochastic filter whose exactness proof
appears in the Sec. -D.

###  -C Proof of Theorem 2 and Proposition 3

Express the filter (17c) as

$\bar{S}_{T}=M_{T}+c_{T}^{\top}(\bar{X}_{0}-m_{0})+\int_{0}^{T}v_{t}^{\top}\,\mathrm{d}\bar{B}_{t}+\int_{0}^{T}w_{t}^{\top}\,\mathrm{d}\bar{W}_{t}$

where $M_{T}=M_{T}(b_{T},u_{t})={\sf E}[\bar{S}_{T}|{\cal Z}_{T}]$ is as
defined in (19). The other three terms model the effects of randomness due to
the initial condition, process noise and the measurement noise, respectively.

Since both the noise terms have zero-mean, these do not affect the terms (i)
and (ii) as introduced in the proof of Thm. 1. The counterpart of (20) now is:

$\displaystyle{\sf E}\Big{[}\big{|}{\sf E}\big{[}\bar{S}_{T}^{2}|{\cal
Z}_{T}\big{]}-(a^{\top}X_{T})^{2}\big{|}^{2}\Big{]}=\;\underbrace{{\sf
var}\big{[}(a^{\top}X_{T})^{2}-M_{T}^{2}\big{]}}_{\text{term (ii)}}\;$
$\displaystyle+\;\underbrace{\big{(}c_{T}^{\top}\Sigma_{0}c_{T}+\int_{0}^{T}v_{t}^{\top}Q_{t}v_{t}+w_{t}^{\top}R_{t}w_{t}\,\mathrm{d}t-{\sf
E}\big{[}(a^{\top}X_{T})^{2}-M_{T}^{2}\big{]}\big{)}^{2}}_{\text{term
(iii)}^{\prime}}$

Since the terms (i) and (ii) are identical to the ones in the proof the Thm.
1, the steps 1 and 2 apply in an identical manner. The optimal solution for
$b_{T}$ and $u_{t}$ is thus the same as before. The optimal $M_{T}$ is given
by $\hat{S}_{T}$ (see (22)).

Upon setting the square term (iii)′ to its minimum value, zero, the scalar
equation for $c_{T}$ is now given by

$c_{T}^{\top}\Sigma_{0}c_{T}=a^{\top}\Sigma_{T}a-\int_{0}^{T}v_{t}^{\top}Q_{t}v_{t}+w_{t}^{\top}R_{t}w_{t}\,\mathrm{d}t$
(25)

This equation has many solutions. We pick one solution by introducing the
following modification of the process $\xi_{t}$:

$\frac{\,\mathrm{d}\xi_{t}}{\,\mathrm{d}t}=\left(-A_{t}^{\top}+\frac{1+\gamma_{2}^{2}}{2}C_{t}^{\top}{\sf
K}_{t}^{\top}-\frac{1-\gamma_{1}^{2}}{2}\Sigma_{t}^{-1}Q_{t}\right)\xi_{t},\;\xi_{T}=a$

where $\gamma_{1}$ and $\gamma_{2}$ are arbitrary constants. Then

$\,\mathrm{d}(\xi_{t}^{\top}\Sigma_{t}\xi_{t})=\gamma_{1}^{2}\xi_{t}^{\top}Q_{t}\xi_{t}\,\mathrm{d}t+\gamma_{2}^{2}\xi_{t}^{\top}{\sf
K}_{t}R_{t}{\sf K}_{t}^{\top}\xi_{t}\,\mathrm{d}t,\quad\xi_{T}=a$

Therefore, upon setting $v_{t}=\gamma_{1}\xi_{t}$ and $w_{t}=\gamma_{2}{\sf
K}_{t}^{\top}\xi_{t}$, one obtains

$\xi_{0}^{\top}\Sigma_{0}\xi_{0}=a^{\top}\Sigma_{T}a-\int_{0}^{T}v_{t}^{\top}Q_{t}v_{t}+w_{t}^{\top}R_{t}w_{t}\,\mathrm{d}t$

and thus

$\displaystyle c_{T}$ $\displaystyle=\xi_{0}=\Psi(0;T)a$ $\displaystyle v_{t}$
$\displaystyle=\gamma_{1}\Psi(t;T)a$ $\displaystyle w_{t}$
$\displaystyle=\gamma_{2}{\sf K}_{t}^{\top}\Psi(t;T)a$

solves (25).

Using these parameters, the estimator is $\bar{S}_{T}=a^{\top}\bar{X}_{T}$
where

$\begin{split}\bar{X}_{t}=&\Phi^{\top}(0;t)m_{0}+\Psi^{\top}(0;t)(\bar{X}_{0}-m_{0})\\\
&+\int_{0}^{t}\Phi^{\top}(s;t){\sf
K}_{s}\,\mathrm{d}Z_{s}+\gamma_{1}\int_{0}^{t}\Psi^{\top}(s;t)\,\mathrm{d}\bar{B}_{s}\\\
&+\gamma_{2}\int_{0}^{t}\Psi^{\top}(s;t){\sf
K}_{s}\,\mathrm{d}\bar{W}_{s}\end{split}$

Its differential form is easily obtained as

$\begin{split}\,\mathrm{d}\bar{X}_{t}=&A_{t}\bar{X}_{t}\,\mathrm{d}t+\gamma_{1}\,\mathrm{d}\bar{B}_{t}+\frac{1-\gamma_{1}^{2}}{2}Q_{t}\Sigma_{t}^{-1}(\bar{X}_{t}-\bar{m}_{t})\,\mathrm{d}t\\\
&+{\sf
K}_{t}\Big{(}\,\mathrm{d}Z_{t}-C_{t}\big{(}\frac{(1+\gamma_{2}^{2})\bar{X}_{t}+(1-\gamma_{2}^{2})\bar{m}_{t}}{2}\big{)}\,\mathrm{d}t+\gamma_{2}\,\mathrm{d}\bar{W}_{t}\Big{)}\end{split}$
(26)

###  -D Proof of the exactness of the filter

The filter (26) is a linear sde, the initial condition $\bar{X}_{0}$ is
Gaussian, and the noise terms are also Gaussian. Therefore, the solution
$\bar{X}_{t}$ is Gaussian for all $t>0$. Thus, in order to show exactness, all
we need to show is that the equations for the conditional mean and the
variance evolve according to the Kalman filter equations.

Upon taking a conditional expectation of (26),

$\,\mathrm{d}\bar{m}_{t}=A_{t}\bar{m}_{t}+{\sf
K}_{t}(\,\mathrm{d}Z_{t}-C_{t}\bar{m}_{t}\,\mathrm{d}t),\quad\bar{m}_{0}=m_{0}$

This is the same as the equation (2a) of the Kalman filter.

Next define the error process $e_{t}:=\bar{X}_{t}-\bar{m}_{t}$. The equation
for the error process is given by

$\begin{split}\,\mathrm{d}e_{t}=&A_{t}e_{t}\,\mathrm{d}t+\frac{1-\gamma_{1}^{2}}{2}Q_{t}\Sigma_{t}^{-1}e_{t}\,\mathrm{d}t\\\
&-{\sf
K}_{t}C_{t}\frac{1+\gamma_{2}^{2}}{2}e_{t}\,\mathrm{d}t+\gamma_{1}\,\mathrm{d}\bar{B}_{t}+\gamma_{2}{\sf
K}_{t}\,\mathrm{d}\bar{W}_{t}\end{split}$

Upon squaring the taking expectations, one obtains the differential equation
for the variance $\bar{\Sigma}_{t}={\sf var}(\bar{X}_{t})$ as follows:

$\begin{split}\frac{\,\mathrm{d}\bar{\Sigma}_{t}}{\,\mathrm{d}t}=&A_{t}\bar{\Sigma}_{t}+\bar{\Sigma}_{t}A_{t}^{\top}+\frac{1-\gamma_{1}^{2}}{2}(Q_{t}\Sigma_{t}^{-1}\bar{\Sigma}_{t}+\bar{\Sigma}_{t}\Sigma_{t}^{-1}Q_{t})\\\
&-\frac{1+\gamma_{2}^{2}}{2}({\sf
K}_{t}C_{t}\bar{\Sigma}_{t}-\bar{\Sigma}_{t}C^{\top}{\sf
K}_{t}^{\top})+\gamma_{1}^{2}Q_{t}+\gamma_{2}^{2}{\sf K}_{t}R_{t}{\sf
K}_{t}^{\top}\end{split}$

with initial condition $\bar{\Sigma}_{0}=\Sigma_{0}$. This is a linear ode (in
$\bar{\Sigma}_{t}$). It thus admits a unique solution. It is straightforward
to verify that $\bar{\Sigma}_{t}=\Sigma_{t}$ in fact solves the equation
(recall that $\Sigma_{t}$ is the solution to the DRE (2b) with initial
condition $\Sigma_{0}$).

Therefore, the filter (26) is exact for all time $t\geq 0$.