# Signalling and Control in Nonlinear Stochastic Systems: An Information State
Approach with Applications

Charalambos D. Charalambous1 and Stelios Louka2 1C. D. Charalambous (Bambos)
is with the Faculty of Department of Electrical and Computer Engineering,
University of Cyprus, Nicosia, Cyprus<EMAIL_ADDRESS>Louka is a Ph.D.
student with the Department of Electrical and Computer Engineering, University
of Cyprus, Nicosia, Cyprus<EMAIL_ADDRESS>
###### Abstract

We consider optimal signalling and control of discrete-time nonlinear
partially observable stochastic systems in state space form. In the first part
of the paper, we characterize the operational control-coding capacity,
$C_{FB}$ in bits/second, by an information theoretic optimization problem of
encoding signals or messages into randomized controller-encoder strategies,
and reproducing the messages at the output of the system using a decoder or
estimator with arbitrary small asymptotic error probability. Our analysis of
$C_{FB}$ is based on realizations of randomized strategies (controller-
encoders), in terms of information states of nonlinear filtering theory, and
either uniform or arbitrary distributed random variables (RVs). In the second
part of the paper, we analyze the linear-quadratic Gaussian partially
observable stochastic system (LQG-POSS). We show that simultaneous signalling
and control leads to randomized strategies described by finite-dimensional
sufficient statistics, that involve two Kalman-filters, and consist of
control, estimation and signalling strategies. We apply decentralized
optimization techniques to prove a separation principle, and to derive the
optimal control part of randomized strategies explicitly in terms of a control
matrix difference Riccati equation (DRE).

##  I Introduction, Main Results, Literature

In this paper we analyze the problem of simultaneous signalling and control
for partially observable discrete-time nonlinear stochastic control systems,
with respect to an average pay-off. Our protocol for simultaneous signalling
and control is a slight generalization of Shannon’s [1] operation definition
for reliable communication of signals or messages over noisy channels, subject
to a total transmitter power constraint of $\kappa\in[0,\infty)$ units of
power, called coding rate and coding capacity, in the theory of information
transmission [2, 3, 4]. By analogy to Shannon [1], we quantify an achievable
control-coding (CC) rate $R$ in bits/second, by the number of messages that
can be encoded into randomized control strategies (consisting of controller,
encoder and estimation strategies) subject to the average pay-off constraint,
and reproduced by the decoder or estimator at the output of the stochastic
control system, with arbitrary small asymptotic error probability. We call the
maximum achievable CC rate the control-coding capacity, $C_{FB}(\kappa)$
bits/second, of the stochastic control system. Embedded in the operational
definition of an achievable CC rate is the dual role of randomized control
strategies,
(1) to control the state of the stochastic control system while meeting the
average pay-off constraint at each time instant and asymptotically, and
(2) to signal the messages encoded into the randomized control strategies
using the stochastic control system as a communication channel, and to
reproduce the messages at the output of the decoder, with arbitrary small
asymptotic error probability.
Our first goal is to characterize $C_{FB}(\kappa)$ as function of
$\kappa\in[0,\infty)$, by an information theoretic optimization problem over
randomized strategies subject to the average pay-off constraint, using
nonlinear filtering theory and sufficient statistics. The analysis of
$C_{FB}(\kappa)$ includes as degenerate cases, many problems of classical
stochastic optimal control theory. In particular, for signalling to occur it
is necessary that $C_{FB}(\kappa)\in(0,\infty)$ for
$\kappa\in(\kappa_{min},\infty)$, where $\kappa_{min}$ is precisely the
minimum asymptotic average pay-off over deterministic control strategies
(i.e., nonrandomized) of the partially observable nonlinear stochastic system
[5] [6, 7, 8].
Our second goal is to characterize $C_{FB}(\kappa)$ for partially observable
stochastic Linear-Quadratic-Gaussian (LQG) control systems. The analysis of of
$C_{FB}(\kappa)$ includes as degenerate cases, many noisy communication
channels with memory, often used in communication applications, such as,
channels with Gaussian noise represented by state space models [9, 10, 11, 12,
13, 14, 15], molecular communication channels [16, 17], communication channels
with finite states known to the encoder and/or the decoder [2, 18, 19, 20, 21,
22, 18], etc.
In the rest of this section, we present the problem formulation, a brief
summary of our main results, followed by their relations to past literature.

###  I-A Problem Formulations and Main Results

####  I-A1 Nonlinear Partially Observable Stochastic System (N-POSS)

We consider a N-POSS with inputs
$A^{n}\stackrel{{\scriptstyle\triangle}}{{=}}\\{A_{1},\ldots,A_{n}\\}$,
$A_{t}:\Omega\rightarrow{\mathbb{A}}$, outputs
$Y^{n}\stackrel{{\scriptstyle\triangle}}{{=}}\\{Y_{1},\ldots,Y_{n}\\}$,
$Y_{t}:\Omega\rightarrow{\mathbb{Y}}$, and states
$X^{n}\stackrel{{\scriptstyle\triangle}}{{=}}\\{X_{1},\ldots,X_{n}\\}$,
$X_{t}:\Omega\rightarrow{\mathbb{X}}$, with conditional distributions,
$\forall
t\in{\mathbb{Z}}_{+}^{n}\stackrel{{\scriptstyle\triangle}}{{=}}\\{1,2,\ldots,n\\}$:

$\displaystyle{\mathbb{P}}\big{\\{}X_{t+1}\in
dx_{t+1}\big{|}Y_{t},X_{t},A_{t}\big{\\}}={\bf P}_{X_{t+1}|Y_{t},X_{t},A_{t}}$
$\displaystyle\hskip 14.45377pt=S_{t+1}(dx_{t+1}|Y_{t},X_{t},A_{t}),$ (I.1)
$\displaystyle{\mathbb{P}}\big{\\{}Y_{t}\in
dy_{t}\big{|}X_{t},A_{t}\big{\\}}={\bf
P}_{Y_{t}|X_{t},A_{t}}=Q_{t}(dy_{t}|X_{t},A_{t}).$ (I.2)

These distributions are induced by the nonlinear recursive equations subject
to an average cost constraint:

$\displaystyle X_{t+1}=f_{t}(X_{t},A_{t},W_{t}),\hskip
7.22743ptX_{1}=x_{1},\hskip 7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n-1},$
(I.3) $\displaystyle Y_{t}=h_{t}(X_{t},A_{t},W_{t}),\hskip 7.22743pt\forall
t\in{\mathbb{Z}}_{+}^{n},$ (I.4) $\displaystyle\frac{1}{n}{\bf
E}\big{\\{}c_{n}(A^{n},X^{n})\big{\\}}\leq\kappa\in[0,\infty),$ (I.5)
$\displaystyle{\bf P}_{X_{1},W^{n}}=P_{X_{1}}\times\prod_{t=1}^{n}{\bf
P}_{W_{t}}\hskip 7.22743pt\mbox{($W^{n}$ indep. and inde. of $X_{1}$)}.$ (I.6)

Here, $W^{n}\stackrel{{\scriptstyle\triangle}}{{=}}\\{W_{1},\ldots,W_{n}\\}$,
$W_{t}:\Omega\rightarrow{\mathbb{W}}$ is the noise process, and
$f_{t}(\cdot),h_{t}(\cdot),c_{n}(\cdot),\forall t$ are measurable functions.
The cost function $c_{n}(\cdot)$ depends on $(A^{n},X^{n})$ to reflect the
cost of controlling $(X^{n},Y^{n})$ via $A^{n}$. Throughout the paper, unless
otherwise stated, $({\mathbb{A}},{\mathbb{Y}},{\mathbb{X}},{\mathbb{W}})$, are
abstract Borel spaces, which include finite-alphabet spaces, finite-
dimensional Euclidean spaces, etc. The N-POSS may correspond to
stable/unstable control or communication channels with state $X^{n}$ [6].

Operational Signalling and Control Protocol. Given the N-POSS, we consider a
code denoted by $\\{(n,{\cal M}^{(n)},\epsilon_{n},\kappa)|n=1,\dots\\}$, as
follows.
(a) Uniformly distr. messages $M:\Omega\rightarrow{\cal
M}^{(n)}\stackrel{{\scriptstyle\triangle}}{{=}}\\{1,2,\ldots,M^{(n)}\\}$.
(b) Controller-Encoder strategies
$g^{n}(\cdot)\stackrel{{\scriptstyle\triangle}}{{=}}(g_{1}(\cdot),\ldots,g_{n}(\cdot))$
mapping messages $M$, and feedback information into inputs111
$(A_{t}=A_{t}^{g},Y_{t}=Y^{g},X_{t}=X_{t}^{g})$ since these are generated from
$g(\cdot)$.,
$A_{1}=g_{1}(M),A_{2}=g_{2}(M,g_{1}(M),Y_{1}),A_{3}=g_{3}(M,g_{1}(M),g_{2}(M,g_{1}(M),Y_{1}),Y^{2}),\ldots$,
of block length $n$, defined by222The superscript on expectation operator
${\bf E}^{g}$ indicates that the corresponding distribution ${\bf P}={\bf
P}^{g}$ depends the encoding strategy $g$.

$\displaystyle{\cal E}_{n}(\kappa)\triangleq\Big{\\{}g_{t}:{\cal
M}^{(n)}\times{\mathbb{A}}^{t-1}\times{\mathbb{Y}}^{t-1}\rightarrow{\mathbb{A}}_{t},\>A_{1}=g_{1}(M),$
$\displaystyle
A_{2}=g_{2}(M,g_{1}(M),Y_{1}),\ldots,A_{n}=g_{n}(M,g^{n-1}(M,Y^{n-2}),Y^{n-1})\Big{|}$
$\displaystyle\mbox{condition {\bf(C1)}) holds,}\hskip
7.22743pt\frac{1}{n}{\bf
E}^{g}\big{\\{}c_{n}(A^{n},X^{n})\big{\\}}\leq\kappa\Big{\\}},$ (I.7)
$\displaystyle\mbox{\bf(C1)}\hskip 7.22743pt{\bf
P}_{A_{t}|A^{t-1},Y^{t-1},W^{t},X^{t}}={\bf
P}_{A_{t}|A^{t-1},Y^{t-1}},\;\forall t\in{\mathbb{Z}}_{+}^{n}.$ (I.8)

Conditional independence (C1)  is equivalent to the Markov chain (MC),
$(W^{t},X^{t})\leftrightarrow(A^{t-1},Y^{t-1})\leftrightarrow A_{t},\forall
t\in{\mathbb{Z}}_{+}^{n}$. We assume $(X_{1},W^{n})$ is independent of the
messages $M$. The information to the controller-encoder strategies is ${\cal
I}_{t}^{g}\stackrel{{\scriptstyle\triangle}}{{=}}\\{M,A^{t-1},Y^{t-1}\\},A_{t}=g_{t}({\cal
I}_{t}^{g}),\forall t\in{\mathbb{Z}}_{+}^{n}$.
(c) Decoder function, $y^{n}\longmapsto d_{n}(y^{n})\in{\cal M}^{(n)}$, with
average error probability ${\bf
P}_{e}^{(n)}=\frac{1}{M^{(n)}}\sum_{i=1}^{M^{(n)}}{\mathbb{P}}\big{\\{}d_{n}(Y^{n})\neq
i|M=i\big{\\}}=\epsilon_{n}\in[0,1]$.
A rate $R$ is called an achievable CC rate, if there exists a controller-
encoder and a decoder sequence satisfying
$\lim_{n\longrightarrow\infty}{\epsilon}_{n}=0$ and
$\liminf_{n\longrightarrow\infty}\frac{1}{n}\log{M}^{(n)}\geq R$. The
operational definition of the CC capacity is
$C_{FB}(\kappa)\triangleq\sup\\{R\big{|}R\>\>\mbox{is achievable}\\}$.

Main Results of N-POSS. Our main results include the following.
1.1) Analysis of the sequential information theoretic characterization of
$C_{FB}$ under Dobrushin’s information stability [23] defined by (see [15])

$\displaystyle
C_{FB}(\kappa)\stackrel{{\scriptstyle\triangle}}{{=}}\lim_{n\longrightarrow\infty}\frac{1}{n}C_{n,FB}(\kappa),$
(I.9) $\displaystyle C_{FB,n}(\kappa)=\sup_{{\cal
P}_{n}\stackrel{{\scriptstyle\triangle}}{{=}}\\{P_{t}(da_{t}|a^{t-1},y^{t-1})\\}_{t=1}^{n},\;\frac{1}{n}{\bf
E}^{P}\\{c_{n}(A^{n},X^{n})\\}\leq\kappa}I(A^{n}\rightarrow Y^{n}),$ (I.10)
$\displaystyle I^{P}(A^{n}\rightarrow
Y^{n})\stackrel{{\scriptstyle\triangle}}{{=}}\sum_{t=1}^{n}I^{P}(A^{t};Y_{t}|Y^{t-1}),$
(I.11) $\displaystyle
I^{P}(A^{t};Y_{t}|Y^{t-1})=H^{P}(Y_{t}|Y^{t-1})-H^{P}(Y_{t}|Y^{t-1},A^{t})$
(I.12)

where $H^{P}(\cdot|\cdot)$ denotes conditional (differential) entropy [3].

Cost-Rate Optimization Problem. A dual optimization problem to
$C_{FB,n}(\kappa)$ is the cost-rate optimization problem defined by

$\displaystyle\kappa_{n}(C)\triangleq\inf_{{\cal P}_{n},\>\mbox{such
that}\>\frac{1}{n}I(A^{n}\rightarrow Y^{n})\>\geq\>C}{\bf
E}^{P}\big{\\{}c_{n}(A^{n},X^{n})\big{\\}}$ (I.13)

where $C\in[0,\infty]$. By [15], $\kappa_{n}(C)$ is a convex non-decreasing
function in $C\in(0,\infty)$. Next, we relate $\kappa_{n}(C)$ to classical
stochastic optimal control.

Relation to Classical Stochastic Optimal Control Problems. Let ${\cal
E}_{n}^{D}$ denote the restriction of randomized strategies ${\cal P}_{n}$ to
the set of deterministic strategies ${\cal
E}_{n}^{D}\triangleq\big{\\{}a_{j}=e_{j}(a^{{j-1}},y^{{j-1}})|j=1,\ldots,n\big{\\}}$.
Define the two stochastic optimal control problems with randomized and
deterministic strategies, respectively, by

$\displaystyle
J_{n}^{SC}(P^{*})\stackrel{{\scriptstyle\triangle}}{{=}}\inf_{{\cal
P}_{n}}{\bf
E}^{P}\big{\\{}c_{n}(A^{n},X^{n})\big{\\}},\;J_{n}^{SC}(e^{*})\stackrel{{\scriptstyle\triangle}}{{=}}\inf_{{\cal
E}_{n}^{D}}{\bf E}^{e}\big{\\{}c_{n}(A^{n},X^{n})\big{\\}}.$

Using the fact that deterministic strategies in $J_{n}^{SC}(e^{*})$ (when they
exists) achieve the performance of randomized strategies in
$J_{n}^{SC}(P^{*})$ (see [24]), we also have (by $I(A^{n}\rightarrow
Y^{n})\geq 0$),

$\displaystyle\frac{1}{n}\kappa_{n}(C)\geq\frac{1}{n}\kappa_{n}(C)\big{|}_{C=0}=\frac{1}{n}J_{n}^{SC}(P^{*})=\frac{1}{n}J_{n}^{SC}(e^{*})=\kappa_{n,min}.$

Consequently, $\kappa_{n,min}\in[0,\infty)$ is the minimum cost required to
control the N-POSS with zero signalling rate. The cost of information
signalling at a rate $C\in(0,\infty)$ is
$\kappa(C)-\kappa(0)\stackrel{{\scriptstyle\triangle}}{{=}}\lim_{n\longrightarrow\infty}\frac{1}{n}\kappa_{n}(C)-\lim_{n\longrightarrow\infty}\frac{1}{n}\kappa_{n}(0)$
provided the supremums and limits exist and they are finite. For a non-zero CC
rate,
$C_{FB}(\kappa)\stackrel{{\scriptstyle\triangle}}{{=}}\lim_{n\longrightarrow\infty}\frac{1}{n}C_{FB,n}(\kappa)\in(0,\infty)$,
it is necessary that $\kappa$ exceeds the critical value
$\kappa_{min}\stackrel{{\scriptstyle\triangle}}{{=}}\kappa(0)$. Hence,
$\kappa_{min}=\lim_{n\longrightarrow\infty}\frac{1}{n}J_{n}^{SC}(e^{*})$ is
precisely the asymptotic minimum cost of the classical stochastic optimal
control with partial observations (i.e., corresponding to zero information
signalling).

Our analysis is based on generalizations of the concepts of the ”information
state” and ”sufficient statistic” often used in stochastic optimal control
theory with deterministic or nonrandomized strategies [25, 26, 5, 27]. In
particular, for the analysis of $C_{FB,n}(\kappa)$ (and hence
$C_{FB}(\kappa)$) we show the following main results.
1.2) Theorem II.2. A sufficient statistic for the input distribution is (i)
the information state defined by the a posteriori conditional distribution
$\xi_{t}(A^{t-1},Y^{t-1})\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
P}_{X_{t}|A^{t-1},Y^{t-1}},\forall t\in{\mathbb{Z}}_{+}^{n}$, and (ii)
independent uniformly distributed RVs
$U^{n}\stackrel{{\scriptstyle\triangle}}{{=}}\\{U_{1},\ldots,U_{n}\\},U_{t}:\Omega\rightarrow{\mathbb{U}}\stackrel{{\scriptstyle\triangle}}{{=}}[0,1]$,
with distribution ${\bf P}_{U_{i}}$, in the sense that, there exists functions
$\mu_{t}^{u}(\cdot)$ such that

$\displaystyle A_{t}={\mu}_{t}^{u}(\xi_{t}(A^{t-1},Y^{t-1}),U_{t}),\hskip
7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n},$ (I.14)
$\displaystyle{P}_{t}(da_{t}|a^{t-1},y^{t-1})={\bf
P}_{U_{t}}\big{(}u_{t}\in{\mathbb{U}}\big{|}{\mu}_{t}^{u}(\xi_{t}(a^{t-1},y^{t-1}),u_{t})\in
da_{t}\big{)}.$

We also show that strategies (I.14) can be replaced by another function
$\mu^{z}(\cdot,Z_{t})$, with $U^{n}$ replaced by arbitrary independent RVs
$Z^{n},Z_{t}:\Omega\rightarrow{\mathbb{Z}}$ with distribution ${\bf
P}_{Z_{i}}$.
1.3) Theorem II.3. Equivalent expressions of $C_{FB,n}(\kappa)$ using
$\mu^{u}(\cdot)$ and $\mu^{z}(\cdot)$, with $I^{P}(A^{n}\rightarrow
y^{n})=I^{\mu^{u}}(U^{n}\rightarrow Y^{n})=I^{\mu^{z}}(Z^{n}\rightarrow
Y^{n})$.

####  I-A2 Linear-Quadratic-Gaussian Decision Model (LQG-POSS)

We analyze (the multiple-input multiple output (MIMO)) LQG-POSS with
$A_{t}:\Omega\rightarrow{\mathbb{R}}^{n_{a}}$,
$X_{t}:\Omega\rightarrow{\mathbb{R}}^{n_{x}}$,
$Y_{t}:\Omega\rightarrow{\mathbb{R}}^{n_{y}}$, correlated noise
$V_{t}:\Omega\rightarrow{\mathbb{R}}^{n_{y}}$, $(n_{a},n_{y},n_{v},n_{x})$
positive integers, defined by

$\displaystyle Y_{t}=D_{t}A_{t}+V_{t},\ \ V_{t}=C_{t}X_{t}+N_{t}W_{t},\ \
\forall t\in{\mathbb{Z}}_{+}^{n},$ (I.15) $\displaystyle
X_{t+1}=F_{t}X_{t}+B_{t}A_{t}+G_{t}W_{t},\hskip 7.22743pt\forall
t\in{\mathbb{Z}}_{+}^{n-1},$ (I.16) $\displaystyle X_{1}\in
G(\mu_{X_{1}},K_{X_{1}}),\hskip 7.22743ptK_{X_{1}}\succeq 0\hskip
7.22743pt\mbox{(pos. semi-definite)},$ (I.17) $\displaystyle W_{t}\in
G(0,K_{W_{t}}),\hskip 7.22743ptK_{W_{t}}\succeq 0,\hskip
7.22743ptN_{t}K_{W_{t}}N_{t}^{T}\succ 0,\hskip 7.22743pt\forall
t\in{\mathbb{Z}}_{+}^{n}$ (I.18) $\displaystyle\frac{1}{n}{\bf
E}\big{\\{}c_{n}(A^{n},X^{n})\big{\\}}\leq\kappa,\hskip
7.22743ptc_{n}(a^{n},x^{n})\stackrel{{\scriptstyle\triangle}}{{=}}\sum_{t=1}^{n}\gamma_{i}(a_{t},x_{t}),$
(I.19)
$\displaystyle\gamma_{t}(a_{t},x_{t})\stackrel{{\scriptstyle\triangle}}{{=}}\langle
a_{t},R_{t}a_{t}\rangle+\langle x_{t},Q_{t}x_{t}\rangle,\;Q_{t}\succeq
0,\;R_{t}\succ 0,\;\forall t$ (I.20)

where $X\in G(\mu_{X},K_{X})$ means the RV $X$ is Gaussian with distribution
${\bf P}_{X}$, having mean $\mu_{X}$ and covariance $K_{X}$, and
$\left\langle\cdot,\cdot\right\rangle$ denotes inner product. We show the
following main results.
2.1) Lemma III.1, Lemma III.2. $C_{FB,n}(\kappa)$ is a maximum entropy
problem, because ${\bf P}_{X_{t}|A^{t-1},Y^{t-1}}$ has conditional covariance
$\Sigma_{t}$, independent of $(A^{t-1},Y^{t-1})$, satisfying a filtering
matrix DRE (see (III.41)) and implies,

$\displaystyle H(Y_{t}|Y^{t-1},A^{t})=H(I_{t}),\hskip
7.22743ptI_{t}\stackrel{{\scriptstyle\triangle}}{{=}}Y_{t}-{\bf
E}\big{\\{}Y_{t}|Y^{t-1},A^{t}\big{\\}},\hskip 7.22743pt\forall
t\in{\mathbb{Z}}_{+}^{n},$ $\displaystyle H(I_{t})=\frac{1}{2}\log\big{(}(2\pi
e)^{n_{y}}\det\big{(}C_{t}\Sigma_{t}C_{t}^{T}+N_{t}K_{W_{t}}N_{t}^{T}\big{)}\big{)}.$
(I.21)

2.2) Theorem III.1. An equivalent characterization of $C_{FB,n}(\kappa)$
corresponding to the optimal strategy of $A^{n}$ is realized by

$\displaystyle
A_{t}=\Gamma_{t}^{1}\hat{X}_{t}+U(Y^{t-1})+Z_{t},\>\>U(Y^{t-1})=\Gamma_{t}^{2}\widehat{\hat{X}}_{t},\>\forall
t\in{\mathbb{Z}}_{+}^{n},$ (I.22)
$\displaystyle\hat{X}_{t}\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
E}\big{\\{}X_{t}\big{|}Y^{t-1},A^{t-1}\big{\\}},\;\widehat{\hat{X}}_{t}\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
E}\big{\\{}\hat{X}_{t}\big{|}Y^{t-1}\big{\\}},\;Z_{t}\in G(0,K_{Z_{t}})$

where $Z^{n}$ is an independent Gaussian process, independent of
$(\hat{X}^{n},\widehat{\hat{X}}^{n})$. The above means, for each $t$,
$\mu_{t}^{z}(\cdot)$ depends on the finite-dimensional sufficient statistic
$\big{(}\hat{X}_{t},\widehat{\hat{X}}_{t},Z_{t}\big{)}$. Moreover,
$C_{FB,n}(\kappa)$ is expressed, as a functional of two matrix DREs of
filtering theory of Gaussian systems, $\Sigma_{t}$ and
$K_{t}\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
E}\big{\\{}\big{(}\hat{X}_{t}-\widehat{\hat{X}}_{t}\big{)}\big{(}\hat{X}_{t}-\widehat{\hat{X}}_{t}\big{)}^{T}\big{|}Y^{t-1}\big{\\}}$,
which is also independent of $Y^{t-1}$.
2.3) Theorem III.1. A decentralized separation principle holds as follows:
(i) The optimal strategy $U(\cdot)$ is explicitly determined from the solution
of a stochastic LQG optimal control problem of minimizing the average cost
[6], and involves the solution of a control matrix DRE $P_{t}\succeq 0$,
evolving backward in time.
(ii) The optimal strategy $(\Gamma^{1},K_{Z})$ is determined by maximizing the
differential entropy of $H(Y^{n})$ subject to the average constraint with
optimal $U(\cdot)$ obtained in (i).
Finally, we discuss the limit convergence,
$C_{FB}\stackrel{{\scriptstyle\triangle}}{{=}}\lim_{n\longrightarrow\infty}\frac{1}{n}C_{FB,n}$,
using the convergence properties of matrix DRE to analogous matrix algebraic
Riccati equations (AREs), as in [28, Section III].

###  I-B Relations to Past Literature

$C_{FB}(\kappa)$ of channels with memory is often characterized by directed
information from the input of the channel $A^{n}$ to the output of the channel
$Y^{n}$, i.e., $I(A^{n}\rightarrow Y^{n})$ optimized over channel inputs with
feedback, $P_{t}(da_{t}|a^{t-1},y^{t-1}),\forall t\in{\mathbb{Z}}_{+}^{n}$
subject to cost constraints [29, 30]. Cover and Pombra [4, Theorem 1]
characterized $C_{FB}(\kappa)$ of additive Gaussian noise (AGN) channels,
$Y_{t}=X_{t}+V_{t},\forall t\in{\mathbb{Z}}_{+}^{n},\frac{1}{n}{\bf
E}\\{\sum_{t=1}^{n}(A_{t})^{2}\\}\leq\kappa$, with nonstationary and
nonergodic jointly Gaussian noise $V^{n}$. $C_{FB}(\kappa)$ of Cover and
Pombra AGN channel is characterized in [12, 31, 31, 14] for the degenerate
case when the noise is asymptotically stationary, with state space realization
of $V^{n}$. $V_{t}=C_{t}X_{t}+W_{t}$, $X_{t+1}=F_{t}X_{t}+W_{t},\forall t$,
such that $P_{t}(da_{t}|a^{t-1},y^{t-1})=P_{t}(da_{t}|x^{t},y^{t-1})$, i.e.,
the encoder distribution at each time $t$, knows the state of the channel
$X^{t}$ (see the hidden assumptions in [13, Theorem 6.1, Lemma 6.1] and [14]).
These assumptions are explicit in the analysis in [12]. When the state $X^{n}$
is not known to the encoder, a sequential version of the Cover and Pombra
$C_{FB}(\kappa)$ is first derived in [28, Theorem II.3] and is different from
[13, Theorem 6.1, Lemma 6.1] and [12, 14]. The method of [28] is used in [32]
to derive $C_{FB}(\kappa)$ for MIMO channels. For finite-state channels with
memory [22, 20], treated $C_{FB}(\kappa)$, when the state of the channel is
known to the encoder.
The N-POSS (I.3)-(I.6) includes the models of the above references. More
importantly, the code does not assume knowledge of the state $X^{n}$ at the
encoder, and $X^{n}$ is affected by $A^{n-1}$. This generality requires
additional concepts to characterize $C_{FB}(\kappa)$. However, in Remark
III.1, we show that from $C_{FB,n}(\kappa)$ of the LQG-POSS, we recover
$C_{FB,n}(\kappa)$ of [28, 32] and [12, 31, 31, 14] as special cases.

##  II Characterization of $C_{FB,n}$ via Information State

First, we introduce the characterization of $C_{FB,n}(\kappa)$ of the N-POSS.

######  Theorem II.1.

($C_{FB,n}$ with randomized strategies)
Consider the N-POSS, the code $\\{(n,{\cal
M}^{(n)},\epsilon_{n},\kappa)|n=1,2,\dots\\}$, and assume the conditions of
the converse/direct coding theorems in [15] hold. Define the randomized
strategies ${\cal P}_{n}(\kappa)$ by

$\displaystyle{\cal
P}_{n}(\kappa)\stackrel{{\scriptstyle\triangle}}{{=}}\Big{\\{}{\bf
P}_{A_{t}|A^{t-1},Y^{t-1}}={P}_{t}(da_{t}\big{|}a^{t-1},y^{t-1}),\forall
t\in{\mathbb{Z}}_{+}^{n}\Big{|}$ $\displaystyle\mbox{ {\bf(C1)} holds and
}\frac{1}{n}{\bf
E}^{P}\big{(}c_{n}(X^{n},A^{n})\big{)}\leq\kappa\Big{\\}}\subseteq{\cal
E}_{n}(\kappa)$

where (C1) follows from the definition of the feedback code. The directed
information from $A^{n}$ to $Y^{n}$ is defined by

$\displaystyle I^{P}(A^{n}\rightarrow
Y^{n})\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
E}^{P}\Big{\\{}\sum_{t=1}^{n}\log\Big{(}\frac{{\bf
P}_{Y_{t}|Y^{t-1},A^{t}}^{P}}{{\bf P}^{P}_{Y_{t}|Y^{t-1}}}\Big{)}\Big{\\}}$
(II.23)

where ${\bf P}_{Y_{t}|Y^{t-1},A^{t}}^{P}$ and ${\bf P}_{Y_{t}|Y^{t-1}}^{P}$,
for $\forall t\in{\mathbb{Z}}_{+}^{n}$ are given by

$\displaystyle{\bf
P}_{Y_{t}|Y^{t-1},A^{t}}^{P}=\int_{{\mathbb{X}}}Q_{t}(dy_{t}|x_{t},a_{t}){\bf
P}_{t}^{P}(dx_{t}|y^{t-1},a^{t-1}),$ (II.24) $\displaystyle{\bf
P}_{Y_{t}|Y^{t-1}}^{P}=\int_{{\mathbb{A}}^{t}\times{\mathbb{X}}^{t}}{Q}_{t}(dy_{t}\big{|}x_{t},a_{t}){P}_{t}(da_{t}|y^{t-1},a^{t-1})$
$\displaystyle\hskip 56.9055pt{\bf
P}_{t}^{P}(dx_{t}\big{|}a^{t-1},y^{t-1}){\bf P}_{t}^{P}(da^{t-1}|y^{t-1}).$
(II.25)

The CC capacity is given by
${C}_{FB}(\kappa)\stackrel{{\scriptstyle\triangle}}{{=}}\lim_{n\longrightarrow\infty}\frac{1}{n}{C}_{FB,n}(\kappa)$,
where ${C}_{FB,n}(\kappa)\stackrel{{\scriptstyle\triangle}}{{=}}\sup_{{\cal
P}_{n}(\kappa)}I^{P}(A^{n}\rightarrow Y^{n})$.

###### Proof.

The derivation is based on of the proof of the converse/direct coding theorems
in [15], hence we omit it. ∎

In $C_{FB,n}(\kappa)$, the fundamental element that determines the properties
of the maximizing elements
$\\{{P}_{t}(da_{t}|a^{t-1},y^{t-1})|t\in{\mathbb{Z}}_{+}^{n}\\}$ is the a
posteriori distribution $\\{{\bf
P}_{t}^{P}(dx_{t}|a^{t-1},y^{t-1})|t\in{\mathbb{Z}}_{+}^{n}\\}$, and its
relation to an informations state, as defined in Appendix VI-A.

###  II-A Information States and Sufficient Statistics

Now we prepare to show the statements of Section I-A.

######  Lemma II.1.

(Consistent family of randomized strategies)
(a) For any element of ${\cal P}_{n}(\kappa)$ there exist independent
uniformly distributed RVs $U^{n}$,
$U_{t}:\Omega\rightarrow{\mathbb{U}}_{t}=[0,1],\forall
t\in{\mathbb{Z}}_{+}^{n}$, and measurable functions ${\mu}_{t}^{u}(\cdot)$ of
$\\{a^{t-1},y^{t-1},u_{t}\\}$, such that

$\displaystyle a_{t}={\mu}_{t}^{u}(a^{t-1},y^{t-1},u_{t}),\hskip
7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n},$ (II.26)
$\displaystyle{P}_{t}(da_{t}|a^{t-1},y^{t-1})={\bf
P}_{U_{t}}\big{(}u_{t}\in[0,1]\big{|}{\mu}_{t}^{u}(a^{t-1},y^{t-1},u_{t})\in
da_{t}\big{)},$ (C2). $U^{n}$ independent RVs, $U_{t}$ independent of
$(W^{t},X_{1})$, $\displaystyle\hskip 14.45377pt\mbox{ ${\bf
P}_{A_{t}|A^{t-1},Y^{t-1},W^{t},X^{t}}={P}_{t}(da_{t}|a^{t-1},y^{t-1}),\forall
t\in{\mathbb{Z}}_{+}^{n}$}.$

Moreover,
$\delta_{t}^{u}\stackrel{{\scriptstyle\triangle}}{{=}}(a^{t-1},y^{t-1},u_{t})$
is a sufficient statistic for ${P}_{t}(da_{t}|a^{t-1},y^{t-1})$.
(b) Similarly to (a) there exist arbitrary independent RVs $Z^{n}$,
$Z_{t}:\Omega\rightarrow{\mathbb{Z}}_{t},\forall t\in{\mathbb{Z}}_{+}^{n}$,
and meas. functions ${\mu}_{i}^{z}(\cdot)$ such that

$\displaystyle a_{t}={\mu}_{t}^{z}(a^{t-1},y^{t-1},z_{t}),\hskip
7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n},$ (II.27)
$\displaystyle{P}_{t}(da_{t}|a^{t-1},y^{t-1})={\bf
P}_{Z_{t}}\big{(}z_{t}\in{\mathbb{Z}}_{t}\big{|}{\mu}_{t}^{z}(a^{t-1},y^{t-1},z_{t})\in
da_{t}\big{)}$ (C3). $Z^{n}$ independent RVs, $Z_{t}$ independent of
$(W^{t},X_{1})$, $\displaystyle\hskip 14.45377pt\mbox{ ${\bf
P}_{A_{t}|A^{t-1},Y^{t-1},W^{t},X^{t}}={P}_{t}(da_{t}|a^{t-1},y^{t-1}),\forall
t\in{\mathbb{Z}}_{+}^{n}$}.$

Moreover,
$\delta_{t}^{z}\stackrel{{\scriptstyle\triangle}}{{=}}(a^{t-1},y^{t-1},z_{t})$
is a sufficient statistic for ${P}_{t}(da_{t}|a^{t-1},y^{t-1})$.

###### Proof.

(a) The existence of $\mu_{t}^{u}$ with the stated properties is an
application of [24, Lemma 1.2], as described in [33, Section III]; (C2) is
inherited from condition (C1), and $\delta_{t}^{u}$ is due to the definition
of Definition VI.1.(b). (b) Follows from properties of quantile representation
of distributions [33, Section III]. ∎

Next, we verify ${\bf P}_{t}^{P}(dx_{t}|a^{t-1},y^{t-1})$ is an information
state using strategies $\mu_{t}^{u}(\cdot),\mu_{t}^{z}(\cdot)$, $\forall
t\in{\mathbb{Z}}_{+}^{n}$ of Lemma II.1.

######  Theorem II.2.

(Information state and separated strategies)
Consider the N-POSS and $C_{n,FB}(\kappa)$ of Theorem II.1.
(a) The a posteriori distribution ${\bf P}^{P}_{t+1}(dx_{t+1}|a^{t},y^{t})$
satisfies the recursion333All distributions are assumed to have probability
densities i.e., ${\bf P}^{P}_{t+1}(dx_{t+1}|a^{t},y^{t})={\bf
p}^{P}_{t+1}(x_{t+1}|a^{t},y^{t})dx_{t+1}$., $\forall
t\in{\mathbb{Z}}_{+}^{n}$:

$\displaystyle{\bf P}_{t+1}^{P}(dx_{t+1}\big{|}a^{t},y^{t})$ (II.28)
$\displaystyle=\frac{{\bf T}_{t+1}(y_{i},a_{t},{\bf
P}_{t}^{P}(\cdot\big{|}a^{t-1},y^{t-1}))(dx_{t+1})}{\int_{{\mathbb{X}}}{\bf
T}_{t+1}(y_{t},a_{t},{\bf
P}_{t}^{P}(\cdot\big{|}a^{t-1},y^{t-1}))(dx_{t+1})},\>{\bf
P}_{1}^{P}(dx_{1}\big{|}y^{0})={\bf P}_{X_{1}}$ $\displaystyle{\bf
T}_{t+1}(y_{t},a_{t},{\bf
P}_{t}^{P}(\cdot\big{|}a^{t-1},y^{t-1}))(dx_{t+1})\stackrel{{\scriptstyle\triangle}}{{=}}\int_{{\mathbb{X}}}S_{t+1}(dx_{t+1}\big{|}y_{t},x_{t},a_{t})$
$\displaystyle\hskip 14.45377pt\otimes
Q_{t}(dy_{t}\big{|}x_{t},a_{t})\otimes{\bf
P}_{t}^{P}(dx_{t}\big{|}a^{t-1},y^{t-1}).$ (II.29)

and $\\{{\bf
P}^{P}_{t+1}(dx_{t+1}|a^{t},y^{t})\big{|}t\in{\mathbb{Z}}_{+}^{n}\\}$ is an
information state.
(b) $\forall t\in{\mathbb{Z}}_{+}^{n}$, distr.
${P}_{t}(da_{t}|a^{t-1},y^{t-1})$ in the definition of distr. ${\bf
P}_{Y_{t}|Y^{t-1}}^{P}$, i.e., (II.25), are induced by $\mu^{u}(\cdot)$ or
$\mu^{z}(\cdot)$ of Lemma II.1.(a), (b).
(c) $C_{FB,n}(\kappa)$ is re-formulated using separated strategies,
$a_{t}=\mu_{t}^{u}(\xi_{t}(a^{t-1},y^{t-1}),u_{t})$ or
$a_{t}=\mu_{t}^{z}(\xi_{t}(a^{t-1},y^{t-1}),z_{t})$,
$\xi_{t}(a^{t-1},y^{t-1})\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
P}_{t}^{P}(dx_{t}|a^{t-1},y^{t-1})$ as follows.

$\displaystyle C_{FB,n}(\kappa)$ (II.30)
$\displaystyle=\sup_{\\{\mu_{t}^{u}(\xi_{t}(a^{t-1},y^{t-1}),u_{t})\\}_{t=1}^{n},\;\frac{1}{n}{\bf
E}^{\mu^{u}}\big{(}c_{n}(A^{n},X^{n})\big{)}\leq\kappa}\sum_{t=1}^{n}I^{\mu^{u}}(A^{t};Y_{t}|Y^{t-1})$
$\displaystyle=\sup_{\\{\mu_{t}^{z}(\xi_{t}(a^{t-1},y^{t-1}),z_{t})\\}_{t=1}^{n},\;\frac{1}{n}{\bf
E}^{\mu^{z}}\big{(}c_{n}(A^{n},X^{n})\big{)}\leq\kappa}\sum_{t=1}^{n}I^{\mu^{z}}(A^{t};Y_{t}|Y^{t-1}).$

###### Proof.

(a) By the existence of probability densities, ${\bf
P}_{t+1}^{P}(dx_{t+1}\big{|}a^{t},y^{t})={\bf
p}_{t+1}^{P}(x_{t+1}\big{|}a^{t},y^{t})dx_{t+1}$, ${\bf
p}_{t+1}^{P}(x_{t+1}\big{|}a^{t},y^{t})=\frac{{\bf
p}_{t+1}^{P}(x_{t+1},a^{t},y^{t})}{{\bf p}_{t}^{P}(a^{t},y^{t})}$. Then ${\bf
p}_{t+1}^{P}(x_{t+1},a^{t},y^{t})=\int_{{\mathbb{X}}_{t}}{\bf
p}_{t+1}^{P}(x_{t+1},x_{t},a^{t},y^{t})$. Using the N-POSS distributions and
condition (C1) we obtain the recursion. By Definition VI.1 the a posteriori
distribution is an information state. (b) Follows from Lemma II.1. (c) Follows
from (a), (b). ∎

Next, we restrict $\mu^{u}(\cdot),\mu^{z}(\cdot)$ to be information lossless
to show $I^{P}(A^{n}\rightarrow
Y^{n})=\sum_{t=1}^{n}I^{\mu^{u}}(U^{t};Y_{t}|Y^{t-1})=\sum_{t=1}^{n}I^{\mu^{z}}(Z^{t};Y_{t}|Y^{t-1})$.

######  Theorem II.3.

(Equivalent characterizations of $C_{FB,n}(\kappa)$)
Consider the N-POSS and $C_{FB,n}(\kappa)$ of Theorem II.1. Define the set of
information lossless strategies,

$\displaystyle{\cal
P}_{n}^{IL,\mu^{u}}(\kappa)\stackrel{{\scriptstyle\triangle}}{{=}}\big{\\{}a_{t}=\mu_{t}^{u}(a^{t-1},y^{t-1},u_{t}),\forall
t\in{\mathbb{Z}}_{+}^{n}\big{|}\mbox{the maps }$
$\mu_{t}^{u}(a^{t-1},y^{t-1},\cdot):{\mathbb{U}}_{t}\rightarrow
a_{t},\forall{\mathbb{Z}}_{+}^{n}$ are bijections, with meas.
$\displaystyle\mbox{inverses, {\bf(C2)} holds,}\;{\bf
E}^{\mu^{u}}\\{c_{n}(A^{n},X^{n})\\}\leq\kappa n\Big{\\}}.$ (II.31)

Similarly, define ${\cal P}_{n}^{IL,\mu^{z}}(\kappa)$ with
$(u_{t},{\mathbb{U}}_{t})$ replaced by $(z_{t},{\mathbb{Z}}_{t})$. Then

$\displaystyle C_{FB,n}(\kappa)=$ $\displaystyle\sup_{{\cal
P}_{n}^{IL,\mu^{u}}(\kappa)}\sum_{t=1}^{n}I(U^{t};Y_{t}|Y^{t-1})$ (II.32)
$\displaystyle=$ $\displaystyle\sup_{\\{{\bf P}_{Z_{t}}\\}_{t=1}^{n},\>{\cal
P}_{n}^{IL,\mu^{z}}(\kappa)}\sum_{t=1}^{n}I(Z^{t};Y_{t}|Y^{t-1}).$ (II.33)

Moreover, in (II.32) and (II.33) ${\cal P}_{n}^{IL,\mu^{u}}(\kappa)$ and
${\cal P}_{n}^{IL,\mu^{z}}(\kappa)$ can be replaced by separated strategies
(see Definition VI.1).

###### Proof.

(II.32), (II.33) follow from the bijection property of the information
lossless strategies and Theorem II.2. ∎

In Section III we use information states, sufficient statistics, and separated
strategies to characterize $C_{FB,n}(\kappa)$ of the LQG-POSS.

##  III Characterization of Feedback Capacity of LQG-POSS via Information
State

We characterize $C_{FB,n}(\kappa)$ of the LQG-POSS (I.15)-(I.20) in several
steps, using Section II, and a generalization of the sufficient statistic
approach in [28, 32].
Step #1. Gaussian $(A^{n},X^{n},Y^{n})$ are optimal.  We show the statement of
Section I-A2, under 2.1).

######  Lemma III.1.

(Calculation of $H^{P}(Y_{t}|Y^{t-1},A^{t})$)
Consider the LQG-DM (I.15)-(I.20). Define $\forall t\in{\mathbb{Z}}_{+}^{n}$,

$\displaystyle\hat{I}_{t}\stackrel{{\scriptstyle\triangle}}{{=}}Y_{t}-{\bf
E}\big{\\{}Y_{t}\big{|}Y^{t-1},A^{t}\big{\\}},\>\mbox{innovations wrt
$Y^{t-1},A^{t}$}$ (III.34)
$\displaystyle\mu_{Y_{t}|Y^{t-1},A^{t}}\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
E}\big{\\{}Y_{t}\big{|}Y^{t-1},A^{t}\big{\\}},\>K_{Y_{t}|Y^{t-1},A^{t}}\stackrel{{\scriptstyle\triangle}}{{=}}cov\big{(}Y_{t},Y_{t}\big{|}Y^{t-1},A^{t}\big{)},$
$\displaystyle\hat{X}_{t}\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
E}\big{\\{}X_{t}\big{|}A^{t-1},Y^{t-1}\big{\\}},\>\Sigma_{t}\stackrel{{\scriptstyle\triangle}}{{=}}cov\big{(}X_{t},X_{t}\big{|}A^{t-1},Y^{t-1}\big{)}.$

(a) The solution of the recursion of Theorem II.2.(a) is conditionally
Gaussian given by

$\displaystyle{\bf P}^{P}_{t}(dx_{t}|A^{t-1},Y^{t-1})\in
G(\hat{X}_{t};\Sigma_{t}),\ \ \forall t\in{\mathbb{Z}}_{+}^{n}$ (III.35)

where $\hat{X}_{t}$ is linear in $(A^{n-1},Y^{t-1})$, $\Sigma_{t}={\bf
E}\big{\\{}(X_{t}-\hat{X}_{t})(X_{t}-\hat{X}_{t})^{T}\big{\\}}$ is independent
of $(A^{t-1},Y^{t-1})$, and satisfy recursions:
(i) $\hat{X}_{t}$ satisfies the generalized Kalman-filter recursion,

$\displaystyle\hat{X}_{t+1}=F_{t}\hat{X}_{t}+B_{t}A_{t}+M_{t}(\Sigma_{t})\hat{I}_{t},\>\hat{X}_{1}=\mu_{X_{1}},\>\forall
t\in{\mathbb{Z}}_{+}^{n-1},$ (III.36) $\displaystyle
M_{t}(\Sigma_{t})\stackrel{{\scriptstyle\triangle}}{{=}}\Big{(}F_{t}\Sigma_{t}C_{t}^{T}+G_{t}K_{W_{t}}N_{t}^{T}\Big{)}\Big{(}N_{t}K_{W_{t}}N_{t}^{T}+C_{t}\Sigma_{t}C_{t}^{T}\Big{)}^{-1},$
$\displaystyle\hat{I}_{t}=Y_{t}-C_{t}\hat{X}_{t}-D_{t}A_{t}=C_{t}\big{(}X_{t}-\hat{X}_{t}\big{)}+N_{t}W_{t},$
(III.37) $\displaystyle\hat{I}_{t}\in G(0,K_{\hat{I}_{t}})\hskip
7.22743pt\mbox{orth. process indep. of $Y^{t-1},A^{t}$},$ (III.38)
$\displaystyle
K_{\hat{I}_{t}}\stackrel{{\scriptstyle\triangle}}{{=}}cov(\hat{I}_{t},\hat{I}_{t})=C_{t}\Sigma_{t}C_{t}^{T}+N_{t}K_{W_{t}}N_{t}^{T}=K_{Y_{t}|Y^{t-1},A^{t}},$
(III.39) $\displaystyle\mu_{Y_{t}|Y^{t-1},A^{t}}=C_{t}\hat{X}_{t}+D_{t}A_{t}.$
(III.40)

(ii) $\Sigma_{t}$ satisfies the matrix DRE,

$\displaystyle\Sigma_{t+1}=F_{t}\Sigma_{t}F_{t}^{T}+G_{t}K_{W_{t}}G_{t}^{T}-\Big{(}F_{t}\Sigma_{t}C_{t}^{T}+G_{t}K_{W_{t}}N_{t}^{T}\Big{)}$
$\displaystyle\hskip
2.84544pt.\Big{(}N_{t}K_{W_{t}}N_{t}^{T}+C_{t}\Sigma_{t}C_{t}^{T}\Big{)}^{-1}\Big{(}F_{t}\Sigma_{t}C_{t}^{T}+G_{t}K_{W_{t}}N_{t}^{T}\Big{)}^{T},$
$\displaystyle\hskip 2.84544pt\hskip 7.22743pt\Sigma_{t}\succeq 0,\hskip
7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n},\hskip
7.22743pt\Sigma_{1}=K_{X_{1}}\succeq 0.$ (III.41)

(b) The conditional entropy $H^{P}(Y_{t}|Y^{t-1},A^{t})$ is given by

$\displaystyle H^{P}(Y_{t}|Y^{t-1},A^{t})=H(\hat{I}_{t}),\hskip
7.22743pt\>\forall t\in{\mathbb{Z}}_{+}^{n}$ (III.42)
$\displaystyle=\frac{1}{2}\log\big{(}(2\pi
e)^{n_{y}}\det\big{(}C_{t}\Sigma_{t}C_{t}^{T}+N_{t}K_{W_{t}}N_{t}^{T}\big{)}\big{)}.$
(III.43)

###### Proof.

All statements follow by verifying that ${\bf
P}^{P}_{t}(dx_{t}|A^{t-1},Y^{t-1})=\frac{1}{(2\pi)^{n_{x}/2}(\det(\Sigma_{t}))^{1/2}}\exp\big{(}-\frac{1}{2}(x_{t}-\hat{X}_{t})\Sigma_{t}^{-1}(x_{t}-\hat{X}_{t})^{T}\big{)},\forall
t\in{\mathbb{Z}}_{+}^{n}$ satisfies recursion (II.28). ∎

######  Lemma III.2.

(Optimality of Gaussian inputs)
Consider the LQG-POSS (I.15)-(I.20). The optimal $P(\cdot|\cdot)\in{\cal
P}_{n}(\kappa)$ which maximizes $I(A^{n}\rightarrow Y^{n})$ is induced by the
realization of jointly Gaussian $A^{n}$, given by

$\displaystyle
A_{t}=\mu_{t}^{z}(Y^{t-1},A^{t-1},Z_{t})=\mu_{t}^{0}(A^{t-1},Y^{t-1})+Z_{t},$
(III.44) $\displaystyle\mu_{t}^{0}(A^{t-1},Y^{t-1})\hskip 7.22743pt\mbox{is
linear in the RVs $(A^{t-1},Y^{t-1})$},$ (III.45) $\displaystyle Z_{t}\in
G(0,K_{Z_{t}}),\;K_{Z_{t}}\succeq 0,\forall t\in{\mathbb{Z}}_{+}^{n}\hskip
7.22743pt\mbox{indep. proc.}$ (III.46) $\displaystyle Z_{i}\hskip
7.22743pt\mbox{indep. of}\hskip
7.22743pt(X^{t},W^{t},A^{t-1},Y^{t-1}),\;\forall t\in{\mathbb{Z}}_{+}^{n}.$
(III.47)

###### Proof.

Follows from Lemma III.1.(b), by the maximum entropy principle, because
$I(A^{n}\rightarrow Y^{n})=H(Y^{n})-\sum_{t=1}^{n}H(\hat{I}_{t})$, and the
average cost constraint is quadratic. By Lemma II.1.(b), one realization of
$A^{n}$ is (III.44)-(III.47). ∎

Step #2. Equivalent Characterization of $C_{FB,n}(\kappa)$ via Information
State and Sufficient Statistics. First, we invoke Lemma III.1, to identify a
sufficient statistic for ${\cal P}_{n}(\kappa)$.

######  Lemma III.3.

Preliminary char. of ${C}_{FB,n}(\kappa)$ for LQG-POSSs
Consider the LQG-POSS (I.15)-(I.20). Then

$\displaystyle Y_{t}=C_{t}{\hat{X}}_{t}+D_{t}A_{t}+\hat{I}_{t},\hskip
7.22743ptt=1,\ldots,n,$ (III.48) $\displaystyle{\bf
P}_{t}(dy_{t}|y^{t-1},a^{t})={\bf P}_{t}(dy_{t}|a_{t},\hat{x}_{t}),$ (III.49)
$\displaystyle{\bf P}_{t}(dy_{t}|y^{t-1})=\int{\bf
P}_{t}(dy|a_{t},\hat{x}_{t}){\bf P}_{t}(da_{t}|\hat{x}_{t},y^{t-1})$
$\displaystyle\hskip 7.22743pt.{\bf P}_{t}(d\hat{x}_{t}|y^{t-1}),$ (III.50)
$\displaystyle P_{t}(da_{t}|a^{t-1},y^{t-1})={\bf
P}_{t}(da_{t}|\hat{x}_{t},y^{t-1})$ (III.51) $\displaystyle
A_{t}=\Gamma_{t}^{1}\hat{X}_{t}+U_{t}(Y^{t-1})+Z_{t},\hskip
7.22743ptU_{t}(Y^{t-1})\stackrel{{\scriptstyle\triangle}}{{=}}\Gamma_{t}^{2}Y^{t-1},$
(III.52) $\displaystyle Z_{t}\in G(0,K_{Z_{t}})\hskip 7.22743pt\mbox{indep.
of}\hskip 7.22743pt(X^{t},\hat{X}^{t},\hat{I}^{t},A^{t-1},Y^{t-1}).$ (III.53)

where $(\Gamma_{t}^{1},\Gamma_{t}^{2})$ are nonrandom. An equivalent
${C}_{FB,n}(\kappa)$ is

$\displaystyle{C}_{FB,n}(\kappa)=\sup_{{\cal
P}_{n}^{\hat{X}}(\kappa)}\sum_{t=1}^{n}I(A_{t},\hat{X}_{t};Y_{t}|Y^{t-1}),$
(III.54) $\displaystyle{\cal
P}_{n}^{\hat{X}}(\kappa)\stackrel{{\scriptstyle\triangle}}{{=}}\Big{\\{}\\{{\bf
P}_{t}(da_{t}|\hat{x}_{t},y^{t-1})\\}_{t=1}^{n}\Big{|}\frac{1}{n}{\bf
E}\big{(}\sum_{t=1}^{n}\hat{\gamma}_{t}(A_{t},\hat{X}_{t})\big{)}\leq\kappa\Big{\\}}$
$\displaystyle\hat{\gamma}_{t}(A_{t},\hat{X}_{t})\stackrel{{\scriptstyle\triangle}}{{=}}\langle
A_{t},R_{t}A_{t}\rangle+\langle\hat{X}_{t},Q_{t}\hat{X}_{t}\rangle+trace\big{(}Q_{t}\Sigma_{t}\big{)}.$
(III.55)

###### Proof.

See Section VI-B. ∎

Step #3. Complete Characterization of $C_{FB,n}(\kappa)$.  Now, we show the
statements of Section I-A2, under 2.2). We use Lemma III.3 to find another
equivalent $Y^{n}$ to compute ${\bf P}_{Y_{t}|Y^{t-1}}$, which is required in
the optimization of $H(Y^{n})$.

######  Theorem III.1.

Complete char. of ${C}_{FB,n}(\kappa)$ for LQG-POSSs
Consider the LQG-POSS (I.15)-(I.20). Define

$\displaystyle I_{t}\stackrel{{\scriptstyle\triangle}}{{=}}$ $\displaystyle
Y_{t}-{\bf E}\big{\\{}Y_{t}\big{|}Y^{t-1}\big{\\}},\>\mbox{$Y^{t}$ generated
by (\ref{cp_16_alt_n_P})},$ (III.56)
$\displaystyle\widehat{\hat{X}}_{t}\stackrel{{\scriptstyle\triangle}}{{=}}$
$\displaystyle{\bf E}\big{\\{}\hat{X}_{t}\Big{|}Y^{t-1}\big{\\}},\;\forall
t\in{\mathbb{Z}}_{+}^{n},\;\widehat{\hat{X}}_{1}\stackrel{{\scriptstyle\triangle}}{{=}}\mu_{X_{1}},\;K_{1}\stackrel{{\scriptstyle\triangle}}{{=}}0,$
$\displaystyle K_{t}\stackrel{{\scriptstyle\triangle}}{{=}}$ $\displaystyle
cov\big{(}\hat{X}_{t},\hat{X}_{t}\Big{|}Y^{t-1}\big{)}={\bf
E}\big{\\{}\big{(}\hat{X}_{t}-\widehat{\hat{X}}_{t}\big{)}\big{(}\hat{X}_{t}-\widehat{\hat{X}}_{t}\big{)}^{T}\big{|}Y^{t-1}\big{\\}}.$

(a) An equivalent representation of $(A^{n},Y^{n})$ is

$\displaystyle
Y_{t}=\big{(}C_{t}+D_{t}\Gamma_{t}^{1}\big{)}\widehat{\hat{X}}_{t}+D_{t}U_{t}(Y^{t-1})+{I}_{t},\hskip
7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n},$ (III.57) $\displaystyle
A_{t}=\Gamma_{t}^{1}\hat{X}_{t}+U_{t}(Y^{t-1})+Z_{t},\hskip
7.22743ptU_{t}(Y^{t-1})\stackrel{{\scriptstyle\triangle}}{{=}}\Gamma_{t}^{2}Y^{t-1},$
(III.58) $\displaystyle Z_{t}\in G(0,K_{Z_{t}})\hskip 7.22743pt\mbox{ind.
of}\hskip
7.22743pt(X^{t},\hat{X}^{t},\widehat{\hat{X}}_{t},\hat{I}^{t},A^{t-1},Y^{t-1})$
(III.59)

where $\widehat{\hat{X}}_{t}$ and $K_{t}$ are given under (i) and (ii) below.
(i) $\widehat{\hat{X}}_{t}$ satisfies the Kalman-filter recursion,

$\displaystyle\widehat{\hat{X}}_{t+1}=F_{t}(\Gamma_{t}^{1})\widehat{\hat{X}}_{t}+B_{t}U_{t}$
$\displaystyle+F_{t}^{CL}(\Sigma_{t},K_{t},\Gamma_{t}^{1})I_{t},\;\widehat{\hat{X}}_{1}=\mu_{X_{1}},\hskip
7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n-1},$ (III.60) $\displaystyle
F_{t}(\Gamma_{t}^{1})\stackrel{{\scriptstyle\triangle}}{{=}}F_{t}+B_{t}\Gamma_{t}^{1},\;C_{t}(\Gamma_{t}^{1})\stackrel{{\scriptstyle\triangle}}{{=}}C_{t}+D_{t}\Gamma_{t}^{1},$
(III.61) $\displaystyle
F_{t}^{CL}(\Sigma_{t},K_{t},\Gamma_{t}^{1})\stackrel{{\scriptstyle\triangle}}{{=}}\Big{(}F_{t}(\Gamma_{t}^{1})K_{t}\big{(}C_{t}(\Gamma_{t}^{1})\big{)}^{T}+B_{t}K_{Z_{t}}D^{T}$
(III.62)
$\displaystyle+M_{t}(\Sigma_{t})K_{\hat{I}_{t}}\Big{)}\big{\\{}K_{\hat{I}_{t}}+D_{t}K_{Z_{t}}D_{t}^{T}+\big{(}C_{t}(\Gamma_{t}^{1})\big{)}K_{t}\big{(}C_{t}(\Gamma_{t}^{1})\big{)}^{T}\big{\\}}^{-1},$
$\displaystyle
I_{t}=Y_{t}-C_{t}\widehat{\hat{X}}_{t}-D_{t}\Gamma_{t}^{1}\widehat{\hat{X}}_{t}-D_{t}U(Y^{t-1})$
$\displaystyle=C_{t}(\Gamma_{t}^{1})\Big{(}\hat{X}_{t}-\widehat{\hat{X}}_{t}\Big{)}+\hat{I}_{t}+D_{t}Z_{t},\hskip
7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n},$ (III.63) $\displaystyle I_{t}\in
G(0;K_{I_{t}}),\;\forall t\in{\mathbb{Z}}_{+}^{n}\;\mbox{orthogonal
innovations proc.}$ $\displaystyle
K_{I_{t}}=K_{Y_{t}|Y^{t-1}}\stackrel{{\scriptstyle\triangle}}{{=}}cov\big{(}I_{t},I_{t}\big{)}=\Big{(}C_{t}(\Gamma_{t}^{1})\Big{)}K_{t}\Big{(}C_{t}(\Gamma_{t}^{1})\Big{)}^{T}$
$\displaystyle+K_{\hat{I}_{t}}+D_{t}K_{Z_{t}}D_{t}^{T},\hskip
7.22743ptK_{\hat{I}_{t}}\hskip 7.22743pt\mbox{given by (\ref{cov_in_noise})}.$
(III.64)

(ii) $K_{t}={\bf E}\big{\\{}\widehat{E}_{t}\widehat{E}_{t}^{T}\big{\\}}$,
$\widehat{E}_{t}=\hat{X}_{t}-\widehat{\hat{X}}_{t}$ satisfies the matrix DRE

$\displaystyle
K_{t+1}=F_{t}(\Gamma_{t}^{1})K_{t}\big{(}F_{t}(\Gamma_{t}^{1}\big{)}^{T}+M_{t}(\Sigma_{t})K_{\hat{I}_{t}}\big{(}M_{t}(\Sigma_{t})\big{)}^{T}+B_{t}K_{Z_{t}}B_{t}^{T}$
$\displaystyle-\Big{(}F_{t}(\Gamma_{t}^{1})K_{t}\big{(}C_{t}(\Gamma_{t}^{1})\big{)}^{T}+B_{t}K_{Z_{t}}D_{t}^{T}+M_{t}(\Sigma_{t})K_{\hat{I}_{t}}\Big{)}$
$\displaystyle.\Big{(}K_{\hat{I}_{t}}+D_{t}K_{Z_{t}}D_{t}^{T}+\big{(}C_{t}(\Gamma_{t}^{1}\big{)}K_{t}\big{(}C_{t}(\Gamma_{t}^{1})\big{)}^{T}\Big{)}^{-1}\Big{(}F_{t}(\Gamma_{t}^{1})K_{t}\big{(}C_{t}(\Gamma_{t}^{1})\big{)}^{T}$
$\displaystyle+B_{t}K_{Z_{t}}B_{t}^{T}+M_{t}(\Sigma_{t})K_{\hat{I}_{t}}\Big{)}^{T},\>K_{t}\succeq
0,\forall t\in{\mathbb{Z}}_{+}^{n},\;K_{1}=0.$ (III.65)

(b) An equivalent characterization of ${C}_{FB,n}(\kappa)$ is

$\displaystyle{C}_{FB,n}(\kappa)=\sup_{{\cal
P}_{n}^{\widehat{\hat{X}}}(\kappa)}\sum_{t=1}^{n}I(A_{t},\hat{X}_{t};Y_{t}|Y^{t-1}),$
(III.66) $\displaystyle
I(A_{t},\hat{X}_{t};Y_{t}|Y^{t-1})=\frac{1}{2}\log\frac{\det(K_{I_{t}})}{\det(K_{\hat{I}_{t}})}=$
(III.67)
$\displaystyle\frac{1}{2}\log\Big{\\{}\frac{\det\Big{(}C_{t}(\Gamma_{t}^{1})K_{t}(C_{t}(\Gamma_{t}^{1})^{T}+K_{\hat{I}_{t}}+D_{t}K_{Z_{t}}D_{t}^{T}\Big{)}}{\det\big{(}K_{\hat{I}_{t}}\big{)}}\Big{\\}},$
(III.68) $\displaystyle{\cal
P}_{n}^{\widehat{\hat{X}}}(\kappa)\stackrel{{\scriptstyle\triangle}}{{=}}\Big{\\{}\\{U_{t},\Gamma_{t}^{1},K_{Z_{t}}\\}_{t=1}^{n}\Big{|}\frac{1}{n}{\bf
E}\big{(}\sum_{t=1}^{n}\widehat{\hat{\gamma}}_{t}(U_{t},\widehat{\hat{X}}_{t},\Gamma_{t}^{1},K_{Z_{t}})\big{)}\leq\kappa\Big{\\}}$
$\displaystyle\widehat{\hat{\gamma}}_{t}(U_{t},\widehat{\hat{X}}_{t},\Gamma_{t}^{1}K_{Z_{t}},)\stackrel{{\scriptstyle\triangle}}{{=}}\langle
U_{t},R_{t}U_{t}\rangle+\langle\widehat{\hat{X}}_{t},Q_{t}(\Gamma_{t}^{1})\widehat{\hat{X}}_{t}\rangle+\langle\widehat{\hat{X}}_{t},L_{t}(\Gamma_{t}^{1})U_{t}\rangle$
$\displaystyle+\langle
L_{t}(\Gamma_{t}^{1})U_{t},\widehat{\hat{X}}_{t}\rangle+trace\Big{(}Q_{t}\Sigma_{t}+Q_{t}(\Gamma_{t}^{1})K_{t}+R_{t}K_{Z_{t}}\Big{)},$
(III.69) $\displaystyle
Q_{t}(\Gamma_{t}^{1})\stackrel{{\scriptstyle\triangle}}{{=}}Q_{t}+(\Gamma_{t}^{1})^{T}R_{t}\Gamma_{t}^{1},\hskip
7.22743ptL_{t}(\Gamma_{t}^{1})\stackrel{{\scriptstyle\triangle}}{{=}}(\Gamma_{t}^{1})^{T}R_{t}.$
(III.70)

###### Proof.

(a) By definition of $I_{t}$ with $Y^{t}$ generated by (III.48), we obtain
(III.57). The rest of equations under (i) and (ii) follow by repeating the
derivation of Kalman-filter equations. (b) (III.68) follows from
$I(A_{t},\hat{X}_{t};Y_{t}|Y^{t-1})=H(Y_{t}|Y^{t-1})-H(\hat{I}_{t})$ from
(III.43), and $H(Y_{t}|Y^{t-1})$ is computed from (III.64). ${\cal
P}_{n}^{\widehat{\hat{X}}}(\kappa)$ follows from Lemma III.3, (III.55) by
${\bf E}\\{\hat{\gamma}_{t}(A_{t},\hat{X}_{t})\\}={\bf E}\\{{\bf
E}\\{\hat{\gamma}_{t}(A_{t},\hat{X}_{t})\big{|}Y^{t-1}\\}\\}$. ∎

#4. Separation Principle. Now, we show the statement of Section I-A2, under
2.3): the strategy $U_{t}(Y^{t-1})=\Gamma_{t}^{2}Y^{t-1},$ is determined
independently of $(\Gamma_{t}^{1}\hat{X}_{t},K_{Z_{t}}),\forall
t\in{\mathbb{Z}}_{+}^{n}$.

######  Theorem III.2.

(Decentralized separation principle)
Consider $C_{FB,n}(\kappa)$ of Theorem III.1.(b). Define the Cost-Rate, i.e.,
dual of $C_{FB,n}(\kappa)$, by

$\displaystyle\kappa_{n}(C)\stackrel{{\scriptstyle\triangle}}{{=}}\inf_{\\{\big{(}\Gamma_{t}^{1},U_{t},K_{Z_{t}}\succeq
0\big{)}\\}_{t=1}^{n}}{\bf
E}\Big{\\{}\sum_{t=1}^{n}\widehat{\hat{\gamma}}_{t}(U_{t},\widehat{\hat{X}}_{t},\Gamma_{t}^{1},K_{Z_{t}})\Big{\\}}$
$\displaystyle\mbox{such that (\ref{kf_m_1}) holds, }\hskip
7.22743pt\frac{1}{2}\sum_{t=1}^{n}\log\frac{\det(K_{I_{t}})}{\det(K_{\hat{I}_{t}})}\geq
nC.$ (III.71)

(a) Decentralized Separation Principle.
(i) The optimal strategy
$\\{U_{t}^{*}(\cdot,\Gamma^{1},K_{Z})|t\in{\mathbb{Z}}_{+}^{n}\\}$ is a
solution of the stochastic optimal control problem

$\displaystyle
J_{n}^{SC}(U^{*},\Gamma^{1},K_{Z})\stackrel{{\scriptstyle\triangle}}{{=}}\inf_{U(\cdot)}{\bf
E}\Big{\\{}\sum_{t=1}^{n}\widehat{\hat{\gamma}}_{t}(U_{t},\widehat{\hat{X}}_{t},\Gamma_{t}^{1},K_{Z_{t}})\Big{\\}}$
(III.72)

and it is given by the following equations.

$\displaystyle
U_{t}^{*}(Y^{t-1},\Gamma^{1},K_{Z})={\Gamma}_{t}^{2,*}\widehat{\hat{X}}_{t},\hskip
7.22743pt\forall t\in{\mathbb{Z}}_{+}^{n-1},$ (III.73)
$\displaystyle{\Gamma}_{t}^{2,*}=-\Big{(}R_{t}+B_{t}^{T}P_{t+1}B_{t}\Big{)}^{-1}\Big{(}\big{(}{L}_{t}(\Gamma_{t}^{1})\big{)}^{T}$
$\displaystyle+B_{t}^{T}P_{t+1}{F}_{t}(\Gamma_{t}^{1})\Big{)},\hskip
7.22743pt{\Gamma}_{n}^{2,*}=-R_{n}^{-1}\big{(}{L}_{n}(\Gamma_{n}^{1})\big{)}^{T}$
(III.74)

where $P_{t}\succeq 0$ satisfies the backward matrix DRE,

$\displaystyle
P_{t}=\big{(}{F}_{t}(\Gamma_{t}^{1})\big{)}^{T}P_{t+1}{F}_{t}(\Gamma_{t}^{1})-\Big{(}\big{(}{F}_{t}(\Gamma_{t}^{1})\big{)}^{T}P_{t+1}B_{t}+{L}_{t}(\Gamma_{t}^{1})\Big{)}$
$\displaystyle.\Big{(}R_{t}+B_{t}^{T}P_{t+1}B_{t}\Big{)}^{-1}\Big{(}\big{(}F_{t}(\Gamma_{t}^{1})\big{)}^{T}P_{t+1}B_{t}$
$\displaystyle+L_{t}(\Gamma_{t}^{1})\Big{)}^{T}+Q_{t}(\Gamma_{t}^{1}),\>\forall
t\in{\mathbb{Z}}_{+}^{n-1},\hskip 7.22743ptP_{n}=Q_{n}(\Gamma_{n}^{1}).$
(III.75)

The optimal cost-rate is given by

$\displaystyle
J_{n}^{SC}(U^{*},\Gamma^{1},K_{Z})=\langle\mu_{X_{1}},P_{1}\mu_{X_{1}}\rangle+\sum_{t=1}^{n}\Big{\\{}trace\Big{(}{F}_{t}^{CL}K_{I_{t}}({F}_{t}^{CL})^{T}P_{t}\Big{)}$
$\displaystyle\hskip
14.45377pt+trace\Big{(}Q_{t}\Sigma_{t}+Q_{t}(\Gamma_{t}^{1})K_{t}+R_{t}K_{Z_{t}}\Big{)}\Big{\\}}.$
(III.76)

(ii) The optimal
$\\{(\Gamma_{t}^{1,*},K_{Z_{t}}^{*})|t\in{\mathbb{Z}}_{+}^{n}\\}$ is
determined from (III.66), with ${\cal P}_{n}^{\widehat{\hat{X}}}(\kappa)$
replaced by $\frac{1}{n}J_{n}^{SC}(U^{*},\Gamma^{1},K_{Z})\leq\kappa$, i.e.,
$U(\cdot)=U^{*}(\cdot)$.

###### Proof.

See Section VI-B. ∎

######  Remark III.1.

The characterizations of $C_{FB,n}(\kappa)$ in [28, 32] correspond to
$B_{t}=0,Q_{t}=0,R_{t}=I,\forall t$. Using these values in $U^{*}(\cdot)$ of
(III.73), (III.74), and $C_{FB,n}(\kappa)$ of Theorem III.1, the optimal input
reduces to
$A_{t}=\Gamma_{t}^{1}\big{(}\hat{X}_{t}-\widehat{\hat{X}}_{t}\big{)}+Z_{t},\forall
t\in{\mathbb{Z}}_{+}^{n}$, with corresponding $C_{FB,n}(\kappa)$, which
coincides with the expression in [28, 32].
Reference [28] discusses relations to the expressions of $C_{FB,n}(\kappa)$
derived in [12, 34, 13, 14].

##  IV Asymptotic Limit

In this section we consider the limit
$C_{FB}(\kappa)=\lim_{n\longleftrightarrow\infty}\frac{1}{n}C_{FB,n}(\kappa)$,
based on Assumptions IV.1.

######  Assumptions IV.1.

Asymptotically time-invariant
(1) The LQG-POSS (I.15)-(I.20) is asymptotically time-invariant (ATI),
$\lim_{n\longrightarrow\infty}(F_{n},B_{n},C_{n},D_{n},N_{n},K_{W_{n}},R_{n},Q_{n})=(F,B,C,D,N,K_{W},R,Q)$.
(2) The controller-encoder strategies are ATI,
$\lim_{n\longrightarrow\infty}(\Gamma_{n}^{1},K_{Z_{n}})=(\Gamma^{1},K_{Z}),K_{Z}\succeq
0$.
(3) The solutions to the matrix DREs $(\Sigma_{n},K_{n},P_{n})$ are such that
$\lim_{n\longrightarrow\infty}(\Sigma_{n},K_{n},P_{n})=(\Sigma,K,P)$, where
$\Sigma\succeq 0,K\succeq 0,P\succeq 0$ are unique stabilizing solutions of
corresponding matrix algebraic Riccati equations (AREs),

$\displaystyle\Sigma=$ $\displaystyle F\Sigma F^{T}-\Big{(}F\Sigma
C^{T}+GK_{W}N^{T}\Big{)}\Big{(}NK_{W}N^{T}+C\Sigma C^{T}\Big{)}^{-1}$
$\displaystyle\Big{(}F\Sigma C^{T}+GK_{W}N^{T}\Big{)}^{T}+GK_{W}G^{T},\hskip
7.22743pt\Sigma\succeq 0,$ (IV.77) similarly for $(K,P)$. (IV.78)

######  Theorem IV.1.

Asymptotic char. of ${C}_{FB}(\kappa)$ for LQG-POSSs
Consider the LQG-POSS (I.15)-(I.20), and suppose Assumptions IV.1 hold. The CC
capacity $C_{FB}(\kappa)$ is given by

$\displaystyle
C_{FB}(\kappa)=\lim_{n\longleftrightarrow\infty}\frac{1}{n}C_{FB,n}(\kappa)=\sup_{{\cal
P}_{\infty}^{\widehat{\hat{X}}}(\kappa)}\frac{1}{2}\log\frac{\det(K_{I})}{\det(K_{\hat{I}})}$
(IV.79) $\displaystyle=\sup_{{\cal
P}_{\infty}^{\widehat{\hat{X}}}(\kappa)}\frac{1}{2}\log\Big{\\{}\frac{\det\Big{(}C(\Gamma^{1})K(C(\Gamma^{1})^{T}+K_{\hat{I}}+DK_{Z}D^{T}\Big{)}}{\det\big{(}K_{\hat{I}}\big{)}}\Big{\\}},$
$\displaystyle{\cal
P}_{\infty}^{\widehat{\hat{X}}}\stackrel{{\scriptstyle\triangle}}{{=}}\Big{\\{}(\Gamma^{1},K_{Z})\big{|}\>K_{Z}\succeq
0,\hskip 7.22743pttrace\Big{(}{F}^{CL}K_{I}({F}^{CL})^{T}P\Big{)}$
$\displaystyle\hskip 14.45377pt\hskip
14.45377pt+trace\Big{(}Q\Sigma+Q(\Gamma^{1})K+RK_{Z}\Big{)}\leq\kappa\Big{\\}},$
s.t. $\Sigma\succeq 0,K\succeq 0,P\succeq 0$ satisfy the matrix AREs

where the limit is the uniform limit over all initial conditions, i.e., is
independent of $\Sigma_{1},P_{1},K_{1}$.
Moreover, sufficient conditions for Assumptions IV.1.(3) to hold, are (i)
detectability and (ii) stabilizability of the AREs.

###### Proof.

Using Theorem III.2 and Theorem III.1.(b) and Assumptions IV.1, we repeat the
proof given in [35] to show the limit over $n\longrightarrow\infty$ and
supremum over the strategies can be interchanged, the asymptotic limit is a
uniform limit over all initial conditions. The fact that (i) detectability and
(ii) stabilizability of the matrix AREs are sufficient for Assumptions
IV.1.(3) to hold follows from the continuous dependence of solutions of matrix
DREs on the parameters. ∎

##  V Conclusion

In this paper we applied the concepts of information state and sufficient
statistic to characterize feedback capacity $C_{FB}$ of nonlinear partially
observable stochastic systems, with input dependent states, which are not
available to the encoder and the decoder. In addition, we applied these
concepts to linear-quadratic-Gaussian partially observable stochastic systems
(LQG-POSS). For LQG-POSS, $C_{FB}$ is expressed in terms of 2 filtering
Riccati equations and 1 control Riccati equation. For certain special cases we
recover recent expressions of $C_{FB}$ that appeared in [28, 32], for Gaussian
channels with memory, without state dependent inputs.

##  VI Appendix

###  VI-A Definition of Information State and Sufficient Statistic

######  Definition VI.1.

(Information state and sufficient statistic)
(a) The a posteriori distribution $\\{{\bf
P}_{t}^{P}(dx_{t}|a^{t-1},y^{t-1})|t\in{\mathbb{Z}}_{+}^{n}\\}$ is called an
information state if the next state ${\bf P}_{t+1}^{P}(dx_{t+1}|a^{t},y^{t})$
is determined from $(y_{t},a_{t})$, the current state ${\bf
P}_{t}^{P}(dx_{t}|a^{t-1},y^{t-1})$, and possibly
${P}_{t}(da_{t}|a^{t-1},y^{t-1})$, $\forall t\in{\mathbb{Z}}_{+}^{n}$.
(b) A statistic
$\\{\delta_{t}|t\in{\mathbb{Z}}_{+}^{n}\\},\delta_{t}:\Omega\rightarrow{\mathbb{D}}$
is called a sufficient statistic for the strategies $P(\cdot|\cdot)\in{\cal
P}_{n}(\kappa)$ if $P_{i}(da_{t}|a^{t-1},y^{t-1})$ is induced by
$A_{t}=\mu_{i}(\delta_{t})$ for some measurable function $\mu_{t}(\cdot)$,
$\forall t\in{\mathbb{Z}}_{+}^{n}$.
(c) The strategies $P(\cdot|\cdot)\in{\cal P}_{n}(\kappa)$ are called
separated strategies if they are generated by
$A_{t}={\mu}_{t}^{z}(\xi_{t}(A^{t-1},Y^{t-1}),Z_{t})$, where
$\xi_{t}(a^{t-1},y^{t-1})\stackrel{{\scriptstyle\triangle}}{{=}}{\bf
P}_{t}^{P}(dx_{t}|a^{t-1},y^{t-1})$, $\mu_{t}^{z}(\cdot)$ is a measurable
function, and $Z_{t}:\Omega\rightarrow{\mathbb{Z}}$ is a RV (responsible for
the randomization), $\forall t\in{\mathbb{Z}}_{+}^{n}$ (i.e.,
$\delta_{t}=(\xi_{t}(A^{t-1},Y^{t-1}),Z_{t})$ is a sufficient statistic). The
set of separated strategies is denoted by ${\cal P}_{n}^{sep}(\kappa)$.

###  VI-B LQG-POSS

Proof of Lemma III.3 We show (III.47), (III.48) last. First, (III.44) is a re-
statement of the first equality in (III.33) in Lemma III.1.(a).(i). The
conditional distributions (III.45), (III.46) follow directly from (III.44),
and the orthogonality of $\hat{I}_{t}$ and $(A^{t},Y^{t-1})$, as follows.

$\displaystyle{\mathbb{P}}\big{\\{}Y_{t}\in
dy\Big{|}Y^{t-1},X^{t}\big{\\}}={\bf P}_{t}(dy|Y^{t-1},A^{t}),$
$\displaystyle=$ $\displaystyle{\bf
P}_{t}(dy|A^{t},Y^{t-1},\hat{X}^{t}),\hskip 7.22743pt\mbox{by
$\hat{X}_{t}={\bf E}\big{\\{}X_{t}\Big{|}Y^{t-1},A^{t-1}\big{\\}}$}$
$\displaystyle=$ $\displaystyle{\bf P}_{t}(dy|A_{t},\hat{X}_{t}),\hskip
7.22743pt\mbox{by (III.44)}.$

The equivalent representation of $\hat{\gamma}_{t}(A_{t},\hat{X}_{t})$ in the
average cost (III.51) is obtain by reconditioning, ${\bf
E}\big{\\{}\sum_{t=1}^{n}{\gamma}_{t}(A_{t},{X}_{t})\big{\\}}=\sum_{t=1}^{n}{\bf
E}\big{\\{}{\bf
E}\big{\\{}{\gamma}_{t}(A_{t},{X}_{t})\big{|}A^{t},Y^{t-1}\big{\\}}\big{\\}}$,
using the quadratic definition of $\gamma_{i}(\cdot)$ of (I.16), Lemma III.1,
and the Markov chain $X_{t}\leftrightarrow(A^{t-1},Y^{t-1})\leftrightarrow
A_{t}$. This shows the new channel (III.44) is linear in $(\hat{X}_{t},A_{t})$
and cost is quadratic in $(\hat{X}_{t},A_{t})$. Since ${\bf
P}_{\hat{X}_{t}|\hat{X}^{t-1},A^{t-1}}={\bf
P}_{\hat{X}_{t}|\hat{X}_{t-1},A_{t-1}}$, and $I^{P}(A^{n}\rightarrow
Y^{n})={\bf E}^{P}\big{\\{}\log\big{(}\frac{{\bf
P}_{t}(dY_{t}|Y^{t-1},A^{t})}{{\bf
P}_{t}(dY_{t}|Y^{t-1})}\big{)}\big{\\}}={\bf E}\big{\\{}\log\big{(}\frac{{\bf
P}_{t}(dY_{t}|A_{t},\hat{X}_{t})}{{\bf
P}_{t}(dY_{t}|Y^{t-1})}\big{)}\big{\\}}$, by Markov decision theory the
optimization over ${\cal P}_{n}(\kappa)$ of $I(A^{n}\rightarrow Y^{n})$ occurs
in the set ${\cal P}_{n}^{\hat{X}}(\kappa)$, i,.e., (III.47). holds. By
(III.47), ${\bf P}_{t}(da_{t}|\hat{x}_{t},y^{t-1})$ is induced by the
realization (III.48). Hence, we obtain (III.50) we use the distributions
(III.45)-(III.47).

Proof of Theorem III.2. (a) In $C_{FB,n}(\kappa)$ defined by (III.62), the
terms $I(A_{t},\hat{X}_{t};Y_{t}|Y^{t-1})$ given by (III.63) do not depend on
the strategy $U_{t}(\cdot),\forall t\in{\mathbb{Z}}_{+}^{n}$, hence we can
apply the person-by-person optimality concepts stated under (i) and (ii). (b)
We express
$\widehat{\hat{\gamma}}_{t}(U_{t},\widehat{\hat{X}}_{t},\Gamma_{t}^{1},K_{Z_{t}})\big{)}$
defined by (III.65) as follows.

$\displaystyle\widehat{\hat{\gamma}}_{t}(U_{t},\widehat{\hat{X}}_{t},\Gamma_{t}^{1},K_{Z_{t}})=$
$\displaystyle\left[\begin{array}[]{c}\widehat{\hat{X}}_{t}\\\
U_{t}\end{array}\right]^{T}\left[\begin{array}[]{cc}Q_{t}(\Gamma_{t}^{1})&L_{t}(\Gamma_{t}^{1})\\\
\big{(}L_{t}(\Gamma_{t}^{1})\big{)}^{T}&R_{t}\end{array}\right]\left[\begin{array}[]{c}\widehat{\hat{X}}_{t}\\\
U_{t}\end{array}\right]$ (VI.86) $\displaystyle+\mbox{additional terms}.$
(VI.87)

Then we have a linear-quadratic Gaussian stochastic optimal control problem,
and its solution is the one stated under (i) [6]. Since we determined
$U^{*}(\cdot)$ it remains to solve $C_{FB,n}(\kappa)$ as stated under (ii).

## References

  * [1] C. E. Shannon, “A mathematical theory of communication,” _Bell Syst. Tech._ , vol. 27, pp. 379–423, July 1948.
  * [2] R. T. Gallager, _Information Theory and Reliable Communication_. John Wiley & Sons, Inc., New York, 1968.
  * [3] S. Ihara, _Information theory for Continuous Systems_. World Scientific, 1993.
  * [4] T. M. Cover and J. A. Thomas, _Elements of Information Theory_ , 2nd ed. John Wiley & Sons, Inc., Hoboken, New Jersey, 2006.
  * [5] P. R. Kumar and P. Varaiya, _Stochastic Systems: Estimation, Identification, and Adaptive Control_. Prentice Hall, 986.
  * [6] P. E. Caines, _Linear Stochastic Systems_ , ser. Wiley Series in Probability and Statistics. John Wiley & Sons, Inc., New York, 1988.
  * [7] A. Bensoussan, _Stochastic control of partially observable systems_. Cambridge University Press, 1992.
  * [8] R. J. Elliott, L. Aggoun, and J. B. Moore, _Hidden Markov Models: Estimation and Control_. Springer-Verlag, Berlin, Heidelberg, New York, 1995.
  * [9] J. Wolfowitz, “Signalling over a gaussian channel with feedback and autoregressive noise,” _Journal of Applied Probability_ , vol. 12, no. 4, pp. 713–723, 1975.
  * [10] T. Cover and S. Pombra, “Gaussian feedback capacity,” _IEEE Transactions on Information Theory_ , vol. 35, no. 1, pp. 37–43, Jan. 1989.
  * [11] S. Yang, A. Kavcic, and S. Tatikonda, “Feedback capacity of finite-state machine channels,” _Information Theory, IEEE Transactions on_ , vol. 51, no. 3, pp. 799–810, March 2005.
  * [12] ——, “On feedback capacity of power-constrained Gaussian noise channels with memory,” _Information Theory, IEEE Transactions on_ , vol. 53, no. 3, pp. 929–954, March 2007.
  * [13] Y.-H. Kim, “Feedback capacity of stationary Gaussian channels,” _IEEE Transactions on Information Theory_ , vol. 56, no. 1, pp. 57–85, 2010.
  * [14] A. Gattami, “Feedback capacity of Gaussian channels revisited,” _IEEE Transactions on Information Theory_ , vol. 65, no. 3, pp. 1948–1960, March 2019\.
  * [15] C. D. Charalambous, C. Kourtellaris, and I. Tziortzis, “Optimal control and signaling strategies of control-coding capacity of general decision models: Applications to gaussian models and decentralized strategies,” _SIAM Journal on Control and Optimization_ , vol. 62, no. 1, pp. 600–629, 2024.
  * [16] L. Galluccio, A. Lombardo, G. Morabito, S. Palazzo, C. Panarello, and G. Schembra, “Capacity of a binary droplet-based microfluidic channel with memory and anticipation for flow-induced molecular communications,” _IEEE Transactions on Communications_ , vol. 66, no. 1, pp. 194–208, January 2018.
  * [17] N. Farsad, E. A. Yilmaz, H. B., C.-B. Chae, , and W. Guo, “A comprehensive survey of recent advancements in molecular communication,” _IEEE Communications Surveys and Tutorials_ , vol. 18, no. 1, pp. 1887–1919, February 2016.
  * [18] J. Chen and T. Berger, “The capacity of finite-state Markov channels with feedback,” _IEEE Transactions on Information Theory_ , vol. 51, no. 3, pp. 780–798, March 2005.
  * [19] S. Tatikonda and S. Mitter, “The capacity of channels with feedback,” _IEEE Transactions on Information Theory_ , vol. 55, no. 1, pp. 323–349, Jan. 2009.
  * [20] H. Permuter, P. Cuff, B. Van Roy, and T. Weissman, “Capacity of the trapdoor channel with feedback,” _IEEE Transactions on Information Theory_ , 2010\.
  * [21] O. Elishco and H. Permuter, “Capacity and coding of the ising channel with feedback,” _IEEE Transactions on Information Theory_ , vol. 60, no. 9, pp. 3138–5149, June 2014.
  * [22] H. H. Permuter, H. Asnani, and T. Weissman, “Capacity of a post channel with and without feedback,” _IEEE Transactions on Information Theory_ , vol. 60, no. 10, pp. 6041–6057, Oct 2014.
  * [23] M. Pinsker, _Information and Information Stability of Random Variables and Processes_. Holden-Day Inc, San Francisco, 1964, translated by Amiel Feinstein.
  * [24] I. I. Gihman and A. V. Skorohod, _Controlled Stochastic Processes_. Springer-Verlag, 1979.
  * [25] S. Striebel, “Sufficient statistics in the optimum control of stochastic systems,” _Journal of Mathematical Analysis and Applications_ , vol. 12, pp. 576–592, 1982.
  * [26] R. E. Mortensen, “Stochastic optimal control with noisy observations,” _International Journal on Control_ , vol. 4, pp. 455–464, 1966.
  * [27] C. D. Charalambous, “The role of information state and adjoint in relating nonlinear output feedback risk-sensitive control and dynamic games,” _IEEE Transactions on Automatic Control_ , vol. 42, no. 8, pp. 1163–1170, 1997.
  * [28] C. D. Charalambous, C. Kourtellaris, and S. Louka, “New formulas of feedback capacity for AGN channels with memory: A time-domain sufficient statistic approach.” [Online]. Available: https://arxiv.org/abs/2010.06226
  * [29] J. L. Massey, “Causality, feedback and directed information,” in _International Symposium on Information Theory and its Applications (ISITA ’90)_ , Nov. 27-30 1990, pp. 303–305.
  * [30] H. Permuter, T. Weissman, and A. Goldsmith, “Finite state channels with time-invariant deterministic feedback,” _IEEE Transactions on Information Theory_ , vol. 55, no. 2, pp. 644–662, Feb. 2009.
  * [31] Y.-H. Kim, “A coding theorem for a class of stationary channels with feedback,” _IEEE Transactions on Information Theory_ , vol. 54, no. 4, pp. 1488–1499, 2008.
  * [32] C. D. Charalambous, C. Kourtellaris, and S. Louka, “Sequential characterization of Cover and Pombra Gaussian feedback capacity: Generalizations to MIMO channels via a sufficient statistic,” in _Proceedings of the 2021 IEEE Information Theory Workshop (ITW)_. IEEE, October 17-21 2021.
  * [33] C. D. Charalambous, C. Kourtellaris, and S. Loyka, “Capacity achieving distributions and separation principle for feedback Gaussian channels with memory: The LQG theory of directed information,” _IEEE Transactions on Information Theory_ , vol. 64, no. 9, pp. 6384–6418, September 2018.
  * [34] Y.-H. Kim, “Feedback capacity of the first-order moving average Gaussian channel,” _IEEE Transactions on Information theory_ , vol. 52, no. 7, pp. 3063–3079, 2006.
  * [35] C. D. Charalambous and S. Louka, “A Riccati-Lyapunov approach to nonfeedback capacity of MIMO Gaussian channels driven by stable and unstable noise,” in _in Proceedings of the 2022 IEEE Information Theory Workshop (ITW)_. IEEE, November 6-9 2022, pp. 184–189, mumbai, India.


# Non-Parallel Voice Conversion for ASR Augmentation

###### Abstract

Automatic speech recognition (ASR) needs to be robust to speaker differences.
Voice Conversion (VC) modifies speaker characteristics of input speech. This
is an attractive feature for ASR data augmentation. In this paper, we
demonstrate that voice conversion can be used as a data augmentation technique
to improve ASR performance, even on LibriSpeech, which contains 2,456
speakers. For ASR augmentation, it is necessary that the VC model be robust to
a wide range of input speech. This motivates the use of a non-autoregressive,
non-parallel VC model, and the use of a pretrained ASR encoder within the VC
model. This work suggests that despite including many speakers, speaker
diversity may remain a limitation to ASR quality. Finally, interrogation of
our VC performance has provided useful metrics for objective evaluation of VC
quality.

Index Terms: Voice Conversion, Automatic Speech Recognition

##  1 Introduction

It is critical for automatic speech recognition (ASR) to be robust to speaker
differences. This is typically addressed by training on speech from a wide
variety of speakers. In this paper, we show that despite being trained on 1000
hours of speech, augmenting speech with diverse speaker characteristics can
improve speech recognition performance.

Voice conversion (VC) models convert input speech from its source speech to
some target speaker, transferring the speaker timbre and other characteristics
while retaining the lexical content of the source speech. While there has been
successful approaches to voice conversion in recent years (Section 2), VC
performance is typically evaluated on clean, in-domain speech. That is, the
same style of speech used in training the VC model is used for evaluation. We
have found that this can lead to VC systems that fail to generalize to speech
and speakers that are substantially different than the training material
(Section 5.3).

Since ASR training data is typically more diverse than VC training data,
constructing a robust VC system is essential for ASR augmentation. ASR systems
have to interact with speech from a variety of recording conditions and a
broad range of speakers including speakers with less common accents.

In service of this robustness we make three important VC design choices.
First, we pursue a non-parallel voice conversion approach (Section 3). By not
requiring parallel data for training, we can train on much more data than we
could otherwise. Second, we use a non-autoregressive decoder. We found that
autoregressive models were susceptible to attention failures which were
catastrophic for their application as a data augmentation technique. Third, we
initialize the VC encoder with a pre-trained ASR encoder. This choice
substantially improves the robustness of the VC model to diverse inputs
(Section 5), while moderately limiting VC naturalness. For ASR augmentation,
we find that this substantial improvement to robustness is more important than
a marginal improvement to naturalness.

The main contribution of this paper are as follows:

  * •

We show that voice conversion can be used to successfully augment speech
recognition training data. We find relative word error rate (WER) improvements
up to $6\%$ when augmenting LibriSpeech which contains 2,456 speakers.

  * •

We show that we can substantially improve the robustness of voice conversion
systems to unseen corpora and different speaker characteristics by
initializing the VC encoder with an ASR encoder.

  * •

We demonstrate VC metrics that can be used for model selection prior to
subjective listening tests.

Figure 1: Schematic diagram of proposed voice conversion system.

##  2 Related Work

Non-parallel voice conversion can be broadly categorized into two major
approaches, the first using Phone posteriogram (PPG) features [1, 2]. PPGs
acts as speaker-invariant representations that can be easily utilized to
achieve conversion. The other main approach is with auto-encoding style
training approaches, utilizing both variational auto-encoder (VAE)[3, 4, 5, 6,
7, 8] and vector-quantized VAE (VQ-VAE) [9, 10, 11, 12] approaches. The auto-
encoding style training approaches try to disentangle speaker speaker
information from the content of source speech, using various methods from
bottlenecking to adversarial training to prevent speaker leakage in the
content or speech encoder. In [7], instead of adversarial training, carefully
designed bottlenecks were coupled with auto-encoding loss to train the VC
system. In [12], this is achieved by VQ-VAE of the speech encoder outputs.

Recent approaches have utilized pre-trained speech encoder to extract speaker
agnostic representation. In [13], ASR trained to predict PPGs are used on
source speech to achieve accent conversion. In [14], a trained ASR encoder is
used as the speech encoder, along with explicit prosody modelling to achieve
speaker and prosody in VC. In [15], accent and speaker disentanglement and
control in VC was achieved in stages, where first stage is accent-agnostic ASR
training, from which the trained speech encoder is used to further conduct VC
training and disentangle accent and speaker.

Previous work on voice conversion and morphing augmentations to help augment
ASR training include Vocal Tract Length Perturbation (VTLP) [16, 17]. More
recent approaches use neural VC models to augment speech. In [18], a VC model
is used to augment adult speech into children speech to help with children
speech ASR. However in this work, the VC model was a Cycle-GAN[19]
architecture trained on the same in-domain data on which the ASR training was
also conducted with. In [20], a VC system was utilized as augmentation for low
resource languages. This system utilizes representations from a pretrained
encoder with CPC[21] objective, instead of one trained on ASR targets. The VC
model augments speech in the very low data regime.

##  3 Non-Parallel Voice Conversion

For our proposed VC model, we utilize an architecture consisting of an
encoder, a bottleneck layer, and a non auto-regressive decoder. Figure 1 shows
the architecture of the VC model.

Encoder: The VC encoder is similar to the encoder in the Conformer[22] ASR
model. Specifically, the encoder consists of sub-sampling convolution layer
that reduces the time dimension of the input speech by 4x. This is then
followed by a stack of conformer layers (16 layers), followed by a output
layer norm layer. We study two versions of encoders, the first being an
conformer encoder that is trained from scratch jointly with the rest of the
system. The second version being a frozen encoder, where the weights are
obtained from an ASR model trained on 960 hours of Librispeech.

Speaker Embedding: Speaker information is provided in the form of one-hot
vectors, which are passed through a learned speaker embedding that maps from
one-hot vectors to a dense 256-dim embedding vector.

Decoder: The decoder is a non-autoregressive decoder composed of RNN layers
followed by convolution layers. Speaker embedding is provided along with the
encoder output to the decoder. Specifically, the dense speaker embedding
vector is copied $T$ times across time and then concatenated together with
encoder outputs to be fed to the decoder. The decoder consists of 2 bi-
directional LSTM layers of size 1024 followed by a stack of convolution
layers. During development, we also explored an auto-regressive, attention-
based decoder. However, we found that during inference even within-domain
speech would sometimes show attention failures leading to extremely poor
conversion [23]. While there are ways to mitigate these failures, since
robustness is necessary for ASR augmentation we pursued the less error-prone,
non-autoregressive approach.

VQVAE Bottleneck: A Vector Quantized Variational Auto Encoding (VQVAE)
Bottleneck [24] is used to quantize encoder outputs into discrete codebook
entries. This restricts information through the VC system during auto-encoding
training. The VQVAE bottleneck typically contain 3 loss terms, the
reconstruction loss, codebook loss and commitment loss.

$\displaystyle L_{codebook}$ $\displaystyle=$
$\displaystyle||sg[z_{e}(x)]-e||^{2}_{2}$ (1) $\displaystyle L_{commit}$
$\displaystyle=$ $\displaystyle\beta||z_{e}(x)-sg[e]||_{2}^{2}$ (2)

Reconstruction loss is computed on the decoder, thus we simply add codebook
and commitment loss from VQVAE to the overall loss objective during training.
We fix VQVAE codebook size to 128 and group size to be 2.

Adversarial Speaker Classification: To disentangle speaker information, we
apply adversarial speaker classification loss on the VQVAE codebook outputs to
further ensure no speaker leakage. Given encoder outputs $m_{enc}(x)$ and
speaker labels $l_{speaker}$, we combine reverse gradient operation $R_{grad}$
along with softmax cross entropy loss as follows:

$L_{adv}=CE(R_{grad}(m_{enc}(x)),l_{speaker})$ (3)

In effect, the training will try to maximize $L_{adv}$, which in effect will
remove as much speaker information from the VQVAE output as possible. We scale
adversarial gradient by weight, and find that this adversarial gradient weight
term requires careful tuning, too small and it does nothing, too large and it
overwhelms the reconstruction loss, see section 5.5 for details.

Objective Function We train on non-parallel data, and only utilize speech
features and speaker embedding for the loss computation. To enable the VC
models to be used as ASR data augmentation, we train our VC model to predict
ASR log-mel features directly. During VC training, the same log-mel features
are used as input and target. We use Huber loss for reconstruction loss. The
model is jointly optimized by minimizing $L_{total}$, with all loss weights
set to 1.0.

$\displaystyle L_{recon}$ $\displaystyle=$
$\displaystyle\left\\{\begin{matrix}\frac{1}{2}(y-\hat{y})^{2}&if\left|(y-\hat{y})\right|<\delta\\\
\delta((y-\hat{y})-\frac{1}{2}\delta)&otherwise\end{matrix}\right.$ (4)
$\displaystyle L_{total}$ $\displaystyle=$ $\displaystyle L_{recon}+\gamma
L_{codebook}+\epsilon L_{commit}+\eta L_{adv}$ (5)

Frozen ASR Encoder In addition to training the full VC model (Figure 1) from
scratch, we replace the encoder network with the encoder of an ASR model. ASR
models implicitly normalize out speaker differences. ASR outputs are not
(explicitly) dependent on speaker characterstics so they are less likely to be
retained in the internal representations of the ASR network. We use an ASR
encoder as the VC encoder, and freeze its weights, training only the decoder,
VQVAE and speaker embedding. By freezing the encoder, it retains the
representations learned during ASR training which leads to improved
robustness.

##  4 Data

VC Corpora: Three TTS corpora were used for training the VC model. The first
LS is the publicly available LibriSpeech Corpus containing 960 Hours of speech
with 2,456 speakers [25]. The second EN-VC is an in-house corpus containing
professional English speakers from 6 English locales, including United States
(en/us), Britain (en/gb), India (en/in), Singapore (en/sg), Nigeria (en/ng)
and Australia (en/au). This corpus contains 58 speakers, totaling around 400
hours of studio quality speech intended for Text-to-speech (TTS) training and
research and hence was recorded at 48KHz. The third corpus SV-VC is, similar
to the second, an in-house Swedish (sv/se) dataset collected for TTS training
and research, which contains 21 hours of across 6 speakers. The data
distribution between speakers is skewed, with speaker hours ranging from 0.5
hours to 40 hours. All datasets are downsampled to 16kHz, and we use 80-dim
mel spectrum as features, with a frame size of 25ms and a 10ms frame shift.

ASR Corpora: For ASR, three training datasets were used in this work. The
first is LS the public LibriSpeech[25] corpus. The other two datasets are in-
house data sets comprising of anonymized short utterances representative of
Google’s voice search (VS) traffic in two languages, English EN-ASR and
Swedish SV-ASR. While the choice of English allows us to analyze two very
different domains, namely, audio books and voice search with similar amounts
of training data( 1000 hours), the choice of Swedish( 6500 hours) allows us to
explore some transfer effects. While the LS corpus used in VC and ASR training
is identical, the in-house VC (EN-VC and SV-VC) and ASR corpora (EN-ASR and
SV-ASR) are completely disjoint.

##  5 Experiments & Results

###  5.1 VC Model Training

The VC model as described in Section 3 is trained with constant learning rate
of 1e-4 for 800k steps with Adam optimizer. Note that the VC model is trained
on studio quality professional speech and has not seen any ASR training data.
When using a frozen ASR encoder, the encoder is initialized from a LibriSpeech
trained ASR encoder. For the LibriSpeech ASR experiments, the VC model is
trained on Librispeech dataset. For in-house datasets EN-ASR and SV-ASR, we
train VC models on our respective internal TTS datasets (Section 4).

For LS-ASR experiments, the VC model is trained on Librispeech data. For EN-
ASR experiments, the VC model is trained on our internal en-us TTS data. For
SV-ASR experiments, we train VC model on both our internal US and Swedish TTS
datasets to have access to more speakers. Note, that the SV VC model is
initialized with the LibriSpeech ASR encoder. This investigates the use of an
ASR encoder as a proxy for a language agnostic representation for VC.

###  5.2 VC for ASR Augmentation

When using VC as an ASR augmentation technique, we follow [26, 27] for ASR
training. Two views of the same data is presented to the ASR model, original
and VC converted speech. Additionally a decoder consistency term is introduced
to force model prediction to be consistent for the two views.

We apply SpecAugment to both views. During VC inference, we uniform randomly
sample speaker ids from the pool of VC training speakers whether LS, EN-VC or
SV-VC to generate augmented versions of the ASR training data. No speaker
information from the ASR training data is used, nor is it available for EN-ASR
or SV-ASR. The VC model is frozen during ASR training and used purely as an
augmentation technique.

###  5.3 VC Quality Analysis

For analyzing VC quality, we train two models on EN-VC, one using a frozen ASR
encoder (ASR Encoder) and one from scratch (VC Encoder).

We train a separate WaveRNN neural vocoder for each VC model that maps from
predicted log-mel features to 16kHz waveform. We analyze conversion quality
with 3 measures, Mean Opinion Score Naturalness (MOS), Speaker Similarity, and
Word Error Rate (WER) scored with in-house ASR system as a proxy for
intelligibility. MOS and Similarity are rated on a 5 point Likert scale. For
all evaluations, we convert from 5 example utterances from 20 source speakers
to 5 target speakers, totaling 500 utterances.

Table 1: MOS naturalness, Speaker Similarity and conversion word-error-rate for In-Domain, In-Locale Conversion Method | MOS | Similarity | WER
---|---|---|---
| Groundtruth
---
4.347 ± 0.049 | - | 10.9
|  VC Encoder
---
4.001 ± 0.069 | 4.227 ± 0.059 | 14.4
|  ASR Encoder
---
3.611 ± 0.075 | 4.419 ± 0.051 | 16.0

In-Domain Conversion Our first analysis evaluates in-domain conversion, where
source speakers and target speakers come from the same US English locale and
corpus (EN-VC) (Table 5.3). When comparing VC using a trained encoder to the
Frozen ASR encoder, we find a higher MOS and lower WER from the converted
speech, but we find lower speaker similarity. This confirms the hypothesis
that the frozen ASR encoder reliably eliminates speaker information from its
output representation.

Cross-Locale Conversion To evaluate cross-local conversion, the source
speakers consists of 5 non-US English locales (see Section 4), converting to 5
US English speakers. Since ASR training and evaluation data often includes a
variety of accents, this evaluates how robust our VC system will be to
differently accented speech.

Table 2 shows that both models result in less natural conversion (MOS) though
the ASR Encoder mitigates this regression. Moreover, the ASR Encoder shows
better robust speaker conversion (Similarity) and better WER.

Table 2:  MOS naturalness, Speaker Similarity and conversion word-error-rate for Cross-Locale Conversion Method | MOS | Similarity | WER
---|---|---|---
| Groundtruth
---
4.347 ± 0.049 | - | 12.0
|  VC Encoder
---
3.344 ± 0.094 | 2.629 ± 0.086 | 39.4
|  ASR Encoder
---
3.281 ± 0.088 | 3.494 ± 0.098 | 34.7

Out-Of-Corpus Conversion For out-of-corpus conversion, the source speech
consists of 20 random speakers from LibriTTS test-other data. The VC training
data is EN-VC. Table 3 shows that when encountering out-of-domain input
speech, the VC Encoder system attains worse MOS score and much higher
conversion WER as compared to the ASR Encoder system. We note that while the
VC system was trained on EN-VC, the ASR Encoder has only seen LibriSpeech
training utterances.

Table 3:  MOS naturalness, Speaker Similarity and conversion word-error-rate for Out-of-Corpus Conversion Method | MOS | Similarity | WER
---|---|---|---
| Groundtruth
---
4.347 ± 0.049 | - | 10.4
|  VC Encoder
---
2.336 ± 0.087 | 2.549 ± 0.08 | 49.2
|  ASR Encoder
---
2.982 ± 0.088 | 4.068 ± 0.071 | 21.0

These three experiments demonstrate that the use of a pretrained, frozen ASR
encoder in VC leads to substantial improvements to VC robustness to accent and
recording condition.

###  5.4 VC Augmentation & ASR Training

We now present our results of using VC as general ASR augmentation strategy
across 3 datasets (Section 4): LS, EN-ASR and SV-ASR. Due to the low quality
conversion of out-of-locale, and out-of-corpus sampled, all VC systems across
3 experiments use a Librispeech-trained ASR encoder as the VC encoder.

For LibriSpeech experiments, we use a conformer RNN transducer model as the
ASR model. The ASR model consists of 16 layers of conformer layers in the
encoder, each with conformer dimension of 16 and 4 attention heads. The
decoder consists of a RNN transducer with RNN dimension of size 320. The
system is trained for up to 120k steps with Adam optimizer, with annealed
learning rate to 5e-4. VC augmentation is described in section 5.2. Table 4
shows that VC augmentation improves WER by 0.2 and 0.1 absolute on test-clean
and test-other. The LibriSpeech training data contains speech from 2,456
speakers, and still the augmentation provided by VC within a consistent data
augmentation framework [26] is able to provide improved performance.

Table 4:  VC Augmentation Results on Librispeech Method | LS test | LS test-other
---|---|---
| Baseline
---
3.0 | 6.8
| \+ VC Augmentation
---
2.8 | 6.7

For EN-ASR experiments, the ASR system consists of a Conformer [22] RNN
transducer model of approximately 600 Million parameters. The ASR encoder
consists of 24 layers of conformer layers each with 1024 dimension and 8
attention heads. The decoder consists of a RNN transducer with 2 Bi-
directional LSTM layers each with 2048 dimension. The encoder has been pre-
trained via self-supervised learning on LibriLight data along with unspoken
text injection via tts as laid out in [28]. The baseline system is fine-tuned
on EN-ASR.

The ASR model used in the Swedish ASR (SV-ASR) experiment is a Conformer
transducer model of approximately 120 Million parameters. The encoder consists
of 12 Conformer[22] layers, each with conformer dimension of 512 encoder
dimension and 8 attention heads. The decoder consists of an embedding based
decoder [29], with embedding dimension of 640 dimension. The model is trained
with random initialization with Adam optimizer for 300k steps, with a learning
rate of 5e-4. VC augmentation is utilized in the set up as described in
section 5.2. The VC model is trained on both EN-VC and SV-VC data. When
initializing the VC encoder with an ASR Encoder, this is the same Librispeech-
trained ASR Encoder used in the EN-ASR models. This addresses the question of
whether an out-of-language ASR model can be used within a VC model. This is
akin to performing PPG Voice Conversion with a phone recognition model trained
on a different language.

Table 5 shows VC augmentation to be an effective augmentation technique for
EN-ASR fine-tuning, improving WER from 6.7% to 6.3%, while the improvement is
smaller for SV-ASR at only 1%.

Table 5:  VC Augmentation Results on EN-ASR and SV-ASR Method | EN-ASR WER | SV-ASR WER
---|---|---
| Baseline
---
6.7 | 13.7
| \+ VC Augmentation
---
6.3 | 13.6

###  5.5 VC Training Quality Metrics

Tuning weights for adversarial losses can be tricky, too small, they do
nothing, too large, the main loss (here reconstruction) is overwhelmed. During
our VC modelling development, we found two measures to be useful for tracking
VC training performance. Table 5.5 shows VC training runs with different
adversarial gradient weights. Also provided is the speaker classification
accuracy, VQVAE codebook perplexity and the reconstruction loss. Based on
these three measures, we identify good candidate VC models as those with low
speaker accuracy, high VQVAE perplexity and low reconstruction loss. While not
narrowly predictive of VC performance, these three measures were able to
identify models that were performing VC well enough to be evaluated by
subjective tests.

Table 6: The effect of adversarial weight on metrics available during VC training. Speaker Adversarial Gradient Weight |  Speaker Acc |  VQVAE Perplexity |  Reconstruction Loss
---|---|---|---
0.0 | 80% | 110 | 0.038
0.1 | 12% | 105 | 0.045
0.5 | 11% | 80 | 0.070
1.0 | 10% | 60 | 0.085

##  6 Conclusion

We demonstrate that even when trained on more than two thousand speakers,
LibriSpeech ASR performance can be improved by Voice Conversion augmentation.
We find similar ($\sim 6\%$) relative gains on an in-house English ASR task,
but smaller improvements ($\sim 1\%$) on a comparable Swedish task. It is
necessary for VC models to be robust to noisy inputs and diverse speakers to
achieve these result. We show that using a pre-trained ASR Encoder to serve as
the encoder in a non auto-regressive, non-parallel VC model is a successful
approach to promoting the necessary VC robustness.

## References

  * [1] L. Sun, K. Li, H. Wang, S. Kang, and H. Meng, “Phonetic posteriorgrams for many-to-one voice conversion without parallel data training,” in _2016 IEEE International Conference on Multimedia and Expo (ICME)_. IEEE, 2016, pp. 1–6.
  * [2] J.-X. Zhang, Z.-H. Ling, and L.-R. Dai, “Non-parallel sequence-to-sequence voice conversion with disentangled linguistic and speaker representations,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 28, pp. 540–552, 2019.
  * [3] C.-C. Hsu, H.-T. Hwang, Y.-C. Wu, Y. Tsao, and H.-M. Wang, “Voice conversion from non-parallel corpora using variational auto-encoder,” in _2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA)_. IEEE, 2016, pp. 1–6.
  * [4] W.-C. Huang, H. Luo, H.-T. Hwang, C.-C. Lo, Y.-H. Peng, Y. Tsao, and H.-M. Wang, “Unsupervised representation disentanglement using cross domain features and adversarial learning in variational autoencoder based voice conversion,” _IEEE Transactions on Emerging Topics in Computational Intelligence_ , vol. 4, no. 4, pp. 468–479, 2020.
  * [5] P. L. Tobing, Y.-C. Wu, T. Hayashi, K. Kobayashi, and T. Toda, “Non-parallel voice conversion with cyclic variational autoencoder,” _arXiv preprint arXiv:1907.10185_ , 2019.
  * [6] H. Kameoka, T. Kaneko, K. Tanaka, and N. Hojo, “Acvae-vc: Non-parallel voice conversion with auxiliary classifier variational autoencoder,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 27, no. 9, pp. 1432–1443, 2019.
  * [7] K. Qian, Y. Zhang, S. Chang, X. Yang, and M. Hasegawa-Johnson, “Autovc: Zero-shot voice style transfer with only autoencoder loss,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 5210–5219.
  * [8] K. Qian, Y. Zhang, S. Chang, M. Hasegawa-Johnson, and D. Cox, “Unsupervised speech decomposition via triple information bottleneck,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 7836–7846.
  * [9] S. Ding and R. Gutierrez-Osuna, “Group latent embedding for vector quantized variational autoencoder in non-parallel voice conversion.” in _INTERSPEECH_ , 2019, pp. 724–728.
  * [10] B. van Niekerk, L. Nortje, and H. Kamper, “Vector-quantized neural networks for acoustic unit discovery in the zerospeech 2020 challenge,” _arXiv preprint arXiv:2005.09409_ , 2020.
  * [11] H. Zhang, “The neteasegames system for voice conversion challenge 2020 with vector-quantization variational autoencoder and wavenet,” _arXiv preprint arXiv:2010.07630_ , 2020.
  * [12] K. Kobayashi, W.-C. Huang, Y.-C. Wu, P. L. Tobing, T. Hayashi, and T. Toda, “crank: An open-source software for nonparallel voice conversion based on vector-quantized variational autoencoder,” in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 5934–5938.
  * [13] G. Zhao, S. Sonsaat, J. Levis, E. Chukharev-Hudilainen, and R. Gutierrez-Osuna, “Accent conversion using phonetic posteriorgrams,” in _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2018, pp. 5314–5318.
  * [14] Z. Wang, X. Zhou, F. Yang, T. Li, H. Du, L. Xie, W. Gan, H. Chen, and H. Li, “Enriching source style transfer in recognition-synthesis based non-parallel voice conversion,” _arXiv preprint arXiv:2106.08741_ , 2021.
  * [15] Z. Wang, W. Ge, X. Wang, S. Yang, W. Gan, H. Chen, H. Li, L. Xie, and X. Li, “Accent and speaker disentanglement in many-to-many voice conversion,” in _2021 12th International Symposium on Chinese Spoken Language Processing (ISCSLP)_. IEEE, 2021, pp. 1–5.
  * [16] X. Cui, V. Goel, and B. Kingsbury, “Data augmentation for deep neural network acoustic modeling,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 23, no. 9, pp. 1469–1477, 2015.
  * [17] J. R. Bellegarda, P. V. de Souza, A. Nadas, D. Nahamoo, M. A. Picheny, and L. R. Bahl, “The metamorphic algorithm: A speaker mapping approach to data augmentation,” _IEEE Transactions on Speech and Audio Processing_ , vol. 2, no. 3, pp. 413–420, 1994.
  * [18] S. Shahnawazuddin, N. Adiga, K. Kumar, A. Poddar, and W. Ahmad, “Voice conversion based data augmentation to improve children’s speech recognition in limited data scenario.” in _Interspeech_ , 2020, pp. 4382–4386.
  * [19] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2223–2232.
  * [20] M. Baas and H. Kamper, “Voice conversion can improve asr in very low-resource settings,” _arXiv preprint arXiv:2111.02674_ , 2021.
  * [21] T. A. Nguyen, M. de Seyssel, P. Rozé, M. Rivière, E. Kharitonov, A. Baevski, E. Dunbar, and E. Dupoux, “The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling,” _arXiv preprint arXiv:2011.11588_ , 2020.
  * [22] A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu _et al._ , “Conformer: Convolution-augmented transformer for speech recognition,” _arXiv preprint arXiv:2005.08100_ , 2020.
  * [23] J. Shen, Y. Jia, M. Chrzanowski, Y. Zhang, I. Elias, H. Zen, and Y. Wu, “Non-attentive tacotron: Robust and controllable neural tts synthesis including unsupervised duration modeling,” 2020. [Online]. Available: https://arxiv.org/abs/2010.04301
  * [24] A. Van Den Oord, O. Vinyals _et al._ , “Neural discrete representation learning,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [25] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in _2015 IEEE international conference on acoustics, speech and signal processing (ICASSP)_. IEEE, 2015, pp. 5206–5210.
  * [26] G. Wang, A. Rosenberg, Z. Chen, Y. Zhang, B. Ramabhadran, Y. Wu, and P. Moreno, “Improving speech recognition using consistent predictions on synthesized speech,” in _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 7029–7033.
  * [27] Z. Chen, A. Rosenberg, Y. Zhang, G. Wang, B. Ramabhadran, and P. J. Moreno, “Improving speech recognition using gan-based speech synthesis and contrastive unspoken text selection.” in _INTERSPEECH_ , 2020, pp. 556–560.
  * [28] Z. Chen, Y. Zhang, A. Rosenberg, B. Ramabhadran, G. Wang, and P. Moreno, “Injecting text in self-supervised speech pretraining,” _arXiv preprint arXiv:2108.12226_ , 2021.
  * [29] M. Ghodsi, X. Liu, J. Apfel, R. Cabrera, and E. Weinstein, “Rnn-transducer with stateless prediction network,” in _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 7049–7053.