# Thermodynamics of nonequilibrium systems with uncertain parameters

Jan Korbel  Section for Science of Complex Systems, CeMSIIS, Medical
University of Vienna, Spitalgasse 23, 1090 Vienna, Austria
<EMAIL_ADDRESS>Science Hub Vienna, Josefstädter Strasse 39, 1080
Vienna, Austria  David H. Wolpert  Santa Fe Institute, Santa Fe, NM, United
States of America<EMAIL_ADDRESS>http://davidwolpert.weebly.com
Science Hub Vienna, Josefstädter Strasse 39, 1080 Vienna, Austria  Arizona
State University, Tempe, AZ, United States of America

###### Abstract

In the real world, one almost never knows the parameters of a thermodynamic
process to infinite precision. Reflecting this, here we investigate how to
extend stochastic thermodynamics to systems with uncertain parameters,
including uncertain number of heat baths / particle reservoirs, uncertainty in
the precise values of temperatures / chemical potentials of those reservoirs,
uncertainty in the energy spectrum, uncertainty in the control protocol, etc.
We formalize such uncertainty with an (arbitrary) probability measure over all
transition rate matrices satisfying local detailed balance. This lets us
define the effective thermodynamic quantities by averaging over all LDB-
obeying rate matrices. We show that the resultant effective entropy violates
the second law of thermodynamics. In contrast to the effective entropy though,
the expected stochastic entropy, defined as the ensemble average of the
effective trajectory-level entropy, satisfies the second law. We then and
explicitly calculate the second-order correction to the second law for the
case of one heat bath with uncertain temperature. We also derive the detailed
fluctuation theorem for expected effective trajectory entropy production for
this case, and derive a lower bound for the associated expected work. Next, to
ground these formal considerations with experimentally testable bounds on
allowed energetics, we derive a bound on the maximal work that can be
extracted from systems with arbitrarily uncertain temperature. We end by
extending previous work on “thermodynamic value of information”, to allow for
uncertainty in the time-evolution of the rate matrix.

##  I Introduction

The microscopic laws of classical and quantum physics provide us a
parameterized set of equations that specify the evolution of a closed system
starting from a specific state. To use those equations we need to know that
specific state, we need to be sure the system is closed, and we need to know
the values of the parameters in the equations. barato2015

Unfortunately, in very many real-world scenarios, we are uncertain about the
precise state of the system, and very often the system is open rather than
closed, subject to uncertain interactions with the external environment.
Statistical physics captures these two types of uncertainty by building on the
microscopic laws of physics in two ways. First, to capture uncertainty about
the state of the system, in statistical physics we replace exact specification
of the system’s state with a probability distribution over states. Second, to
capture uncertain interactions between the system and the external
environment, we add randomness to the dynamics, in a precisely parameterized
form 111We note though there is a substantial literature in which adopts an
“inclusive Hamiltonian” framework, in which the external environment is
finite, and the joint dynamics of the system-environment is explicitly
modeled. This has been explored either with explicit Hamiltonian dynamics,
with only randomness in the initial state kawai_dissipation:_2007 ;
esposito2010entropy ; ptaszynski2018first  or at least approximate Hamiltonian
dynamics seifert2016first ; strasberg2017stochastic ; talkner2020HMF;
talkner2020comment ; strasberg2020measurability ..

In particular, in classical stochastic thermodynamics seifert2012stochastic ;
van2015ensemble , we model the system as a probability distribution evolving
under a continuous-time Markov chain (CTMC), with a precisely specified rate
matrix. Typically we require that the CTMC obeys local detailed balance (LDB).
This means that the rate matrix of the CTMC has to obey certain restrictions
which are parameterized by the energy spectrum of the system, the number of
thermodynamic reservoirs in the external environment perturbing the system’s
dynamics, and the temperatures and chemical potentials of those reservoirs.
Often we also allow the rate matrix of the CTMC to change in time, subject to
those restrictions, according to a precisely defined “protocol”.

However, in addition to uncertainty about the state of the system, and
uncertainty about interactions with the external environment, there is an
additional unavoidable type of uncertainty in all real-world systems:
uncertainty about the parameters in the equations governing the dynamics. In
the context of stochastic thermodynamics, this means that even if we impose
LDB, we will never know the reservoir temperatures and chemical potentials to
infinite precision (often even being unsure about the number of such
reservoirs), we will never know the energy spectrum to infinite precision, and
more generally we will never know the rate matrix and its time-dependence to
infinite precision.

In this paper we consider how stochastic thermodynamics (and nonequilibrium
statistical physics more generally) needs to be modified to capture this third
type of uncertainty, in addition to the two types of uncertainty it already
captures. We emphasize that this third type of uncertainty, concerning the
parameters of the thermodynamic process, is unavoidable in essentially all
real-world scenarios. Nonetheless, almost nothing is known about its
thermodynamic consequences 222We note though that it is already known that if
we do not account for all thermodynamic reservoirs, we invariably undercount
the total of entropy production in a process esposito2010three ..

To clarify our precise focus in this paper, we note that there has been some
preliminary research on how to modify stochastic thermodynamics in the case
that we, the experimentalist, are not able to view all state transitions in
the system as it evolves bisker2017hierarchical ;
shiraishi_ito_sagawa_thermo_of_time_separation.2015 . In contrast to our
focus, this uncertainty concerns what is observed as the system evolves,
whereas the uncertainty we consider involves the parameters governing that
evolution. Similarly, some models consider either spatial matsuo2000stochastic
or temporal fiore2019entropy  variation of temperature and other parameters,
but they assume that this evolution is known.

Probably the closest research to what we consider in this paper is sometimes
called _superstatistics_. It has long been known that an average over Gibbs
distributions cannot be written as some single Gibbs distribution (Thm. 1 in
wolpert1993use ). This means that even equilibrium statistical physics must be
modified when there is uncertainty in the temperature of a system. The
analysis of these modifications was begun by Beck and Cohen
beck2003superstaitstic , who developed an effective theory for thermodynamics
with temperature fluctuations. They considered a system coupled to a bath,
which is in a _local equilibrium_ under the slow evolution of the temperature
of the bath. The main assumption they exploit is _scale-separation_ : while
for short time scales, the equilibrium distribution is a canonical
distribution with inverse temperature $\beta$, the long-scale behavior is
determined by the _superposition_ of canonical distributions with some
distribution of temperatures $f(\beta)$. The resulting _superstatistical_
distribution $p(E)=\int\mathrm{d}\beta f(\beta)\exp(-\beta E)$ was later
identified with the distribution corresponding to generalized entropic
functionals hanel2011generalized ; garcia2011superstatistics . Later
interpretations of superstatistics were not based on the notion of local
equilibria but rather on the Bayesian approach to systems with uncertain
temperature sattin2006bayesian ; davis2018temperature . These are conceptually
closer to the focus of this paper.

The rest of the paper is organized as follows: in the next section, we
introduce necessary notation and briefly recall the main results of stochastic
thermodynamics. In Section III, we present the general framework of expected
thermodynamics for systems with uncertain transition rates. First we note that
the expected probability distribution is not Markovian. We then show that the
expected entropy does not satisfy the second law. On the other hand, the
_expected stochastic entropy_ , defined as the ensemble average of the
expected trajectory entropy, does satisfy the second law.

For the remainder of the paper, we focus on the simple case where we know for
certain that there is a single (heat) reservoir, but are uncertain about its
temperature, which is constant during the evolution of the system. The
uncertainty in temperature implies uncertainty in the transition rate matrix,
since we mean assume that the actual rate matrix, whatever it is, satisfies
LDB. In Section IV, we consider the special case where the temperature
distribution is sharply peaked. We show that the expected entropy flow can be
smaller than expected inverse temperature times expected heat and explicitly
calculate the correction to the second law. Furthermore, we derive the
detailed fluctuation theorem for the expected trajectory entropy production.
Finally, for the case of equilibrium initial and final distributions, we show
that the expected work is not lower-bounded by the expected difference in the
equilibrium Helmholtz free energy, but derive a correction to the lower bound.

In the final two sections we consider an issue foundational to stochastic
thermodynamics: How much work can be extracted from a system during a process
that takes it from its initial distribution to a given, target distribution.
First, in Section V, we consider this issue when we are uncertain both about
the initial distribution and about the temperature of the system as it
evolves. We focus on how that uncertainty changes the results of the standard
analysis of this issue, in which we suppose a {quench; equilibrate;
semistatically-evolve} process is applied to the system immediately after the
initial distribution is generated parrondo2015thermodynamics ;
hasegawa2010generalization . Then in Section VI, we simplify this scenario.
First, we suppose that the temperature during the dynamics is perfectly known,
so that the only uncertainty is in the initial distribution. However, we also
suppose that there is a (perfectly known) delay between when the initial
distribution is generated, $t_{i}$, and the time $\tau$ when the {quench;
equilibrate; semistatically-evolve} process can begin, during which time the
system evolves according to a (perfectly known) rate matrix. Specifically, we
analyze how the minimal EP changes as the length of the delay changes, which
causes the distributional uncertainty at the beginning of the quench process
to change. We then present preliminary results of how this analysis changes
when there is uncertainty about the rate matrix taking the initial
distribution to the time $\tau$ distribution, in addition to (and possibly
statistically coupled with) the uncertainty about the initial distribution.

We end with a discussion section in which we describe just a few of the myriad
directions for future work.

##  II Terminology and notation

In conventional stochastic thermodynamics, with no uncertainty about the
thermodynamic parameters, we consider a system coupled to $N$ heat and/or
particle reservoirs, with temperatures $\beta_{1},\dots,\beta_{N}$ and
chemical potentials $\mu_{1},\dots,\mu_{N}$. As shorthand write
$\theta^\nu=(\beta_{\nu},\mu_{\nu})$ and
$\bm{\theta}=(\theta^1,\dots,\theta^N)$. We write the rate matrix for going
from state $x^{\prime}$ to state $x$ as $K_{x,x^{\prime}}$, with the time
index implicit. So the master equation is
$\dot{p}_{x}(t)=\sum_{x^{\prime}}K_{x,x^{\prime}}p_{x^{\prime}}(t)$. As usual,
we assume that the rate matrix can be decomposed as
$K_{x,x^{\prime}}=\sum_{\nu=1}^{N}K_{x,x^{\prime}}^{\nu}$ where the separate
matrices $K_{x,x^{\prime}}^{\nu}$ each satisfy LDB.

The internal energy is written as $U=\sum_{x}p_{x}\epsilon_{x}$, and the first
law of thermodynamics is written in the usual way, as
$\dot{U}=\dot{W}+\dot{Q}+\dot{W}_{chem}$, where
$\dot{W}(t)=\sum_{x}\dot{\epsilon}_{x}(t)p_{x}(t)$. The heat flow $\dot{Q}$ is
decomposed into heat flows to particular reservoirs:

$\displaystyle\dot{Q}(t)$ $\displaystyle=\sum_{\nu}\dot{Q}_{\nu}(t)$ (1)
$\displaystyle=\sum_{\nu}\sum_{x}(\epsilon_{x}(t)-\mu_{\nu}n_{x})K_{x,x^{\prime}}^{\nu}p_{x}(t)$
(2)

where $n_{x}$ is the number of particles in the system, as specified by the
state $x$. (For simplicity, we ignore the possibility of more than one
distinguishable type of particle.) The chemical work is then given by
$\dot{W}_{chem}=\sum_{\nu}\sum_{x}\dot{p}_{x}(t)\mu_{\nu}n_{x}$. The Shannon
entropy is written as $S=-\sum_{x}p_{x}\ln p_{x}$. The second law of
thermodynamics can be expressed as $\dot{S}=\dot{S}_{i}+\dot{S}_{e}$ where
$\dot{S}_{i}\geq 0$ and $\dot{S}_{e}=\sum_{\nu}\beta_{\nu}\dot{Q}^{\nu}$.

Throughout this paper, for simplicity we assume that any time-extended
thermodynamic process we refer to occurs in some interval $[t_{i},t_{f}]$
where we have no uncertainty about those starting and ending times. We write a
generic stochastic trajectory across that interval as $x(t)$. So the
stochastic energy at time $\tau$ is $\epsilon(t)=\epsilon_{x(t)}$, and the
first law of thermodynamics on the trajectory level for any time $t$ is
$\frac{\mathrm{d}}{\mathrm{d}t}e(t)=\dot{w}(t)+\dot{q}(t)+\dot{w}_{chem}(t)$,
where

$\displaystyle\dot{w}(t)$
$\displaystyle=\sum_{x}\delta_{x,x(t)}\dot{\epsilon}_{x(t)},$ (3)
$\displaystyle\dot{q}(t)$
$\displaystyle=\sum_{\nu}\dot{q}^\nu(t)=\sum_{\nu}\dot{\delta}_{x,x(t)}(\epsilon_{x(t)}-\mu_{\nu}n_{x(t)}),$
(4) $\displaystyle\dot{w}_{chem}$
$\displaystyle=\sum_{\nu}\dot{\delta}_{x,x(t)}\mu_{\nu}n_{x(t)}$ (5)

are called the stochastic work, stochastic heat, and stochastic chemical work,
respectively.

As usual, the stochastic entropy along a trajectory is defined as
$s_{x(\tau)}=-\ln p_{x(\tau)}$. Stochastic entropy production is then

$\displaystyle\dot{s}_{i}=\dot{s}-\dot{s}_{e}$ (6)

where due to LDB, $\dot{s}_{e}=\sum_{\nu}\beta_{\nu}\dot{q}^\nu$. Due to LDB,
stochastic entropy production fulfills the _detailed fluctuation theorem_ ,
$P(\Delta s_{i})/\tilde{P}(-\Delta s_{i})=e^{\Delta s_{i}}$, where $\tilde{P}$
denotes the probability under a time-reversed protocol. By averaging these
trajectory-level quantities over all trajectories, we recover the ensemble-
level versions. For more details, see the Appendix.

##  III Effective thermodynamics of the system with uncertain transition rates

As described in the introduction, we are interested in how the conventional
laws of stochastic thermodynamics concerning the evolution of a system change
when there is uncertainty about the parameters of that evolution, but we
assume LDB holds, whatever those parameters are. We start with comments on the
general case where the transition rates of the systems are not known with
infinite precision. Such uncertainty must arise whenever we do not know exact
number of heat reservoirs, their temperatures/and or chemical potentials, but
can even arise when we do know those quantities exactly, and even when we
impose LDB. (For simplicity, we not consider uncertainty in the energy
spectrum or how it varies in time. 333Uncertainty in the energy spectrum and /
or the time-dependence of the rate matrix can also be included in the
analysis, at the cost of more complex notation.) However, note even if we knew
those quantities to infinite precision, LDB does not uniquely fix the rate
matrix $K$, and so there can still be uncertainty concerning $K$.

In the Appendix, we review why the general form of a transition matrix
satisfying LDB can be written as

$\displaystyle
K(N;\bm{\theta},\mathbf{R})\;=\;\sum_{\nu}K^\nu(\theta^\nu,R^\nu)$
$\displaystyle\;\;=\sum_{\nu=1}^N\left[R^\nu\,(\Pi^\nu(\theta^\nu))^{-1}\mathrm{diag}\left(R^\nu\,(\Pi^\nu(\theta^\nu))^{-1}\cdot\textbf{1}\right)\right]$
(7)

where each $R^\nu$ is an arbitrary symmetric matrix and the associated
$\Pi^\nu$ is a diagonal matrix whose diagonal elements are the entires in the
equilibrium distribution of the reservoir $\nu$.

We now consider a probability measure $\mathrm{d}F_{K}$ over the rate matrices
obeying Eq. 7, which quantifies our uncertainty about the temperatures /
chemical potential of the reservoirs, number of reservoirs, etc. We suppose
that before a system starts to evolve, its rate matrix is generated by
sampling $\mathrm{d}F_{K}$, and that the system then evolves with that rate
matrix, without ever again sampling $\mathrm{d}F_{K}$. However, the
experimentalist is still uncertain what that randomly generated rate matrix is
as they observe the system evolving. We emphasize that this scenario differs
from one where the system couples / decouples from the reservoirs in a random
process, or where the reservoirs undergo temperature fluctuations during its
evolution, as for example as considered in superstatistics
beck2003superstaitstic .

We write the associated _effective_ value of a (perhaps implicitly time-
dependent) generic quantity $X^{K}$ as

$\displaystyle\overline{X}:$ $\displaystyle=$
$\displaystyle\int\mathrm{d}F_{K}\,X^K$ (8) $\displaystyle=$
$\displaystyle\sum_{N=1}^\infty{\mathcal{P}}_{N}\int\prod_{\nu=1}^N\mathrm{d}\mathbf{R}^\nu\mathrm{d}\bm{\theta}^\nu\,f^N(\bm{\theta},\mathbf{R})\,X^{K(N;\bm{\theta},\mathbb{S})}$

where ${\mathcal{P}}_{N}$ is the probability distribution over the number of
reservoirs and $f^N(\bm{\theta},\mathbf{R})$ is the distribution over all
possible LDB-obeying rate matrices for a system coupled with $N$ reservoirs.
We use the term “effective value” to refer to expectations formed this way by
integrating over all possible values of $N,\bm{\theta},\mathbf{R}$. We instead
use the term _average_ value to mean expectations formed by integrating over
all possible ensemble trajectories, and denote such a value generically as
$\langle\cdot\rangle$.

We emphasize that there are other types of uncertainty concerning
thermodynamic parameters, reflecting imperfect knowledge of energy levels,
chemical potentials, the control protocol driving the energy spectrum, etc.
For simplicity, in this paper we do not consider these other types of
parameter uncertainty. However, it is straight-forward to generalize some of
the results below to capture these other types of uncertainty as well.

Using this notation, the effective distribution at any time $t$ is

$\overline{p}_{x}(t):=\int\mathrm{d}F_{K}p^{K}_{x}(t)\,.$ (9)

and the time evolution of $\bar{p}$ is given by

$\dot{\overline{p}}_{x}(t)=\sum_{x^{\prime}}\overline{J}_{x,x^{\prime}}=\sum_{x^{\prime}}\int\mathrm{d}F_{K}K_{x,x^{\prime}}p^{K}_{x^{\prime}}(t)$
(10)

The corresponding transition matrix at time $t$ can be formally defined as

$\mathcal{K}_{x,x^{\prime}}(p^K(t)):=\frac{\int\mathrm{d}F_{K}K_{x,x^{\prime}}p^{K}_{x^{\prime}}(t)}{\int\mathrm{d}F_{K}p^{K}_{x^{\prime}}(t)}$
(11)

depends on all $p^K$. However, this matrix will change if the distributions
$p^K_{x^{\prime}}(t)$ change, i.e., is not only a function of the state $x$.
Accordingly, it results in non-Markovian dynamics strasberg2019non . Note as
well that $\overline{K}=\int\mathrm{d}F_{K}\,K$, the effective transition rate
matrix, is not directly related to the dynamics of the effective probability
distribution. In particular $\overline{p}_{x}(t)\neq p_{x}^{\overline{K}}(t)$
in general.

Next, write the effective internal energy as
$\overline{U}=\sum_{x}\overline{p}_{x}\epsilon_{x}$. The first law of
thermodynamics in terms of this effective internal energy can be expressed as
$\dot{\overline{U}}=\dot{\overline{W}}+\dot{\overline{Q}}+\dot{\overline{W}}_{chem}$.
Similarly, we can introduce the effective entropy as
$\bar{S}=-\sum_{x}\overline{p_{x}\ln p_{x}}$ and write down the second law of
thermodynamics as

$\dot{\bar{S}}=\dot{\bar{S}}_{i}+\dot{\bar{S}}_{e}$ (12)

where

$\displaystyle\dot{\bar{S}}_{i}$
$\displaystyle=\int\mathrm{d}F_{K}\dot{S}^K_{i}\geq 0$ (13)
$\displaystyle\dot{\bar{S}}_{e}$
$\displaystyle=\int\mathrm{d}F_{K}\dot{S}^K_{e}$ (14)
$\displaystyle=\overline{\sum_{\nu=1}^N\beta^K_{\nu}\dot{Q}^{\nu,K}}$ (15)

(Note that we average over $N$, and therefore over the number of terms in the
sum in Eq. 15.) $\dot{\bar{S}}_{i}$ is the effective entropy production rate,
and $\dot{\bar{S}}_{e}$ is the effective entropy flow rate.

Note that the effective entropy production is non-negative, as in conventional
(no uncertainty) stochastic thermodynamics, since each $\dot{S}^K_{i}$ is non-
negative. However, there is no simple relation between the effective entropy
flow rate and the effective heat flow rate, given by

$\dot{\overline{Q}}=\overline{\sum_{\nu=1}^N\dot{Q}^{\nu,K}}$ (16)

(This discrepancy between the two rates would exist even if we knew with
certainty that $N=1$; it reflects the fact that there is statistical coupling
between $\beta^K$ and $\dot{Q}^K$ when $\beta^K$ is a random variable.) In
particular, the derivative of the entropy is not necessarily lower-bounded by
the heat flow rate, which means that this ensemble version of the second law
of thermodynamics is not as consequential as the standard version.

We now turn our focus to trajectory-level thermodynamics. To begin, note that
for a given trajectory $x(t)$ and given energy spectrum, the stochastic energy
does not depend on the transition rate matrix. Therefore stochastic work and
stochastic total heat (heat plus chemical work) are independent of the rate
matrix.

Next, define the effective stochastic entropy at time $\tau$ as

$\overline{s}(x(\tau))=-\overline{\log\,p_{x(\tau)}}$ (17)

Note that the effective stochastic entropy is not the same as the stochastic
entropy of the effective distribution, $-\log\bar{p}_{x}(t)$. If we average
the trajectory-level decomposition of entropy, Eq. 6, over all possible
transition rate matrices, we obtain

$\dot{\bar{s}}(x(\tau))=\dot{\bar{s}}_{i}(x(\tau))+\dot{\bar{s}}_{e}(x(\tau))$
(18)

where

$\displaystyle\dot{\bar{s}}_{e}(x(\tau))$
$\displaystyle=\int\mathrm{d}F_{K}{\dot{s}}^K_{e}(x(\tau))$
$\displaystyle=\sum_{N=1}^\infty{\mathcal{P}}_{N}\sum_{\nu=1}^N\overline{\beta_{\nu}(\epsilon_{x(\tau)}-\mu_{\nu}n_{x(\tau)})}$
$\displaystyle=\sum_{N=1}^\infty{\mathcal{P}}_{N}\sum_{\nu=1}^N\overline{\beta_{\nu}}(\epsilon_{x(\tau)}-\overline{\mu_{\nu}}n_{x(\tau)})$
$\displaystyle=\sum_{N=1}^\infty{\mathcal{P}}_{N}\sum_{\nu=1}^N\overline{\beta^\nu}\,\dot{\overline{q^\nu}}(x(\tau))$
(19)

where in the last line we assume that $\mathrm{Cov}(\beta^\nu,\mu^\nu)=0$,
i.e., that our uncertainty about the temperature is statistically independent
of our uncertainty about the chemical potential.

By averaging Eq. 18 over all possible trajectories under the effective
probability distribution $\overline{p}_{x(\tau)}$, we obtain

$\langle\dot{\bar{s}}\rangle_{\bar{p}}=\langle\dot{\bar{s}}_{i}\rangle_{\bar{p}}+\langle\dot{\bar{s}}_{e}\rangle_{\bar{p}}$
(20)

where we define $\langle
f(x(\tau))\rangle_{\bar{p}}:=\sum_{x(\tau)}f(x(\tau))\bar{p}_{x(\tau)}$. Note
that

$\dot{\bar{S}}_{e}=\overline{\langle\dot{s}_{e}\rangle_{p^K}}\neq\langle\dot{\bar{s}}_{e}\rangle_{\bar{p}}=\sum_{N=1}^\infty{\mathcal{P}}_{N}\sum_{\nu=1}^N\overline{\beta^\nu}\,\dot{\overline{Q^\nu}}$
(21)

As shorthand, write the effective stochastic entropy as

$\mathcal{S}:=\langle{\bar{s}}\rangle_{\bar{p}}=-\sum_{x}\overline{p}_{x}\,\cdot\,\overline{\ln
p_{x}}$ (22)

With help of Eq. 20 we can show that the effective stochastic entropy also
satisfies the second law. In addition, the effective stochastic entropy
exceeds the stochastic entropy, $\overline{S}$:

$\dot{\mathcal{S}}\geq\sum_{N=1}^\infty{\mathcal{P}}_{N}\sum_{\nu=1}^N\overline{\beta^\nu}\,\dot{\overline{Q^\nu}}\,.$
(23)

##  IV System coupled to one reservoir with sharply peaked temperature
distribution

In this section we consider the simple scenario where we are certain that the
system is coupled to a single heat reservoir, i.e., $\bm{\theta}=\beta$ but
are uncertain about the temperature of that reservoir. Via LDB, this
uncertainty in temperature causes uncertainty in the rate matrix, and
therefore uncertainty in the stochastic thermodynamics. For simplicity, we
only consider the effect of this specific type of uncertainty on expectation
values. We indicate such an effective value of a general thermodynamic
variable $X(\beta)$ (possibly depending on the trajectory $x(t)$ defined in
the time interval $[t_{i},t_{f}]$) as

$\bar{X}=\int\mathrm{d}F_{\beta}X(\beta)$ (24)

To begin, we make a second-order expansion of $X$ in $\beta-\bar{\beta}$ to
obtain

$\bar{X}\approx
X(\bar{\beta})+\frac{\mathrm{Var}\beta}{2}\frac{\partial^2X}{\partial\beta^2}|_{\bar{\beta}}$
(25)

In particular, the effective heat flow to second order is

$\bar{Q}\approx
Q(\bar{\beta})+\frac{\mathrm{Var}\beta}{2}\frac{\partial^2Q}{\partial\beta^2}|_{\bar{\beta}}$
(26)

Moreover, $\bar{S}_{e}$ can be expanded as

$\displaystyle\bar{S}_{e}$ $\displaystyle=$ $\displaystyle\overline{\beta
Q}=\int\mathrm{d}\beta\,p(\beta)\,\beta Q(\beta)$ (27) $\displaystyle\simeq$
$\displaystyle\int\mathrm{d}\beta\,p(\beta)\beta\left[Q(\bar{\beta})+(\beta-\bar{\beta})\frac{\partial
Q}{\partial\beta}|_{\bar{\beta}}+\frac{(\beta-\bar{\beta})^2}{2}\frac{\partial^2Q}{\partial\beta^2}|_{\bar{\beta}}\right]$
$\displaystyle=$
$\displaystyle\bar{\beta}Q(\bar{\beta})+\mathrm{Var}\beta\frac{\partial
Q}{\partial\beta}|_{\bar{\beta}}+\frac{\overline{\beta^3}-\overline{\beta}\,\overline{\beta^2}}{2}\frac{\partial^2Q}{\partial\beta^2}|_{\bar{\beta}}$

By plugging in for $Q(\bar{\beta})$ from Eq. 26, we obtain

$\displaystyle\bar{S}_{e}=\bar{\beta}\bar{Q}+\mathrm{Var}\beta\frac{\partial
Q}{\partial\beta}|_{\bar{\beta}}+\frac{\overline{\beta^3}-\bar{\beta}^3}{2}\frac{\partial^2Q}{\partial\beta^2}|_{\bar{\beta}}$
$\displaystyle=\bar{\beta}\bar{Q}-m_{2}(\beta)C_{V}+m_{3}(\beta)\left(C_{V}^{\prime}+\frac{C_{V}^{\prime\prime}}{2\bar{\beta}}\right)$
(28)

where $C_{V}=\frac{\partial Q}{\partial T}$ is the specific heat at
$\bar{T}=\frac{1}{\bar{\beta}}$ and

$m_{k}(\beta)=\frac{\overline{\beta^k}}{\bar{\beta}^k}-1$ (29)

is the _relative $k$-th moment_. It is straightforward to show from Jensen’s
inequality that $m_{k}(\beta)\geq 0$.

Note that for many cases the effective entropy flow is smaller than effective
inverse temperature times effective heat. Consider, for example the case where
$C_{V}=const>0$, i.e., positive specific heat independent of temperature.)
This can be interpreted as a violation of the second law for effective
thermodynamic quantities. Formally, it arises because $\beta$ and $Q$ are
statistically coupled, so $\overline{\beta
Q}\neq\overline{\beta}\;\overline{Q}$. It is important to emphasize that this
is not a _physical violation_ of the second law. Experimentally, it would
arise if we repeatedly prepare the system the same (imprecise) way, and
interpret the resulting empirical average values of thermodynamic quantities
as an average under one, fixed distribution, being repeatedly sampled in each
experiment, when in fact the distribution itself varies from one experiment to
the next.

We now use these preliminary results to investigate the trajectory
thermodynamics when the distribution over temperature is highly peaked. First,
we note that the effective change of entropy production $\bar{\Delta s}_{i}$
across a trajectory $x(\tau)$ starting at $t_{0}$ and ending at $t_{f}$ can be
expressed as

$\Delta\bar{s}_{i}=\overline{\ln\frac{\mathcal{P}^\beta(x(\tau))}{\tilde{\mathcal{P}}^\beta(\tilde{x}(\tau))}}$
(30)

where $\mathcal{P}^\beta$ is the probability of observing trajectory $x(\tau)$
for inverse temperature $\beta$, $\tilde{x}(\tau)=x(t_{f}-\tau)$ is the time-
reversed trajectory $\tilde{\mathcal{P}}$ is the probability for the time-
reversed protocol $\tilde{\lambda}(x)$. We assume that for each $\beta$, we
are certain about the initial and final distribution. By using expansion Eq.
25 we can express the effective entropy production change as:

$\Delta\bar{s}_{i}=\Delta
s_{i}(\bar{\beta})+\frac{\mathrm{Var}\beta}{2}\frac{\partial^2\Delta
s_{i}}{\partial\beta^2}|_{\bar{\beta}}\,.$ (31)

Furthermore, due to the fact that the trajectory entropy flow is a linear
function of $\beta$, we can use the following identity:

$\left.\frac{\partial^2\Delta
s_{i}}{\partial\beta^2}\right|_{\beta=\bar{\beta}}=\left.\frac{\partial^2\Delta
s}{\partial\beta^2}\right|_{\beta=\bar{\beta}}\,.$ (32)

An interesting special case is where the initial and final distribution are
equilibrium distributions, as for example in Crooks’ theorem. So the
equilibrium distribution for each $\beta$ is given by the Boltzmann
distribution with corresponding temperature.

In this special case,

$\displaystyle\frac{\partial^2s}{\partial\beta^2}\left.\right|_{\beta=\bar{\beta}}$
$\displaystyle=$
$\displaystyle\frac{\partial^2}{\partial\beta^2}\left(\beta\epsilon_{x}+\ln
Z\right)\left.\right|_{\beta=\bar{\beta}}$ (33) $\displaystyle=$
$\displaystyle\mathrm{Var}(U)_{\bar{\beta}}=\sum_{x}\pi^{\bar{\beta}}_{x}\epsilon^2_{x}-\left(\sum_{x}\pi^{\bar{\beta}}\epsilon^2_{x}\right)^2\,.$

so we finally obtain

$\Delta\bar{s}_{i}=\Delta
s_{i}(\bar{\beta})+\frac{1}{2}\mathrm{Var}(\beta)\,\Delta\mathrm{Var}(U)_{\bar{\beta}}\,.$
(34)

Depending on $\Delta\mathrm{Var}(U)_{\bar{\beta}}$, the trajectory entropy
production can be smaller or bigger than for the case of effective
temperature.

Expressing $\Delta s_{i}(\bar{\beta})$ from Eq. 31 and plugging it into the
DFT for the entropy production change with effective temperature
$\ln\frac{\mathcal{P}^{\bar{\beta}}(x(\tau))}{\tilde{\mathcal{P}}^{\bar{\beta}}(\tilde{x}(\tau))}=\Delta
s_{i}(\bar{\beta})$, it is possible to obtain the detailed fluctuation theorem
for $\bar{s}_{i}$ as

$\displaystyle P^{\bar{\beta}}(\Delta
s_{i})=\int\mathcal{D}x(\tau)\mathcal{P}^{\bar{\beta}}(x(\tau))\delta\left(\Delta\bar{s}_{i}-\overline{\ln\frac{\mathcal{P}^\beta(x(\tau))}{\tilde{\mathcal{P}}^\beta(\tilde{x}(\tau))}}\right)$
$\displaystyle=e^{\Delta
s_{i}-\frac{1}{2}\mathrm{Var}(\beta)\,\Delta\mathrm{Var}(U)_{\bar{\beta}}}\int\mathcal{D}x(\tau)\tilde{\mathcal{P}}^{\bar{\beta}}(\tilde{x}(\tau))\delta\left(-\Delta\bar{s}_{i}-\overline{\ln\frac{\tilde{\mathcal{P}}^\beta(\tilde{x}(\tau))}{\mathcal{P}^\beta(x(\tau))}}\right)=e^{\Delta
s_{i}-\frac{1}{2}\mathrm{Var}(\beta)\,\Delta\mathrm{Var}(U)_{\bar{\beta}}}\tilde{P}^{\bar{\beta}}(-\Delta
s_{i})$ (35)

so we obtain

$\frac{P^{\bar{\beta}}(\Delta s_{i})}{\tilde{P}^{\bar{\beta}}(-\Delta
s_{i})}=e^{\Delta
s_{i}-\frac{1}{2}\mathrm{Var}(\beta)\,\Delta\mathrm{Var}(U)_{\bar{\beta}}}$
(36)

Using this result to average over the ensemble of trajectories gives us the
integrated fluctuation theorem

$\left\langle e^{\Delta
s_{i}-\frac{1}{2}\mathrm{Var}(\beta)\,\Delta\mathrm{Var}(U)_{\bar{\beta}}}\right\rangle_{P^{\bar{\beta}}}=1$
(37)

By using Jensen inequality, we finally obtain

$\displaystyle\langle\Delta\bar{s}_{i}\rangle_{\bar{\beta}}\geq\frac{1}{2}\mathrm{Var}(\beta)\,\Delta\mathrm{Var}(U)_{\bar{\beta}}$
(38)

So depending on whether $\Delta\mathrm{Var}(U)_{\bar{\beta}}$ is positive or
negative (i.e., whether the internal energy variance is increased or decreased
during the experiment), the effective entropy production will be positive – or
negative.

Finally, by plugging $\Delta s_{i}^{\bar{\beta}}=\bar{\beta}(w-\Delta
F(\bar{\beta}))$ into Eq. 34 (where $F(\bar{\beta})$ is the equilibrium free
energy for temperature $\bar{\beta}$), we obtain

$\Delta\bar{s}_{i}=\bar{\beta}\left(w-\Delta
F(\bar{\beta})\right)+\frac{1}{2}\mathrm{Var}(\beta)\,\Delta\mathrm{Var}(U)_{\bar{\beta}}\,.$
(39)

By taking the ensemble average under the effective probability distribution,
we obtain

$\bar{W}\geq\Delta
F(\bar{\beta})-\frac{1}{2}\frac{\mathrm{Var}(\beta)}{\bar{\beta}}\Delta\mathrm{Var}(U)_{\bar{\beta}}\,.$
(40)

Next, we can expand the effective equilibrium free energy as

$\bar{F}=F(\bar{\beta})+\frac{1}{2}\mathrm{Var}(\beta)\frac{\partial^2F}{\partial\beta^2}|_{\bar{\beta}}$
(41)

where

$\displaystyle\frac{\partial^2F}{\partial\beta^2}|_{\bar{\beta}}$
$\displaystyle=$
$\displaystyle\frac{\partial^2}{\partial\beta^2}\left(-\frac{1}{\beta}\ln
Z(\beta)\right)|_{\bar{\beta}}$ (42) $\displaystyle=$
$\displaystyle\frac{2F(\bar{\beta})}{\bar{\beta}^2}-\frac{2U(\bar{\beta})}{\bar{\beta}^2}-\frac{\mathrm{Var}(U)_{\bar{\beta}}}{\bar{\beta}}$

By expressing $F(\bar{\beta})$ from Eq. (42) and plugging it into Eq. 40, we
obtain

$\bar{W}\geq\Delta\bar{F}+\bar{\beta}m_{2}(\beta)\,\Delta S(\bar{\beta})$ (43)

##  V Optimal work extraction from a system with uncertain temperature

In this and the following section we consider the foundational issue of the
minimal thermodynamic costs necessary to evolve an (uncertain) initial
distribution to a (perfectly known) desired ending distribution, by
appropriate choice of a (perfectly known) time-dependent Hamiltonian. For
simplicity, we also assume in both sections that we know with certainty that
there is only one reservoir during the system’s evolution, and there are no
particle interchanges between the system and that reservoir during that
evolution. (So the reservoir is a heat bath.)

While we can impose an arbitrary time-dependent Hamiltonian $H_{t}(x)$ on the
system, since we assume that the system obeys LDB at all times with respect to
that Hamiltonian, the rate matrix at any given moment will depend on the
temperature. For completeness, we assume that we have zero uncertainty about
this dependence, i.e., there is a single-valued function mapping
$(H_{t}(x),\beta)\rightarrow K_{x,x^{\prime}}(t)$ which we can set arbitrarily
(subject to the constraint of LDB), and which we know with zero uncertainty.
Accordingly, our choice of $H_{t}$ would determine the full function
$K_{x,x^{\prime}}(t)$ with zero uncertainty, if it weren’t for uncertainty
about the temperature.

We start in this section by considering the case where in addition to being
uncertain about the initial distribution, we are also uncertain about the
temperature of the heat reservoir coupled to the system during its evolution.
We further suppose that the (uncertain) initial distribution is specified by
the temperature of the heat reservoir that will govern the system’s evolution
from that distribution. However, we allow this specification to be flexible;
we do not require that the initial distribution be an equilibrium for the heat
reservoir.

Formally, in this section we suppose that we have an initial set of
distributions, one for each allowed temperature $\beta$, written as
$p_{x}(t_{i}|\beta)$. As mentioned above, we do not assume that each of those
distributions is at equilibrium for the associated temperature. Physically, we
imagine a set of different physical apparatuses, each connected to its own
reservoir with (inverse) temperature $\beta$, and each with its own
distribution, $p_{x}(t_{i}|\beta)$, where we randomly pick one of those
apparatuses, without knowing which one we have picked, by randomly sampling a
distribution $\mathrm{d}F_{\beta}=f(\beta)\mathrm{d}\beta$. Write the
associated $\beta$-averaged initial distribution over $x$ as

$\displaystyle\bar{p}_{x}(t_{i})=\sum_{\beta}f(\beta)p_{x}(t_{i}|\beta)$ (44)

After choosing one of the apparatuses this way, we apply some protocol, ending
with some effective distribution

$\displaystyle\bar{p}_{x}(t_{f})=\sum_{\beta}f(\beta)p_{x}(t_{f}|\beta)$ (45)

We also have a target ending distribution, ${p^*}_{x}$, and want to evolve
$\bar{p}_{x}(t_{i})$ to that target distribution. We are interested in
minimizing the (dissipated) work required to do this. To simplify the
analysis, we assume that there is one and only one Hamiltonian
$H^*_{t_{f}}(x)$ such that the target distribution is a $\beta$-average of the
associated equilibrium distribution for Hamiltonian $H^*_{t_{f}}$, i.e.,

$\displaystyle p^*_{x}$ $\displaystyle=\sum_{\beta}f(\beta)\dfrac{e^{-\beta
H^*_{t_{f}}(x)}}{Z(H^*_{t_{f}},\beta)}$ (46)
$\displaystyle=\sum_{\beta}f(\beta)\dfrac{e^{-\beta
H^*_{t_{f}}(x)}}{\sum_{x^{\prime}}e^{-\beta H^*_{t_{f}}(x^{\prime})}}$ (47)

If the protocol changes $p_{x}(t_{i})\rightarrow p^*_{x}$ in finite time, then
in general it will not end with each distribution $p_{x}(t_{f}|\beta)$ equal
to the associated Boltzmann distribution. (So if we were to watch the system
evolve after the ending time $t=1$, we would see it change from $p^*$.) Even
if the protocol takes infinite time though, in general there will be more than
one protocol that sends the initial distribution $p_{x}(t_{i})$ to
$p_{x}(t_{f})=p^*_{x}$. Whatever the protocol is, for each $\beta$ there will
be an associated amount of EP that would arise if that apparatus were the one
chosen in our initial sampling process. The $\beta$-average of those EPs is
the $\beta$-average of the potentially extractable work that will be
dissipated due to our not being able to tailor the protocol to match the
actual apparatus and its inverse temperature $\beta$. Our goal is to find
which protocol generates minimal $\beta$-averaged EP while ending with the
target distribution.

In this section we restrict attention to a protocol that comprises a
conventional {quench; equilibrate; semistatically-evolve} process
parrondo2015thermodynamics ; hasegawa2010generalization . Note that in the
intermediate step, after the initial quench, we wait an arbitrarily long time,
to allow the system to thermalize for the new (post-quench) Hamiltonian, for
whatever $\beta$ the system happens to have. So whatever semi-static evolution
the system undergoes after that step, it will generate no EP, i.e., all of the
$\beta$-averaged EP is generated in the thermalization step.

Define $H^*_{t_{i}}(x)$ as the post-quench Hamiltonian that minimizes that
$\beta$-averaged EP. To solve for $H^*_{t_{i}}(x)$, first note that for each
$\beta$, the EP generated as $P_{t_{i}}(x|\beta)$ relaxes to the associated
equilibrium is given the Kullback Liebler divergence between two distributions
over $X$:

$\displaystyle D\left(p_{X}({t_{i}}|\beta)\;\bigg{|}\bigg{|}\;\dfrac{e^{-\beta
H^*_{t_{i}}(X)}}{Z(H^*_{t_{i}},\beta)}\right)$ (48)

The $\beta$-averaged EP is the $\beta$-average of this KL divergence. Since
the $\beta$-averaged entropy of $p_{x}(t_{i}|\beta)$ is independent of
$H^*_{t_{i}}(x)$, this means that $H^*_{t_{i}}(x)$ is the minimizer of

$\displaystyle\sum_{\beta}f(\beta)\left[\sum_{x}p_{x}(t_{i}|\beta)\beta
H^*_{t_{i}}(x)+\ln Z(H^*_{t_{i}},\beta)\right]$ (49)

Differentiating Eq. 63 with respect to $H^*_{t_{i}}(x)$ for each $x$ gives a
set of coupled equations, one for each $x$:

$\displaystyle\sum_{\beta}f(\beta)\beta\left[p_{x}(t_{i}|\beta)-\dfrac{e^{-\beta
H^*_{t_{i}}(x)}}{\sum_{x^{\prime}}e^{-\beta
H^*_{t_{i}}(x^{\prime})}}\right]=0$ (50)

or if we define $g(x):=\sum_{\beta}f(\beta)\beta p_{x}(t_{i}|\beta)$,

$\displaystyle g(x)=\sum_{\beta}f(\beta)\beta\dfrac{e^{-\beta
H^*_{t_{i}}(x)}}{\sum_{x^{\prime}}e^{-\beta H^*_{t_{i}}(x^{\prime})}}$ (51)

(Compare to Eq. 47.)

The effective reversible work expended during the semi-static evolution is the
change in equilibrium (!) free energy during the semi-static evolution,

$\displaystyle W^{ss}$
$\displaystyle:=\sum_{\beta}f(\beta)\left[F_{\beta}(H^*_{t_{f}})-F_{\beta}(H^*_{t_{i}})\right]$
(52) $\displaystyle=\sum_{\beta}f(\beta)\beta^{-1}\left[\ln
Z(\beta,H^*_{t_{i}})-\ln Z(\beta,H^*_{t_{f}})\right]$ (53)

The associated change in effective energy is

$\displaystyle\Delta
E^{ss}:=\sum_{\beta,x}f(\beta)\left[H^*_{t_{f}}(x)\dfrac{e^{-\beta
H^*_{t_{f}}(x)}}{Z(H^*_{t_{f}},\beta)}-H^*_{t_{i}}(x)\dfrac{e^{-\beta
H^*_{t_{i}}(x)}}{Z(H^*_{t_{i}},\beta)}\right]$ (54)

The difference of those is the total heat during the semi-static evolution,
$Q^{ss}$.

If we plug the optimal $H^*_{t_{i}}(x)$ solving Eq. 51 into Eq. 48, we get a
formula for the $\beta$-averaged dissipated work during the full protocol,
$W^{diss}$. In addition, if we write the initial, pre-quench Hamiltonian as
$H^-_{t_{i}}(x)$, then the work during the quench is

$\displaystyle
W^q:=\sum_{\beta,x}f(\beta)p_{x}(t_{i}|\beta)\left[H^*_{t_{i}}(x)-H^-_{t_{i}}(x)\right]$
(55)

(In general, the dissipated work during the full protocol will differ from the
actual work, since the change in entropy as the system relaxes to its
equilibrium distribution is work that could have been extracted but wasn’t.)

If we add $W^{q}$ to $W^{ss}$, we get the total reversible work during the
full protocol. Now adding the $\beta$-averaged dissipated work gives the total
work during the full protocol,

$\displaystyle W^f=W^{ss}+W^q+W^{diss}$ (56)

which provides a bound to the question motivating our analysis, of how much
work can be extracted. (The reason this is only a bound is that we have not
proven that in the scenario considered here, the {quench; equilibrate;
semistatically-evolve} process is optimal.)

In addition, the total change in internal energy during the full protocol is

$\displaystyle\Delta E^f$ $\displaystyle:=\Delta E^{ss}+W^q$
$\displaystyle\qquad+\sum_{\beta,x}f(\beta)H^*_{t_{i}}(x)\left[\dfrac{e^{-\beta
H^*_{t_{i}}(x)}}{Z(\beta,H^*_{t_{i}})}-p_{x}(t_{i}|\beta)\right]$ (57)
$\displaystyle=W^q+\sum_{\beta,x}f(\beta)\left[H^*_{t_{f}}(x)\dfrac{e^{-\beta
H^*_{t_{f}}(x)}}{Z(H^*_{t_{f}},\beta)}-p_{x}(t_{i}|\beta)\right]$ (58)

Now recall that in conventional, equilibrium thermodynamics, if we move a
system isothermally and semi-statically from one equilibrium to another, the
difference between the work expended and the change in internal energy is the
change in entropy (up to a multiplicative factor given by the temperature). We
would like an analogous interpretation to hold here. That means we want to be
able to interpret the difference $W^f-\Delta E^f$ as the change in a
generalized entropy during the full protocol, i.e., we want there to be some
functional $S^\dagger$ such that we can write

$\displaystyle S^\dagger(p({t_{i}}))-S^\dagger(p^*)$
$\displaystyle\;\;=W^{diss}+W^{ss}-\sum_{\beta,x}f(\beta)\left[H^*_{t_{f}}(x)\dfrac{e^{-\beta
H^*_{t_{f}}(x)}}{Z(H^*_{t_{f}},\beta)}-p_{x}(t_{i}|\beta)\right]$ (59)

##  VI Dynamics of the thermodynamic value of information

We now investigate a more controlled version of the scenario considered in the
previous section. Just like in the previous section, here we are uncertain
about the initial distribution. However, in contrast to the scenario
considered in the previous section, here we are certain that the system
evolves while coupled to a (single) heat reservoir with temperature $\beta=1$.
To emphasize this distinction, here we index the initial set of distributions
by an uncertain parameter $\alpha$ (not necessarily equal to a temperature),
writing it as $p_{x}(t_{i}|\alpha)$. So the $\alpha$-averaged distribution
over $x$ is

$\displaystyle p_{x}(t_{i})=\sum_{\alpha}f(\alpha)p_{x}(t_{i}|\alpha)$ (60)

Physically, we imagine a process where we first randomly sample $f(\alpha)$,
choosing one of the distributions, but don’t know which one was chosen. As in
the previous section, we want to apply a standard {quench; equilibrate;
semistatically-evolve} process to that system, in order to change the
(unknown) distribution $p_{x}(t_{i}|\alpha)$, whatever it is, to the
(perfectly known) desired ending distribution $p_{x}(t_{f})=P^*_{x}$ no matter
what $\alpha$ is. However, in contrast to the analysis of the previous
section, here we consider the case where the evolution takes place while the
system is coupled to some single fixed heat reservoir at temperature
$k_{B}T=1$, no matter what $\alpha$ is. In addition, we assume that there is a
(perfectly known) delay between when the initial distribution is generated and
when the {quench; equilibrate; semistatically-evolve} process begins, during
which time the system evolves according to a known rate matrix.

Our ultimate interest is in how the length of that delay affects the EP. To
begin though, we consider the simple case where the delay is zero, i.e., the
{quench; equilibrate; semistatically-evolve} process begins immediately after
the initial distribution is randomly generated. First, note that if the
distributions $p_{x}(t_{i}|\alpha)$ are not all the same, there is no single
Hamiltonian $H(x)$ we can quench to so that every $p_{x}(t_{i}|\alpha)$ (one
for each $\alpha$) is at equilibrium for that $H(x)$. So as in the previous
section, whatever $H(x)$ we quench to will mean that the equilibration step,
before the semistatic evolution, will result in a nonzero $\alpha$-average of
the (conventionally defined) EP of each of the separate distributions
$p_{x}(t_{i}|\alpha)$ as that distribution relaxes to the equilibrium
distribution defined by $H(x)$ and that $\alpha$. So all the EP arises while
the system is equilibrating.

Define $H^\dagger(x)$ as the Hamiltonian that minimizes the $\alpha$-average
of this initial EP. To solve for $H^\dagger_{t_{i}}$, first note that for each
$\alpha$, whatever Hamiltonian $H$ we use, the EP generated as
$p_{x}(t_{i}|\alpha)$ relaxes to equilibrium is

$\displaystyle D\left(p_{x}(t_{i}|\alpha)\;||\;\dfrac{e^{-H(x)}}{Z(H)}\right)$
(61)

where as usual the partition function is

$\displaystyle Z(H):=\sum_{x}e^{-H(x)}$ (62)

The $\alpha$-averaged EP is the $\alpha$-average of this KL divergence. Since
the $\alpha$-averaged entropy of $P_{t_{i}}(x|\alpha)$ is independent of
$H(x)$, $H^\dagger_{t_{i}}(x)$ is the minimizer of

$\displaystyle\sum_{x}p_{x}({t_{i}})\left[H(x)+\ln Z(H)\right]=\ln
Z(H)+\sum_{x}p_{x}({t_{i}})H(x)$ (63)

where

$\displaystyle p_{x}({t_{i}}):=\sum_{\alpha}f(\alpha)p_{x}(t_{i}|\alpha)$ (64)

Differentiating Eq. 63 with respect to $H(x)$ for each $x$ gives

$\displaystyle p_{x}{t_{i}}$
$\displaystyle=\dfrac{e^{-H(x)}}{\sum_{x^{\prime}}e^{-H(x^{\prime})}}$ (65)

So

$\displaystyle H^\dagger_{t_{i}}(x)=-\ln P_{x}({t_{i}})$ (66)

Note that this is the exact result one would have derived if one had simply
marginalized over the uncertainty about the initial distribution and then
applied the conventional calculation of what quenched Hamiltonian results in
minimal EP.

Write $S\left(P_{t_{i}}(X|\alpha)\right)$ for the conditional entropy of $X$
given $\alpha$ under joint distribution
$P_{t_{i}}(X,\alpha)=f(\alpha)p_{t_{i}}(X|\alpha)$. Then we can write the EP —
all generated in the stage where the system equilibrates — as

$\displaystyle\sum_{\alpha}P(\alpha)D\left(P_{t_{i}}(X|\alpha)\;||\;\dfrac{e^{-H^\dagger_{t_{i}}(X)}}{Z(H^\dagger_{t_{i}})}\right)$
$\displaystyle\qquad\qquad\qquad=\langle
H^\dagger_{t_{i}}\rangle_{P_{t_{i}}}-S\left(P_{t_{i}}(X|\alpha)\right)+\ln
Z(H^\dagger_{t_{i}})$ (67)
$\displaystyle\qquad\qquad\qquad=S\left(P_{t_{i}}(X)\right)-S\left(P_{t_{i}}(X|\alpha)\right)$
(68) $\displaystyle\qquad\qquad\qquad=I_{P_{t_{i}}}(X;\alpha)$ (69)

which is just the Jensen Shannon divergence of the set of distributions
$\\{P(x|\alpha)\\}$, weighted according to $P(\alpha)$.

This Jensen Shannon divergence is just the usual expression for the
“thermodynamic value of information”, i.e., for the amount that the minimal
extractable work changes if we can measure some statistic $\alpha$ concerning
the initial distribution of the state of the system compared to how much we
can extract if we can’t measure that statistic.

As a variation of the scenario considered above, suppose that we are delayed
in when we can start the quench process, until some time $\tau>0$ after
$P(\alpha)$ is sampled. In other words, suppose that after $\alpha$ is
randomly chosen, but before the quench process starts, the system evolves
according to a known rate matrix, $W^{x^{\prime}}_{x}(t)$, that is the same
for all $\alpha$, up to some time $\tau$, when the quench process starts. If
we take the time-derivative of this value of information with respect to
$\tau$ we get

$\displaystyle\dfrac{dI_{P_{\tau}}(X;\alpha)}{d\tau}$
$\displaystyle=\dfrac{dS\left(P_{\tau}(X)\right)}{d\tau}-\dfrac{dS\left(P_{\tau}(X|\alpha)\right)}{d\tau}$
(70)
$\displaystyle=\sum_{\alpha}P(\alpha)\sum_{x,x^{\prime}}W^{x^{\prime}}_{x}(\tau)P_{\tau}(x^{\prime}|\alpha)\left[\ln\dfrac{\sum_{\alpha^{\prime}}P({\alpha^{\prime}})W^{x^{\prime}}_{x}(\tau)P_{\tau}(x^{\prime}|{\alpha^{\prime}})}{\sum_{\alpha^{\prime}}P({\alpha^{\prime}})W^{x}_{x^{\prime}}(\tau)P_{\tau}(x|{\alpha^{\prime}})}-\ln\dfrac{W^{x^{\prime}}_{x}(\tau)P_{\tau}(x^{\prime}|\alpha)}{W^{x}_{x^{\prime}}(\tau)P_{\tau}(x|\alpha)}\right]$
(71)

This difference in derivatives of entropies equals the difference between two
EP rates, distinguished by whether we know $\alpha$ or not; the EF rate
doesn’t depend on our knowledge about $\alpha$, and so the two EF rates (one
for knowing $\alpha$, one for averaging over $\alpha$) cancel.

In addition, if we multiply and divide by $P(\alpha)$ inside the rightmost
logarithm in Eq. 71, and then apply the log-sum inequality, we see that
$I_{P_{\tau}}(X;\alpha)$ is monotonically increasing in time. (We can derive
the same result with the data-processing inequality.) Physically, this means
that the minimal dissipated work that would arise if we used our {quench;
equilibrate; semistatically-evolve} process grows the longer we wait to use
it, assuming the system is undergoing $\alpha$-independent dynamics.

Next, suppose that $\alpha$ indexes not just the distribution, but also the
rate matrix, i.e., suppose that we are unsure about the dynamics as well as
the distribution. Then we must write $W^{x^{\prime}}_{x}(\alpha,\tau)$ instead
of $W^{x^{\prime}}_{x}(\tau)$, and the dynamics of the value of information
becomes

$\displaystyle\dfrac{dI_{P_{\tau}}(X;\alpha)}{d\tau}$
$\displaystyle=\sum_{\alpha}P(\alpha)\sum_{x,x^{\prime}}W^{x^{\prime}}_{x}(\alpha,\tau)P_{\tau}(x^{\prime}|\alpha)\left[\ln\dfrac{\sum_{\alpha^{\prime}}P({\alpha^{\prime}})W^{x^{\prime}}_{x}(\alpha^{\prime},\tau)P_{\tau}(x^{\prime}|{\alpha^{\prime}})}{\sum_{\alpha^{\prime}}P({\alpha^{\prime}})W^{x}_{x^{\prime}}(\alpha^{\prime},\tau)P_{\tau}(x|{\alpha^{\prime}})}-\ln\dfrac{W^{x^{\prime}}_{x}(\alpha,\tau)P_{\tau}(x^{\prime}|\alpha)}{W^{x}_{x^{\prime}}(\alpha,\tau)P_{t}(x|\alpha)}\right]$
(72)

Now the data-processing inequality doesn’t apply. However, we can still use
the log-sum inequality, to again conclude that $I_{P_{\tau}}(X;\alpha)$ is
monotonically increasing in time.

A natural question is whether adding uncertainty to the rate matrix can cause
the rate of increase in time of $I_{P_{\tau}}(X;\alpha)$ to shrink, e.g.,
formalized as whether the following quantity can be negative:

$\displaystyle\sum_{\alpha}P(\alpha)\sum_{x,x^{\prime}}\left[W^{x^{\prime}}_{x}(\alpha,\tau)P_{\tau}(x^{\prime}|\alpha)\ln\dfrac{\sum_{\alpha^{\prime}}P({\alpha^{\prime}})W^{x^{\prime}}_{x}(\alpha^{\prime},\tau)P_{\tau}(x^{\prime}|{\alpha^{\prime}})}{\sum_{\alpha^{\prime}}P({\alpha^{\prime}})W^{x}_{x^{\prime}}(\alpha^{\prime},\tau)P_{\tau}(x|{\alpha^{\prime}})}-\overline{W}^{x^{\prime}}_{x}(\tau)P_{\tau}(x^{\prime}|\alpha)\ln\dfrac{\sum_{\alpha^{\prime}}P({\alpha^{\prime}})\overline{W}^{x^{\prime}}_{x}(\tau)P_{\tau}(x^{\prime}|{\alpha^{\prime}})}{\sum_{\alpha^{\prime}}P({\alpha^{\prime}})\overline{W}^{x^{\prime}}_{x}(\tau)P_{\tau}(x|{\alpha^{\prime}})}\right]$
(73)

where
$\overline{W}^{x^{\prime}}_{x}(\tau):=\sum_{\alpha^{\prime\prime}}P(\alpha^{\prime\prime})W^{x}_{x^{\prime}}(\alpha^{\prime\prime},\tau)$.
In the special case where all distributions $P(x|\alpha)$ are the same, so
that the only uncertainty is in the rate matrix, the expression in Eq. 73
equals $0$. Nonetheless, by applying the log-sum inequality to Eq. 72 for
$P_{t}(x|\alpha)$ independent of $\alpha$ we see that the rate of increase of
the value of information is still non-negative, even when our only uncertainty
concerns the rate matrix.

##  VII Discussion and Future work

In any real-world experimental test of a system there are three major types of
uncertainty: Uncertainty about the state of the system; uncertainty about the
state of the external environment that the system is interacting with;
uncertainty about the parameters of the equations governing the dynamics of
the system and its interaction with its external environment. In (classical)
stochastic thermodynamics, the first type of uncertainty is addressed by
replacing specification of the system’s state, and the second one is typically
addressed by assuming the environment is infinite, at equilibrium, and
evolving far faster than does the system (“separation of time scales”). In
essence, the entire field of stochastic thermodynamics concerns the
consequences of those two types of uncertainty for the dynamics of energy and
particle counts in the system. However, very little attention (if any) has
been paid so far to the third type of uncertainty. Here we begin an
investigation of the consequences of that third type of uncertainty, showing
that it entails major modifications to the standard results previously derived
in stochastic thermodynamics.

Our investigations have only scratched the surface of issues involved with
this third type of uncertainty. Some of the more immediate questions to be
addressed in future work include:

  1. 1.

For pedagogical simplicity we did not explicitly model uncertainty in the
energy spectrum and / or associated dynamics of the rate matrix in the
analysis in Section III. Future work involves analyzing the relationship
between the thermodynamic consequences of those uncertainties and the
thermodynamic consequences of the uncertainties that we did explicitly
consider in Section III.

  2. 2.

How does the analysis in Section IV change if we include low-order expansions
of other parameters besides temperature?

  3. 3.

The approach outlined in Section V defines generalized entropy using the total
work expended during the {quench; equilibrate; semistatically-evolve} process
that takes $(P_{t_{i}}(x),H^-_{t_{i}}(x))\rightarrow(P^*(x),H^*_{t_{f}}(x))$
with minimal dissipated work. Is that process the same as the one that
minimizes the total work during any (quench-thermalize-evolve) process that
takes $(P_{t_{i}}(x),H^-_{t_{i}}(x))\rightarrow(P^*(x),H^*_{t_{f}}(x))$?

  4. 4.

Is there any alternative protocol which, like the {quench; equilibrate;
semistatically-evolve} process discussed above is ignorant of $\beta$, and
which also maps $P_{t_{i}}(x)\rightarrow P^*(x)$ in infinite time, but has
less $\beta$-averaged EP than the quench-thermalize-evolve protocol?

  5. 5.

Is there any such process which finishes in finite time, so that
$P_{t_{f}}(x|\beta)$ is not an equilibrium distribution for $H_{t_{f}}(x)$ for
all $\beta$?

  6. 6.

Is there any alternative protocol to the {quench; equilibrate; semistatically-
evolve} process considered in Section VI which is also ignorant of $\alpha$,
and which also maps each possible initial distribution $P_{t_{i}}(x|\alpha)$
to $P_{t_{f}}(x)$, but has less $\alpha$-averaged EP than
$I_{P_{t_{i}}}(X;\alpha)$?

## References

  * (1) C. Beck and E.G.D. Cohen, _Superstatistics_ , Physica A 322 (2003), 267–275.
  * (2) Gili Bisker, Matteo Polettini, Todd R Gingrich, and Jordan M Horowitz, _Hierarchical bounds on entropy production inferred from partial information_ , Journal of Statistical Mechanics: Theory and Experiment 2017 (2017), no. 9, 093210.
  * (3) Sergio Davis and Gonzalo Gutiérrez, _Temperature is not an observable in superstatistics_ , Physica A: Statistical Mechanics and its Applications 505 (2018), 864–870.
  * (4) Massimiliano Esposito, Katja Lindenberg, and Christian Van den Broeck, _Entropy production as correlation between system and reservoir_ , New Journal of Physics 12 (2010), no. 1, 013013.
  * (5) Massimiliano Esposito and Christian Van den Broeck, _Three faces of the second law. i. master equation formulation_ , Physical Review E 82 (2010), no. 1, 011143.
  * (6) Carlos E. Fiore and Mário J. de Oliveira, _Entropy production and heat capacity of systems under time-dependent oscillating temperature_ , Phys. Rev. E 99 (2019), 052131.
  * (7) Vladimir García-Morales and Katharina Krischer, _Superstatistics in nanoscale electrochemical systems_ , Proceedings of the National Academy of Sciences 108 (2011), no. 49, 19535–19539.
  * (8) Rudolf Hanel, Stefan Thurner, and Murray Gell-Mann, _Generalized entropies and the transformation group of superstatistics_ , Proceedings of the National Academy of Sciences 108 (2011), no. 16, 6390–6394.
  * (9) H-H Hasegawa, J Ishikawa, K Takara, and DJ Driebe, _Generalization of the second law for a nonequilibrium initial state_ , Physics Letters A 374 (2010), no. 8, 1001–1004.
  * (10) R. Kawai, J. M. R. Parrondo, and Christian Van den Broeck, _Dissipation: the phase-space perspective_ , Physical Review Letters 98 (2007), no. 8, 080602.
  * (11) Miki Matsuo and Shin ichi Sasa, _Stochastic energetics of non-uniform temperature systems_ , Physica A 276 (2000), no. 1, 188–200.
  * (12) We note though there is a substantial literature in which adopts an “inclusive Hamiltonian” framework, in which the external environment is finite, and the joint dynamics of the system-environment is explicitly modeled. This has been explored either with explicit Hamiltonian dynamics, with only randomness in the initial state kawai_dissipation:_2007 ; esposito2010entropy ; ptaszynski2018first  or at least approximate Hamiltonian dynamics seifert2016first ; strasberg2017stochastic ; talkner2020HMF; talkner2020comment ; strasberg2020measurability .
  * (13) We note though that it is already known that if we do not account for all thermodynamic reservoirs, we invariably undercount the total of entropy production in a process esposito2010three .
  * (14) Uncertainty in the energy spectrum and / or the time-dependence of the rate matrix can also be included in the analysis, at the cost of more complex notation.
  * (15) Juan MR Parrondo, Jordan M Horowitz, and Takahiro Sagawa, _Thermodynamics of information_ , Nature Physics 11 (2015), no. 2, 131–139.
  * (16) Krzysztof Ptaszyński, _First-passage times in renewal and nonrenewal systems_ , Physical Review E 97 (2018), no. 1, 012127.
  * (17) Fabio Sattin, _Bayesian approach to superstatistics_ , The European Physical Journal B-Condensed Matter and Complex Systems 49 (2006), no. 2, 219–224.
  * (18) Udo Seifert, _Stochastic thermodynamics, fluctuation theorems and molecular machines_ , Reports on Progress in Physics 75 (2012), no. 12, 126001.
  * (19) , _First and second law of thermodynamics at strong coupling_ , Physical Review Letters 116 (2016), no. 2, 020601.
  * (20) Ito S. Kawaguchi K. Shiraishi, N. and T. Sagawa, _Role of measurement-feedback separation in autonomous Maxwell’s demons_ , New Journal of Physics (2015).
  * (21) Philipp Strasberg and Massimiliano Esposito, _Stochastic thermodynamics in the strong coupling regime: An unambiguous approach based on coarse graining_ , Physical Review E 95 (2017), no. 6, 062101.
  * (22) , _Non-markovianity and negative entropy production rates_ , Physical Review E 99 (2019), no. 1, 012120.
  * (23) , _Measurability of nonequilibrium thermodynamics in terms of the hamiltonian of mean force_ , Physical Review E 101 (2020), no. 5, 050101\.
  * (24) Peter Talkner and Peter Hänggi, _Comment on" measurability of nonequilibrium thermodynamics in terms of the hamiltonian of mean force”_ , arXiv preprint arXiv:2006.16592 (2020).
  * (25) Christian Van den Broeck and Massimiliano Esposito, _Ensemble and trajectory thermodynamics: A brief introduction_ , Physica A: Statistical Mechanics and its Applications 418 (2015), 6–16.
  * (26) David H Wolpert et al., _On the use of evidence in neural networks_ , Advances in neural information processing systems (1993), 539–539.

##  Appendix A Brief review of stochastic thermodynamics

$\dot{p}_{x}(t)=\sum_{x^{\prime}}K_{x,x^{\prime}}p_{x^{\prime}}(t)=\sum_{x^{\prime}}J_{x,x^{\prime}}\,.$
(74)

Transition rates can be decomposed as

$K_{x,x^{\prime}}=\sum_{\nu=1}^{N}K_{x,x^{\prime}}^{\nu}\,.$ (75)

The local equilibrium distribution is defined as

$\pi^{\nu}_{x}=\frac{\exp(-\beta_{\nu}(\epsilon_{x}-\mu_{\nu}n_{x}))}{Z^{\nu}}\,.$
(76)

Transition rates fulfill local detailed balance, so

$\frac{K^{\nu}_{x,x^{\prime}}}{K^{\nu}_{x^{\prime}x}}=\frac{\pi^{\nu}_{x}}{\pi^{\nu}_{x^{\prime}}}=e^{-\beta_{\nu}[(\epsilon_{x}-\epsilon_{x^{\prime}})-\mu_{\nu}(n_{x}-n_{x^{\prime}})]}\,.$
(77)

Thus, local flows

$J_{x,x^{\prime}}^{\nu}=K_{x,x^{\prime}}^{\nu}p_{x^{\prime}}(t)-K_{x^{\prime}x}^{\nu}p_{x}(t)$
(78)

vanish for local equilibrium distributions.

We define internal energy of the system as

$U=\sum_{x}p_{x}\epsilon_{x}.$ (79)

The first law of thermodynamics can be formulated as follows:

$\dot{U}=\dot{W}+\dot{Q}+\dot{W}_{chem}$ (80)

where

$\displaystyle\dot{W}$ $\displaystyle=$
$\displaystyle\sum_{x}p_{x}\dot{\epsilon}_{x}$ (81) $\displaystyle\dot{Q}$
$\displaystyle=$
$\displaystyle\sum_{\nu}\dot{Q}^{\nu}=\sum_{\nu}\sum_{x}\dot{p}_{x}\left(\epsilon_{x}-\mu_{\nu}n_{x}\right)$
(82) $\displaystyle\dot{W}_{chem}$ $\displaystyle=$
$\displaystyle\sum_{\nu}\dot{W}_{chem}^{\nu}=\sum_{\nu}\sum_{x}\dot{p}_{x}\mu_{\nu}n_{x}$
(83)

Entropy of the system is defined as

$S=-\sum_{x}p_{x}\ln p_{x}$ (84)

Second law of thermodynamics can be expressed as follows:

$\dot{S}=\dot{S}_{i}+\dot{S}_{e}$ (85)

where

$\displaystyle\dot{S}_{i}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\sum_{x,x^{\prime}\nu}J_{x,x^{\prime}}^{\nu}\ln\frac{w_{x,x^{\prime}}^{\nu}p_{x}}{K_{x^{\prime}x}^{\nu}p_{x^{\prime}}}\geq
0$ (86) $\displaystyle\dot{S}_{e}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\sum_{x,x^{\prime}\nu}J_{x,x^{\prime}}^{\nu}\ln\frac{w_{x,x^{\prime}}^{\nu}}{K_{x^{\prime}x}^{\nu}}$
(87) $\displaystyle=$ $\displaystyle\sum_{\nu}\beta_{\nu}\dot{Q}^{\nu}\,.$

### Trajectory thermodynamics and fluctuation theorems:

We consider a stochastic trajectory $x(\tau)$. We define stochastic energy as
$\epsilon(\tau)=\epsilon_{x(\tau)}$. We also consider that particle levels
$n_{x}$ are independent of time. The first law of thermodynamics can be
expressed on the trajectory level as

$\frac{\mathrm{d}}{\mathrm{d}t}e(\tau)=\dot{w}(\tau)+\dot{q}(\tau)+\dot{w}_{chem}(\tau)$
(88)

where

$\displaystyle\dot{w}$ $\displaystyle=$
$\displaystyle\sum_{x}\delta_{x,x(\tau)}\dot{\epsilon}_{x(\tau)}$ (89)
$\displaystyle\dot{q}+\dot{w}_{chem}(\tau)$ $\displaystyle=$
$\displaystyle\sum_{x}\dot{\delta}_{x,x(\tau)}\epsilon_{x(\tau)}$ (90)

It is important to stress that the first law of thermodynamics is the pure
energetic balance which does depends neither on the number of reservoirs nor
on their parameters. Heat and chemical work can be decomposed into
contributions from particular reservoirs

$\displaystyle\dot{q}=\sum_{\nu}\dot{q}^\nu(\tau)=\sum_{\nu}\dot{\delta}_{x,x(\tau)}(\epsilon_{x(\tau)}-\mu_{x}n_{x(\tau)})$
(91)
$\displaystyle\dot{w}_{chem}=\sum_{\nu}\dot{w}^\nu_{chem}(\tau)=\sum_{\nu}\dot{\delta}_{x,x(\tau)}\mu_{x}n_{x(\tau)}$
(92)

Stochastic entropy along a trajectory $x(\tau)$ can be defined as

$s_{x(\tau)}=-\ln p_{x(\tau)}\,.$ (93)

It is straightforward to show that stochastic entropy fulfills the second law
of thermodynamics, i.e.,

$\dot{s}=\dot{s}_{i}+\dot{s}_{e}$ (94)

where

$\dot{s}_{e}=\sum_{\nu}\beta_{\nu}\dot{q}^\nu$ (95)

By averaging over the trajectory ensemble, we obtain the ensemble second law
of thermodynamics in the form $\langle\dot{s}_{i}\rangle=\dot{S}_{i}\geq 0$.

Stochastic entropy production satisfies the well-known fluctuation theorem,
which is a consequence of the following expression:

$\ln\frac{\mathcal{P}_{x(\tau)}}{\tilde{\mathcal{P}}_{\tilde{x}(\tau)}}=\Delta
s_{i}$ (96)

where $\mathcal{P}_{x(\tau)}$ is the probability of stochastic trajectory
$x(\tau)$ for a protocol $\lambda(\tau)$ and
$\tilde{\mathcal{P}}_{\tilde{x}(\tau)}$ is the probability of observing time-
reversed trajectory $\tilde{x}(\tau)=x(T-\tau)$ for a time-reversed protocol
$\tilde{\lambda}(\tau)=\lambda(T-\tau)$. It is straightforward to show that
$\Delta s_{i}$ obeys the detailed fluctuation theorem (DFT)

$\frac{P(\Delta s_{i})}{\tilde{P}(-\Delta s_{i})}=e^{\Delta s_{i}}$ (97)

as well as integrated fluctuation theorem (IFT)

$\left\langle e^{-\Delta s_{i}}\right\rangle=1\,.$ (98)

##  Appendix B General form of transition matrix satisfying local detailed
balance

We now focus on the most general form of transition matrix which fulfills
local detailed balance. We denote transition rate matrix as
$K=\\{K_{x,x^{\prime}}\\}_{x,x^{\prime}}$ and probability distribution
$p(t)=\\{p_{x}(t)\\}_{x}$. We start with the most general form of a transition
matrix satisfying detailed balance, which is a special case of local detailed
balance when a system is coupled to one reservoir. We define the equilibrium
distribution $\pi=\\{\pi_{x}\\}_{x}$ as the solution of

$K\pi=0.$ (99)

We define $\Pi=\mathrm{diag}(\pi)$ where $\mathrm{diag}$ denotes a diagonal
matrix with diagonal elements equal to elements of the vector. Requirement of
detailed balance from Eq. (77) can be in the matrix notation rewritten as

$K\Pi=\left(K\Pi\right)^T$ (100)

We consider the following decomposition of the transition rate matrix:

$K=R\,\Pi^{-1}$ (101)

Then, we obtain from Eq. (100) that $R=R^T$, i.e., the matrix is symmetric. It
is actually possible to show that Eq. (102) is the most general decomposition
of the transition rate matrix satisfying detailed balance. The normalization
of the transition rate matrix is obtained in the diagonal elements of $R$. For
a general matrix $R^{\prime}$, the general transition rate matrix can be
expressed as

$K=R^{\prime}\Pi^{-1}-\mathrm{diag}\left(R^{\prime}\Pi^{-1}\cdot\textbf{1}\right)$
(102)

where $\cdot$ denotes standard application of matrix to a vector and
$\textbf{1}=(1,\dots,1)$. In the following, we always assume that the matrix
$R$ is chosen so that $K$ is normalized ($\sum_{x}K_{x,x^{\prime}}=0$).
Consequently, the probability flow can be expressed as follows:

$J_{x,x^{\prime}}=R_{x,x^{\prime}}\left(\frac{p_{x^{\prime}}(t)}{\pi_{x^{\prime}}}-\frac{p_{x}(t)}{\pi_{x}}\right)$
(103)

from which we immediately see that the system satisfies detailed balance. A
matrix satisfying local detailed balance (LDB-obeying matrix) is then a sum of
matrices satisfying detailed balance, so

$K=\sum_{\nu}K^\nu=\sum_{\nu=1}^NR^\nu\,\left(\Pi^\nu\right)^{-1}$ (104)

We now assume the local equilibrium distribution in the form of Boltzmann
distribution. Then, $\Pi^\nu$ is parameterized by
$\alpha^\nu=(\beta^\nu,\mu^\nu)$ (we consider that the energy levels are known
and fixed). We also denote $\mathbf{R}=\\{R^\nu\\}_{\nu=1}^N$, then a general
transition matrix satisfying local detailed balance can be characterized by
the following variables

$K(N;\bm{\theta},\mathbf{R})=\sum_{\nu=1}^NR^\nu(\Pi^\nu)^{-1}-\mathrm{diag}\left(R^\nu(\Pi^\nu)^{-1}\cdot\textbf{1}\right)$
(105)