# A fluctuation theorem for time-series of signal-response models
with the backward transfer entropy

Andrea Auconi1,2, Andrea Giansanti2,3 and Edda Klipp1<EMAIL_ADDRESS>berlin.de $1$ Theoretische Biophysik, Humboldt-Universität zu Berlin,
Invalidenstraße 42, D-10115 Berlin, Germany
$2$ Dipartimento di Fisica, Sapienza Università di Roma, Rome, Italy
$3$ INFN, Sezione di Roma 1, Rome, Italy

###### Abstract

The irreversibility of trajectories in stochastic dynamical systems is linked
to the structure of their causal representation in terms of Bayesian networks.
We consider stochastic maps resulting from a time discretization with interval
$\tau$ of signal-response models, and we find an integral fluctuation theorem
that sets the backward transfer entropy as a lower bound to the conditional
entropy production. We apply this to a linear signal-response model providing
analytical solutions, and to a nonlinear model of receptor-ligand systems. We
show that the observational time $\tau$ has to be fine-tuned for an efficient
detection of the irreversibility in time-series.

††preprint: APS/123-QED

##  I Introduction

The irreversibility of a process is the possibility to infer the existence of
a time’s arrow looking at an ensemble of realizations of its dynamicsJarzynski
(2011); Parrondo  _et al._ (2009); Feng and Crooks (2008). This concept is
formalized in thermodynamics as dissipated work or entropy productionJarzynski
(1997); Crooks (1999); Evans and Searles (2002), a quantity that relates the
probability of paths with their time-reversal conjugatesKawai  _et al._
(2007).

Fluctuation theorems have been developed to describe the statistical
properties of the entropy production and its relation to information-theoretic
quantities in both Hamiltonian and Langevin dynamicsJarzynski (2000); Chernyak
_et al._ (2006); Ito and Sagawa (2013). Particular attention was given to
measurement-feedback controlled modelsSagawa and Ueda (2012, 2010) inspired by
the Maxwell demonSzilard (1964), a gedanken-experiment in which physical work
is extracted from thermodynamic systems using information. An information
engine of this kind has been experimentally realized with a colloidal particle
in an electric potentialToyabe  _et al._ (2010).

Here, we are interested in the stochastic dynamics of autonomous
(uncontrolled) systems, where the irreversibility of trajectories results from
nonconservative forcesChernyak  _et al._ (2006).

We propose a generalization of the entropy production to the case of time-
series resulting from a discretization with interval $\tau$ of continuous
models. We call it mapping irreversibility and we use the symbol
$\varPhi_{\tau}$. Our motivation is a future use of the stochastic
thermodynamics framework in the analysis of time-series data in biology and
finance.

A key quantity here is the observational time $\tau$, and the detection of the
irreversibility of processes is based on a fine-tuning of this parameter. We
discuss this point with a model of receptor-ligand systems, where the entropy
production measures the robustness of signaling.

We define signal-response models as continuous-time stationary processes
characterized by the absence of feedback. In the bidimensional case they
consist of a fluctuating signal $x$ and a dynamic response $y$. In a recent
workAuconi  _et al._ (2017) we studied the information processing properties
of linear multidimensional signal-response models and defined a measure of
causal influence. Such definition is meaningful only in linear signal-response
models.

The backward transfer entropy $T_{y\rightarrow x}(-\tau)$ is the standard
transfer entropyCover and Thomas (2012) calculated in the ensemble of time-
reversed trajectories, and it was shown to be related to the divergence of the
dynamics from a hidden Markov modelIto (2016).

On a bivariate framework with dynamic variables $x$ and $y$, we define the
conditional entropy production $\varPhi_{\tau}^{y|x}$ of $y$ given $x$ with
observational time $\tau$ (or conditional mapping irreversibility) as the
difference between the entropy production of the two-dimensional system
$(x,y)$ and the entropy production of the variable $x$ alone (with
observational time $\tau$),
$\varPhi_{\tau}^{y|x}=\varPhi_{\tau}^{xy}-\varPhi_{\tau}^{x}$.

We find an integral fluctuation theorem for signal-response models that
involves the backward transfer entropy, and that is valid for any stationary
(nonlinear) signal-response model. From this it follows the II law of
thermodynamics for signal-response models, i.e. that the backward transfer
entropy $T_{y\rightarrow x}(-\tau)$ of the response $y$ on the past of the
signal $x$ is a lower bound to the conditional mapping irreversibility
$\varPhi_{\tau}^{y|x}$:

$\displaystyle\varPhi_{\tau}^{y|x}\geq T_{y\rightarrow x}(-\tau).$ (1)

This is our main result, and it shows the connection between the
irreversibility of a process and the information flows towards the past
between variables. For the basic linear response model (BLRM discussed in
Auconi  _et al._ (2017)), in the limit of small observational time $\tau$ the
backward transfer entropy converges to the causal influence.

The paper is structured as follows. In section II.A we state the setting and
formalism we use for the stochastic thermodynamics of time-series in the
bivariate case, we motivate why we are interested in autonomous systems, we
define the (conditional) mapping irreversibility, we introduce the spatial
density of entropy production and we review the general integral fluctuation
theoremSeifert (2012). In section II.B we derive the fluctuation theorem for
signal-response models involving the backward transfer entropy, and in section
II.C we show how it applies to a dynamic linear signal-response model and to a
biological model of receptor-ligand systems. In the Discussion section we
review the results and we motivate further our definition of conditional
mapping irreversibility comparing it with a possible alternative. We provide
an Appendix section where the analytical solutions for the entropy production
and for the backward transfer entropy in the BLRM are discussed.

##  II RESULTS

###  II.1 Bivariate time-series stochastic thermodynamics

####  II.1.1 Setting and definition of causal representations

Let us consider an ensemble of trajectories generated by a continuous-time
stochastic process composed of interacting variables stimulated by noise. The
stochastic differential equations describing such kind of systems can be
represented as Bayesian networks, giving the probabilistic solution for the
evolution as a function of the observational time interval $\tau$. Still,
there are multiple ways of decomposing the joint probability distribution of
states at the two instants $t$ and $t+\tau$. We say that a Bayesian network is
a causal representation of the dynamics if conditional probabilities are
expressed in a way that variables at time $t+\tau$ depend on variables at the
same time instant or on variables at the previous time instant $t$ (and not
vice-versa), and that the dependence structure is done in order to minimize
the total number of conditions on the probabilities. This corresponds to
minimizing the number of links in the Bayesian network describing the dynamics
with observational time $\tau$. This is our preferred setting to describe
information flows and causalityAuconi  _et al._ (2017), and to develop
integral fluctuation theorems for the entropy production in stochastic
thermodynamics.

In this paper we restrict ourselves to the bidimensional case with two
stochastic dynamical variables $x$ and $y$. They have an interaction,
described by the functions $g_{x}(x,y)$ and $g_{y}(x,y)$, which is in general
asymmetric ($g_{x}(x,y)\neq\pm g_{y}(x,y)$). Taking Brownian motion $dW$ as
noise stimulus, the stochastic differential equation for the general case is
written in the Ito representation(Shreve, 2004) as:

$\displaystyle\begin{cases}dx=g_{x}(x,y)dt+h_{x}(x,y)\sqrt{D_{x}}\,\,dW_{x}\\\
dy=g_{y}(x,y)dt+h_{y}(x,y)\sqrt{D_{y}}\,\,dW_{y}\end{cases}$ (2)

where $D_{x}$ and $D_{y}$ are diffusion coefficients, and the $h$ functions
are accounting for the case of multiplicative noise. Brownian motions are
characterized by $\left\langle dW_{i}dW_{j}\right\rangle=\delta_{ij}dt$, for
any $dt>0$.

We define the global variable $\zeta_{\tau}^{xy}$ as a couple of two
successive states of system $(x,y)$ separated by a time interval $\tau$,
$\zeta_{\tau}^{xy}\equiv(x(t)=x_{t},y(t)=y_{t},x(t+\tau)=x_{t+\tau},y(t+\tau)=y_{t+\tau})\equiv
f_{\tau}^{xy}(x_{t},y_{t},x_{t+\tau},y_{t+\tau})\equiv(x_{t},y_{t},x_{t+\tau},y_{t+\tau})$.
The functional form $f_{\tau}^{xy}$ is particularly convenient for the
specification of the backward global variable $\widetilde{\zeta_{\tau}^{xy}}$.
This is defined as the time-reversed conjugate of the global variable
$\zeta_{\tau}^{xy}$, meaning the inverted couple of the same two successive
states, $\widetilde{\zeta_{\tau}^{xy}}\equiv
f_{\tau}^{xy}(x_{t+\tau},y_{t+\tau},x_{t},y_{t})\equiv(\widetilde{x_{t}},\widetilde{y_{t}},\widetilde{x_{t+\tau}},\widetilde{y_{t+\tau}})$.
Correspondences of the type $\widetilde{x_{t}}=x_{t+\tau}$ are possible only
when states at both times $t$ and $t+\tau$ are given. The probability density
associated to the global variable $p(\zeta_{\tau}^{xy})$ is equal to the
probability of the starting point $p(x_{t},y_{t})$ times the path-integral of
the indicator of the trajectories that start and arrive in the states
specified by $\zeta_{\tau}^{xy}$.

####  II.1.2 Preliminary discussion of controlled systems

A protocolJarzynski (1997) for influencing the $y$ dynamics controlling the
$x$ variable, $\lambda(x)\equiv(x_{t},x_{t+\tau})$, is defined here as a
couple of successive states of the $x$ variable, $x_{t},x_{t+\tau}$, that are
kept fixed regardless of the dynamics of the $y$. Similarly to what is
postulated by Ito-Sagawa for discrete-time dynamics on Bayesian networks(Ito
and Sagawa, 2013), we assume that a detailed fluctuation theorem can be
written for causal representations of time-series as a generalization of the
detailed fluctuation theorem for single trajectories of controlled
systems(Seifert, 2005). We define the entropy production with observational
time $\tau$ (and we call it ”mapping irreversibility”)
$\varphi^{y|\lambda(x)}_{\tau}$ of system $y$ controlled by system $x$ with
protocol $\lambda(x)\equiv(x_{t},x_{t+\tau})$ for a particular realization
$(y_{t},y_{t+\tau})$ as:

$\displaystyle\varphi^{y|\lambda(x)}_{\tau}=\ln\left(\frac{p(y_{t})}{p(y_{t+\tau})}\right)+\ln\left(\frac{p_{\lambda}(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})}{p_{\lambda}(\widetilde{y_{t+\tau}}|\widetilde{y_{t}},\widetilde{x_{t}},\widetilde{x_{t+\tau}})}\right),~{}~{}~{}~{}~{}$
(3)

where the two terms on the right hand side correspond respectively to the
entropy change of system $y$ and the entropy change in the thermal bath
attached to $y$.

This is a generalization of the physical entropy production of continuous
trajectories to the case of time-series. It describes the irreversibility of a
transition in the stochastic map as a function of the observational time
$\tau$. The physical entropy production rate of the original trajectory is
found in the limit $\tau\rightarrow 0$. With this definition for time-series
(Eq.3) we are able to treat more general systems of differential equations
with mixed deterministic and stochastic variables, for which the continuous
entropy production diverges. This is the case of Maxwell demon’s deterministic
strategies, when considering the demon as part of the system. In addition, we
can now approach the data analysis of time-series in the framework of
stochastic thermodynamics, where we needed a definition of entropy production
that does not rely on a continuous-time limit.

Note that the ensemble average of the entropy production is performed over the
actual probability of the system $y$ and controlling variable $x$,
$p(\zeta_{\tau}^{xy})$, where the control protocol
$\lambda(x)\equiv(x_{t},x_{t+\tau})$ may be influenced by the system $y$
dynamics. In $p_{\lambda}(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$ the control
protocol $\lambda(x)$ fixes the two states of the controlling variable $x$ at
times $t$ and $t+\tau$ as in $p(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$, but at
intermediate instants the control variable $x$ can only follow trajectories
that are compatible with the dynamics of system $x$ uninfluenced by $y$, that
means in a different model where no measurement is performed and $y$ is not
present in the differential equation for the evolution of $x$. This reduced
dynamics without measurement is described by probability
$p_{\lambda}(\zeta_{\tau}^{xy})\equiv p(\zeta_{\tau}^{y|\lambda(x)})\cdot
p_{\lambda}(x_{t},x_{t+\tau})$.

A general fluctuation theorem connecting the entropy production in a
thermodynamic system with the information used by controlling variables has
been formulated in the case of fixed Bayesian networksIto and Sagawa (2013).
With the word ”fixed” here we mean independent of a time discretization. There
it is assumed that a fixed Bayesian network is the complete description of the
dynamics, so that feedbacks between variables over time arise only from the
structure of the directed graph. This assumption was crucial because the
detailed fluctuation theorem (Eq.3) holds only in the absence of feedback
control, and in general one would have to calculate the probabilities of
transitions in both the real dynamics $p(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$
and the reduced dynamics $p_{\lambda}(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$, and
for a fixed Bayesian network these two coincide. In general, when the Bayesian
network is a causal representation of a continuous-time dynamics, the two
situations with and without feedback correspond to two different Bayesian
networks, and the transition probabilities associated to these two different
models are different, $p_{\lambda}(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})\neq
p(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$, and also not related in a simple way.
Therefore the general fluctuation theorem for fixed Bayesian networksIto and
Sagawa (2013) does not hold for causal representations.

More explicitly, if we have knowledge of the state of a continuous time system
$(x,y)$ at time $t$, that is $(x_{t},y_{t})$, then our estimate on the
evolution of the variable $y$ at time $t+\tau$, $y_{t+\tau}$, would be
influenced by a further knowledge on the evolution of the variable $x$ at time
$t+\tau$, $x_{t+\tau}$, not only because $x$ is directly influencing $y$ ($x$
is present in the differential equation for $y$), but also because if $y$ is
directly influencing $x$ ($y$ is present in the differential equation for $x$)
then the knowledge of the $x$ transition $x_{t},x_{t+\tau}$ gives information
on $y$ at times in between $t$ and $t+\tau$ and therefore also on
$y_{t+\tau}$. Then the probability $p(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$ is
different from the same probability calculated in a model in which $y$ does
not influence $x$ ($y$ is not present in the differential equation for $x$),
$p_{\lambda}(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})\neq
p(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$. We say that these two models, i.e. one
with feedback and one without feedback, have two different causal
representations. They correspond to two different Bayesian networks with
different transition probabilities.

This is the case for most situations where Bayesian networks are built as a
discretization of continuous time models, and this is also the case of
measured time-series data from an experiment. For Langevin systems, it was
shown(Ito and Sagawa, 2013) that in the limit $\tau\rightarrow 0$ these two
probabilities coverge to each other, and the fluctuation theorem for fixed
Bayesian dynamics can be used for the entropy production rate.

We note that the Ito-Sagawa definitionIto and Sagawa (2013) of entropy
production for Bayesian networks differs from our Eq.3 also in the temporal
order in which conditioning is preformed. This is discussed in the Discussion
section and in Appendix C.

####  II.1.3 The general fluctuation theorem on autonomous systems

In this paper we study the probabilistic dynamics of systems in the absence of
controlling variables. Our main interest is to relate the irreversibility of
trajectories to the information flows between variables over time.

We define the mapping irreversibility, i.e. the stochastic entropy production
with finite observational time $\tau$, for the autonomous uncontrolled system
$(x,y)$ as a function of the global variable $\zeta_{\tau}^{xy}$ as:

$\displaystyle\varphi_{\tau}^{xy}=\ln\left(\frac{p(\zeta_{\tau}^{xy})}{p(\widetilde{\zeta_{\tau}^{xy}})}\right).$
(4)

This quantity satisfies the integral fluctuation theoremSeifert (2012), i.e.:

$\displaystyle\left\langle
e^{-\varphi_{\tau}^{xy}}\right\rangle_{p(\zeta_{\tau}^{xy})}=\int_{\Omega}d\zeta_{\tau}^{xy}p(\widetilde{\zeta_{\tau}^{xy}})=1,$
(5)

where $d\zeta_{\tau}^{xy}=dx_{t}dy_{t}dx_{t+\tau}dy_{t+\tau}$,
$\widetilde{dx_{t}}=dx_{t+\tau}$, and $\Omega$ is the whole space of the
global variable.

From the convexity of the exponential function it follows that the entropy
production $\varPhi_{\tau}^{xy}$, which is the ensemble average of the
stochastic entropy production $\varphi_{\tau}^{xy}$, is non-negative. This is
the II law of Thermodynamics for the bivariate system $(x,y)$:

$\displaystyle\varPhi_{\tau}^{xy}=\left\langle\varphi_{\tau}^{xy}\right\rangle_{p(\zeta_{\tau}^{xy})}\geq
0.$ (6)

We define the conditional mapping irreversibility of $y$ given $x$ as the
difference between the mapping irreversibility of the system $(x,y)$ and the
mapping irreversibility of system $x$ alone:

$\displaystyle\varPhi_{\tau}^{y|x}=\varPhi_{\tau}^{xy}-\varPhi_{\tau}^{x}=\left\langle\ln\left(\frac{p(y_{t},y_{t+\tau}|x_{t},x_{t+\tau})}{p(\widetilde{y_{t}},\widetilde{y_{t+\tau}}|\widetilde{x_{t}},\widetilde{x_{t+\tau}})}\right)\right\rangle_{p(\zeta_{\tau}^{xy})}.~{}~{}~{}~{}$
(7)

In the general case of a complete causal representation resulting from the
dynamics (Fig.1), where all edges are present in the Bayesian network, nothing
more can be said than the II of Thermodynamics (Eq.6). As an example, this
happens when the evolution of each variable is influenced by all other
variables (Eq.2).

Figure 1:  Complete causal representation. The arrows represent the way we
decompose the joint probability density. In the complete case we have
$p(\zeta_{\tau}^{xy})=p(x_{t})\cdot p(y_{t}|x_{t})\cdot
p(x_{t+\tau}|x_{t},y_{t})\cdot p(y_{t+\tau}|x_{t},y_{t},x_{t+\tau})$.

We argue that more informative fluctuation theorems arise as a consequence of
missing edges in the causal representation of the dynamics in terms of
Bayesian networks. In the bivariate case there is only one class of
continuous-time models for which informative fluctuation theorems for causal
representations can be written: the signal-response models.

####  II.1.4 The spatial density of entropy production

Let us also use an equivalent representation of the entropy production in
terms of backward probabilitiesIto and Sagawa (2016) defined as
$p_{B}(\zeta_{\tau}^{xy})=p(x(t)=x_{t},y(t)=y_{t},x(t-\tau)=x_{t+\tau},y(t-\tau)=y_{t+\tau})$.
For stationary processes it holds
$p_{B}(\zeta_{\tau}^{xy})=p(\widetilde{\zeta_{\tau}^{xy}})$ and
$\varphi^{xy}_{\tau}=\ln(\frac{p(\zeta_{\tau}^{xy})}{p_{B}(\zeta_{\tau}^{xy})})$.

We introduce here the spatial density of entropy production (with observatinal
time $\tau$) for stationary processes as:

$\displaystyle\psi(x_{t},y_{t})=\int^{\infty}_{-\infty}\int^{\infty}_{-\infty}dx_{t+\tau}dy_{t+\tau}p(\zeta_{\tau}^{xy})\varphi^{xy}_{\tau}=$
$\displaystyle=\int^{\infty}_{-\infty}\int^{\infty}_{-\infty}dx_{t+\tau}dy_{t+\tau}p(\zeta_{\tau}^{xy})\ln\left(\frac{p(\zeta_{\tau}^{xy})}{p_{B}(\zeta_{\tau}^{xy})}\right)$
$\displaystyle=p(x_{t},y_{t})\int^{\infty}_{-\infty}\int^{\infty}_{-\infty}dx_{t+\tau}dy_{t+\tau}p(x_{t+\tau},y_{t+\tau}|x_{t},y_{t})*$
$\displaystyle*\ln\left(\frac{p(x(t+\tau)=x_{t+\tau},y(t+\tau)=y_{t+\tau}|x(t)=x_{t},y(t)=y_{t})}{p(x(t-\tau)=x_{t+\tau},y(t-\tau)=y_{t+\tau}|x(t)=x_{t},y(t)=y_{t})}\right)$
(8)

The spatial density of entropy production $\psi(x_{t},y_{t})$ tells us which
situations $(x_{t},y_{t})$ contribute more to the irreversibility of the
macroscopic process. $\psi(x_{t},y_{t})$ is proportional to the distance
(precisely to the Kullback–Leibler divergenceCover and Thomas (2012)) of the
distribution of future states $p(x_{t+\tau},y_{t+\tau}|x_{t},y_{t})$ to the
distribution of past states $p(x_{t-\tau},y_{t-\tau}|x_{t},y_{t})$ of the same
condition $(x_{t},y_{t})$.

###  II.2 The fluctuation theorem for signal-response models

If the system $(x,y)$ is such that the variable $y$ does not influence the
dynamics of the variable $x$, then we are dealing with signal-response models
(Fig.2). The stochastic differential equation for signal-response models is
written in the Ito representation(Shreve, 2004) as:

$\displaystyle\begin{cases}dx=g_{x}(x)dt+h_{x}(x)\sqrt{D_{x}}\,\,dW_{x}\\\
dy=g_{y}(x,y)dt+h_{y}(x,y)\sqrt{D_{y}}\,\,dW_{y}\end{cases}$ (9)

The absence of feedback is written in $\frac{\partial g_{x}}{\partial
y}=\frac{\partial h_{x}}{\partial y}=0$. As a consequence the conditional
probability satisfies $p(y_{t}|x_{t},x_{t+\tau})=p(y_{t}|x_{t})$, and the
corresponding causal representation is incomplete, see the Bayesian network in
Fig.2.

Figure 2:  Causal representation of signal-response models. The joint
probability density is decomposed into $p(\zeta_{\tau}^{xy})=p(x_{t})\cdot
p(y_{t}|x_{t})\cdot p(x_{t+\tau}|x_{t})\cdot
p(y_{t+\tau}|x_{t},y_{t},x_{t+\tau})$.

For signal-response models we can provide a lower bound on the entropy
production that is more informative than Eq.6, and that involves the backward
transfer entropy $T_{y\rightarrow x}(-\tau)$. The backward transfer entropyIto
(2016) is defined as the standard transfer entropy for the ensemble of time-
reversed trajectories. The stochastic counterpart as a function of
$\zeta_{\tau}^{xy}\setminus y_{t}$ is defined as:

$\displaystyle T^{st}_{y\rightarrow
x}(-\tau)=\ln\left(\frac{p(x_{t}|y_{t+\tau},x_{t+\tau})}{p(x_{t}|x_{t+\tau})}\right),$
(10)

where $st$ stands for stochastic.

Then by definition $T_{y\rightarrow x}(-\tau)=\left\langle
T^{st}_{y\rightarrow x}(-\tau)\right\rangle_{p(\zeta_{\tau}^{xy})}$. We keep
the same symbol $T_{y\rightarrow x}$ as the standard transfer entropy because
in stationary processes the backward transfer entropy is the standard transfer
entropy (calculated on forward trajectories) for negative shifts $-\tau$. It
measures information flows towards past, and in stationary processes it does
not depend on the instant $t$ but only on the observational time $\tau$.

The fluctuation theorem for signal-response models is:

$\displaystyle\left\langle
e^{-\varphi_{\tau}^{xy}+\varphi_{\tau}^{x}+T^{st}_{y\rightarrow
x}(-\tau)}\right\rangle_{p(\zeta_{\tau}^{xy})}=$
$\displaystyle=\int_{\Omega}d\zeta_{\tau}^{xy}p(\widetilde{y_{t+\tau}}|\widetilde{x_{t}},\widetilde{y_{t}},\widetilde{x_{t+\tau}})p(x_{t},x_{t+\tau},y_{t+\tau})=1,~{}~{}~{}~{}$
(11)

where we used the signal-response property of no feedback
$p(\widetilde{y_{t}}|\widetilde{x_{t}},\widetilde{x_{t+\tau}})=p(\widetilde{y_{t}}|\widetilde{x_{t}})$,
the correspondence $dy_{t}=\widetilde{dy_{t+\tau}}$, and the normalization
$\int^{\infty}_{-\infty}d\widetilde{y_{t+\tau}}p(\widetilde{y_{t+\tau}}|\widetilde{x_{t}},\widetilde{y_{t}},\widetilde{x_{t+\tau}})=1$.

From the convexity of the exponential it follows the II law for signal-
response models (Eq.1):

$\displaystyle\varPhi_{\tau}^{y|x}=\varPhi_{\tau}^{xy}-\varPhi_{\tau}^{x}\geq
T_{y\rightarrow x}(-\tau),$

which is the main result of our paper.

###  II.3 Applications

####  II.3.1 The basic linear response model

We study the II law for signal-response models (Eq.1) in the basic linear
response model (BLRM), whose information processing properties for the forward
trajectories are already discussed in (Auconi  _et al._ , 2017). The BLRM is
composed of a fluctuating signal $x$ described by the Ornstein-Uhlenbeck
process(Uhlenbeck and Ornstein, 1930; Gillespie, 1996), and a dynamic linear
response $y$ to this signal:

$\displaystyle\begin{cases}dx=-\frac{x}{t_{rel}}dt+\sqrt{D}\,\,dW\\\
\frac{dy}{dt}=\alpha x-\beta y\end{cases}$ (12)

Note that the response $y$ is not coupled to a thermal bath, $D_{y}=0$, while
the signal is, $D_{x}=D>0$. For any finite interval $\tau$, this corresponds
to the limit of weak coupling $D_{y}\rightarrow 0$.

This model allows analytical representations for the mapping irreversibility
$\varPhi_{\tau}^{xy}$ (calculated in Appendix A) and the backward transfer
entropy $T_{y\rightarrow x}(-\tau)$ (calculated in Appendix B). We find that,
once the observational time $\tau$ is specified, $\varPhi_{\tau}^{xy}$ and
$T_{y\rightarrow x}(-\tau)$ are both functions of just the two parameters
$t_{rel}$ and $\beta$, which describe respectively the time scale of the
fluctuations of the signal and the time scale of the response to a
deterministic input.

Since the signal is a time-symmetric (reversible) process,
$\varPhi_{\tau}^{x}=0$, the backward transfer entropy $T_{y\rightarrow
x}(-\tau)$ is the lower bound on the total entropy production
$\varPhi_{\tau}^{xy}$ in the BLRM.

The plot in Fig.3 shows the mapping irreversibility $\varPhi_{\tau}^{xy}$ and
the backward transfer entropy $T_{y\rightarrow x}(-\tau)$ as a function of the
observational time $\tau$. In the limit of small $\tau$, the entropy
production diverges because of the deterministic nature of the response
dynamics (the standard deviation on the determination of the velocity
$\frac{dy}{dt}$ due to instantaneous movements of the signal vanishes as
$\alpha\sqrt{D}\sqrt{dt}\rightarrow 0$). The backward transfer entropy
$T_{y\rightarrow x}(-\tau)$ instead vanishes for $\tau\rightarrow 0$ because
the Brownian motion has nonzero quadratic variationShreve (2004) and is the
dominating term in the signal dynamics for small time intervals. In the limit
of large observational time intervals $\tau\rightarrow\infty$ the entropy
production is asymptotically double the backward transfer entropy, that is its
lower bound given by the II law for signal-response models (Eq.1),
$\frac{\varPhi_{\tau}^{xy}}{T_{y\rightarrow x}(-\tau)}\rightarrow 2$. This
limit of $2$ is valid for any choice of the parameters in the BLRM.

Figure 3:  Mapping irreversibility $\varPhi_{\tau}^{xy}$, backward transfer
entropy $T_{y\rightarrow x}(-\tau)$ and causal influence $C_{x\rightarrow
y}(\tau)$ in the the BLRM as a function of the observational time interval
$\tau$. The parameters are $\beta=0.2$ and $t_{rel}=10$. All graphs are
produced using RR Core Team (2014).

Interestingly, for small observational time $\tau\rightarrow 0$, the causal
influence of the signal on the evolution of response (defined in (Auconi  _et
al._ , 2017)) converges to the backward transfer entropy of the response on
the past of the signal $C_{x\rightarrow y}(\tau)\rightarrow T_{y\rightarrow
x}(-\tau)$. For large observational time $\tau\rightarrow\infty$ instead the
causal influence converges to the standard (forward) transfer entropy
$C_{x\rightarrow y}(\tau)\rightarrow T_{y\rightarrow x}(\tau)$. Also in this
limit $\tau\rightarrow\infty$, the causal influence is an eighth of the
entropy production $\frac{\varPhi_{\tau}^{xy}}{C_{x\rightarrow
y}(\tau)}\rightarrow 8$ for any choice of the parameters in the BLRM.

Let us now consider the spatial density of entropy production
$\psi(x_{t},y_{t})$ in the BLRM for small and large observational time $\tau$.
In Fig.4 we choose a $\tau$ smaller than the characteristic response time
$\frac{1}{\beta}$ and also smaller than the characteristic time of
fluctuations $t_{rel}$. In the limit $\tau\rightarrow 0$ the signal dynamics
is dominated by noise and the entropy production is mainly given by movements
of the response $y$. The two spots correspond to the points where the product
of the density $p(x_{t},y_{t})$ times the absolute value of the instant
velocity $\dot{y}$ is larger. For longer intervals
$\tau\gtrapprox\frac{1}{\beta}$ (that is the case of Fig.5) the history of the
signal becomes relevant because it determined the present value of the
response, therefore the irreversibility density is also distributed on those
points of the diagonal (corresponding to roughly $\dot{y}=0$) where the
absolute value of the response $y$ is big enough. Also as a consequence, in
this regime the backward transfer entropy is a meaningful lower bound on the
entropy production, that is Eq.1.

Figure 4:  Spatial density of entropy production $\psi(x_{t},y_{t})$ for the
BLRM at $\tau=0.5<\frac{1}{\beta}<t_{rel}$. The parameters are $\beta=0.2$ and
$t_{rel}=10$. Both $\psi(x_{t},y_{t})$ and the space $(x,y)$ are expressed in
units of the standard deviation of the dynamics. Figure 5:  Spatial density of
entropy production $\psi(x_{t},y_{t})$ for the BLRM at
$\tau=25>t_{rel}>\frac{1}{\beta}$. The parameters are $\beta=0.2$ and
$t_{rel}=10$. Both $\psi(x_{t},y_{t})$ and the space $(x,y)$ are expressed in
units of the standard deviation of the dynamics.

####  II.3.2 Receptor-ligand systems

The Receptor-Ligand interaction is the fundamental mechanism of molecular
recognition in biology and is a recurring motif in signaling pathwaysKlipp
_et al._ (2016); Kholodenko (2006). The fraction of activated receptors is
part of the cell’s representation of the outside world, it is the cell’s
estimate on the concentration of ligands in the environment, upon which it
bases its protein expression and response to external stress.

If we could experimentally keep the concentration of ligands fixed we would
still get a fluctuating number of activated receptors due to the intrinsic
stochasticity of the macroscopic description of chemical reactions. Recent
studies allowed a theoretical understanding of the origins of the macroscopic
”noise” (i.e. the output variance in the conditional probability
distributions), and also raised questions about the optimality of the input
distributions in terms of information transmissionBialek and Setayeshgar
(2005); Tkačik  _et al._ (2008); Crisanti  _et al._ (2018); Waltermann and
Klipp (2011).

Here we consider the dynamical aspects of information processing in receptor-
ligand systemsDi Talia and Wieschaus (2012); Nemenman (2012), where the
response is integrated over time. If the perturbation of the receptor-ligand
binding on the concentration of free ligands is negligible, that is in the
limit of high ligand concentration, receptor-ligand systems can be modeled as
nonlinear signal-response modelsDi Talia and Wieschaus (2014). We write our
model of receptor-ligand systems in the Ito representationShreve (2004) as:

$\displaystyle\begin{cases}dx=-(x-1)dt+x\,dW_{x}\\\
dy=k_{on}(1-y)\frac{x^{h}}{1+x^{h}}dt-
k_{off}ydt+y(1-y)dW_{y}~{}~{}~{}\end{cases}$ (13)

The fluctuations of the ligand concentration $x$ are described by a mean-
reverting geometric Brownian motion, with an average $\braket{x}=1$ in
arbitrary units. The response, that is the fraction of activated receptors
$y$, is driven by a Hill-type interaction with the signal with cooperativity
coefficient $h$, and chemical bound/unbound rates $k_{on}$ and $k_{off}$. For
simplicity, the dynamic range of the response is set to be coincident with the
mean value of the ligand concentration, that means to choose a Hill constant
$K=\braket{x}=1$. The form of the $y$ noise is set by the biological
constraint $0<y<1$. For simplicity, we choose a cooperativity coefficient of
$h=2$, that is the lower order of sigmoidal functions.

The mutual information between the concentration of ligands and the fraction
of activated receptors in a cell is a natural choice for quantifying its
sensory propertiesTkačik  _et al._ (2009). Here we argue that, in the case of
signal-response models, the conditional entropy production is the relevant
measure, because it quantifies how the dynamics of the signal produces
irreversible transitions in the dynamics of the response, which is closely
related to the concept of causation. Besides, our measure of causal
influenceAuconi  _et al._ (2017) has yet not been generalized to the nonlinear
case, while the entropy production has a consistent thermodynamical
interpretationSeifert (2012).

We simulated the receptor-ligand model of Eq.13, and we evaluated numerically
the mapping irreversibility $\varPhi_{\tau}^{xy}$ and the backward transfer
entropy $T_{y\rightarrow x}(-\tau)$ using a multivariate Gaussian
approximation for the conditional probabilities
$p(x_{t+\tau},y_{t+\tau}|x_{t},y_{t})$ (details in Appendix D). The II law for
signal response models sets $\varPhi_{\tau}^{xy}\geq T_{y\rightarrow
x}(-\tau)$ and proves to be a useful tool for receptor-ligand systems, as it
is seen if Fig.6. Note that the numerical estimation of the entropy production
requires many more samples compared to the backward transfer entropy:
$\varPhi_{\tau}^{xy}$ depends on $\zeta_{\tau}^{xy}$ (4 dimensions) while
$T_{y\rightarrow x}(-\tau)$ depends on $\zeta_{\tau}^{xy}\setminus y_{t}$ (3
dimensions). In a real biological experimental setting the sampling process is
expensive, and the backward transfer entropy is therefore a useful lower bound
for the entropy production, and an interesting characterization of the system
to be used when the number of samples is not large enough.

Figure 6:  Entropy production and backward transfer entropy in our model of
receptor-ligand systems (Eq.13). The parameters are $k_{on}=5$, $k_{off}=1$,
$h=2$ and $t_{rel}=10$.

The intrinsic noise of the response $y(1-y)dW_{y}$ is the dominant term in the
response dynamics for small intervals $\tau$. This makes both
$\varPhi_{\tau}^{xy}$ and $T_{y\rightarrow x}(-\tau)$ vanish in the limit
$\tau\rightarrow 0$. In the limit of large observational time $\tau$, as it is
also the case for the BLRM and in any stationary process, the entropy
production for the corresponding time-series $\varPhi_{\tau}^{xy}$ and all the
information measures are vanishing, because the memory of the system is damped
exponentially over time by the relaxation parameter $k_{off}$ ($\beta$ in the
BLRM). Therefore in order to better detect the irreversibility of a process
one must choose an appropriate observational time $\tau$. In the receptor-
ligand model of Eq.13 with parameters $k_{on}=5$, $k_{off}=1$, $h=2$ and
$t_{rel}=10$ we see that the optimal observational time is around $\tau\approx
0.5$ (see Fig.6). Here for ”optimal” we mean the observational time that
corresponds to the highest mapping irreversibility $\varPhi_{\tau}^{xy}$, but
one might also be interested in inferring the entropy rate (that is
$\frac{\varPhi_{\tau}^{xy}}{\tau}$ in the limit $\tau\rightarrow 0$) looking
at time-series data with finite sampling interval $\tau$. We do not treat this
problem here.

##  III DISCUSSION

This work is based on our definition of mapping irreversibility (Eq.4), that
is the entropy production of time-series obtained from a discretization with
observational time $\tau$ of continuous dynamics.

For controlled models the entropy production takes the form of Eq.3. This
definition is different from the one used by Ito-Sagawa(Ito and Sagawa, 2013,
2016). Their alternative definition of discrete entropy production of system
$y$ controlled by protocol $\lambda(x)$ in a single time step is:

$\displaystyle\eta^{y|\lambda(x)}_{\tau}=\ln\left(\frac{p(y_{t})}{p(y_{t+\tau})}\right)+\ln\left(\frac{p_{\lambda}(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})}{p_{\lambda}(\widetilde{y_{t+\tau}}|\widetilde{y_{t}},x_{t},x_{t+\tau})}\right).~{}~{}~{}~{}~{}~{}$
(14)

When feedbacks enter the dynamics at shorter scales compared to the
observational time $\tau$, as it is the case in our setting of causal
representations, the transition probabilities in the reduced dynamics (where
no feedback is performed) $p_{\lambda}(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$ are
different compared to the transition probabilities in the original dynamics
$p(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})$. This is not problematic in their
setting though, because in a fixed Bayesian network the feedbacks arise as
edges paths in the network and are not deriving from an underlying continuous
dynamics.

The difference with our definition (Eq.3) is in the way the discrete protocol
$\lambda(x)$ is applied. They perform the conditioning for the backward
transitions on the same exact states as in the forward transitions. In our
setting of bivariate causal representations, their conditioning for the
backward transitions would be on the original states $(x_{t},x_{t+\tau})$
rather than on the time reversal conjugates
$(\widetilde{x_{t}},\widetilde{x_{t+\tau}})$ (compare Eq.3 and Eq.14). This
difference becomes evident in the limit $\tau\rightarrow\infty$ as we discuss
in Appendix C for the BLRM. There, where no feedback on the signal is
performed by the response, the alternative irreversibility (Eq.14) has a
finite limit (instead of zero) for large time intervals
$\tau\rightarrow\infty$,
$\left\langle\eta^{y|\lambda(x)}_{\tau}\right\rangle_{p(\zeta_{\tau}^{xy})}\rightarrow\beta
t_{rel}$. Note that in this limit the forward time-series are
indistinguishable from their time-reversed conjugates, and
$\left\langle\eta^{y|\lambda(x)}_{\tau}\right\rangle_{p(\zeta_{\tau}^{xy})}$
is greater than zero because of the correlation $\braket{xy}$ being greater
than zero. In addition, we note that because of this way of conditioning, the
term
$\eta^{y|\lambda(x)}_{\tau}-\ln\left(\frac{p(y_{t})}{p(y_{t+\tau})}\right)$
does not have a simple relation to the total entropy production of the system
$(x,y)$ like our Eq.7.

A modification of the alternative definition $\eta^{y|\lambda(x)}_{\tau}$ in
which the reduced dynamics dependence is removed, can be written as:

$\displaystyle\eta^{y|x}_{\tau}=\ln\left(\frac{p(y_{t+\tau}|y_{t},x_{t},x_{t+\tau})}{p(\widetilde{y_{t+\tau}}|\widetilde{y_{t}},x_{t},x_{t+\tau})}\right)+\ln\left(\frac{p(y_{t})}{p(y_{t+\tau})}\right).~{}~{}~{}~{}~{}$
(15)

For this modified version, a fluctuation theorem can be written, and it is the
Ito-Sagawa fluctuation theorem proved in (Ito and Sagawa, 2013). In stationary
systems it involves the difference between forward and backward transfer
entropies:

$\displaystyle\braket{e^{-\eta_{\tau}^{y}{x}+T^{st}_{y\rightarrow
x}(-\tau)-T^{st}_{y\rightarrow x}(\tau)}}{}{p(\zeta_{\tau}^{xy})}=1.$ (16)

The two different definitions for controlled systems Eq.3 and Eq.14, with
different ways of applying the conditioning on the protocol, still converge
for Langevin systems in the limit $\tau\rightarrow 0$, because the
discretization scheme enters the entropy production with terms that are
vanishing faster than $\tau$ as it is shown in (Ito and Sagawa, 2013).

Nevertheless, we argue that in the more general case of time-series the Ito-
Sagawa definition should be modified into Eq.3 for controlled system and to
Eq.4 for autonomous systems.

We used our definition to study the irreversibility of stochastic maps
resulting from a time discretization with observational time $\tau$ of
continuous models. While for autonomous systems in the general case the only
statement we can provide is the II law of thermodynamics (Eq.6), a more
informative lower bound on the entropy production is found for signal-response
models (Eq.1). This sets the backward transfer entropy as a lower bound to the
entropy production, and describes the connection between the irreversibility
of stochastic trajectories and the information flows towards past between
variables.

We restrict ourselves to the bivariate case here, but we conjecture that
fluctuation theorems for multidimensional stochastic autonomous dynamics
should arise in general as a consequence of missing arrows in the (non
complete, see e.g. Fig.2) causal representation of the dynamics in terms of
Bayesian networks.

In our opinion, a general relation connecting the incompleteness of the causal
representation of the dynamics and fluctuation theorems is still lacking.

We also introduced a discussion about the observational time $\tau$ in data
analysis. In a biological model of receptor-ligand systems we showed that it
has to be fine-tuned for a robust detection of the irreversibility of the
process, which is related to the concept of causation(Auconi  _et al._ , 2017)
and therefore to the efficiency of the biological coupling between signalling
and response.

## Acknowledgements

We thank Wolfgang Giese, Valentina T Tovazzi, Friedemann Uschner, Björn
Goldenbogen, Ana Bulovic, Roman Rainer, Matthias Reis and Martin Seeger for
useful discussions. We thank Marco Scazzocchio for numerical methods and
algorithms expertise. We thank Jesper Romers for discussions of mathematical
aspects.

Work at Humboldt-Universität zu Berlin was supported by the DFG
(Graduiertenkolleg 1772 for Computational Systems Biology).

## APPENDIX A: MAPPING IRREVERSIBILITY IN THE BLRM

Let us consider an ensemble of stochastic trajectories generated with the BLRM
(Eq.12). The mapping irreversibility $\varPhi_{\tau}^{xy}$ here is the
Kullback-Leibler divergence(Cover and Thomas, 2012) between the probability
density $p(\zeta_{\tau}^{xy})$ of couples of successive states
$\zeta_{\tau}^{xy}$ separated by a time interval $\tau$ of the original
trajectory and the probability density
$p_{B}(\zeta_{\tau}^{xy})=p(\widetilde{\zeta_{\tau}^{xy}})$ of the same
couples of successive states $\zeta_{\tau}^{xy}$ of the time-reversed
conjugate of the original trajectory (Eq.4). For the sake of clarity, we use
here in this appendix the full formalism rather than the compact one based on
the functional form $f_{\tau}^{xy}$.

The time-reversed density of a particular couple of successive states,
$(x(t)=\gamma,y(t)=\delta)$ and $(x(t+\tau)=\mu,y(t+\tau)=\xi)$, is equivalent
to the original density of the exchanged couple of states,
$(x(t)=\mu,y(t)=\xi)$ and $(x(t+\tau)=\gamma,y(t+\tau)=\delta)$. Therefore the
density
$p(\widetilde{\zeta_{\tau}^{xy}})=p(x(t)=\mu,y(t)=\xi,x(t+\tau)=\gamma,y(t+\tau)=\delta)$
is the transpose of the density
$p(\zeta_{\tau}^{xy})=p(x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu,y(t+\tau)=\xi)$.

The mapping irreversibility for the BLRM is then written as:

$\displaystyle\varPhi_{\tau}^{xy}=\braket{\varphi_{\tau}^{xy}}_{p(\zeta_{\tau}^{xy})}=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\gamma
d\delta d\mu
d\xi~{}p\left(x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu,y(t+\tau)=\xi\right)*$
$\displaystyle*\ln\left(\frac{p(x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu,y(t+\tau)=\xi)}{p(x(t)=\mu,y(t)=\xi,x(t+\tau)=\gamma,y(t+\tau)=\delta)}\right).$
(17)

The BLRM is ergodic, therefore the densities $p(\zeta_{\tau}^{xy})$ and
$p(\widetilde{\zeta_{\tau}^{xy}})$ can be empirically sampled looking at a
single infinitely-long trajectory.

The causal structure of the BLRM (and of any signal-response model, see Fig.2)
is such that the evolution of the signal is not influenced by the response,
$p(x(t+\tau)|x(t),y(t))=p(x(t+\tau)|x(t))$. Then we can write the joint
probability densities $p(\zeta_{\tau}^{xy})$ of couples of successive states
over a time interval $\tau$ of the original trajectory as:

$\displaystyle p(\zeta_{\tau}^{xy})\equiv
p(x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu,y(t+\tau)=\xi)=$
$\displaystyle=p(x(t)=\gamma)\cdot p(y(t)=\delta|x(t)=\gamma)\cdot
p(x(t+\tau)=\mu|x(t)=\gamma)\cdot
p(y(t+\tau)=\xi|x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu).~{}~{}~{}~{}$ (18)

We need to evaluate all these probabilities. Since we are dealing with linear
models, these are all Gaussian distributed, and we will calculate only the
expected value and the variance of the relevant variables involved.

The system is Markovian,
$p(x(t+\tau)|x(t+t^{\prime}),x(t))=p(x(t+\tau)|x(t+t^{\prime}))$ with $0\leq
t^{\prime}\leq\tau$, and the Bayes rule assumes the form
$p(x(t+t^{\prime})|x(t),x(t+\tau))=\frac{p(x(t+t^{\prime})|x(t))p(x(t+\tau)|x(t+t^{\prime}))}{p(x(t+\tau)|x(t))}$.
Then we calculate the conditional expected value for the signal $x(t+\tau)$
given a condition for its past $x(t)$ and another condition for its future
$x(t+\tau)$ as:

$\displaystyle\braket{x(t+t^{\prime})}{x(t),x(t+\tau)}=x(t)e^{-\frac{t^{\prime}}{t_{rel}}}\frac{1-e^{-\frac{2(\tau-t^{\prime})}{t_{rel}}}}{1-e^{-\frac{2\tau}{t_{rel}}}}+x(t+\tau)e^{-\frac{\tau-t^{\prime}}{t_{rel}}}\frac{1-e^{-\frac{2t^{\prime}}{t_{rel}}}}{1-e^{-\frac{2\tau}{t_{rel}}}}.$
(19)

Now we can calculate the full-conditional expectation of the response:

$\displaystyle\braket{y(t+\tau)}{x(t),y(t),x(t+\tau)}=y(t)e^{-\beta\tau}+\alpha\int_{0}^{\tau}dt^{\prime}e^{-\beta(\tau-t^{\prime})}\braket{x(t+t^{\prime})}{x(t),x(t+\tau)}=$
$\displaystyle=y(t)e^{-\beta\tau}+\alpha\frac{e^{-\beta\tau}}{1-e^{-\frac{2\tau}{t_{rel}}}}\left(x(t)(\frac{e^{\tau(\beta-\frac{1}{t_{rel}})}-1}{\beta-\frac{1}{t_{rel}}}-\frac{e^{\tau(\beta-\frac{1}{t_{rel}})}-e^{-\frac{2\tau}{t_{rel}}}}{\beta+\frac{1}{t_{rel}}})+x(t+\tau)(\frac{e^{\beta\tau}-e^{-\frac{\tau}{t_{rel}}}}{\beta+\frac{1}{t_{rel}}}-\frac{e^{\tau(\beta-\frac{2}{t_{rel}})}-e^{-\frac{\tau}{t_{rel}}}}{\beta-\frac{1}{t_{rel}}})\right).$
(20)

One can immediately check that the limits for small and large time intervals
$\tau$ verify respectively $\lim\limits_{\tau\rightarrow
0}\braket{y(t+\tau)}{x(t),y(t),x(t+\tau)}=y(t)$ and
$\lim\limits_{\tau\rightarrow\infty}\braket{y(t+\tau)}{x(t),y(t),x(t+\tau)}=x(t+\tau)\frac{\alpha
t_{rel}}{\beta t_{rel}+1}=\braket{y(t+\tau)}{x(t+\tau)}$.

The causal order for the evolution of the signal is such that
$p(x(t+t^{\prime\prime})|x(t),x(t+t^{\prime}),x(t+\tau))=p(x(t+t^{\prime\prime})|x(t+t^{\prime}),x(t+\tau))$
if $0\leq t^{\prime}\leq t^{\prime\prime}\leq\tau$. Then we can calculate:

$\displaystyle\braket{x(t+t^{\prime})x(t+t^{\prime\prime})}{x(t),x(t+\tau)}_{t^{\prime\prime}\geq
t^{\prime}}=$
$\displaystyle=\int_{-\infty}^{\infty}dx(t+t^{\prime})p(x(t+t^{\prime})|x(t),x(t+\tau))x(t+t^{\prime})\braket{x(t+t^{\prime\prime})}{x(t+t^{\prime}),x(t+\tau)}=$
$\displaystyle=\braket{x(t+t^{\prime})}{x(t),x(t+\tau)}*$
$\displaystyle*\left(x(t+\tau)e^{-\frac{\tau-t^{\prime\prime}}{t_{rel}}}\frac{\sigma^{2}_{t^{\prime\prime}-t^{\prime}}}{\sigma^{2}_{\tau-t^{\prime}}}+e^{-\frac{t^{\prime\prime}-t^{\prime}}{t_{rel}}}\frac{\sigma^{2}_{\tau-t^{\prime\prime}}}{\sigma^{2}_{\tau-t^{\prime}}}\left(\frac{1}{x(t)\frac{e^{-\frac{t^{\prime}}{t_{rel}}}}{\sigma^{2}_{t^{\prime}}}+x(t+\tau)\frac{e^{-\frac{\tau-t^{\prime}}{t_{rel}}}}{\sigma^{2}_{\tau-t^{\prime}}}}+\frac{x(t)\frac{e^{-\frac{t^{\prime}}{t_{rel}}}}{\sigma^{2}_{t^{\prime}}}+x(t+\tau)\frac{e^{-\frac{\tau-t^{\prime}}{t_{rel}}}}{\sigma^{2}_{\tau-t^{\prime}}}}{\frac{1}{\sigma^{2}_{t^{\prime}}}+\frac{e^{-\frac{2(\tau-t^{\prime})}{t_{rel}}}}{\sigma^{2}_{\tau-t^{\prime}}}}\right)\right).$
(21)

Let us write the full-conditional expectation of the squared response as a
function of the expectations we just calculated:

$\displaystyle\braket{y^{2}(t+\tau)}{x(t),y(t),x(t+\tau)}=y^{2}(t)e^{-2\beta\tau}+2\alpha
y(t)e^{-2\beta\tau}\int_{0}^{\tau}dt^{\prime}e^{\beta
t^{\prime}}\braket{x(t+t^{\prime})}{x(t),x(t+\tau)}+$
$\displaystyle+\alpha^{2}e^{-2\beta\tau}\int_{0}^{\tau}\int_{0}^{\tau}dt^{\prime}dt^{\prime\prime}e^{\beta(t^{\prime}+t^{\prime\prime})}\braket{x(t+t^{\prime})x(t+t^{\prime\prime})}{x(t),x(t+\tau)}.$
(22)

A relevant feature of linear response models is that the conditional variances
do not depend on the particular values of the conditioning variables(Auconi
_et al._ , 2017). Here we consider the full-conditional variance
$\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}=\braket{y^{2}(t+\tau)}{x(t),y(t),x(t+\tau)}-\braket{y(t+\tau)}{x(t),y(t),x(t+\tau)}^{2}$,
and it will be independent of the conditions $x(t)$, $y(t)$, and $x(t+\tau)$.
Then the remaining terms in $\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}$ sum
up to:

$\displaystyle\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}=2\frac{\alpha^{2}e^{-2\beta\tau}}{\sigma^{2}_{\tau}}\int_{0}^{\tau}dt^{\prime\prime}\sigma^{2}_{\tau-t^{\prime\prime}}e^{t^{\prime\prime}(\beta-\frac{1}{t_{rel}})}\int_{0}^{t^{\prime\prime}}dt^{\prime}\sigma^{2}_{t^{\prime}}e^{t^{\prime}(\beta+\frac{1}{t_{rel}})}=2\alpha^{2}\sigma^{2}_{x}\frac{e^{-2\beta\tau}}{1-e^{-\frac{2\tau}{t_{rel}}}}*$
(23)
$\displaystyle*\left(-\frac{2}{t_{rel}}\frac{\beta+\frac{1}{t_{rel}}-\frac{2}{t_{rel}}e^{\tau(\beta-\frac{1}{t_{rel}})}-(\beta-\frac{1}{t_{rel}})e^{-\frac{2\tau}{t_{rel}}}}{(\beta+\frac{1}{t_{rel}})^{2}(\beta-\frac{1}{t_{rel}})^{2}}+\frac{\frac{1}{t_{rel}}e^{2\beta\tau}-\beta-\frac{1}{t_{rel}}+\beta
e^{-\frac{2\tau}{t_{rel}}}}{2\beta(\beta+\frac{1}{t_{rel}})^{2}}-\frac{\frac{1}{t_{rel}}e^{2\tau(\beta-\frac{1}{t_{rel}})}-\beta+(\beta-\frac{1}{t_{rel}})e^{-\frac{2\tau}{t_{rel}}}}{2\beta(\beta-\frac{1}{t_{rel}})^{2}}\right),$
(24)

where we used the fact that
$\braket{x(t+t^{\prime})x(t+t^{\prime\prime})}{x(t),x(t+\tau)}$ is symmetric
in $t^{\prime}$ and $t^{\prime\prime}$. We recall that for functions with the
symmetry $f(t^{\prime},t^{\prime\prime})=f(t^{\prime\prime},t^{\prime})$ it
holds:
$\int_{0}^{\tau}\int_{0}^{\tau}dt^{\prime}dt^{\prime\prime}f(t^{\prime},t^{\prime\prime})=2\int_{0}^{\tau}dt^{\prime}\int_{0}^{t^{\prime}}dt^{\prime\prime}f(t^{\prime},t^{\prime\prime})$.

The limits for small and large time intervals $\tau$ verify respectively
$\lim\limits_{\tau\rightarrow 0}\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}=0$
and
$\lim\limits_{\tau\rightarrow\infty}\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}=\alpha^{2}\sigma^{2}_{x}\frac{t_{rel}}{\beta(\beta
t_{rel}+1)^{2}}=\sigma^{2}_{y(t)|x(t)}$.

The factorization of the probability density $p(\zeta_{\tau}^{xy})$ into
conditional densities (Eq.APPENDIX A: MAPPING IRREVERSIBILITY IN THE BLRM)
leads to a decomposition of the mapping irreversibility. Here we show that in
the BLRM all of these terms are zero except for the two terms corresponding to
the full-conditional density of the evolution of the response in the original
trajectory and in the time-reversed conjugate.

For a stationary stochastic process like the BLRM it holds
$p(x(t)=\gamma,y(t)=\delta)=p(x(t+\tau)=\gamma,y(t+\tau)=\delta)$, then these
two terms cancel:

$\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\gamma
d\delta d\mu
d\xi~{}p(x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu,y(t+\tau)=\xi)\cdot\ln(p(x(t)=\gamma,y(t)=\delta))=$
$\displaystyle=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\gamma
d\delta~{}p(x(t)=\gamma,y(t)=\delta)\cdot\ln(p(x(t)=\gamma,y(t)=\delta))=$
$\displaystyle=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\gamma
d\delta~{}p(x(t+\tau)=\gamma,y(t+\tau)=\delta)\cdot\ln(p(x(t)=\gamma,y(t)=\delta))=$
$\displaystyle=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\mu
d\xi~{}p(x(t+\tau)=\mu,y(t+\tau)=\xi)\cdot\ln(p(x(t)=\mu,y(t)=\xi))=$
$\displaystyle=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\gamma
d\delta d\mu
d\xi~{}p(x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu,y(t+\tau)=\xi)\cdot\ln(p(x(t)=\mu,y(t)=\xi)).$
(25)

The contribution from the signal in the mapping irreversibility is also zero
since the Ornstein-Uhlenbeck process is reversible,
$p(x(t)=\gamma,x(t+\tau)=\mu)=p(x(t)=\mu,x(t+\tau)=\gamma)$:

$\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\gamma
d\mu~{}p(x(t)=\gamma,x(t+\tau)=\mu)~{}\ln\left(\frac{p(x(t+\tau)=\mu|x(t)=\gamma)}{p(x(t+\tau)=\gamma|x(t)=\mu)}\right)=0.$
(26)

The mapping irreversibility is therefore:

$\displaystyle\varPhi_{\tau}^{xy}=\braket{\varphi_{\tau}^{xy}}_{p(\zeta_{\tau}^{xy})}=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\gamma
d\delta d\mu d\xi~{}p(x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu,y(t+\tau)=\xi)*$
$\displaystyle*\ln\left(\frac{p(y(t+\tau)=\xi|x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu)}{p(y(t+\tau)=\delta|x(t)=\mu,y(t)=\xi,x(t+\tau)=\gamma)}\right)=$
$\displaystyle=-\frac{1}{2}+\frac{1}{2\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}d\gamma
d\delta d\mu d\xi~{}p(x(t)=\gamma,y(t)=\delta,x(t+\tau)=\mu,y(t+\tau)=\xi)*$
$\displaystyle*(\delta-\braket{y(t+\tau)}{x(t)=\mu,y(t)=\xi,x(t+\tau)=\gamma})^{2},$
(27)

where in the last passage we exploited the fact that all the probability
densities are Gaussian distributed. Solving the integrals we get the mapping
irreversibility for the BLRM as a function of the time interval $\tau$:

$\displaystyle\varPhi_{\tau}^{xy}=\frac{1}{2}(e^{-2\beta\tau}-1)+\frac{2\alpha^{2}\sigma^{2}_{x}t_{rel}^{2}}{\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}}\frac{(e^{-\beta\tau}-e^{-\frac{\tau}{t_{rel}}})^{2}(e^{-2\beta\tau}+1-2e^{-\tau(\beta+\frac{1}{t_{rel}})})}{(\beta^{2}t_{rel}^{2}-1)^{2}(1-e^{-\frac{2\tau}{t_{rel}}})}.$
(28)

$\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}$ is proportional to
$\alpha^{2}\sigma^{2}_{x}$, therefore the mapping irreversibility
$\varPhi_{\tau}^{xy}$ is a function of just the two parameters $t_{rel}$ and
$\beta$ (and of the observational time $\tau$).

Figure 7:  Numerical verification of the analytical solution for the entropy
production $\varPhi_{\tau}^{xy}$ with observational time $\tau$ in the BLRM.
The parameters are $\beta=0.2$ and $t_{rel}=10$.

## APPENDIX B: backward transfer ENTROPY IN THE BLRM

In the BLRM, where all densities are Gaussian distributed, the backward
transfer entropy is equivalent to the time-reversed Granger causality(Barnett
_et al._ , 2009):

$\displaystyle T_{y\rightarrow
x}(-\tau)=I(x(t),y(t+\tau)|x(t+\tau))=\ln\left(\frac{\sigma_{y|x}}{\sigma_{y(t+\tau)|x(t),x(t+\tau)}}\right)$
(29)

We have to calculate the conditional variance
$\sigma_{y(t+\tau)|x(t),x(t+\tau)}$. Let us recall the relation for the value
of the response as a function of the whole past history of the signal
trajectory:

$\displaystyle y(t+\tau)=\alpha
e^{-\beta(t+\tau)}\left(\int_{-\infty}^{t}dt^{\prime}x(t^{\prime})e^{\beta
t^{\prime}}+\int_{t}^{t+\tau}dt^{\prime}x(t^{\prime})e^{\beta
t^{\prime}}\right)$ (30)

Then we write the conditional squared response as

$\displaystyle\braket{y^{2}(t+\tau)}{x(t),x(t+\tau)}=2\alpha^{2}e^{-2\beta(t+\tau)}(e^{2\beta
t}\int_{0}^{\tau}\int_{0}^{t^{\prime\prime}}dt^{\prime\prime}dt^{\prime}e^{\beta(t^{\prime}+t^{\prime\prime})}\braket{x(t+t^{\prime})x(t+t^{\prime\prime})}{x(t),x(t+\tau)}_{t^{\prime\prime}\geq
t^{\prime}}+$
$\displaystyle+\int_{-\infty}^{t}\int_{-\infty}^{t^{\prime\prime}}dt^{\prime\prime}dt^{\prime}\braket{x^{2}(t^{\prime\prime})}{x(t)}e^{-\frac{t^{\prime\prime}-t^{\prime}}{t_{rel}}+\beta(t^{\prime}+t^{\prime\prime})}+\int_{-\infty}^{t}\int_{t}^{t+\tau}dt^{\prime}dt^{\prime\prime}\braket{x(t^{\prime})}{x(t)}\braket{x(t^{\prime\prime})}{x(t),x(t+\tau)}e^{\beta(t^{\prime}+t^{\prime\prime})})~{}~{}$
(31)

Since $\sigma^{2}_{y(t+\tau)|x(t),x(t+\tau)}$ is expected to be independent of
$x(t)$ and $x(t+\tau)$, then the remaining terms sum up to:

$\displaystyle\sigma^{2}_{y(t+\tau)|x(t),x(t+\tau)}=\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}+\sigma^{2}_{y|x}e^{-2\beta\tau},$
(32)

where $\sigma^{2}_{y|x}=\frac{\sigma^{2}_{x}\alpha^{2}}{\beta
t_{rel}(\beta+\frac{1}{t_{rel}})^{2}}$ was already calculated in Auconi  _et
al._ (2017). Then the backward transfer entropy is:

$\displaystyle T_{y\rightarrow
x}(-\tau)=I\left(x(t),y(t)|x(t+\tau)\right)=-\frac{1}{2}\ln\left(\frac{\sigma^{2}_{y(t+\tau)|x(t),y(t),x(t+\tau)}}{\sigma^{2}_{y|x}}+e^{-2\beta\tau}\right).$
(33)

## APPENDIX C: LARGE OBSERVATIONAL TIME LIMIT OF THE ALTERNATIVE DEFINITION OF
ENTROPY PRODUCTION IN THE BLRM.

We are in the BLRM and the absence of feedback implies
$p_{\lambda}(\zeta_{\tau}^{xy})=p(\zeta_{\tau}^{xy})$. Still the conditioning
in the alternative definition (Eq.14) is made on a different protocol compared
to our definition of mapping irreversibility in controlled systems (Eq.3). The
effect of these different choices is best seen in the limit
$\tau\rightarrow\infty$.

In the limit of large observational time, the probability density
$p(\zeta_{\tau}^{xy})$, our definition $\varphi^{y|\lambda(x)}_{\tau}$ and the
alternative definition $\eta^{y|\lambda(x)}_{\tau}$ decompose respectively
into:

$\displaystyle
p(\zeta_{\tau}^{xy})=p(x(t)=x_{t},y(t)=y_{t},x(t+\tau)=x_{t+\tau},y(t+\tau)=y_{t+\tau})$
$\displaystyle\rightarrow p(x(t)=x_{t},y(t)=y_{t})\cdot
p(x(t+\tau)=x_{t+\tau},y(t+\tau)=y_{t+\tau}),$ (34)

$\displaystyle\varphi^{y|\lambda(x)}_{\tau}\rightarrow\ln\left(\frac{p(y_{t+\tau}|x_{t+\tau})}{p(\widetilde{y_{t+\tau}}|\widetilde{x_{t+\tau}})}\right)=\ln\left(\frac{p(y_{t+\tau}|x_{t+\tau})}{p(y_{t}|x_{t})}\right),$
(35)

$\displaystyle\eta^{y|\lambda(x)}_{\tau}\rightarrow\ln\left(\frac{p(y_{t+\tau}|x_{t+\tau})}{p(\widetilde{y_{t+\tau}}|x_{t+\tau})}\right),$
(36)

where we neglected the change of entropy in the $y$ system, that is
$\ln\left(\frac{p(y_{t})}{p(y_{t+\tau})}\right)$, because its ensemble average
vanishes in the BLRM.

$\left\langle\varphi^{y|\lambda(x)}_{\tau}\right\rangle_{p(\zeta_{\tau}^{xy})}$
is vanishing in stationary systems in the limit $\tau\rightarrow\infty$ as a
consequence of the factorization of $p(\zeta_{\tau}^{xy})$.

The ensemble average of the alternative definition is not vanishing and it is
calculated as:

$\displaystyle\braket{\eta^{y}{x}_{\tau}}{=}-\frac{1}{2}\left(1+\ln(2\pi\sigma^{2}_{y|x})\right)+\frac{1}{2}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}dx_{t+\tau}dy_{t}p(x_{t+\tau})p(y_{t})\left(\ln(2\pi\sigma^{2}_{y|x})+\frac{(\widetilde{y_{t+\tau}}-\braket{\widetilde{y_{t+\tau}}}{x_{t+\tau}})^{2}}{\sigma^{2}_{y|x}}\right)=$
$\displaystyle=-\frac{1}{2}+\frac{1}{2}\frac{\sigma^{2}_{y}}{\sigma^{2}_{y|x}}\left(1+\frac{\beta
t_{rel}}{\beta t_{rel}+1}\right)=\beta t_{rel},$ (37)

where we used the relations
$p(\widetilde{y_{t+\tau}}|x_{t+\tau})=p(y(t+\tau)=y_{t}|x(t+\tau)=x_{t+\tau})$,
$\braket{\widetilde{y_{t+\tau}}}{x_{t+\tau}}=\braket{y_{t+\tau}}{x_{t+\tau}}=\braket{y(t+\tau)}{x(t+\tau)=x_{t+\tau}}=\braket{y(t)}{x(t)=x_{t+\tau}}=x_{t+\tau}\frac{\alpha
t_{rel}}{\beta t_{rel}+1}$, and $\sigma^{2}_{y}=(1+\beta
t_{rel})\sigma^{2}_{y|x}=\sigma^{2}_{x}\frac{\alpha^{2}t_{rel}}{\beta(\beta
t_{rel}+1)}$ (such quantities and also the mutual information and standard
transfer entropy in the BLRM are discussed in (Auconi  _et al._ , 2017)).

## APPENDIX D: NUMERICAL ESTIMATION OF THE ENTROPY PRODUCTION IN THE BIVARIATE
GAUSSIAN APPROXIMATION.

We calculate numerically the mapping irreversibility $\varPhi_{\tau}^{xy}$ as
an average of the spatial density of entropy production $\psi(x_{t},y_{t})$,
$\varPhi_{\tau}^{xy}=\int^{\infty}_{-\infty}\int^{\infty}_{-\infty}dx_{t}dy_{t}\psi(x_{t},y_{t})$.
In the computer algorithm the $(x,y)$ space is dicretized in boxes $(i,j)$,
and for each box we estimate the conditional correlation $C_{xy|i,j}$ of
future values $(x_{t+\tau},y_{t+\tau})$, the conditional correlation
$\widetilde{C_{xy|i,j}}$ of past values $(x_{t-\tau},y_{t-\tau})$, the
expected values for both variables in future ($\braket{x}{i,j}$,
$\braket{y}{i,j}$) and past states ($\braket{\widetilde{x}}{i,j}$,
$\braket{\widetilde{y}}{i,j}$), and the standard deviations on those
$\sigma_{x|i,j}$, $\widetilde{\sigma_{x|i,j}}$, $\sigma_{y|i,j}$,
$\widetilde{\sigma_{y|i,j}}$. The spacial density evaluated in the box $(i,j)$
is then calculated as the bidimensional Kullback-Leibler divergence in the
Gaussian approximationDuchi (2007):

$\displaystyle\psi(i,j)=P(i,j)~{}\frac{1}{2}[\ln(\frac{\widetilde{\sigma_{x|i,j}^{2}}\widetilde{\sigma_{y|i,j}^{2}}(1-\widetilde{C^{2}_{xy|i,j}})}{\sigma_{x|i,j}^{2}\sigma_{y|i,j}^{2}(1-C^{2}_{xy|i,j})})-2+\frac{\frac{\sigma_{x|i,j}^{2}}{\widetilde{\sigma_{x|i,j}^{2}}}+\frac{\sigma_{y|i,j}^{2}}{\widetilde{\sigma_{y|i,j}^{2}}}-2\frac{\sigma_{x|i,j}\sigma_{y|i,j}}{\widetilde{\sigma_{x|i,j}}\widetilde{\sigma_{y|i,j}}}C_{xy|i,j}\widetilde{C_{xy|i,j}}}{1-\widetilde{C^{2}_{xy|i,j}}}+$
$\displaystyle+\frac{\widetilde{\sigma^{2}_{y|i,j}}(\braket{\widetilde{x}}{i,j}-\braket{x}{i,j})^{2}+\widetilde{\sigma^{2}_{x|i,j}}(\braket{\widetilde{y}}{i,j}-\braket{y}{i,j})^{2}-2\widetilde{C_{xy|i,j}}\widetilde{\sigma_{x|i,j}}\widetilde{\sigma_{y|i,j}}(\braket{\widetilde{x}}{i,j}-\braket{x}{i,j})(\braket{\widetilde{y}}{i,j}-\braket{y}{i,j})}{\widetilde{\sigma^{2}_{x|i,j}}\widetilde{\sigma^{2}_{y|i,j}}(1-\widetilde{C^{2}_{xy|i,j}})}]$
(38)

The effect of the finite width of the discretization is attenuated by
estimating all the quantities taking into account the starting point
$(x_{t},y_{t})$ within the box $(i,j)$, subtracting the difference to the mean
values for each box. For example, when we sample for the estimate of the
conditional average $\braket{x_{t+\tau}}{i}$ we would collect samples
$x_{t+\tau}-(x_{t}-\braket{x_{t}}{i})$.

## References

  * Jarzynski (2011) C. Jarzynski, Annu. Rev. Condens. Matter Phys. 2, 329 (2011).
  * Parrondo  _et al._ (2009) J. M. Parrondo, C. Van den Broeck, and R. Kawai, New Journal of Physics 11, 073008 (2009).
  * Feng and Crooks (2008) E. H. Feng and G. E. Crooks, Physical review letters 101, 090602 (2008).
  * Jarzynski (1997) C. Jarzynski, Physical Review Letters 78, 2690 (1997).
  * Crooks (1999) G. E. Crooks, Physical Review E 60, 2721 (1999).
  * Evans and Searles (2002) D. J. Evans and D. J. Searles, Advances in Physics 51, 1529 (2002).
  * Kawai  _et al._ (2007) R. Kawai, J. Parrondo, and C. Van den Broeck, Physical review letters 98, 080602 (2007).
  * Jarzynski (2000) C. Jarzynski, Journal of Statistical Physics 98, 77 (2000).
  * Chernyak  _et al._ (2006) V. Y. Chernyak, M. Chertkov, and C. Jarzynski, Journal of Statistical Mechanics: Theory and Experiment 2006, P08001 (2006).
  * Ito and Sagawa (2013) S. Ito and T. Sagawa, Physical review letters 111, 180603 (2013).
  * Sagawa and Ueda (2012) T. Sagawa and M. Ueda, Physical Review E 85, 021104 (2012).
  * Sagawa and Ueda (2010) T. Sagawa and M. Ueda, Physical review letters 104, 090602 (2010).
  * Szilard (1964) L. Szilard, Systems Research and Behavioral Science 9, 301 (1964).
  * Toyabe  _et al._ (2010) S. Toyabe, T. Sagawa, M. Ueda, E. Muneyuki, and M. Sano, Nature Physics 6, 988 (2010).
  * Auconi  _et al._ (2017) A. Auconi, A. Giansanti, and E. Klipp, Physical Review E 95, 042315 (2017).
  * Cover and Thomas (2012) T. M. Cover and J. A. Thomas,  _Elements of information theory_ (John Wiley & Sons, 2012).
  * Ito (2016) S. Ito, Scientific reports 6 (2016).
  * Seifert (2012) U. Seifert, Reports on Progress in Physics 75, 126001 (2012).
  * Shreve (2004) S. E. Shreve,  _Stochastic calculus for finance II: Continuous-time models_ , Vol. 11 (Springer Science & Business Media, 2004).
  * Seifert (2005) U. Seifert, Physical review letters 95, 040602 (2005).
  * Ito and Sagawa (2016) S. Ito and T. Sagawa, Mathematical Foundations and Applications of Graph Entropy 3, 63 (2016).
  * Uhlenbeck and Ornstein (1930) G. E. Uhlenbeck and L. S. Ornstein, Physical review 36, 823 (1930).
  * Gillespie (1996) D. T. Gillespie, Physical review E 54, 2084 (1996).
  * R Core Team (2014) R Core Team,  _R: A Language and Environment for Statistical Computing_ , R Foundation for Statistical Computing, Vienna, Austria (2014).
  * Klipp  _et al._ (2016) E. Klipp, W. Liebermeister, C. Wierling, A. Kowald, and R. Herwig,  _Systems biology: a textbook_ (John Wiley & Sons, 2016).
  * Kholodenko (2006) B. N. Kholodenko, Nature reviews Molecular cell biology 7, 165 (2006).
  * Bialek and Setayeshgar (2005) W. Bialek and S. Setayeshgar, Proceedings of the National Academy of Sciences of the United States of America 102, 10040 (2005).
  * Tkačik  _et al._ (2008) G. Tkačik, C. G. Callan, and W. Bialek, Proceedings of the National Academy of Sciences 105, 12265 (2008).
  * Crisanti  _et al._ (2018) A. Crisanti, A. De Martino, and J. Fiorentino, Physical Review E 97, 022407 (2018).
  * Waltermann and Klipp (2011) C. Waltermann and E. Klipp, Biochimica et Biophysica Acta (BBA)-General Subjects 1810, 924 (2011).
  * Di Talia and Wieschaus (2012) S. Di Talia and E. F. Wieschaus, Developmental cell 22, 763 (2012).
  * Nemenman (2012) I. Nemenman, Physical biology 9, 026003 (2012).
  * Di Talia and Wieschaus (2014) S. Di Talia and E. F. Wieschaus, Biophysical journal 107, L1 (2014).
  * Tkačik  _et al._ (2009) G. Tkačik, A. M. Walczak, and W. Bialek, Physical Review E 80, 031920 (2009).
  * Barnett  _et al._ (2009) L. Barnett, A. B. Barrett, and A. K. Seth, Physical review letters 103, 238701 (2009).
  * Duchi (2007) J. Duchi, Berkeley, California (2007).