# Time-irreversibility tests for random-length time series: the matching-time
approach applied to DNA

R. Salgado-García<EMAIL_ADDRESS>Centro de Investigación en Ciencias-IICBA,
Physics Department, Universidad Autónoma del Estado de Morelos. Avenida
Universidad 1001, colonia Chamilpa, CP 62209, Cuernavaca Morelos, Mexico.

###### Abstract

In this work we implement the so-called matching time estimators for
estimating the entropy rate as well as the entropy production rate for
symbolic sequences. These estimators are based on recurrence properties of the
system, which have been shown to be appropriate to test irreversibility
specially when the sequences have large correlations or memory. Based on limit
theorems for matching-times we derive a maximum likelihood estimator for
entropy rate assuming that we have a set of moderately short symbolic time-
series of finite random duration. We show that the proposed estimator has
several properties that makes it adequate to estimate entropy rate and entropy
production rate (or to test irreversibility) when the sample sequences have
different lengths such as the coding sequences of DNA. We test our approach in
some controlled examples of Markov chains. We also implement our estimators in
genomic sequences to show that the degree of irreversibility coding sequences
of human DNA is significantly larger than the corresponding non-coding
sequences.

††preprint: AIP/123-QED

> Time irreversibility of a time series gives statistical information about
> the nature of the underlying process such as the presence of nonlinear
> correlations Daw, Finney, and Kennel (2000). Although this concept has been
> mainly used in physics to determine how much a given process is far from
> thermodynamic equilibrium Latora and Baranger (1999); Porporato, Rigby, and
> Daly (2007); Roldán and Parrondo (2012); Gaspard (2004), it has been adapted
> and implemented in other disciplines to study the time irreversibility of a
> wide range of different phenomena, such as for instance, the heart beat time
> series Costa, Goldberger, and Peng (2005), DNA sequences Salgado-Garcia and
> Maldonado (2021); Provata, Nicolis, and Nicolis (2014), financial time
> series of assets Flanagan and Lacasa (2016) or even pieces of classical
> music González-Espinoza, Martínez-Mekler, and Lacasa (2020). The problem of
> determining whether or not is a series time irreversible is not a trivial
> task due to the finite duration of the measurements (or the time series).
> Furthermore, in many cases such time series might be even of random
> duration. The DNA coding sequences, music scores and even heart beat time
> seres are some examples of this situation. In this case, estimating the
> degree of irreversibility requiere a different approach from the standard
> tools already developed. Here we propose a method that profits of the random
> duration of the time-series when we have access to a sample set of
> different-length time series.

##  I Introduction

Real life measurements of certain variables results in time-series of finite
duration. When there are no restrictions in the number of sample date being
acquired the time series collected can have the same length and these can be
used to perform statistical analysis without worrying about the statistical
errors in each time-series. This is because the error is proportional to the
inverse of the square root of sample size. However, time series collected from
a certain process are not always of the same length, and even might be of
random duration. This is the case, for example, of DNA coding sequences, since
the genes codifying for proteins have different lengths Li and Graur (1991).
Another examples of this situation is the case of symbolic time series coming
from musical pieces González-Espinoza, Martínez-Mekler, and Lacasa (2020), the
heart beat time series Costa, Goldberger, and Peng (2005) or any other
biological signal whose measurements might be abruptly interrupted by
uncontrolled external or internal influences. It is clear that the statistical
errors in these cases are different due to randomness in the duration of the
series and even this phenomenon might cause a bias if the estimator is
dependent on the length of the time-series as in the case of the recurrence-
time statistics Kontoyiannis (1998); Chazottes and Redig (2005); Chazottes and
Ugalde (2005); Maldonado (2015); Salgado-Garcia and Maldonado (2021).

In this work we are mainly interested in determine the degree of
irreversibility of a process by analyzing a sample set of random-length time
series, i.e. finite-time realizations of random duration of the corresponding
process. This is made through the use of the recurrence-time statistics
Kontoyiannis (1998); Chazottes and Redig (2005); Maldonado (2015); Salgado-
Garcia and Maldonado (2021) and in particular of the matching-time estimator
for entropy rate Kontoyiannis (1998). This estimator has been less studied in
practical applications and in this work we show that it is a suitable tool for
analyzing this class of inhomogeneous time series. To estimate the entropy
rate and _reversed entropy rate_ (i.e., the relative entropy of the process
with respect to the invariant measure of the time-reversed process Chazottes
and Redig (2005)) we assume that the matching-time obtained from every time
series is normally distributed, i.e., we assume the validity of the central
limit theorem for finite, but sufficiently large, time series. This hypothesis
allows us to implement the maximum likelihood method to obtain an estimator
for a sample set of random-length time series. Thereafter it is possible to
determine the degree of irreversibility of the process by comparing the
entropy rate and the reversed entropy rate estimated by this approach.

This work is organized as follows. In Sec. II we resume the statistical
properties of the matching-time estimator for entropy rate, such as the
central limit theorem which was proved in Ref. Kontoyiannis, 1998. In this
section we also introduce the maximum likelihood estimator for the entropy
rate when considering as sample set of random length time series. In Sec. III
we test the proposed estimator in two different scenarios: in the case in
which the symbolic sequences generated from a reversible Markov chain and from
an irreversible Markov chain. In both cases we obtain symbolic sequences of
finite random duration using a given distribution of lengths. In Sec. IV we
implement the proposed estimator to determine the degree of irreversibility of
coding DNA sequences of human genome, in which every gene is seen as a
symbolic time series of random duration. Finally in Sec. V we summarize the
main results of our study as well as the main conclusions of this work.

##  II Matching-time estimators for entropy rate and entropy production rate

###  II.1 Matching times

Let $\mathbf{X}:=\\{X_{n}\,:\,n\in\mathbb{m}_{0}\\}$ be a discrete-valued
stationary ergodic process generated by the law $\mathbb{P}$, whose
realizations are infinite sequences of symbols taken from a finite set $A$,
i.e., the set of all posible realizations is a subset of $A^{\mathbb{N}}$.
Here we denote by $\mathbf{x}=(x_{0}x_{1}x_{2}x_{3}\dots)$ an infinite
realization of the process $\mathbf{X}$. Let $\ell$ be a positive integer, we
denote by $x_{0}^{\ell-1}$ the string of the first $\ell$ symbols of the
realization $\mathbf{x}$. A finite string $\mathbf{a}:=a_{1}a_{2}a_{3}\ldots
a_{\ell}$ comprised of $\ell$ symbols will be called either $\ell$-word or
$\ell$-block. We say that the $\ell$-word $\mathbf{a}$ “occurs” at the $k$th
site of the trajectory $\mathbf{x}$, if $x_{k}^{k+\ell-1}=\mathbf{a}$.

The _entropy rate_ $h$ of the process $\mathbf{X}$ (also called Kolmogorov-
Sinai entropy or KS entropy) is defined as rate at which the entropy of
$n$-blocks grows with $n\in\mathbb{N}_{0}$ Kontoyiannis (1998); Chazottes and
Redig (2005),

$h:=\lim_{n\to\infty}\frac{1}{n}\sum_{x_{0}^{n}\in{A}^{n}}\mathbb{P}(X_{0}^{n-1}=x_{0}^{n-1})\log(\mathbb{P}(X_{0}^{n-1}=x_{0}^{n-1})).$
(1)

Analogously, the _reversed entropy rate_ $h_{\mathrm{R}}$ of the process
$\mathcal{X}$ is defined as Kontoyiannis (1998); Chazottes and Redig (2005),

$h_{\mathrm{R}}:=\lim_{n\to\infty}\frac{1}{n}\sum_{x_{0}^{n-1}\in{A}^{n}}\mathbb{P}(X_{0}^{n-1}=x_{0}^{n-1})\log(\mathbb{P}(X_{0}^{n-1}=x_{n-1}^{0})).$
(2)

The _entropy production rate_ $e_{\mathrm{p}}$ of the process $\mathcal{X}$
quantifies the degree of irreversibility of the process $\mathcal{X}$ and is
defined as Kontoyiannis (1998); Chazottes and Redig (2005),

$e_{\mathrm{p}}:=\lim_{n\to\infty}\frac{1}{n}\sum_{x_{0}^{n-1}\in{A}^{n}}\mathbb{P}(X_{0}^{n-1}=x_{0}^{n-1})\log\left(\frac{\mathbb{P}(X_{0}^{n-1}=x_{0}^{n-1})}{\mathbb{P}(X_{0}^{n-1}=x_{n-1}^{0})}\right).$
(3)

According to Ref. Gaspard, 2004 the entropy production rate can be obtained as
the difference between the reversed entropy rate and the entropy rate,
$e_{\mathrm{p}}=h_{\mathrm{R}}-h$, for Markov processes and for more general
systems Maes (1999). This fact allows to state that the difference between the
reversed entropy rate and entropy rate, $h_{\mathrm{R}}-h$, can be used as an
irreversibility index in systems in which the entropy production rate cannot
be obtained directly from the observed time-series. Now we proceed to state
the corresponding estimators for $h$ and $h_{\mathrm{R}}$ based on the so-
called _matching times_.

Given a finite realization $\mathbf{x}:=x_{0}x_{1},x_{2},\dots x_{t}$ up to
time $t$ of the process $\mathcal{X}$, the _matching time_ $L_{t}^{+}$ is
defined as the shortest $\ell$ such that the $\ell$-word ${x}_{0}^{\ell-1}$
does not reappears in $\mathbf{x}$. Specifically we have that

$L_{t}^{+}(\mathbf{x}):=\min\\{\ell\,:\,{x}_{0}^{\ell-1}\not={x}_{j}^{j+\ell-1},\,\forall
j=1,2,\dots,t-\ell+1\\}.$ (4)

Analogously we define the _reversed matching time_ $L_{t}^{-}$ as the shortest
$\ell$ such that the _time-reversed_ $\ell$-word $\mathbf{x}_{\ell-1}^{0}$
does not appears in $\mathbf{x}$, i.e.,

$L_{t}^{-}(\mathbf{x}):=\min\\{\ell\,:\,{x}_{\ell-1}^{0}\not={x}_{j}^{j+\ell-1},j=1,2,\dots,t-\ell+1\\}.$
(5)

According to Kontoyiannis Kontoyiannis (1998), $L_{t}^{+}$ satisfy a law of
large numbers, in the sense that $L_{t}^{+}/\log(t)$ converges $\mathbb{P}$
almost surely to $1/h$ for $t\to\infty$, where $h$ is the entropy rate of the
process $X$. Analogously the reversed matching times converges $\mathbb{P}$
almost surely to the reversed entropy rate Chazottes and Redig (2005), i.e.,
$L_{t}^{-}/\log(t)\to 1/h_{\mathrm{R}}$ for $t\to\infty$. In a similar way,
these estimators for the entropy rate and reversed entropy rate also satisfy a
central limit theorem in the following form,

$\displaystyle\sqrt{~\frac{h^{3}\log(t)}{\sigma^{2}}}\left(\frac{L_{t}^{+}}{\log(t)}-\frac{1}{h}\right)$
$\displaystyle\to$ $\displaystyle\mathcal{N}(0,1),$ (6)
$\displaystyle\sqrt{~\frac{h_{R}^{3}\log(t)}{\sigma_{R}^{2}}}\left(\frac{L_{t}^{-}}{\log(t)}-\frac{1}{h_{R}}\right)$
$\displaystyle\to$ $\displaystyle\mathcal{N}(0,1),$ (7)

in distribution as $t\to\infty$. Here $\sigma$ and $\sigma_{\mathrm{R}}$ are a
constants depending on the process (see Ref. Kontoyiannis, 1998 for details)
and in this case can be interpreted as parameters to be estimated related to
the statistical error due to the finiteness of the size of the sample
sequence.

Next, we introduce the random variables $X^{+}$ and $X^{-}$ as follows

$\displaystyle X^{+}_{t}:=\frac{L_{t}^{+}}{\log(t)},$ (8) $\displaystyle
X^{-}_{t}:=\frac{L_{t}^{-}}{\log(t)},$ (9)

whose realizations can be interpreted as observation of an approximation to
the inverse entropy rate and reversed entropy rate, respectively, for a time
series of finite length $t$.

Notice that both $X^{+}_{t}$ and $X^{-}_{t}$ satisfies the central limit
theorem, as we stated in Eqs. (6) and (7), which allows us to assume that the
distribution of $X^{+}_{t}$ and $X^{-}_{t}$ is normal for finite, but
sufficiently large, $t$. This assumption implies that the probability
distribution function of $X_{t}^{+}$, which we denote by $f_{+}(s)$, and the
probability distribution function of $X_{t}^{-}$, denoted by $f_{-}(s)$, can
be approximated by the normal distribution as follows,

$\displaystyle f_{+}(x)$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2\pi\varrho_{+}^{2}}}\exp\left(-\frac{(x-1/h)^{2}}{2\varrho_{+}^{2}}\right),$
(10) $\displaystyle f_{-}(x)$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2\pi\varrho_{-}^{2}}}\exp\left(-\frac{(x-1/h_{\mathrm{R}})^{2}}{2\varrho_{-}^{2}}\right),$
(11)

respectively, for $t\gg 1$. Here $\varrho^{2}_{+}$ and $\varrho^{2}_{-}$ are
the variance of $X_{t}^{+}$ and $X_{t}^{-}$ respectively, which depend on the
length of sample sequence $t$. Explicitly $\varrho^{2}_{+}$ and
$\varrho^{2}_{-}$ are defined as

$\displaystyle\varrho^{2}_{+}$ $\displaystyle:=$
$\displaystyle\frac{\sigma^{2}}{h^{3}\log(t)},$ (12)
$\displaystyle\varrho^{2}_{-}$ $\displaystyle:=$
$\displaystyle\frac{\sigma_{R}^{2}}{h_{R}^{3}\log(t)}.$ (13)

Under this assumption, it is clear that the mean and variance of $X_{t}^{+}$
are given by

$\mathbb{E}[X_{t}^{+}]=\frac{1}{h},\qquad\mbox{Var}(X_{t}^{+})=\varrho^{2}_{+}.$
(14)

Analogously we also have that

$\mathbb{E}[X_{t}^{-}]=\frac{1}{h_{\mathrm{R}}},\qquad\mbox{Var}(X_{t}^{-})=\varrho_{-}^{2}.$
(15)

###  II.2 Estimation procedure

Now let us state the problem we are facing. We will assume that we have a set
$\mathcal{W}:=\\{\mathbf{x}_{i},:\,|\mathbf{x}_{i}|=t_{i},1\leq i\leq m\\}$ of
$m$ finite sample sequences (finite-time observations of the process
$\mathcal{X}$). The sample sequences are assumed to have different lengths
$t_{i}$ and we wish to estimate the entropy rate and the reversed entropy
rate. It is reasonable to assume that the sequence lengths $t_{i}$ are
independent realizations of certain random variable whose distribution is
denoted by $g(t)$. The collection of all the sequence lengths $t_{i}$ will be
denoted by $\mathcal{T}$, i.e.,

$\mathcal{T}:=\\{t_{i}\,:~\,1\leq i\leq m\\}.$ (16)

It is clear that each sequence in $\mathcal{W}$ gives us a sample of the
matching time and a sample of the reversed matching time by applying eq. (4)
and eq. (5) respectively. We denote by

$\mathcal{L}^{+}:=\\{\ell_{i}^{+}=L_{t_{i}}^{+}(\mathbf{x}_{i})\,:\,\mathbf{x}_{i}\in\mathcal{W},\,1\leq
i\leq m\\},$ (17)

the set of all resulting matching times by applying Eq. (4) to each word in
$\mathcal{W}$. Analogously we denote by

$\mathcal{L}^{-}:=\\{\ell_{i}^{-}=L_{t_{i}}^{-}(\mathbf{x}_{i})\,:\,\mathbf{x}_{i}\in\mathcal{W},\,1\leq
i\leq m\\},$ (18)

the set of all reversed matching times obtained by applying Eq. (4) to each
word in $\mathcal{W}$.

The sample sets of matching times, $\mathcal{L}^{+}$ and $\mathcal{L}^{-}$,
give in turn _inhomogeneous_ sample sets of entropy rate and reversed entropy
rate as follows,

$\displaystyle\mathcal{X}^{+}$ $\displaystyle:=$
$\displaystyle\left\\{x_{i}^{+}:=\frac{\ell_{i}}{\log(t_{i})}\,:\,\ell_{i}\in\mathcal{L}^{+},1\leq
i\leq m\right\\}$ (19) $\displaystyle\mathcal{X}^{-}$ $\displaystyle:=$
$\displaystyle\left\\{x_{i}^{-}:=\frac{\ell_{i}}{\log(t_{i})}\,:\,\ell_{i}\in\mathcal{L}^{-},1\leq
i\leq m\right\\}$ (20)

We say that these sample sets are _inhomogeneous_ in the sense that the
collected samples comes from _different_ distributions, since the
distriibution itself depends on $t$, the length of the time series. In other
words, we can think of $\mathcal{X}^{+}$ and $\mathcal{X}^{-}$ as a sets of
realizations of random variables that are independent but not identically
distributed, contrary to what is commonly assumed in statistics. Despite these
sample sets are made up of realizations of different distributions, such
distributions depend parametrically on the entropy rate and reversed entropy
rate respectively, as it can be appreciated in Eqs. (10) and (11) . This fact
allows us to implement some estimators for $h$ and $h_{\mathrm{R}}$. In
Appendix A we show that the maximum likelihood estimators for these quantities
are given by,

$\displaystyle\hat{h}$ $\displaystyle=$
$\displaystyle\frac{\frac{1}{m}\sum_{j=1}^{m}\log(t_{i})}{\frac{1}{m}\sum_{j=1}^{m}\ell_{i}^{+}}$
(21) $\displaystyle\hat{h}_{\mathrm{R}}$ $\displaystyle=$
$\displaystyle\frac{\frac{1}{m}\sum_{j=1}^{m}\log(t_{i})}{\frac{1}{m}\sum_{j=1}^{m}\ell_{i}^{-}}$
(22)

where $\ell_{i}^{+}$ and $\ell_{i}^{-}$ are sample matching times from
$\mathcal{L}^{+}$ and $\mathcal{L}^{-}$ respectively. We should stress the
fact that these estimations do not depend on the specific distribution of the
sequences length $t_{i}$, as we proved in Appendix A. This allows us to apply
our formulas in several possible scenarios in which the sequence lengths are
randomly distributed.

The parameters $\sigma^{2}$ and $\sigma_{\mathrm{R}}^{2}$ can also be
estimated and these quantities allow to obtain the estimated error in the
estimation due to the finiteness of the sample sequences. In Appendix A we
also show that,

$\displaystyle\hat{\sigma}^{2}$ $\displaystyle=$
$\displaystyle\hat{h}^{2}\left(\hat{h}\hat{c}^{+}-\hat{a}^{+}\right),$ (23)
$\displaystyle\hat{\sigma}^{2}_{\mathrm{R}}$ $\displaystyle=$
$\displaystyle\hat{h}_{R}^{2}\left(\hat{h}_{R}\hat{c}^{-}-\hat{a}^{-}\right),$
(24)

where the sample functions $\hat{a}^{\pm}$ and $\hat{c}^{\pm}$ are defined as

$\displaystyle\hat{a}^{\pm}:=\frac{1}{m}\sum_{i=1}^{m}\ell_{i}^{\pm},$ (25)
$\displaystyle\hat{c}^{\pm}:=\frac{1}{m}\sum_{i=1}^{m}\frac{(\ell^{\pm}_{i})^{2}}{\log(t_{i})}.$
(26)

Once we have an expression for $\sigma^{2}$ and $\sigma_{\mathrm{R}}^{2}$, the
errors in the estimation due to the finiteness of the sample sequence can be
obtained from the central limit theorem through equations (6) and (7). In
Appendix A we show that the average errors due to the finiteness of the sample
sequences are given by,

$\displaystyle\hat{\varepsilon}=\frac{\hat{\sigma}^{2}}{{\hat{h}}}\frac{1}{m}\sum_{i=1}^{m}\frac{1}{{\log(t_{i})}},$
(27)
$\displaystyle\hat{\varepsilon}_{\mathrm{R}}=\frac{\hat{\sigma}_{R}^{2}}{{\hat{h}_{R}}}\frac{1}{m}\sum_{i=1}^{m}\frac{1}{{\log(t_{i})}}.$
(28)

These expressions for $\hat{\varepsilon}$ and allow us evaluate how large is
the error due to the fact that the limit $t\to\infty$ has not been reached and
due to the fact the sample sequences are inhomogeneous in their length.

##  III Numerical experiments

In this section we will perform numerical simulations in order to test the
proposed procedure to obtain estimation for the entropy rate and the reversed
entropy rate and consequently, an estimation for the entropy production rate.
We consider the case of a three-states Markov chain which, depending on a
parameter, can be reversible or irreversible, thus allowing to test the
estimator in both situations.

###  III.1 Markov chain model

To test the estimator for entropy rate we use a three-states Markov chain that
is a minimal model of an irreversible stochastic model Jiang, Qian, and Qian
(2004). Actually, depending on a parameter, the chain can be reversible or
irreversible, thus allowing to test the estimator in both situations, showing
that our method allows to estimate the entropy rate. Consequently, we can use
the estimator as an index of irreversibility to apply in real situations.

Let $\\{X_{t}\in\mathbf{S}:t\in\mathbb{N}\\}$ be a discrete-time stochastic
process with state space $\mathbf{S}:=\\{1,2,3\\}$. We define the process as a
three state Markov chain by introducing the stochastic matrix
$M:\mathbf{S}\times~\mathbf{S}\to[0,1]\subset\mathbb{R}$, defined as

$M=\left(\begin{array}[]{ccc}0&p&1-p\\\ 1-p&0&p\\\ p&1-p&0\end{array}\right),$
(29)

where $p$ is a parameter such that $p\in[0,1]$. It is easy to see that this
matrix is doubly stochastic and therefore the invariant distribution
$\mathbf{\pi}=\mathbf{\pi}M$ is given by
$\pi=(\frac{1}{3},\frac{1}{3},\frac{1}{3})$. Moreover, it is easy to compute
the entropy rate and the time-reversed entropy rate, which are given by Jiang,
Qian, and Qian (2004),

$\displaystyle h(q)$ $\displaystyle=$ $\displaystyle-q\log(q)-(1-q)\log(1-q),$
(30) $\displaystyle h_{R}(q)$ $\displaystyle=$
$\displaystyle-(1-q)\log(q)-q\log(1-q).$ (31)

Additionally, the corresponding entropy production rate is given by

$e_{p}(q)=(2q-1)\log\left(\frac{q}{1-q}\right).$ (32)

It is clear from the above formulas that the chain is reversible only for the
case $p=1/2$. Otherwise, the process is irreversible with a degree of
irreversibility increasing as $p$ deviates from $p=1/2$.

###  III.2 Entropy estimation tests for fixed length time series

Figure 1: Estimates of entropy rate and entropy production rate for a
reversible Markov chain. (a) Using $m=10^{4}$ realizations of fixed length $t$
we estimate the entropy rate and the reversed entropy rate using the matching-
time estimators given in Eqs. (21) and (21) (solid and dashed lines
respectively). The red line stands for the exact value of the entropy rate. We
can observe that the error in the estimation for small $t$ (less that $10^{3}$
time steps) is of the same order of magnitude as for large $t$ (for $10^{6}$
time steps). (b) We should notice that the entropy production rate can be
accurately predicted for moderately small $t$. In this case, due to
reversibility, the entropy rate and the reversed entropy rate are the same and
therefore the entropy production rate is zero.

The first test we implement consists in obtaining $m$ realizations of the
process of fixed (non random) length $t$. This numerical experiment is
designed to test how accurate is the convergence of the entropy rate through
the matching time estimator without considering the randomness in the length
of the time series. For the first numerical experiment we simulate the above
defined Markov chain for $p=0.5$. In this case we expect that the entropy rate
and the reversed entropy rate be equal, thus obtaining a vanishing entropy
production rate. Then we simulate trajectories (time-series) of several
lengths, ranging from $t=500$ time steps up to $t=10^{6}$ time steps. For each
fixed length $t$ we obtain $m=10^{4}$ different realizations of the process
and for every realization we compute the corresponding matching-time and
reversed matching-time. This procedure gives us a sample set of $m=10^{4}$
realizations of matching-times and a set of $m=10^{4}$ realizations of
reversed matching-times. These sample sets are then used to obtain the
corresponding estimations of the entropy rate and the reversed entropy rate.
In Fig. 1a we show the behavior of the estimated entropy rate and reversed
entropy rate as a function of the length of the series. We can observe that
the error for short sequences (i.e., for $t\sim 10^{3}$ or below) is
approximately of the same order of magnitude as for large sequences ($t\sim
10^{6}$).This means that the matching-time estimator for entropy rate has some
stability in its accuracy when the length of the time series changes up to
three order of magnitude, making it feasible the entropy estimations for time
series of random length varying from moderately short lengths (around $10^{3}$
time steps) up to large lengths without worrying about the introduction of
large errors for considering small length time series. This effect is more
evident when looking at the estimation of the entropy production rate. For
$p=0.5$ the Markov chain is reversible and the corresponding entropy
production rate is zero. In Fig. 1b we display the behavior of the estimated
entropy production rate as a function of the size of the series $t$. We should
notice that in this case the convergence is fast, since the estimated entropy
production rate goes to zero (in the average) for time series lengths of the
order of $10^{3}$ time steps. This property makes the matching-time estimator
adequate to test the irreversibility of a series even for moderately short
length time series.

Figure 2: Estimates of entropy rate and entropy production rate for an
irreversible Markov chain. (a) Using $m=10^{4}$ realizations of fixed length
$t$ we estimate the entropy rate and the reversed entropy rate using the
matching-time estimators given in Eqs. (21) and (21) (solid and dashed lines
respectively). The solid red line and the dashed red lines stands for the
exact value of the entropy rate and the reversed entropy rated respectively.
We can observe that the error in both of these estimations for small $t$ (less
that $10^{3}$ time steps) is of the same order of magnitude as for large $t$
(for $10^{6}$ time steps). (b) We should notice that the estimated entropy
production rate (solid line) can be accurately predicted for moderately small
$t$. In this case the exact value of entropy production rate is denoted by the
solid red line, which is the difference between $h$ and $h_{\mathrm{R}}$.

The second test we implement consists in estimating the entropy rate and
reversed entropy rate for time series coming from an irreversible Markov
chain. For this case we use the three-states Markov chain model introduced
above for $p=0.60$. As in the case $p=0.50$ we perform the estimation of
entropy rate without considering the randomness in the length of the time
series. We simulate trajectories of several lengths, ranging from $t=500$ to
$t=10^{6}$ time steps. For each fixed length $t$ we obtain $m=10^{4}$
different realizations of the process and for every realization we compute the
corresponding matching-time and reversed matching-time. These sample sets are
then used to obtain the corresponding estimations of the entropy rate and the
reversed entropy rate through the matching-times as described in Sec. II.2. In
Fig. 2a we show the behavior of the estimated entropy rate and reversed
entropy rate as a function of the length of the series. As in the case
$p=0.50$, for the case $p=0.60$ we see that the error for short sequences (for
$t\sim 10^{3}$) is approximately of the same order of magnitude as for large
sequences ($t\sim 10^{6}$), implying that the accuracy of the estimations is
stable a we move from short to large lengths. In Fig. 2b we show the behavior
of the estimated entropy production rate as a function of the series length
$t$. We can see that the accuracy of $\hat{e}_{\mathrm{p}}$ also maintains
within the same order of magnitude as we increase the series length from
moderately short ($t\sim 10^{3}$) to large ($t\sim 10^{6}$). According to
these numerical test, it seem that the entropy rate estimator based on
matching-times are adequate for testing irreversibility for random length
time-series.

Figure 3: Random length distribution. We show the histogram of $10^{4}$
realizations of random lengths (black bars) using the Gamma distribution model
(solid red line) given in Eqs. (34) and (33). The parameters are chosen as
$k=1$ and $\lambda=1/1921$. The latter corresponds to the inverse mean length
of the coding sequences of the human genome which where analyzed to test
irreversibility. This choice was done to mimic the statistical properties of
the real random lengths in order to implement control test with artificial
sequences. Notice that the histogram starts at $t_{\mathrm{min}}=10^{3}$. This
is because we ignore all sequences below $t_{\mathrm{min}}$ for performing the
entropy estimations.

###  III.3 Entropy estimation tests for random length time series

The next numerical experiment is designed to test the estimators for $h$ and
$h_{\mathrm{R}}$ given in Eqs. (21) and (21), but now considering random
length time series. To this end we propose a model distribution to generate
the random length time series of the three-states Markov chain. The model we
use to generate random lengths is a discrete Gamma distribution Chakraborty
and Chakravarty (2012) whose probability function $g(t)$, for $t\in\mathbb{N}$
is defined as

$g(t)=G(t)-G(t-1),$ (33)

where $G(x)$ is the (cumulative) distribution function of a (continuous)
random variable with Gamma distribution,

$G(x):=\int_{0}^{x}\frac{\lambda\left(\lambda x\right)^{k}e^{-\lambda
x}}{\Gamma(k)}$ (34)

This model was chosen because it reproduce the main statistical features of
the length distribution of real coding sequences of human genome (see Sec. IV
below). Based on observations of real genomic sequences we choose the
parameter $k=1$ and $\lambda=1/1921$, the latter being the inverse of the mean
length coding sequences of human genome. In Fig. 3 we show the histogram of
$m=10^{4}$ realization of random length using this model. As we will see
below, this histogram mimics the main qualitative behavior of the histogram of
the random lengths coming from coding sequences of the human DNA.

Next we generate the random length time series. First we generate a random
length $t$ using the Gamma model distribution. Then, we start to generate the
time series by simulating the Markov chain and stopping the process after $t$
time steps. It is worth mentioning that we implemented an additional condition
on $t$. Actually we limited the value of $t$ to lie in the interval
$10^{3}\leq t\leq 1.5\times 10^{4}$. This condition is imposed to avoid $t$
values too short, in order to have a better control of the error, which is
according to the tests implemented in Sec. III.2.

Figure 4: Estimations of entropy rates and reversed entropy rate for a Markov
chain. (a) We show the exact entropy rate $h$ (solid line) as a function of
the parameter $p$ of the Markov chain defined in Sec. III.1. We estimate $h$
(open circles) using the method of matching-times for random lengths
introduced in Sec. III.2 using the Gamma model for generating the random
lengths. We also show the corresponding estimations of entropy rate but using
fixed length time series (open squares) for a length of $t=10^{6}$ time steps
(b) The same as in panel (a) for the reversed entropy rate. We should notice
that both estimations, fixed and random length time series, gives estimations
consistent with the exact corresponding values.

We then use this procedure to generate $m=10^{4}$ time series using the above
described Markov chain for several values of the parameter $p$, ranging from
$p=0.01$ to $p=0.99$. Next, for every fixed $p$ we use the entropy estimators
(21) and (21) to estimate $h$ and $h_{\mathrm{R}}$. In Fig 4 we show the
estimations of the entropy rate (Fig. 4a) and the reversed entropy rate (Fig.
4b) using this procedure (open circles) compared with their exact counterparts
(solid black lines) using the corresponding formulas given in Eqs. (30) and
(31). In order to analyze the performance of these estimations based on random
length time series we also display entropy rate (open squares in Fig. 4a) and
the reversed entropy rate (open squares in Fig. 4b) estimations using fixed-
length time series as it was done in Sec. III.2. The length used to obtain the
corresponding entropy estimations was $t=10^{6}$ time steps. It is important
to stress that for random-length estimations we used time series whose length
lie in the interval $10^{3}\leq t\leq 1.5\times 10^{4}$). Notice that despite
the large difference in the lengths of the time series used for the
estimations (a difference of around two orders of magnitude), we have that the
accuracy of the random-length estimations is comparable to the one of the
fixed-length estimations. Moreover, as we can appreciate from Fig. 4, these
estimations are both comparable with the corresponding exact entropy rates.
This allows us to state that the proposed estimators based on matching-times
seem to be adequate to implement it in real scenarios in which the time series
are moderately short and random.

##  IV Testing irreversibility of DNA sequences

Next we turn out to the problem of determining if the coding sequences of
human DNA is reversible or not and if this irreversibility (if any) is
comparable to the corresponding degree of irreversibility of the whole human
genome. It is important to remark the statistical properties of DNA has been
previously studied from several points of view Salgado-García and Ugalde
(2016), and in particular the irreversible character of human genome has been
analyzed in Refs. Provata, Nicolis, and Nicolis, 2014; Salgado-Garcia and
Maldonado, 2021. It is clear that understanding this characteristic might give
us a deeper understanding of the physics of DNA, such as protein diffusion
along the DNA Gorman and Greene (2008); Mirny  _et al._ (2009); Salgado-García
(2019); Cocho  _et al._ (2003) among other phenomena.

Figure 5: Random length distribution for coding sequences of _Homo sapiens_.
We show the frecuency histogram as a function of the length of the coding
sequences. We should notice that the most frequent length we found in coding
sequences is nearly $10^{3}$ bp. Actually, the mean length of the coding
sequences is $1970$ bp. For the entropy estimation analysis we only take into
account sequences larger than $t_{\mathrm{min}}=10^{3}$ bp. Shorter sequences
are excluded to fulfill as most as possible the validity of the central limit
theorem.

In Refs.Provata, Nicolis, and Nicolis, 2014; Salgado-Garcia and Maldonado,
2021 it was found that the entropy production rate estimated from human genome
(including coding and non-coding sequences) was around $e_{\mathrm{p}}\approx
0.07$, a value which is significantly non-zero. These studies reveal that real
genomic sequences might be indeed irreversible, however still remains open the
question if the coding part of the genome has a degree of irreversibility
larger or lower that the whole genome. In order to shed some light on this
question we analyzed the irreversibility of the coding part of the human
genome with the technique introduced in this work. To do this, we obtained the
coding sequences of all the chromosomes of _Homo sapiens_ from the GenBank
database Benson  _et al._ (1997).

First we should stress the fact that the coding sequences of human genome have
different lengths which can be considered as random. The corresponding empiric
distribution can be appreciated in Fig. 5 where we display the frequency
histogram as a function of the length of the sequence in base-pairs (bp). It
is clear that the most frequent length we found in the sample set of coding
sequences is of the order of $t\sim 10^{3}$ bp; actually, the mean length
$\bar{t}$ of the coding sequences is $\bar{t}=1970$ bp. For our analysis we
discard the sequences whose length is lower than $t_{\mathrm{min}}=10^{3}$ bp
. The latter was done to avoid errors due to short sequences since we need
that the sequences be large enough to satisfy the central limit theorem.
Taking into account only sequences larger than $10^{3}$ bp coming from all the
chromosomes, we analyzed $80671$ coding sequences with a mean length $3360$
bp. In Fig. 6a we show the mean length $t$ of the coding sequences from every
chromosome. In Fig. 6b we observe the number of coding sequences contained in
every chromosome. We can see that there are chromosomes with a low number of
coding sequences such as the chromosomes $13$, $21$ and $Y$. This fact might
introduce statistical errors in estimations. Despite this fact, as we will see
below, the entropy estimations even for these chromosomes gives estimations
consistent with the estimations from the rest of chromosomes.

Figure 6:  Mean length and number of sample sequences for _Homo sapiens_. (a)
We show the mean length of the coding sequences extracted from every
chromosome.For the analysis we took into account only coding sequences larger
than $10^{3}$ bp. (b) Number of coding sequences contained in every
chromosome. We notice that there are chromosomes with a low number of coding
sequences such as the chromosomes $13$, $21$ and $Y$. This fact might
introduce statistical errors in estimations. Despite this fact, the entropy
estimations even for these chromosomes gives estimations consistent with the
estimations from the rest of chromosomes.

Next we proceed to estimate the entropy rate and the reversed entropy rate for
every chromosome separately. In Fig. 7a we can see the estimations of $h$
(black bars) and $h_{\mathrm{R}}$ (red bars) for every human chromosome. We
should see that the reversed entropy rate is significantly larger than the
entropy rate thus allowing us to see that there is certain irreversibility of
the coding sequences. In Fig. 7b we display the estimated entropy production
rate ($e_{\mathrm{p}}=h_{\mathrm{R}}-h$) for every chromosome. We can see that
the entropy production rate is around $0.2$ and actually the mean
$\hat{e}_{\mathrm{p}}$ we obtain from these data is $e_{\mathrm{p}}\approx
0.1928$ In Table 1 we resume the values of the estimations of entropy rate and
entropy production rate for both coding and non-coding sequences of _Homo
sapiens_. The values for non-coding sequences was obtained from Refs. Provata,
Nicolis, and Nicolis, 2014; Salgado-Garcia and Maldonado, 2021.

We clearly see that the estimated entropy production rate for coding sequences
that we obtained here is larger than twice the entropy production rate
reported for the analyzed sequences of _Homo sapiens_ which include both
coding and non-coding. Since the non-coding sequences in _Homo sapiens_ is
around the $98\%$ of the whole genome we can say that the estimations reported
in Refs. Provata, Nicolis, and Nicolis, 2014; Salgado-Garcia and Maldonado,
2021 are representative of the non-coding genomic material. Therefore, our
results allow to state that the degree of irreversibility of coding sequences
is significantly larger than the non-coding material. This would mean that the
non-coding part of the genome might possibly underwent several processes that
lead to lose its irreversibility, such as random mutations or any other
process (or thermodynamic force) that drives the non-coding genetic material
to the equilibrium in the sense of reversibility.

Figure 7: Entropy estimations for coding sequences of _Homo sapiens_. (a) We show the estimations of the entropy rate (black bars) and the reversed entropy rate (red bars) for the coding part of every chromosome. We should notice that the reversed entropy rate is larger than the entropy rate, which means that the coding sequences are spatially irreversible. (b) We show the behavior of the estimated entropy production rate for every chromosome. We can observe that the value of $e_{\mathrm{p}}$ deviates moderately on the chromosomes, thus allowing us to say that the mean value of the entropy production rate that we estimate is representative of the coding part of the human genome.  Table 1: Entropy rate and entropy production rate for coding and non-coding DNA of _Homo sapiens_.  | coding | non-coding
---|---|---
$\hat{h}$ | $1.3334$ |  $1.277$111From Ref. Salgado-Garcia and Maldonado, 2021
$\hat{h}_{\mathrm{R}}$ | $1.5262$ |  $1.352$111From Ref. Salgado-Garcia and Maldonado, 2021
$\hat{e}_{\mathrm{p}}$ | $0.1928$ |  $0.075$222From Ref. Provata, Nicolis, and Nicolis, 2014. The corresponding value reported in Ref. Salgado-Garcia and Maldonado, 2021 is $e_{\mathrm{p}}\approx 0.077$.
$\mbox{STD}(e_{\mathrm{p}})/\hat{e}_{\mathrm{p}}$ |  $0.1442$333The standard deviation of $e_{\mathrm{p}}$ is computed with respect to the estimated values from the chromosomes. It represents the mean deviation of the entropy production rate from one chromosome to another. | $-$

##  V Conclusions

We have introduced the maximum likelihood estimator of entropy rate based on
the recurrence properties of the system. In particular we made use of the so-
called matching times which gives an estimator for the inverse entropy rate.
According to the central limit theorem Kontoyiannis (1998), this estimator
converges to the normal distribution if it is appropriately normalized. We
used this fact to obtain a maximum likelihood estimator when we have time
series of several size i.e., when we have a sample set of time series of
random duration. We showed through numerical simulations that the proposed
estimator gives accurate enough estimations of entropy rate and reversed
entropy rate in the case of Markov chain even for moderately short time series
(of around $t^{3}$ time steps). Once we tested the estimator using time series
of random length obtained from both, a reversible and an irreversible Markov
chain, we proceed to implement our method to determine the degree of
irreversibility of coding sequences of human genome. We observed that the
coding sequences of _Homo sapiens_ have an entropy production rate
$\hat{e}_{\mathrm{p}}=0.1928$. Interestingly we saw that this value for
$\hat{e}_{\mathrm{p}}$ does not deviates too much from one chromosome to
another, i.e., the degree of irreversibility of all the coding part of all the
chromosomes of the human genome is approximately the same. This fact suggest
that the irreversibility character of the coding sequences might be result of
a evolutive process that lead the coding sequences to its current degree of
irreversibility. Moreover, other studies revealed that the non-coding part of
the human genome has a degree of irreversibility that is lower than the one we
report here for the coding sequences. This would mean that the non-coding part
of the genome, might underwent some process that lead it to lower its degree
of irreversibility in the course of evolution, i.e., the non-coding part of
the genome might possibly be under the influence of some thermodynamic force
that drives the sequences to the equilibrium in the sense of irreversibility.
Clearly, understanding the origin of irreversibility of real genomes requiere
a much more deeper statistical analysis from the point of view of genome
evolution.

###### Acknowledgements.

This work was supported by CONACYT through grant FORDECYT-
PRONACES/1327701/2020.

##  Appendix A Maximum Likelihood estimator for inhomogeneous samples.

Let us start by assuming that we have a normal random variable $X$ whose
distribution depends on a value $t$. We assume this parameter $t$ is a
realization of a random variable $T$ whose distribution is denoted by
$g(t,\theta)$, where $\theta$ is a (unknown) parameter to be estimated. In
terms of $t$, the mean and variance of $X$ are given by

$\displaystyle\mathbb{E}[X]$ $\displaystyle=$ $\displaystyle\frac{1}{h},$ (35)
$\displaystyle\mbox{Var}(X)$ $\displaystyle=$
$\displaystyle\frac{\sigma^{2}}{h^{3}\log(t)},$ (36)

and the corresponding probability density function of $X$ can be written as,

$f(x;h,\sigma,t):=\frac{h^{3}\log(t)}{\sqrt{2\pi\sigma^{2}}}\exp\left(-\frac{\left(x-\frac{1}{h}\right)^{2}h^{3}\log(t)}{2\sigma^{2}}\right).$
(37)

Since the parameter $t$ is a realization of the random variable $T$, it is
clear that $f(x;h,\sigma,t)$ is the conditioned distribution of $X$ given $T$.
Therefore, assuming that $T$ is independent, it is clear that the joint
probability density function $f_{X,Y}(x,y)$ of $(X,T)$ is given by

$f_{X,Y}(x,y)=f(x;h,\sigma,t)g(t,\theta).$ (38)

where the variables $h$, $\sigma^{2}$, and $\theta$ are parameters to be
estimated from a given sample.

Next we consider a sample set $\mathcal{S}$ of realizations of $(X,T)$ as
follows,

$\mathcal{S}:=\\{(x_{i},t_{i}):1\leq i\leq m\\},$ (39)

which will be used to estimate $h$, $\sigma^{2}$, and $\theta$ by means of the
maximum likelihood method.

Once we have stated the joint density function of $(X,T)$, the likelihood
function for the sample set $\mathcal{S}$ can be written as follows,

$L(h,\sigma^{2};\mathcal{S})=\prod_{i=1}^{m}f(x_{i};h,\sigma,t_{i})g(t_{i},\theta)\Delta
x.$ (40)

The log-likelihood function is therefore given by

$\displaystyle\log L$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{m}\bigg{[}-\frac{\left(x_{i}-\frac{1}{h}\right)^{2}h^{3}\log(t_{i})}{2\sigma^{2}}+\frac{1}{2}\log\left(\frac{h^{3}\log(t_{i})}{\sigma^{2}}\right)$
(41) $\displaystyle+$
$\displaystyle~\log\left(g(t_{i},\theta)\right)+\log\left(\frac{\Delta
x}{\sqrt{2\pi}}\right)\bigg{]}.$

After some calculations it is easy to see that the first derivative of $\log
L$ with respect to $h$, $\sigma^{2}$ and $\theta$ can be expressed as,

$\displaystyle\frac{\partial\log L}{\partial h}$ $\displaystyle=$
$\displaystyle-\frac{3}{2}\sum_{i=1}^{m}\frac{\left(x_{i}-\frac{1}{h}\right)^{2}h^{2}\log(t_{i})}{\sigma^{2}}-\sum_{i=1}^{m}\frac{\left(x_{i}-\frac{1}{h}\right)h\log(t_{i})}{\sigma^{2}}$
(42) $\displaystyle+$ $\displaystyle\frac{3}{2}\frac{m}{h},\quad$
$\displaystyle\frac{\partial\log L}{\partial\sigma^{2}}$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{m}\frac{\left(x_{i}-\frac{1}{h}\right)^{2}h^{3}\log(t_{i})}{2\sigma^{4}}-\frac{m}{2\sigma^{2}},$
(43) $\displaystyle\frac{\partial\log L}{\partial\theta}$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{m}\frac{\partial\log
g(t_{i};\theta)}{\partial\theta}.$ (44)

Now, to maximize the log-likelihood function we equate to zero the above
partial derivatives. Solving these equations will give us the maximum
likelihood estimations for every parameter. Then we obtain,

$\displaystyle\frac{m\sigma^{2}}{h}$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{m}\left(x_{i}-\frac{1}{h}\right)^{2}h^{2}\log(t_{i})+\frac{2}{3}\sum_{i=1}^{m}\left(x_{i}-\frac{1}{h}\right)h\log(t_{i}),$
$\displaystyle m\sigma^{2}$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{m}\left(x_{i}-\frac{1}{h}\right)^{2}h^{3}\log(t_{i}),\qquad$
(46) $\displaystyle 0$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{m}\frac{\partial\log
g(t_{i};\theta)}{\partial\theta}.\qquad$ (47)

Notice that the last equation involves uniquely the parameter $\theta$. This
means that the maximum likelihood estimation for $\theta$ can be obtained
separately from the problem of estimating $h$ and $\sigma^{2}$. This was
actually expected due to the fact that $T$ is an independent random variable
and therefore the parameters of the model can be estimated without the
information about the realizations $x_{i}$. The latter means that the
distribution of random duration of the time series can be estimated
independently of the estimations of any other property of the series, which in
the present case is entropy rate and the entropy production rate of the
process.

For the sake of clearness we introduce the following short-hand notations

$\displaystyle\hat{a}:=\frac{1}{m}\sum_{i=1}^{m}x_{i}\log(t_{i}),$ (48)
$\displaystyle\hat{b}:=\frac{1}{m}\sum_{i=1}^{m}\log(t_{i}),$ (49)
$\displaystyle\hat{\xi}:=\frac{1}{m}\sum_{i=1}^{m}\log(t_{i})\left(x_{i}-\frac{1}{h}\right)^{2}.$
(50)

In terms of $\hat{a}$, $\hat{b}$, and $\hat{\xi}$ we can rewrite Eqs.
(LABEL:eq:to-solve-1) and (46) as

$\displaystyle\frac{\sigma^{2}}{h}$ $\displaystyle=$ $\displaystyle
h^{2}\hat{\xi}+\frac{2}{3}\left(\hat{a}h-\hat{b}\right),$ (51)
$\displaystyle\sigma^{2}$ $\displaystyle=$ $\displaystyle
h^{3}\hat{\xi},\qquad$ (52)

Next, from Eq. (52) we see that $\hat{\xi}=\sigma^{2}/h^{3}$, which can be
substituted into Eq. (51) to obtain,

$\frac{\sigma^{2}}{h}=\frac{\sigma^{2}}{h}+\frac{2}{3}\left(\hat{a}h-\hat{b}\right),$
(53)

which implies that $\hat{a}h-\hat{b}=0$, thus obtaining the maximum likelihood
estimation $\hat{h}$ for the parameter $h$,

$\hat{h}:=\frac{\hat{b}}{\hat{a}}=\frac{\frac{1}{m}\sum_{i=1}^{m}\log(t_{i})}{\frac{1}{m}\sum_{i=1}^{m}x_{i}\log(t_{i})}.$
(54)

Recalling that $x_{i}$ is actually defined as $x_{i}=\ell_{i}/\log(t_{i})$, we
can see that $\hat{h}$ can alternatively written as

$\hat{h}=\frac{\frac{1}{m}\sum_{i=1}^{m}\log(t_{i})}{\frac{1}{m}\sum_{i=1}^{m}\ell_{i}},$
(55)

which was the formula anticipated in Eqs. (21) and (22).

Now, once we have an estimation for $h$, we can obtain an estimation for
$\sigma^{2}$ through the expression (52). In order to obtain a simplified
formula for this parameter, let us perform some calculation. Firs notice that
$\hat{\xi}$ defined in eq. (50) can be written as,

$\displaystyle\hat{\xi}$ $\displaystyle=$
$\displaystyle\frac{1}{m}\sum_{i=1}^{m}\log(t_{i})\left(x_{i}^{2}-2\frac{x_{i}}{h}+\frac{1}{h^{2}}\right)$
(56) $\displaystyle=$
$\displaystyle\frac{1}{m}\sum_{i=1}^{m}x_{i}^{2}\log(t_{i})-\frac{2}{h}\frac{1}{m}\sum_{i=1}^{m}x_{i}\log(t_{i})+\frac{1}{h^{2}}\frac{1}{m}\sum_{i=1}^{m}\log(t_{i})$
$\displaystyle=$
$\displaystyle\hat{c}-\frac{2\hat{a}^{2}}{h}+\frac{\hat{b}}{h^{2}},$

where we defined $\hat{c}$ as

$\displaystyle\hat{c}$ $\displaystyle:=$
$\displaystyle\frac{1}{m}\sum_{i=1}^{m}x_{i}^{2}\log(t_{i}).$ (57)

Thus, in terms of the sample functions $\hat{a}$, $\hat{b}$, and $\hat{c}$ we
have that an estimation for the parametes $\sigma^{2}$ can be written as

$\hat{\sigma}^{2}=\hat{h}^{3}\hat{c}-2\hat{h}^{2}\hat{a}+\hat{h}\hat{b}.$ (58)

We can simplify the above expression if we recall that
$\hat{h}=\hat{b}/\hat{a}$. Further calculations allows us to write

$\hat{\sigma}^{2}=\hat{h}^{2}\left(\hat{h}\hat{c}-\hat{a}\right).$ (59)

It is clear that the parameter $\sigma$ is associated to the statistical error
of $X$ and therefore it would be involved in the error associated to the
estimation of $h$. Actually we will not obtain the exact expression of the
standard deviation of the estimator $\hat{h}$ but we will made an
approximation that will allows us to known the error due to the finiteness of
the sample sequences $\\{t_{i}\\}$. Since $\sigma/\sqrt{h^{3}\log(t)}$ is an
approximation to the error of the inverse entropy rate, it is clear that

$\frac{1}{h}\approx\frac{1}{\hat{h}}\pm\frac{\hat{\sigma}}{\sqrt{\hat{h}^{3}\log(t)}}.$
(60)

Taking the inverse of the above equality we obtain

$h\approx\left(\frac{1}{\hat{h}}\pm\frac{\hat{\sigma}}{\sqrt{\hat{h}^{3}\log(t)}}\right)^{-1},$
(61)

thus, assuming that $\sigma/\sqrt{h^{3}\log(t)}$ is small, we can expand the
right-hand side of the above equation up to first order. Then we obtain

$h\approx\hat{h}\pm\frac{\hat{\sigma}}{\sqrt{\hat{h}\log(t)}}.$ (62)

Finally, since $t$ is a random variable we average the squared error over all
the sequence length of the sample, thus obtaining that

$\hat{\epsilon}^{2}=\frac{\hat{\sigma}^{2}}{{\hat{h}}}\frac{1}{m}\sum_{i=1}^{m}\frac{1}{{\log(t_{i})}}.$
(63)

This last expression allows to see that our estimator is consistent with the
central limit theorem in the sense that error vanishes if we made $t_{i}$ tend
to infinity. Otherwise, for finite $t$ the error remains finite no matter how
many samples we use, i.e., no matter how long is $m$. The latter, as we can
see, is a consequence of the fact that there is an intrinsic error due to the
finiteness of $t$.

## Data Availability

The data that supports the findings of this study are available within the
article.

## References

## References

  * Daw, Finney, and Kennel (2000) C. Daw, C. Finney, and M. Kennel, “Symbolic approach for measuring temporal “irreversibility”,” Physical Review E 62, 1912 (2000).
  * Latora and Baranger (1999) V. Latora and M. Baranger, “Kolmogorov-sinai entropy rate versus physical entropy,” Phys. Rev. Lett. 82, 520 (1999).
  * Porporato, Rigby, and Daly (2007) A. Porporato, J. R. Rigby, and E. Daly, “Irreversibility and fluctuation theorem in stationary time series,” Phys. Rev. Lett. 98, 094101 (2007).
  * Roldán and Parrondo (2012) E. Roldán and J. M. R. Parrondo, “Entropy production and kullback-leibler divergence between stationary trajectories of discrete systems,” Phys. Rev. E 85, 031129 (2012).
  * Gaspard (2004) P. Gaspard, “Time-reversed dynamical entropy and irreversibility in markovian random processes,” J. of Stat. Phys. 117, 599–615 (2004).
  * Costa, Goldberger, and Peng (2005) M. Costa, A. L. Goldberger, and C.-K. Peng, “Broken asymmetry of the human heartbeat: loss of time irreversibility in aging and disease,” Phys. Rev. Lett. 95, 198102 (2005).
  * Salgado-Garcia and Maldonado (2021) R. Salgado-Garcia and C. Maldonado, “Estimating entropy rate from censored symbolic time series: A test for time-irreversibility,” Chaos: An Interdisciplinary Journal of Nonlinear Science 31, 013131 (2021).
  * Provata, Nicolis, and Nicolis (2014) A. Provata, C. Nicolis, and G. Nicolis, “DNA viewed as an out-of-equilibrium structure,” Phys. Rev. E 89, 052105 (2014).
  * Flanagan and Lacasa (2016) R. Flanagan and L. Lacasa, “Irreversibility of financial time series: a graph-theoretical approach,” Phys. Lett. A 380, 1689–1697 (2016).
  * González-Espinoza, Martínez-Mekler, and Lacasa (2020) A. González-Espinoza, G. Martínez-Mekler, and L. Lacasa, “Arrow of time across five centuries of classical music,” Phys. Rev. Research 2, 033166 (2020).
  * Li and Graur (1991) W.-H. Li and D. Graur,  _Fundamentals of molecular evolution_ , 576.5 L5 (1991).
  * Kontoyiannis (1998) I. Kontoyiannis, “Asymptotic recurrence and waiting times for stationary processes,” J. Theor. Prob. 11, 795–811 (1998).
  * Chazottes and Redig (2005) J.-R. Chazottes and F. Redig, “Testing the irreversibility of a Gibbsian process via hitting and return times,” Nonlinearity 18, 2477–2489 (2005).
  * Chazottes and Ugalde (2005) J.-R. Chazottes and E. Ugalde, “Entropy estimation and fluctuations of hitting and recurrence times for Gibbsian sources,” Discrete Continuous Dynamical Systems Ser. B 5, 565–586 (2005).
  * Maldonado (2015) C. Maldonado, “Fluctuation bounds for entropy production estimators in gibbs measures,” J. of Phys. A: Math. and Theor. 48, 045003 (2015).
  * Maes (1999) C. Maes, “The fluctuation theorem as a Gibbs property,” J. Stat. Phys. 95, 367–392 (1999).
  * Jiang, Qian, and Qian (2004) D.-Q. Jiang, M. Qian, and M.-P. Qian,  _Mathematical Theory of Nonequilibrium Steady States_ (Springer, 2004).
  * Chakraborty and Chakravarty (2012) S. Chakraborty and D. Chakravarty, “Discrete gamma distributions: Properties and parameter estimations,” Communications in Statistics-Theory and Methods 41, 3301–3324 (2012).
  * Salgado-García and Ugalde (2016) R. Salgado-García and E. Ugalde, “Symbolic complexity for nucleotide sequences: a sign of the genome structure,” J. of Phys. A: Math. and Theor. 49, 445601 (2016).
  * Gorman and Greene (2008) J. Gorman and E. C. Greene, “Visualizing one-dimensional diffusion of proteins along dna,” Nature Structural & Molecular Biology 15, 768–774 (2008).
  * Mirny  _et al._ (2009) L. Mirny, M. Slutsky, Z. Wunderlich, A. Tafvizi, J. Leith, and A. Kosmrlj, “How a protein searches for its site on DNA: the mechanism of facilitated diffusion,” Journal of Physics A: Mathematical and Theoretical 42, 434013 (2009).
  * Salgado-García (2019) R. Salgado-García, “Noise-induced rectification in out-of-equilibrium structures,” Phys. Rev. E 99, 012128 (2019).
  * Cocho  _et al._ (2003) G. Cocho, A. Cruz, G. Martínez-Mekler, and R. Salgado-García, “Replication ratchets: polymer transport enhanced by complementarity,” Physica A: Statistical Mechanics and its Applications 327, 151–156 (2003).
  * Benson  _et al._ (1997) D. A. Benson, M. S. Boguski, D. J. Lipman, and J. Ostell, “Genbank,” Nucleic acids research 25, 1–6 (1997).