The long and short of templated copying

Jenny Poulton
Imperial College London
Department of Bioengineering
PhD Thesis

###### Abstract

Templated copying is the central operation by which biology produces complex
molecules. Cells copy sequence information from DNA to RNA and on into
proteins, which are the molecules responsible for the function and regulation
of cellular systems. In the templated copying process the template catalyses
the formation of a second molecule carrying the same sequence. Traditionally,
people have ignored the separation of the template and copy at the end of the
process, but separation is necessary and fundamentally changes the
thermodynamics of the process.

In general, creating an accurate polymer costs free energy. Omitting
separation, this cost can be compensated for by the extra free energy released
by “correct” copy/template bonds. Separation requires these bonds be broken,
so true copying requires an input of free energy. Equally the fact that
copy/template bonds are temporary means there is no thermodynamic bias towards
accuracy, instead copying relies only on kinetic effects to promote accuracy.
In general, transducing energy can only happen reversibly when the
transduction process is quasistatic and time varying; something that cannot be
true when you are relying on kinetic discrimination. Copying is a far from
equilibrium process. This thesis explores the consequences of this
observation. We start in the limit of infinite length copies where the costs
of accuracy represent hard thermodynamic bounds and then moves to the finite
length limit where these same limits can be understood as kinetic barriers. We
then discuss copying systems as non-equilibrium steady states, which can be
analysed as information engines moving free energy between out-of-equilibrium
baths.

††preprint: AIP/123-QED

## Copyright and originality statement

This work is the work of Jenny Poulton, and any parts which are adaptations of
other work are clearly referenced.

The copyright of this thesis rests with the author. Unless otherwise
indicated, its contents are licensed under a Creative Commons Attribution-Non
Commercial 4.0 International Licence (CC BY-NC). Under this licence, you may
copy and redistribute the material in any medium or format. You may also
create and distribute modified versions of the work. This is on the condition
that: you credit the author and do not use it, or any derivative works, for a
commercial purpose. When reusing or sharing this work, ensure you make the
licence terms clear to others by naming the licence and linking to the licence
text. Where a work has been adapted, you should indicate that the work has
been changed and describe those changes. Please seek permission from the
copyright holder for uses of this work that are not included in this licence
or permitted under UK Copyright Law.

###### Contents

  1. I Self assembly
  2. II Templating
  3. III Synthetic Copying Systems
  4. IV Theoretical Modelling of Chemical Systems
  5. V Solving Markov models
    1. V.1 Transition matrices
    2. V.2 Absorbing state matrices
    3. V.3 Rate matrices: for the continuous time case
    4. V.4 Embedding
    5. V.5 Monte Carlo simulations
    6. V.6 Using methods in combination
  6. VI Stochastic Thermodynamics
    1. VI.1 Thermodynamics of trajectories
    2. VI.2 Thermodynamic reversibility
    3. VI.3 Transition State Theory
  7. VII Information Theory
  8. VIII Thermodynamics of information
  9. IX Thermodynamics of copying
  10. X Aims and Outline
  11. I Models and Methods
    1. I.1 Model definition
    2. I.2 Model analysis
      1. I.2.1 Properties of the growing monomer
      2. I.2.2 Final state of the polymer
      3. I.2.3 Linking the growing polymer to the final sequence
      4. I.2.4 Corroboration with simulations
  12. II Results
    1. II.1 General thermodynamic bounds
    2. II.2 Behaviour of specific systems
  13. III Conclusion
  14. I Model of oligomerisation
  15. II Properties of general oligomerisation systems
  16. III An explicit model of dimerisation to explore the consequences of the general result in Eq. 2
  17. IV Finding the fluxes through the network using a transition matrix
  18. V Is there necessarily a thermodynamic cost to accuracy?
  19. VI Template copying as an inherently non-equilibrium information engine
  20. VII Kinetic convergence on thermodynamic constraints for infinite-length polymers
  21. VIII Model of copy production for oligomers of length $|s|>2$
  22. IX Conclusion
  23. I Fundamental constraints on a generic cyclical oligomerisation system
  24. II A specific model of dimerisation
  25. III A system in which the template and destroyer act independently between fixed monomer and oligomer baths.
  26. IV Template/destroyer system as a three-way information engine
  27. V General discussion of a system in which the concentration oligomers is not externally controlled
  28. VI Two explicit models for a template/destroyer system which maintains a steady-state of oligomers where $|{\bf s}|\geq 2$
  29. VII Conclusion
  30. I Kinetic Proofreading
  31. II Turn order in copying systems
  32. III Synthetic copying systems
  33. IV Conclusion

###### List of Figures

  1. 1 Image adapted from[1], here trapezium shaped capsomers first form triangular units, which then form into a complex spherical covering. The spherical covering is highly symmetric and identical capsomers can occupy multiple locations in the structure without changing its final shape. This allows the programming of the final shape of the spherical covering to be programmed into a very small number of subunits (here one).
  2. 2 DNA origami is often created from a single long scaffold which is pinned into place with staple molecules which are complementary to sections of the scaffold (matching colour ends). Notice that in this case, while we have folded the scaffold into this configuration using the available staples, there are many redundant configurations that this mixture of scaffold and staple could fold into. Imagine, for example if the yellow ends of the two middle staples were to swap, the scaffold would fold into an entirely different configurations. To design out redundancy, we would have to have many more than two unique staples. This is the core of the problem of targeting specific complex molecules with self assembly.
  3. 3 A generic copying process. Coarse grained steps on left show a monomer coming out of solution and attaching to the left-most free site on the template. The copy then grows on the template, site by site, with the tail detaching sequentially, before the final copy polymer detaches from the rightmost site on the template. Detailed sub-steps on right in which a monomer comes out of solution, is polymerised into the chain, and then the previous monomer detaches from the template. In this generic procedure we are enforcing an specific ordering of steps, which is a simple analogy for the process of translation and transcription. The mechanisms by which this state order is enforced is beyond the scope of the main part of this work, but will be discussed further in the conclusion.
  4. 4 The process of RNA translation is two-fold. First, enzymes called RNA synthetases catalyse the binding of tRNA codons of length three to their matching amino acid. RNA synthetases are specific, they are complementary to a specific pairing of tRNA and amino-acids[2]. This creates a pool of dimers of one codon and one amino acid. The tRNA/amino acid dimer then binds with the exposed mRNA and the amino acid is bound into the growing protein chain, catalysed nonspecifically by the ribosome. Thus the process of copying RNA into proteins involves both the copying of the information in a synthetase into a dimer, and the information in a long strand of mRNA into a protein.
  5. 5 a) A representative configuration of a given pair of neighbouring macrostates, the first a growing copy attached by its final monomer to the template, and the second, in which a monomer has come out of solution and bound to the next empty site on the template. b) Each of these macrostates contains many microstates, for example changes in the physical configuration of the trailing end. c) A hypothetical energy landscape for the transition, with a free-energy barrier between the two states, topped by the “activated complex” referred to in transition rate theory. A plausible trajectory through the system is shown; the system initially quickly moves between microstates within the left hand basin, before it gains enough energy due to the fluctuating environment to overcome the barrier, releasing the energy as it travels into the right hand stable basin, where it remains.
  6. 6 a) The transition matrix for the discrete state-space, discrete time network shown. Note that all quantities are probabilities, and all columns sum to one. The long time limit is the zeroth eigenvector of the matrix. b) The rate matrix for the discrete state-space, continuous time network shown. Note that all quantities are rates, and all columns sum to zero. The long time limit is again the zeroth eigenvector of the matrix. c) The absorbing state matrix for the discrete state-space, discrete time network shown. States C and E are absorbing states, and A, B and E are transient states. Thus ${\bf M}$ is $3\times 3$ and ${\bf A}$ is $2\times 3$.
  7. 7 The probability distributions (red) for system with a time varying free energy landscape (black). The system shown undergoes the following protocol; the system is switched to a single monostable well by lowering the free energy barrier, allowing the system to equilibrate, and then the barrier is raised. If this system is initialised with all trajectories in the left hand well, then after this protocol half will be in the right hand well and half will be in the left hand well. If we reverse this protocol starting with the trajectories in the mixed final state, we will not return to all trajectories being in the left hand well, but instead the trajectories would remain with half in the left and half in the right.
  8. 8 A venn diagram showing the additive relationship between various forms of entropy and mutual information. We can straightforwardly see from here that $I(X;Y)=H(X)-H(X|Y)=H(Y)-H(Y|X)$ or $I(X;Y)=H(X,Y)-H(X|Y)-H(Y|X)$ etc.
  9. 9 (a) The Maxwell’s demon paradox[3]. A box containing a uniform gas at equilibrium is partitioned down the middle. That partition contains a door which is operated by a “demon”. Without spending any energy, the demon can selectively allow particles to cross the partition. It allows fast particles from left to right, while blocking slow particles, and allows slow particles from right to left, while blocking fast particles. This results in the system now having a cold section and a hot section, which can hypothetically be used to do work. Thus the system has gone from an equilibrium state, to an out-of-equilibrium state without work being done on it, breaking the second law. (b) The solution to the Maxwell’s demon paradox is best understood by simplifying the system into a Szilard engine[4]. Here a box contains a single particle gas. A partition is dropped into the box, confining the particle to one side of the box. The particle then pushes the partition back to the edge of the box. If a weight is coupled to the partition, then the particle can lift the weight doing a maximum of $k_{B}T\ln{2}$ of work. This problem is identical to that posed by Maxwell, fluctuations in an equilibrium gas are being harnessed to do work, without a corresponding input of work. The key to exorcising the demon comes from the observation that in order to extract the work, the position of the pulley must be correlated with the position of the particle. If it was always attached on e.g. the right hand side, then half the time the particle would lift the weight, but half the time it would lower it, on average doing no work. The minimum cost of correlating two bits of information is $k_{B}T\ln{2}$, exactly the same as the maximum amount of work that can be extracted from the system. Thus the second law is restored and the demon is exorcised.
  10. 1 Free-energy landscapes for simple examples of (a) templated self-assembly, in which the monomers remain bound to the template during the copy process; and (b) persistent copying, in which the monomers detach from the template after they have been incorporated into the polymer. Both diagrams show the addition of three monomers to a growing polymer, driven by a chemical free energy of backbone polymerisation $\Delta G_{\rm pol}$. In each case, two scenarios are considered: the addition of two incorrect monomers, followed by a correct one (top), and the addition of three correct monomers (bottom). Local minima in the landscape represent macrostates following complete incorporation of monomers; intermediate configurations, illustrated schematically for the first transition, are part of the effective barriers. In templated self-assembly, the chemical free-energy cost of previously incorporated mismatches is retained as the daughter grows [5, 6, 7, 8, 9, 10]. Thus in (a), each mismatch in the daughter increases the chemical free energy by $\Delta G_{\rm D}$ relative to the perfect match. In persistent copying (b), the chemical free-energy penalty for incorporating wrong momomers is only temporary; it arises when the incorrect monomer is added to the growing polymer, but is lost when that monomer subsequently detaches from the template. As a result, the overall chemical free-energy change of creating an incorrect polymer is the same as that for a correct one. Analyzing the consequences of this constraint, which is a generic feature of copying but does not arise in templated self assembly, is the essence of this work. The figure also shows that in our specific model, incorporating a wrong monomer after a correct one tends to reduce the chemical free-energy drop to $\Delta G_{\rm pol}-\Delta G_{\rm TT}$, and incorporating a correct monomer after an incorrect one tends to increase it to $\Delta G_{\rm pol}+\Delta G_{\rm TT}$; however, adding a wrong monomer to a wrong one, and adding a correct monomer to a correct one, does not change the free-energy drop $\Delta G_{\rm pol}$.
  11. 2 Transitions of an arbitrary polymer $\&$. To relate the final chain to the growing chain it is useful to consider fluxes through interfaces in this transition diagram. Using the tip and combined probabilities, along with relative propensities it is possible to describe the fluxes through interfaces 4-7 in terms of properties of the growing chain. Equally by considering errors and conditional errors and taking fractions of the overall growth velocity, it is possible to find the fluxes through interfaces 4-7 in terms of properties of the final chain and growth velocity.
  12. 3 The transition diagram for the Markov process describing the sequence of monomers found by stepping forward along a completed chain.
  13. 4 Errors obtained from Gillespie simulations are indistinguishable from the analytical results obtained using Gaspard’s method for all three mechanisms.
  14. 5 Error probability $\epsilon$ as a function of $\Delta G_{\rm{\rm{pol}}}$ for all three mechanisms: (a) over a wide range of $\Delta G_{\rm pol}$, and (b) within the entropy-driven region $\Delta G_{\rm pol}\leq 0$. The temporary thermodynamic mechanism is always least accurate, and the combined mechanism most accurate. All mechanisms have no accuracy in the limit of $\Delta G_{\rm pol}\rightarrow-\ln{2}$, and are far from the fundamental bound on single-site accuracy imposed by $H_{\rm ss}=-\epsilon\ln\epsilon-(1-\epsilon)\ln(1-\epsilon)\geq-\Delta G_{\rm pol}$ except at $-\Delta G_{\rm pol}\approx 0$.
  15. 6 Efficiencies $\eta_{\rm ss}$ (a) and $\eta$ (b) plotted against $\Delta G_{\rm pol}$ show sharp peaks at $\Delta G_{\rm pol}=0$ as $\Delta G\rightarrow\infty$ in all three cases. In the combined case we see a second peak in $\eta$, and a shoulder in $\eta_{\rm ss}$ at $\Delta G_{\rm pol}=-0.48121$. (c) Both of these peaks in $\eta$ tend to unity at $\Delta G\rightarrow\infty$.
  16. 1 (a) A model for templated self-assembly on an infinitely long template. Here bonds retained between copy and template stabilise the low entropy state. (b) A model of persistent copying on an infinitely long template. Association of the first monomer, and complete dissociation of the complete polymer are omitted. (c) A model of persistent copying on a finite oligomer template, including complete dissociation of full length oligomers and single monomers into surrounding baths. In all cases the polymer/oligomer grows sequentially and sequence-specifically on the template, with the copy process oligomer separating from the template behind its leading edge [11].
  17. 2 (a) Templated self-assembly on an infinite length polymer and (b) templated copying with sequential separation. Growth on the template in all cases has a a constant slope with a gradient that depends on the average free-energy of adding an arbitrary monomer to the growing chain ($\Delta G_{\rm pol}$) and the accuracy. The entropy cost between a perfectly accurate copy and an unbiased copy is $\ln{2}$, increasing the gradient of the slope. In templated self-assembly, favourable copy/template bonds can overcome the cost of correlating copy and template, meaning that accuracy is favoured with a shallower gradient. For templated copying, accuracy increases the gradient due to the cost of correlating the two sequences without compensating interactions.
  18. 3 Minimal model of oligomerisation illustrated with dimers. A template (squares) interacts with baths of monomers and dimers of a second species (circles). Monomers can bind to the template and dimerize, while dimers binding to the template can be destroyed or interconverted. The standard dimerisation free-energy is $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{\rm dim}$ for all sequences, but the free-energy change of matching and non-matching varieties of monomer coming out of solution and binding to the template at the reference concentration are $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}-[1]$ and $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}-[2]$, respectively, allowing selectivity. Rate constants $k_{\rm cat}$ and $k_{\rm on}$ define the dynamics.
  19. 4 A dimer copying model shows finite accuracy in the equilibrium limit. (a) The net production rate $J({\rm s})$ against chemical driving $-\Delta G_{\rm chem}$ for each of the four dimers 1,1, 1,2, 2,1 and 2,2 relative to a template of sequence 1,1, with $\Delta G_{\rm disc}=\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}-\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}=10,\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}+\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}=0$ and unbiased monomer and oligomer baths. All $J({\rm s})$ pass through zero at the equilibrium point $\Delta G_{\rm chem}=0$ but quickly separate when $\Delta G_{\rm chem}\neq 0$ (inset). $J({1,2})$ and $J({2,1})$ overlap. At larger driving, accurate copies are preferentially produced. (b) Error fraction at which incorrect type 2 monomers are incorporated into dimers, $\epsilon$, against $\Delta G_{\rm chem}$ for various $\Delta G_{\rm disc}$ and $\Delta G_{r}+\Delta G_{w}=0$. The system has finite accuracy, $\epsilon\neq 0.5$, as $\Delta G_{\rm chem}\rightarrow 0^{-}$, with the accuracy dependent on $\Delta G_{\rm disc}$.
  20. 5 The dimer copying model can show perfect accuracy in the equilibrium limit. (a) We plot the net production rate $J_{\rm s}$ against chemical driving $-\Delta G_{\rm chem}$ for each of the four dimers 1,1, 1,2, 2,1 and 2,2 relative to a template of sequence 1,1 and with $\Delta G_{\rm disc}=\Delta G_{w}^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}-\Delta G_{r}^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}=2$, but $\frac{\Delta G_{r}^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}+\Delta G_{w}^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}}{2}=5$. All oligomers have the same (fixed) concentration $[S_{\rm tot}]/4$, and all monomers have the same (fixed) concentration $[M_{\rm tot}]/2$. All $J_{\rm s}$ pass through zero at the equilibrium point $\Delta G_{\rm chem}=0$ but separate quickly (inset) at non-zero driving. Here accurate copies are preferentially produced but all fluxes are very low. (b) Error fraction at which incorrect type 2 monomers are incorporated into dimers, $\epsilon$, against $-\Delta G_{\rm chem}$ for various $\Delta G_{\rm disc}$ with $\frac{\Delta G_{r}^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}+\Delta G_{w}^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}}{2}=5$. The system tends towards perfect accuracy, $\epsilon\rightarrow 0$, as $\Delta G_{\rm disc}\rightarrow\infty$, even as the system tends to the equilibrium point $\Delta G_{\rm chem}\rightarrow 0^{-}$.
  21. 6 Oligomer copying as an information engine with no equilibrium point. (a) Total flux $J_{\rm{tot}}$ against driving $-\Delta G_{\rm chem}$ for a range of discrimination free energies $\Delta G_{\rm disc}=\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}-\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}$, with $[1]=[2]=0.1$, $[1,1]=[1,2]=[2,1]=0.001$ and $[2,2]=0.1$. $\Delta G_{\rm disc}$ is varied with $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}+\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}=0$ fixed. The point $J_{\rm tot}=0$ at which there is no net dimerisation varies within the allowed white range despite the fact that the overall dimerisation free-energy is independent of $-\Delta G_{\rm disc}$. Specificity for ${\bf s}_{\rm min}=1,1$ makes growth easier and pushes $J_{\rm tot}=0$ to the lower limit, and specificity for ${\bf s}_{\rm max}=2,2$ has the opposite effect. (b) Phase plot of the information engine. Here the leftmost purple boundary is the transition from $J_{\rm tot}$ negative to positive. Here we fix $\Delta G_{\rm disc}=5$, $[S_{\rm tot}]=1$, $[M_{\rm tot}]=1$, $t(s)=0.25$ for all $s$ and vary $\Delta G_{\rm chem}$. We further vary $p(s)=0.25+d,0.25,0.25,0.25-d$ by varying $d$. There is a regime in which chemical work is used to specifically produce sequences of high free-energy and a regime in which specific production of low free-energy sequences is used to drive oligomerisation against a chemical load.
  22. 7 free-energy profiles for (a) a templated copying process on an infinite lengths template and (b) a finite length template. The gradient of the free-energy profile during the bulk of the copying process is set by the free-energy of dimerisation, the chemical driving due to the relative concentrations of the monomer and oligomer baths and the accuracy of the oligomer; an accurate oligomer required an extra $\ln{2}$ to add a monomer in the bulk more than a unbiased oligomer. However in the case of a finite length template, the initiation and termination steps can provide a theoretically unlimited adjustment to the overall free-energy of the process. It is intuitive that the initial rise and final drop off represent the entropy reduction due to isolating a free monomer on the template and the entropy increase due to releasing a polymer into solution. What is less intuitive is how these two steps depend on the composition of the monomer and polymer baths. In some cases creating the most accurate oligomer can be thermodynamically favourable compared to other oligomers.
  23. 8 Thermodynamic restrictions in an infinite-length model become kinetic restrictions for longer oligomers. (a) Net flux per unit empty template $\tilde{J}_{tot}$ of oligomer production and (b) the net fraction of error creation for a range of lengths $|{\bf s}|$. We vary $\Delta G_{\rm dim}^{-\kern-4.0pt{\ominus}\kern-4.0pt-}$, with $[M_{\rm tot}]=1$, $[S_{\rm tot}]$ chosen to give $\Delta G_{\rm chem}=0$ at $\Delta G_{\rm dim}^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}=-5$, $p(s)=t(s)$ unbiased and $\Delta G_{\rm disc}=8$ at $\Delta G_{r}+\Delta G_{w}=0$. Also plotted in (d) is the thermodynamic constraint on accuracy for an infinite length polymer, set by requiring a non-positive slope of the free-energy profile, $-\epsilon\ln{\epsilon}-(1-\epsilon)\ln{(1-\epsilon)}\geq\Delta G_{\rm dim}^{-\kern-4.0pt{\ominus}\kern-4.0pt-}+\ln 2$, and the actual error rate obtained for an infinite-length copy for these parameters [11]. Short oligomers overcome kinetic barriers to produce copies with $\Delta G_{\rm dim}^{-\kern-4.0pt{\ominus}\kern-4.0pt-}>0$ and $\epsilon$ below the infinite-length limit and thermodynamic constraint. At each length the simulation is averaged over $10,000,000$ trajectories.
  24. 1 free-energy profiles for an infinite length polymer (a) and a finite length polymer (b). The free-energy required for a polymer to grow on a template (the gradient) is dependent on the free-energy change for the extension of the polymer by one, and the accuracy of the polymer (an entropic consideration). The gradient is steeper for a more accurate polymer. However, depending on the composition of the surrounding monomer and polymer baths, the process of final detachment can provide a theoretically unlimited adjustment to the overall free-energy of the process. Therefore, it is possible for a finite length polymer to be produced, through leveraging thermal fluctuation, at a free-energy per monomer below that of the minimum required for a infinite length copying process to progress.
  25. 2 Our system now has two pathways for creating an oligomer, a template pathway (circles) and a destroyer pathway (hexagons). The free-energy required to create a polymer via the template path is $\Delta G^{|{\bf s}|}_{\rm dim}$ and via the destroyer pathway $\Delta G^{|{\bf s}|}_{\rm dim}-\Delta G^{|{\bf s}|}_{\rm fuel}$. Therefore, the system can exhibit cycles, with oligomers on average created by the template pathway, and destroyed by the destroyer pathway when $\Delta G^{|{\bf s}|}_{\rm fuel}<0$.
  26. 3 A specific example of a minimal model of the mechanism of the destroyer type catalysts which are coupled to fuel. The destroyer (squares) interacts with baths of monomers and dimers of a second species (circles). Monomers can bind to the template and dimerize, while dimers binding to the template can be destroyed or inter-converted. The standard dimerisation free-energy is $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{\rm dim}$ for all sequences, but the free-energy of matching and non-matching varieties of monomer coming out of solution and binding to the template are $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}+[1]$ and $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}+[2]$, respectively, allowing selectivity. $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}$ and $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}$ are the free energies of forming a copy template bond that is a match or mismatch respectively, measured at the reference concentration. Rate constants $k_{\rm cat}$ and $k_{\rm on}$ set the speed of the dynamics. The destroyer has identical dynamics to the template, as described in ch. 2 except the final step in which a dimer is forced off the template is coupled to the consumption of a fuel molecule which causes a free-energy change of $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{\rm fuel}$ at the reference concentration.
  27. 4 (a) The four dimer production fluxes from a template (solid lines) and a destroyer (dashed lines). (b) The net flux of dimer creation through the template and destroyer for each of the four dimer types. Here $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{\rm fuel}=5$ and $\Delta G_{\rm diff}=2$. The monomer and polymer baths, $p({\bf s})$ and $t({\bf s})$, are unbiased and $M_{\rm tot}=P_{\rm tot}=1$. Alongside the fact that $|{\bf s}|=2$ this mean that $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{\rm dim}=\Delta G^{|{\bf s}|}_{\rm dim}=\Delta G_{\rm chem}$ and $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{\rm fuel}=\Delta G^{|{\bf s}|}_{\rm fuel}$. Because the monomer and polymer baths are unbiased, individually all the fluxes pass through zero at the same point, $\Delta G_{\rm dim}=0$ for the template and $\Delta G_{\rm dim}=5$ for the destroyer. However, when summing the fluxes, $J(\textbf{s})=J^{T}(\textbf{s})+J^{D}(\textbf{s})$, there is a region (shaded grey) in which $J([1,1])>0$ and $J([2,2])<J([1,2])=J([2,1])<0$, where the system net produces the most accurate dimer while destroying all other dimers.
  28. 5 The system is capable of acting as a three-way engine interconverting information, the oligomerisation free-energy $\Delta G^{|{\bf s}|}_{\rm dim}$ and the cycle free-energy $\Delta G^{|{\bf s}|}_{\rm fuel}$. Here the monomer bath is unbiased and $M_{\rm tot}=1$, and $\Delta G_{\rm diff}=5$ for both the template and destroyer. In (a) we fix $\Delta G_{\rm fuel}=2$ and vary $\Delta G_{\rm dim}$ and $d$ (refer to main text) which varies $\Delta G_{\rm inf}$. Here $\alpha<0$ throughout, but $\beta$ and $\gamma$ become positive at various points. In (b) we fix $\Delta G_{\rm dim}=2$ and vary $\Delta G_{\rm fuel}$ and $d$. Here we see that at any given point, no more than two of $\alpha$, $\beta$ and $\gamma$ are positive. In (c) we fix $d=0.249$ and vary $\Delta G_{\rm dim}$ and $\Delta G_{\rm fuel}$. Again, we see that at any given point, no more than two of $\alpha$, $\beta$ and $\gamma$ are positive.
  29. 6 Left, a coarse-grained model in which the polymer grows sequentially, left to right with the tail separating as it grows. Right, an explicit set of sub-steps is required to calculate the time taken for an interaction between template/destroyer and a growing oligomer to resolve. One simple configuration of sub-steps is defined here and used in this work. In these steps, a monomer comes out of solution and binds to the template, the monomer is then polymerised into the growing chain and then the previous final monomer detaches from the chain.
  30. 7 We plot the error and the entropy change for systems of length 2, 3, 4 and 5, for three different ratios of $k^{T}_{\rm on}:k^{D}_{\rm on}$. Here there is off-rate discrimination alone via the template, $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}=8$, $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}=-8$, $\Delta G_{\rm diff}=16$ and $k^{1}_{\rm on}=k^{2}_{\rm on}=k_{on}$ for the template. The destroyer has no specificity $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}=\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}=\Delta G_{\rm diff}=0$. $\Delta G_{\rm dim}=0$ throughout. The system is highly stochastic unless the destroyer is slowed considerably compared to the template. While the system fails to beat $\Delta S<\Delta G^{|{\bf s}|}_{\rm fuel}$, we do observe that when $k_{\rm on}^{T}=10,000,000k_{\rm on}^{T}$ that the error for $|{\bf s}|=4$ and $|{\bf s}|=5$ is the same; a signature that could imply the conjectured constraint would be beaten at longer lengths.
  31. 8  We plot the error and the entropy change for systems of length 2, 3, 4 and 5, for three different ratios of $k^{T}_{\rm on}:k^{D}_{\rm on}$. The system contains a single template and destroyer. The template has moderate specificity using on- and off-rate discrimination, $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}=3$, $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}=-3$, $\Delta G_{\rm diff}=6$, $k^{1}_{\rm on}=e^{\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}}$ and $k^{1}_{\rm on}=e^{\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}}$. The destroyer has no specificity $\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{r}=\Delta G^{-\kern-4.0pt{\ominus}\kern-4.0pt-}_{w}=\Delta G_{\rm diff}=0$. $\Delta G_{\rm dim}=0$ throughout.While the system fails to beat $\Delta S<\Delta G^{|{\bf s}|}_{\rm fuel}$, we do observe that in all three ratios of $k_{\rm on}^{T}:k_{\rm on}^{T}$ that the error improves as $|{\bf s}|$ increase. This is a signature that could imply the conjecture constraint would be beaten at longer lengths.
  32. 1 On the left we see a copying process without proofreading. A monomer comes out of solution and binds to the template, is polymerised into the growing chain and the previous final monomer detaches. Accuracy is generated because correct monomers are more likely to stay bound to the template for long enough to be polymerised into the copy polymer. On the right we see a copying process with proofreading. A monomer comes out of solution, is activated, is polymerised into the growing chain and the previous final monomer detaches. Monomers now have two opportunities to fall off, before and after activation. As long as the activation step is driven; either by the process of activation releasing free-energy into the surroundings, or by a higher concentration of inactive than active monomers. The system is now capable of cycling, meaning the system is no longer tightly coupled. This means that the amount of free-energy it takes to generate the polymer no longer exactly predicts the state of the polymer, unlike a system without futile cycles.
  33. 2 Drawing out the full state tree we see that in the non-proofreading case there are two paths forward: adding a correct or incorrect match, and only one paths backwards: removing the final monomer. Thus the polymer can grow from purely entropic driving. Adding kinetic proofreading adds extra paths backwards, reducing the capacity for entropic driving.
  34. 3 The models used throughout this thesis have enforced well defined turn order. An empty template exists in a bath of monomers. A monomer comes out of solution and attaches at the leftmost site on the template. Monomers come out of solution, join the leftmost available site and are polymerised into the chain. The previous final monomer must separate from the template before the next monomer can come out of solution. The polymer grows to full length and can only fall off into solution from the final position on the template.
  35. 4 A selection of plausible copy/template configurations if strict turn order is relaxed. Products can remain bound to the template (product inhibition) or small fragmentary polymers can be formed. Avoiding these requires careful engineering of the copy/template interactions.
  36. 5 A proposed solution to the problem of product inhibition. If the process of forming a backbone bond between consecutive copy monomer weakens the copy template bond at the penultimate site, then it helps force the copy polymer off the template behind the tip, while keeping one strong copy/template bond at the tip.
  37. 6 a) Toehold mediated strand displacement. A substrate strand $S$ contains a sequence $\beta$ (blue) is bound to a complementary strand $A$ containing $\beta^{*}$ (blue). A toehold sequence $\gamma$ (pink) is left uncovered by $A$. An invader strand $I$ containing both $\beta^{*}$ and $\gamma^{*}$ attaches to the toehold and then slowly displaces the initial strand $A$. This process releases free energy due to the bonds between $I$ and $S$ in the toehold section. b) Handhold mediated strand displacement. This reaction allows the invader strand $I$ to interact with both the substrate strand $S$ and the initially bound strand $A$. The substrate strand $S$ contains both the sequence $\beta$ (blue) and the toehold $\gamma$ (pink). The initially bound strand contains $\beta^{*}$ and also a handhold sequence $\omega$ (green). The invader strand initially binds to the handhold on $A$ via the sequence $\omega^{*}$ which brings the sequence $\beta^{*}$ into proximity to the toehold on the substrate. The strand displacement reaction occurs and the handhold detaches. If we label $A$ the template, $S$ the final monomer attached to the template and $I$ the new monomer, we can see that this reaction performs templated copying.

## Introduction

##  I Self assembly

Biological systems rely on complex processes requiring complex molecules,
which must be assembled from available materials without external
manipulation[12]. An example of a complex biological molecule is a virus
capsid; a protective protein covering which surrounds viral genetic material.
As illustrated schematically in fig. 1, this protective covering is highly
symmetric and is made up of a large number of similar subunits called
capsomers. Due to local interactions between capsomers the capsid is able to
self assemble. This means that the capsid structure is the ground state of the
system, and favourable bonds between capsomers causes them to assemble
themselves into a specific structure. This process is completely autonomous;
it requires no external manipulation and is encoded into the capsomers
themselves[13].

Figure 1: Image adapted from[1], here trapezium shaped capsomers first form
triangular units, which then form into a complex spherical covering. The
spherical covering is highly symmetric and identical capsomers can occupy
multiple locations in the structure without changing its final shape. This
allows the programming of the final shape of the spherical covering to be
programmed into a very small number of subunits (here one).

One reason that capsids are well suited to self assembly is that they are
highly symmetric, and identical capsomers can occupy multiple different
locations within the capsid structure without changing the structure of the
capsid. These properties are not ubiquitous to complex biological molecules.

Another group of complex molecules is proteins, which have diverse roles in
the body including acting as enzymes, antibodies, message carriers, providing
structure, transport and storage. Capsomers are themselves complex proteins.
Proteins are made up of subunits called amino-acids, which come in a library
of 20 distinct types[14]. Proteins have very complex and precisely defined
folded structures, lacking symmetry, which allow the protein to perform its
complex functions. As an example, many enzymes form shapes which have spaces
which closely fit a specific substrate. These active sites can be used to
bring reactant molecules close to each other and encourage their
transformation into useful product molecules[14]. Even very small differences
in the composition of a protein can cause it to misfold, and this can have
serious consequences for organisms. In the human body misfolding is
responsible for diseases such as Alzheimer’s, Parkinson’s and type 2
diabetes[15].

Figure 2: DNA origami is often created from a single long scaffold which is
pinned into place with staple molecules which are complementary to sections of
the scaffold (matching colour ends). Notice that in this case, while we have
folded the scaffold into this configuration using the available staples, there
are many redundant configurations that this mixture of scaffold and staple
could fold into. Imagine, for example if the yellow ends of the two middle
staples were to swap, the scaffold would fold into an entirely different
configurations. To design out redundancy, we would have to have many more than
two unique staples. This is the core of the problem of targeting specific
complex molecules with self assembly.

In order to understand the difficulty in self assembling complex molecules
without symmetry we will first consider synthetic self assembly mechanisms. It
is possible to design synthetic self assembly mechanisms to target complex
structures. In these systems, a pool of subunits is allowed to combine and in
doing so relax into a ground state which has been pre-designed through
specific interactions between the subunits[16, 17, 18]. An example of this is
DNA origami[19, 20, 21], in which a long single strand of DNA is folded into a
specific topology by specially designed “staples” which have favourable bonds
with two different regions of the long strand as shown as fig. 2. It should be
noted here that in order to assemble one specific structure accurately, a
large library of subunits is required. For a structure with no symmetry, then
you need $n$ unique molecules for a $n$-particle assembly. This is in order to
prevent competing ground states–and therefore configurations–the system might
relax into different from the one being targeted[22, 12]. In structures that
have high symmetry, like the virus capsid, it is possible to reduce the number
of subunits, but it is not the case for most biomolecules. To illustrate the
issue of competing ground states; the process showed in fig. 2 has multiple
ground states, and so would not reliably form the molecule shown. Thus, every
target molecule would require a different and distinct set of pre-designed
subunits which themselves would have to be created. Given there are around
50,000 distinct functional proteins, often made up of hundreds of amino-
acids[15], predesigning interactions between subunits is clearly not an
optimal strategy for living organisms in many situations.

##  II Templating

Biology manages to solve the problem of targeting these complex structures
using a library of only 20 amino-acids. The assembly method now relies on
interactions between amino-acids and a template, here mRNA. The mRNA sequence
encodes the order in which amino-acids will be arranged in the final protein
chain. Favourable interactions between amino-acids and RNA codons allow a
specific polymer to be formed. The sequence of the polymer is therefore
determined via the favourable interaction with the template, and the amino-
acids themselves are only necessary to encode the folding of the polymer into
its functional form given the sequence. This requires fewer components than
encoding both the sequence and the folding. Given there are $20^{100}\approx
1\times 10^{130}$ ways to combine a library of 20 amino-acids into a chain of
length 100, the use of templates to target one of only 50,000 functional
protein sequences, which then fold into complex molecules, is clearly vital to
the functioning of the body[15].

We therefore argue that templated copying is central to the development of
biological complexity; there is no other way for an organism to realistically
manufacture the variety of complex molecules required for it to function.
Unsurprisingly, therefore, organisms have developed sophisticated machinery to
perform templated copying.

To aid understanding we outline a generic template copying scheme as shown in
fig. 3, which is a rough analogy to the processes of RNA transcription and
translation. On the left side we see a monomer come out of solution and attach
to the first site of a template. The copy then grows on the template, site by
site, with the tail detaching sequentially before the final copy polymer
detaches at the final step. On the right we zoom into the details of a single
step, in which a monomer comes out of solution, is polymerised into the chain,
and then the previous monomer detaches from the template. In this generic
procedure we are enforcing an specific ordering of steps, which is a simple
analogy for the process of RNA translation and transcription. The mechanisms
by which this state order is enforced is beyond the scope of the main part of
this work, but will be discussed further in the conclusion.

Figure 3: A generic copying process. Coarse grained steps on left show a
monomer coming out of solution and attaching to the left-most free site on the
template. The copy then grows on the template, site by site, with the tail
detaching sequentially, before the final copy polymer detaches from the
rightmost site on the template. Detailed sub-steps on right in which a monomer
comes out of solution, is polymerised into the chain, and then the previous
monomer detaches from the template. In this generic procedure we are enforcing
an specific ordering of steps, which is a simple analogy for the process of
translation and transcription. The mechanisms by which this state order is
enforced is beyond the scope of the main part of this work, but will be
discussed further in the conclusion.

The central dogma of molecular biology describes the way information is
transferred via templating from DNA via RNA to proteins. DNA is able to self
replicate by using itself as a template. DNA forms a double stranded
structure, with two long strands made of DNA nucleotides spiralling around
each other in a shape known as the double helix. There are four DNA
nucleotides; adenine, cytosine, guanine and thymine. These nucleotides have
complementary interactions: adenine and thymine form one attractive base pair,
and cytosine and guanine form the other. The two strands of DNA are therefore
complementary to each other, but encoding the same information[14].

These complementary interactions are central to DNA’s ability to be used as a
template. The first step of the process is to separate the two strands of DNA
which is accomplished by a helicase enzyme. DNA nucleotides then come out of
solution and bind with the exposed strand which acts as a template. The
stability of that interaction dictates whether the nucleotide is incorporated
permanently into the new strand via the formation of a backbone bond, or
whether the nucleotide falls back off the exposed strand into solution
first[23]. Thus the more stable complementary nucleotide is more likely to be
incorporated than a mismatch. This, in essence, is the source of all accuracy
in templated copying.

RNA transcription is a very similar mechanism to DNA self replication except
that RNA is a single stranded molecule. RNA also has four bases, but here
uracil takes the place of thymine. Here, once again, the two strands of DNA
must be separated, this time by a polymerase enzyme. RNA nucleotides then come
out of solution, bind to the exposed DNA strand, are polymerised into the
chain and then separate from the DNA strand. Only active regions of the DNA
are copied into RNA, and active regions code for many types of RNA. A
proportion of RNA strands are functional complex molecules. These include the
ribosome, and other structures which support RNA translation, as well as
catalytic RNA which has specific enzymatic roles in an organism[14].

RNA translation is the process by which the sequence of messenger RNA is then
copied into proteins via the ribosome in a process called translation. Thus
the structure of proteins is directly coded for by short sections of the DNA
sequence.

Translation is slightly more complex as it is necessary to go from an alphabet
of four RNA nucleotides, to one of 20 amino-acids. Instead of a one-to-one
mapping as in DNA self-replication and RNA transcription, it takes an RNA
codon of length three to encode for a single amino acid. As shown in fig. 4,
molecules of tRNA which are complementary to the three bases of the exposed
mRNA codon, act as intermediaries, binding first to the amino-acid
corresponding to the exposed mRNA codon using a synthetase enzyme[2], and then
to the exposed mRNA codon. This brings the amino-acid alongside the growing
protein chain, where it is polymerised into the chain.

It should be noted that the creation of a hybrid tRNA/amino acid is itself a
form of templating. The synthetase enzyme is specific and corresponds to a
specific pairing of amino acid and tRNA.

Figure 4: The process of RNA translation is two-fold. First, enzymes called
RNA synthetases catalyse the binding of tRNA codons of length three to their
matching amino acid. RNA synthetases are specific, they are complementary to a
specific pairing of tRNA and amino-acids[2]. This creates a pool of dimers of
one codon and one amino acid. The tRNA/amino acid dimer then binds with the
exposed mRNA and the amino acid is bound into the growing protein chain,
catalysed nonspecifically by the ribosome. Thus the process of copying RNA
into proteins involves both the copying of the information in a synthetase
into a dimer, and the information in a long strand of mRNA into a protein.

While stabilising interactions between copy and template are central to the
generation of accuracy in templated copying, it is important to note that
templated copying is only an effective solution to the issue of creating
complex molecules if the template can be recycled[24]. Without this, we would
be faced with the prospect of predesigning the template for every assembly, a
challenge no easier to solve than designing the target molecule directly. The
DNA sequence acts as a stable store of information, which can be readily
accessed by the cell, and which does not have to be created from scratch for
every process. Indeed the evolution of DNA genes through successive
generations of an organism is itself a form of templating, albeit one with
extra steps which cause variation.

Beyond this, in the vast majority of cases the purpose of templated copying is
to create a second functional molecule which can act separately to its
template. In the case of DNA self replication, the two DNA strands end up in
different cells, and mRNA and proteins are required to be able to function
separately from their template within and without the cell.

This separation of copy and template can happen as part of the process copy
creation in the case of RNA transcription and translation, leaving the copy
and template separated as a by-product of creating the copy. Alternatively, in
the case of DNA, the copying process involves separating the two strands of
DNA in the helix (which represent copy and template from the previous round of
copying) which are then used as templates to create another double stranded
copy/template double helix system. Separation is therefore vital and the very
interactions between complementary molecules which promote accuracy, now make
it difficult for copy and template to separate[25]. Understanding this is key
to understanding templated copying.

##  III Synthetic Copying Systems

The ability to design a synthetic templated copying system, without leveraging
existing biological machinery, is an essential step in humanity’s
understanding of the development of biological complexity. However, success in
building synthetic self replicators has been very limited.

The earliest work focused on the accumulation of monomers on a template and
subsequent polymerisation, but did not demonstrate subsequent separation of
copy and template[26, 27]. While this does not fulfil our requirements for a
templated copying system, instead achieving templated self-assembly, it does
have some interesting uses. Templated self-assembly has been explored as a
computational paradigm[28], with information being propagated between layers
of a polymer in predetermined ways to generate patterns or perform functions.
Erik Winfree is perhaps the best-known name in this field, demonstrating the
production of self-assembled and algorithmically patterned two-dimensional
lattices of DNA in his PhD thesis[29], and expanding into other complex
algorithms using the same technique[30].

A system by which monomers settle on a template, polymerise and then
dissociate from the template completely autonomously, i.e. without external
manipulation, has been demonstrated by G. von Kiedrowski’s group in their 1994
paper “Self-replication of complementary nucleotide-based oligomers”[31]. In
this system, monomers comprising of three nucleotides bind on a six-nucleotide
dimer template, polymerise into a six-nucleotide dimer, and then dissociate.
They demonstrate accumulation of product above that caused by the spontaneous
binding of the two three-nucleotide monomers into a six-nucleotide dimer in
the absence of a template. They acknowledge that product inhibition due to
complementarity between copy and template allows at best parabolic growth of
dimer concentration. Product inhibition due to cooperativity grows with
length[32, 33, 34] because the entropy of a single long monomer in solution is
much lower than multiple short molecules in solution. It is therefore telling
that this paper only succeeds in the accumulation of dimers, and above this,
only in a system where the monomers are very short strands of nucleotides.

In order to achieve accumulation of products other than dimers, people have
resorted to various novel methods to enforce separation. One method has been
used by the group of S. Otto in his paper “Exponential self-replication
enabled through a fibre elongation/breakage mechanism”[34]. Here the
replicating unit is a hexamer, which accumulates into long strands, with the
end of the strand acting as a template for the next hexamer. Through constant
agitation of the mixture, these strands break, causing an exponential growth
of free ends and therefore templates. A similar method has been used by R.
Schulman’s group[35] in which water currents are use to break apart crystals
which build up information in layers.

Other systems require several different reaction environments in order to
perform one copying iteration of template interaction, polymerisation and
separation. Various groups have achieved the copying of dimers and
trimers[36], heptamers[37], and even 24mers[38]. In “Chemical self-replication
of palindromic duplex DNA”[38], Li and Nicolaou use a double strand of DNA as
a template, with DNA fragments as the copy monomers. At low pH these DNA
fragments settle onto the copy, a chemical reagent is added in order to bring
about the formation of backbone bond, and then the pH is raised in order to
force the single strand of DNA off the double strand template. The pH is then
lowered and the single strand copy accumulates complementary molecules. The
chemical reagent is added for a second time to encourage backbone bond
formation and the system now has two double stranded templates. This process
requires three different environments (two of which are used twice) arranged
into a sequence of five steps, with external manipulation required to move
between each environment.

A cellular system does not have the luxury of moving between many different
environments in order to enforce separation. Therefore, if the aim is to
understand the process which underlies the development of biological
complexity, some form of autonomy in the process seems essential. After all,
as previously stated, biological systems rely on complex processes requiring
complex molecules which must self-assemble in an autonomous manner. In order
to understand the development of the complex machinery required for templated
copying in a biological setting, it seems essential to probe the design of
autonomous synthetic copying mechanisms.

One particularly compelling piece of work is that by Dieter Braun’s group[39].
Here they use a spatially non-uniform environment, namely a capillary which
has a temperature gradient over it. This sets up a convection current, where
monomers settle on a polymer in the cool region, polymerise and then separate
in the hot region. They have successfully demonstrated the accumulation of
polymers of length 143[40], an extremely impressive achievement. Despite still
requiring two different environments (hot and cold), because a thermal vent is
a plausible candidate for an origin of life scenario[33], this work is of
interest to those who wish to probe early life-like systems. It should however
be noted that the accumulated polymers are random, and do not reflect the
sequence of the seeded polymer.

It is therefore clear, given our failure to achieve synthetic copiers, that we
do not fully understand the process of templated copying. This thesis is
inspired by an observation that none of the theoretical literature explicitly
considers the separation of copy and template, in many cases actively omitting
it[5, 6, 7, 8, 9, 10, 41, 42]. Given the issue of product inhibition is well-
known by those performing experiments in the field, it seems critical that
this aspect of copying is included in theoretical models of copying. Previous
theoretical works omitting separation have successfully highlighted a range of
phenomena, such as entropy-driven growth [5, 10] and the possibility of using
kinetic proofreading [23, 43] to enhance accuracy [5, 9]. However the failure
to consider separation as part of the copying process has led to the erroneous
suggestion that there is a so-called “energetic limit”[8] in which accuracy
can be generated due to permanent thermodynamic stabilisation of complementary
monomers in the copy and template. As soon as separation is explicitly
considered, it is clear that these stabilising forces must be temporary. My
thesis will explore this in more detail.

##  IV Theoretical Modelling of Chemical Systems

Figure 5: a) A representative configuration of a given pair of neighbouring
macrostates, the first a growing copy attached by its final monomer to the
template, and the second, in which a monomer has come out of solution and
bound to the next empty site on the template. b) Each of these macrostates
contains many microstates, for example changes in the physical configuration
of the trailing end. c) A hypothetical energy landscape for the transition,
with a free-energy barrier between the two states, topped by the “activated
complex” referred to in transition rate theory. A plausible trajectory through
the system is shown; the system initially quickly moves between microstates
within the left hand basin, before it gains enough energy due to the
fluctuating environment to overcome the barrier, releasing the energy as it
travels into the right hand stable basin, where it remains.

Biological copying systems, such as polymerases interacting with a DNA
template during RNA transcription, are inherently unpredictable, or
stochastic. In order for the reaction to progress, resources must be acquired
from the surrounding environment via collision and it is not possible to
predict the exact moment that two molecules collide, even in an environment of
relatively consistent resource density. For RNA transcription these resources
include RNA nucleotides, the monomer required to build the copy, and ATP,
which is a fuel source for the process. Even the process of the polymerase
first interacting with the DNA strand must occur stochastically.

We can imagine a hypothetical energy landscape for the general process of
making a templated copy, which starts with a single monomer about to bind to
an empty template and finishes with a full-length copy polymer having just
separated from the template. Every iteration of the process will move through
the energy landscape differently but on average they will behave in
predictable ways. The field of stochastic thermodynamics is designed to
analyse such systems.

The free energy landscape for a single copy being built on a single template
can be accurately modelled as a series of meta-stable macrostates separated by
high free-energy barriers. For example, as shown in fig. 5a, a growing polymer
on a template, attached by its final monomer, would be a meta-stable
macrostate, as would the same system with a monomer having come out of
solution and bound to the template at the next available site. While there are
in fact lots of different microstates within each of these meta-stable
macrostates e.g. multiple different physical arrangements of the trailing tail
(fig. 5b), transitions between these microstates within the meta-stable
macrostates are fast, while transitions between meta-stable macrostates are
slow (fig. 5c)[44, 45]. Given this state space, it is reasonable to consider
that we are working in a discrete state setting, with our trajectories
spending most of their times in the stable basins, and only occasionally
moving between neighbouring macrostates.

Stochastic thermodynamics gives us tools to evaluate discrete state systems
which include many biological systems such as molecular motors[46, 47],
membrane transport[48], population dynamics[49, 50] and disease spreading
models[51]. There are two broad classes of models with discrete state space;
the first is for settings with both discrete state space and discrete time,
and the second is for settings with discrete state space but continuous time.
While in order to understand the full dynamics of a templated copying system
it is necessary to work in the discrete state space and continuous time limit,
there are specific cases where it is sufficient to work in the discrete state
space and discrete time limit. I will therefore give an overview of methods of
solving both classes of system.

In the discrete state space and discrete time case, systems can be analysed by
discrete Markov state models. Given that systems spend a long time in meta-
stable free-energy basins, with fast transitions between microstates within
the meta-stable basin, the system quickly forgets how any particular
trajectory entered the meta-stable state. Therefore the next step between
meta-stable basins is dependent only on the current basin, and not on any
previous steps taken. This is the condition for a system being Markovian and
is true in both the discrete and continuous time cases. If a system has both
temporal homogeneity and is ergodic then there will be a unique steady state
for a given system, giving the probabilities of being in each of the
macrostates of the system in the long time limit[52]. Temporal homogeneity is
the property that the probability of a given transition $i$ to $j$,
$P_{i\rightarrow j}$ is dependent only on relative time, not on absolute time.
This means that the probability that a transition happen in one second is not
dependent on whether that is between second 2 and 3 of a process or between
second 12 and 13. Ergodicity is the property that it is possible to reach
every possible other state from any given state in a finite number of steps,
so that there are no regions of the state space that are isolated from others.

In order to find the unique steady state of a system, it is necessary to
quantify the probabilities of each possible transition between two meta-stable
basins in a certain unit of time. In the discrete time case, we model time as
a series of discrete “ticks” of length $\Delta t$, after which exactly one
transition has happened (including the possibility of a transition to the
current state). An example of a truly discrete time system is a turn-based
board game; after each round, one tick has happened and each player (each
representing a single trajectory) has taken one turn’s worth of action with a
given probability. Many systems are not truly discrete in time, but can be
discretised by integrating the continuous probability of making any given move
from any given basin (or failing to make a move) in the time $\Delta t$. We
can use these probabilities in our discrete time Markov state model.

##  V Solving Markov models

Figure 6: a) The transition matrix for the discrete state-space, discrete time
network shown. Note that all quantities are probabilities, and all columns sum
to one. The long time limit is the zeroth eigenvector of the matrix. b) The
rate matrix for the discrete state-space, continuous time network shown. Note
that all quantities are rates, and all columns sum to zero. The long time
limit is again the zeroth eigenvector of the matrix. c) The absorbing state
matrix for the discrete state-space, discrete time network shown. States C and
E are absorbing states, and A, B and E are transient states. Thus ${\bf M}$ is
$3\times 3$ and ${\bf A}$ is $2\times 3$.

###  V.1 Transition matrices

One common method of solving discrete time Markov state models is by using a
transition matrix[44, 53]. For a network which has well defined probabilities
for moving between discrete states in a discrete unit of time, each location
in the transmission matrix ${\bf T}$ represents the probability of moving from
the state represented by the row, to the state represented by the column in
that given time, ie $T_{ij}$ is the probability of moving from state $j$ to
state $i$. An example transition matrix for a system is shown in fig. 6a. If
we represent a given initial state by a probability vector ${\bf P}(0)$ where
the $i$th position in ${\bf P}(0)$ represents the probability of starting in
state $i$ then the probability distribution after one tick is just ${\bf
P}(\Delta t)={\bf T}{\bf P}(0)$. For a given number of ticks $n$ the
probability distribution is ${\bf P}(n\Delta t)={\bf T}^{n}{\bf P}(0)$ and the
long time steady state is just the zeroth eigenvector of ${\bf T}$.

Return to our hypothetical energy landscape for a single copy growing on a
single template, which starts with a single monomer about to bind to an empty
template and finish with a full length copy polymer having just separated from
the template. It is clear than in this case, what we are most interested in is
not the probability density of being in any given intermediate state at any
given time, but instead the probability of us ending up with one of a number
of empty template end states, with either the first monomer dissociating from
the template or a full length copy dissociating from the template. Methods
such as these also gives us tools to calculate the probability of ending up
any given final state, given a starting state (such as the first monomer bound
to the otherwise empty template).

###  V.2 Absorbing state matrices

In many systems, what we are most interested in is not the probability density
of being in any given intermediate state at any given time, but instead the
probability of us ending up with one of a number of designated “endpoint”
states. Stochastic thermodynamics also gives us tools to calculate the
probability of ending up any given final state, given a starting state.

To do this we use an absorbing state matrix similar to a transition matrix, in
which the probability of leaving the absorbing state is zero[44]. These
matrices can be split into four parts as shown in fig. 6c, known as ${\bf M}$,
${\bf A}$, ${\bf I}$ and ${\bf 0}$. Here, ${\bf M}$ is a $t\times t$ matrix
where $t$ is the number of transient states, ${\bf T}$ is a $r\times t$ matrix
where $r$ is the number of absorbing states and ${\bf I}$ and ${\bf 0}$ are
identity and zero matrices respectively. It is straightforward to calculate
the fundamental matrix ${\bf F}=({\bf I}-{\bf M})^{-1}$ which gives the
expected number of times a system passes through state $i$ given it started in
state $j$. This fundamental matrix ${\bf F}$ can be used to calculate a whole
range of properties of the system, including the matrix of probabilities that
the system ends in absorbing state $j$, given it started in transient state
$i$ which is given by ${\bf Z}={\bf F}.{\bf A}$. This is key to our analysis
of the creation of short oligomer copies being created with respect to short
oligomer templates.

###  V.3 Rate matrices: for the continuous time case

In the continuous time limit we use not a probability based transition matrix,
but a rate matrix which defines the master equation for each state[44]. A
master equation describes a system in terms of flows in and out of a state.
The probability of being in a single state $i$ at a given time $t$ is denoted
$P_{i}(t)$. The master equation for the rate of change of the probability of
being in state $i$ is then,

$\displaystyle\frac{dp_{i}(t)}{dt}=\sum_{j\neq i}W_{j\leftarrow
i}P_{j}(t)-\sum_{j\neq i}W_{i\leftarrow j}P_{i}(t).$ (1)

where the transition frequency from state $i$ to state $j$ is given as
$W_{j\leftarrow i}$[54]. We can roll the entire set of master equations for a
system into a rate matrix ${\bf W}$ (fig. 6, b), where $W_{ij}=W_{i\leftarrow
j}$ is the flow from state $j$ to state $i$ for $i\neq j$ and where the
diagonals $W_{ii}$ are the sum over the negative of the other values in the
column[52]. The solution to the master equation $\frac{d{\bf P}}{dt}=-{\bf
W}{\bf P}$ is just ${\bf P}(t)=e^{{\bf W}t}{\bf P}(0)$, where ${\bf P}$ is the
vector of probabilities, so by diagonalising ${\bf W}$ we can
straightforwardly find the steady state of the system[52].

###  V.4 Embedding

It is possible to embed a continuous time process into a discrete time
process, which will have the same probability of visiting a given sequence of
states, but may not hold any information about the timing of the process. This
most commonly done in one of two ways, one in which the properties of the
continuous time process is broken down into a series of discrete “ticks”, and
one in which normalised rates are used as probabilities of taking a step.

In the first of these two methods, the continuous system is integrated over a
defined time period in order to calculate the probabilities of any given
transition, or failure to transition. These probabilities are then used as the
probabilities in the discrete time transition matrix. This retains some
information about timings, albeit limited to the resolution of the length of a
single “tick”.

The second method is arguably simpler than the first. Here for a given state,
the rates of transitioning out of that state are normalised and used as
probabilities in the discrete time transition matrix. This method can give the
same probability for a given sequence of transitions and states as the
continuous time case, but all information about timings is lost[44].

###  V.5 Monte Carlo simulations

It can often be the case that systems are not analytically tractable using the
various matrix methods above. Systems with more than a small number of
intermediate states quickly become ungainly to work with analytically. In
these cases we can use Monte-Carlo simulations of a process to generate many
individual stochastic trajectories, the average properties of which should
converge on our analytical results for a sufficiently large amount of
trajectories[44].

The basic mechanism of a Monte-Carlo simulation is straightforward. Like the
transition matrices above, we start by defining the probabilities of where the
system will move to at the next step, given your current position. We then
draw a random number, and use this to choose which of the possible steps it
might take. For example if from your current state you can go to one of 2
states with equal probability, a random number of less than or equal to
$\frac{1}{2}$ would take you to the first state, and of greater than
$\frac{1}{2}$ would take you to the second state. You then update the time, by
adding an exponentially distributed random number, distributed around the
inverse of the sum of the rates from the current state to all adjoining states
$\frac{1}{\sum_{j}k_{ij}}$ where $i$ is the current state. If a system is
initialised in a given state, it is possible to observe the trajectory it
traces through the different states. Multiple iterations of this allows you to
calculate the average properties of such a system.

###  V.6 Using methods in combination

It is often the case in copying systems that several of these methods will be
used in combination. As an example an absorbing state matrix being used to
calculate the final state probabilities of an underlying network which
represents a small part of a more complex systems, which are the used in a
further analytical calculation or simulation of a higher level system. This is
especially useful for systems with a high level of modularity.

Over the next sections I will outline some important concepts in stochastic
thermodynamics and information theory, which give us constraints on our
models, and allow us to interpret them as physically consistent systems.

##  VI Stochastic Thermodynamics

###  VI.1 Thermodynamics of trajectories

Consider a specific trajectory, which encompasses several steps, in a system
which is in a stationary free energy landscape. Throughout this subsection we
are not considering systems undergoing time dependent protocols unless
explicitly stated otherwise, because we are interested in autonomous systems
typical of biochemistry. Here the system starts in state $0$, and waits until
time $t_{1}$ until transitioning to state $1$, and then waits until time
$t_{2}$ until transitioning to state two, all the way up to waiting time
$t_{n}$ to transition from state $n-1$ to n. There exists a time reversal of
this trajectory in which the system starts in state $n$, and waits time
$t_{n}$ to transition to state $n-1$ and so on, all the way back to waiting
time $t_{1}$ to transition from state one to zero[55, 56].

In the case that this trajectory is markovian, as is the case throughout this
thesis where the system undergoes many fast transitions between microstates
within a meta-stable macrostate, and only rarely transitions between meta-
stable macrostates, then the probability of the forward and backward
trajectories, given their starting states are,

$P_{0\rightarrow n|0}(t)=P_{0}({\rm wait}=t_{0})P_{{0}\rightarrow{1}}\times
P_{1}({\rm wait}=t_{1})P_{{1}\rightarrow{2}}\times...\times P_{n-1}({\rm
wait}=t_{n-1})P_{{n-1}\rightarrow{n}},\\\ $ (2) $P_{0\rightarrow
n|0}(t)=\prod_{i=1}^{n}P_{i-1}({\rm wait}=t_{i-1})P_{{i-1}\rightarrow{i}},\\\
$ (3)

for the forward trajectory and

$P_{n\rightarrow 0|n}(t)=P_{n}({\rm wait}=t_{n})P_{{n}\rightarrow{n-1}}\times
P_{n-1}({\rm wait}=t_{n-1})P_{{n-1}\rightarrow{n-2}}\times...\times P_{1}({\rm
wait}=t_{1})P_{{1}\rightarrow{0}},\\\ $ (4) $P_{n\rightarrow
0|n}(t)=\prod_{i=1}^{n}P_{i}({\rm wait}=t_{i})P_{t_{i}\rightarrow t_{i-1}},\\\
$ (5)

for the reverse trajectory. Here $P_{n}({\rm wait}=t_{n})$ is the probability
of the system spending time $t_{n}$ in state $n$ before transitioning and
$P_{n\rightarrow n+1}$ is the instantaneous probability of transitioning from
state $n$ to $n+1$, which is identical to the rate of transition
$k_{n\rightarrow n+1}$. Each individual transition in the path corresponds to
an amount of entropy released into the environment. We denote the individual
contributions to the “environmental entropy” of the path $\Delta s^{i+}_{\rm
env}=\ln{\frac{P_{n\rightarrow n+1}}{P_{n+1\rightarrow n}}}$ for the
individual reactions in the forward trajectory or $\Delta s^{i-}_{\rm
env}=-\Delta s^{(i+1)+}_{\rm env}$ for the reverse trajectory. Thus for the
forward trajectory the total entropy released into the environment $\Delta
S_{\rm env}=\sum_{i=0}^{n}\Delta s^{i+}_{\rm env}$. This entropy change
accounts for the entropy due to the transitions between states, but it is also
necessary to consider the individual entropies of the initial and final
states. Because the system has been coarse grained, the initial and final
states are macrostates which themselves contain multiple microstates. There is
thus an internal entropy of the initial and final states, and we denote the
difference between them $\Delta S_{\rm mac}$[57].

This leads us neatly to fundamental assumption of stochastic thermodynamics;
that the relative probabilities of a pair of forward and backward trajectories
are given by

$k_{B}\ln{\frac{P_{0\rightarrow n|0}(t)}{P_{n\rightarrow 0|n}(t)}}=\Delta
S_{\rm env+mac}=\Delta S_{\rm env}+\Delta S_{\rm mac}.$ (6)

Here $\Delta S_{\rm env}$ is the entropy change of the environment due to the
forward trajectory and $\Delta S_{\rm mac}=S(z(t))-S(z(0))$ is the difference
between the macrostate entropy of the macrostate in which the trajectory
begins and the entropy of the macrostate in which the trajectory ends[58, 54,
59, 60, 45, 55, 56]. In a system without coarse graining, $\Delta S_{\rm mac}$
would be zero because a microstate has no internal entropy. Observe that in
the ratio $\frac{P_{0\rightarrow n}(t)}{P_{n\rightarrow 0}(t)}$ the waiting
times cancel, leaving just the ratio of the rates. This equation is basis of a
large part of stochastic thermodynamics, from which you can derive fluctuation
relations, uncertainty relations and the second law[60].

As a result the rates can be given by

$\displaystyle\frac{k_{\rm forward}}{k_{\rm backward}}=e^{\Delta S_{\rm
env+mac}},$ (7)

where $k_{\rm forward}=\prod_{i=1}^{n}k_{i-1\rightarrow i}$ and $k_{\rm
forward}=\prod_{i=1}^{n}k_{i\rightarrow i-1}$[45]. Given that a simple
molecular system such as a templated copying system can only exchange energy
in the form of heat with its environment, then the entropy change of the
environment takes the form of heat dissipated into the environment (the
canonical ensemble), ie $T\Delta S_{\rm env}(z(t))=\Delta U_{\rm env}=-\Delta
U_{\rm mac}$[60], where $\Delta U_{\rm mac}$ is the heat dissipated free
energy difference between the initial and final macrostate. This allows us to
write the entropy change of the environment and the macrostate as

$\displaystyle\Delta S_{\rm env+mac}=T\Delta S_{\rm env}+T\Delta S_{\rm mac},$
(8) $\displaystyle\Delta S_{\rm env+mac}=-\Delta U_{\rm mac}+T\Delta S_{\rm
mac},$ (9) $\displaystyle\Delta S_{\rm env+mac}=T\Delta S_{\rm env}+T\Delta
S_{\rm mac}$ (10) $\displaystyle\Delta S_{\rm env+mac}=-\Delta G_{\rm mac},$
(11)

where $\Delta G_{\rm mac}$ is the change in free energy of the system due to
changing from one macrostate to another[60]. This tells us that for any
trajectory, including individual transitions,

$\displaystyle\frac{k_{\rm forward}}{k_{\rm backward}}=e^{-\Delta G_{\rm
mac}}.$ (12)

This result will underly the design of the models throughout this thesis.

###  VI.2 Thermodynamic reversibility

Figure 7: The probability distributions (red) for system with a time varying
free energy landscape (black). The system shown undergoes the following
protocol; the system is switched to a single monostable well by lowering the
free energy barrier, allowing the system to equilibrate, and then the barrier
is raised. If this system is initialised with all trajectories in the left
hand well, then after this protocol half will be in the right hand well and
half will be in the left hand well. If we reverse this protocol starting with
the trajectories in the mixed final state, we will not return to all
trajectories being in the left hand well, but instead the trajectories would
remain with half in the left and half in the right.

It is worth taking a moment to talk about different types of reversibility.
Microscopic reversibility is related to eq. 7. It requires that if there is a
trajectory that moves through a system via a specific path, that there is an
equivalent trajectory which can move through the system along the same path,
but in the opposite direction. The probability of the two trajectories is
related by eq. 7. This concept is closely related to that of detailed balance,
which gives the probability of changing states without reference to a specific
pair of trajectories. It is characterised by the relation that for states $a$
and $b$ that the instantaneous probabilities of transitioning between the
states are related by $P(a\rightarrow b)=P(b\rightarrow a)e^{\beta\Delta G}$,
where $\Delta G$ is the difference in free energy between states $a$ and
$b$[58, 59, 56].

Thermodynamic reversibility is a property of an ensemble of transitions, which
are moving through a system as it undergoes a protocol. An example of a
protocol illustrated in fig. 7 could be on a double well potential. The
protocol would be the switching of the system to a single monostable well by
lowering the free energy barrier, allowing the system to equilibrate, and then
raising the barrier again. If this system is initialised with all trajectories
in the left hand well, then after this protocol half will be in the right hand
well and half will be in the left hand well. If we reverse this protocol
starting with the trajectories in the mixed final state, we will not return to
all trajectories being in the left hand well, but instead the trajectories
would remain with half in the left and half in the right. Thus the process is
thermodynamically irreversible This is despite the fact that some individual
trajectories will return exactly to the state they came from (microscopic
reversibility). Thermodynamically reversible process do not increase the
entropy of the universe, but processes which are irreversible do[60, 45, 61,
56].

In the case of a copying system our protocol is very simple; merely “do
nothing”. This is because our trajectories are moving through an unchanging
free energy landscape. In these cases, the system is inevitably
thermodynamically irreversible if a non-zero net evolution occurs[60].

Returning to

$k_{B}\ln{\frac{P_{0\rightarrow n|0}(t)}{P_{n\rightarrow 0|n}(t)}}=\Delta
S_{\rm env+mac},$ (13)

and noting that $P_{0\rightarrow n|0}(t)$ is conditional on the system
starting in state zero. If we add $k_{B}\ln{\frac{P_{0}(0)}{P_{n}(t)}}$, where
$P_{n}(t)$ is the probability of being in state $n$ at time $t$, to both sides
of the equation we get

$k_{B}\ln{\frac{P_{0\rightarrow n}(t)}{P_{n\rightarrow 0}(t)}}=\Delta S_{\rm
env+mac}+k_{B}\ln{\frac{P_{0}(0)}{P_{n}(t)}},$ (14)

where $P_{0\rightarrow n|0}(t)P_{0}(0)=P_{0\rightarrow n}(t)$ in general[54,
45]. If we now average over all possible trajectories $0\rightarrow n$ then we
get,

$k_{B}\sum_{0,n}P_{0\rightarrow n}(t)\ln{\frac{P_{0\rightarrow
n}(t)}{P_{n\rightarrow 0}(t)}}=\sum_{0,n}P_{0\rightarrow n}(t)\Delta S_{\rm
env+mac}+k_{B}\sum_{0,n}P_{0\rightarrow n}(t)\ln{\frac{P_{0}(0)}{P_{n}(t)}},$
(15) $k_{B}\sum_{0,n}P_{0\rightarrow n}(t)\ln{\frac{P_{0\rightarrow
n}(t)}{P_{n\rightarrow 0}(t)}}=\Delta S_{\rm
env+mac}+k_{B}P_{0}(0)\ln{P_{0}(0)}-k_{B}P_{n}(t)\ln{P_{n}(t)},$ (16)

The term on the left hand side is the Kullback Leibler divergence between the
distribution of forward and backward trajectories. This quantity is discussed
in more detail below, but essentially is a non-negative quantity which
quantifies how different two distributions are. The first term on the right
hand side is the entropy change of the environment and the macrostates and the
rightmost two terms are the differences between the Shannon entropy of the
initial and final states. Thus the right hand side is the total entropy change
of the system during the trajectory. Shannon entropy is also discussed in more
detail below but is functionally identical to thermodynamic entropy.

Thus the right hand side is exactly the total entropy production for a
trajectory. Due to Jenson’s inequality[62], the left hand side is non-
negative, and only reaches zero when the process is thermodynamically
reversible, ie forwards and backwards transitions have the same
probability.[55]. Thus, a thermodynamically irreversible process generates
entropy.

###  VI.3 Transition State Theory

In defining the rates of reaction, it is important to understand what a
transition entails. In order to leave the reactant basin, the system must gain
enough free energy to reach the top of the barrier for long enough to cross
the barrier, only to lose it again as it progresses to the stable product
basin. It is assumed that it takes much longer for the system to fluctuate in
such a way that it gains enough free energy to leave the reactant basin than
it does to relax to the product basin. This is called the separation of
timescales.

Transition state theory[44] can be used to estimate the value of the rates of
transition $k_{a\rightarrow b}$ or $k_{b\rightarrow a}$ between states $a$ and
$b$. It considers a hypothetical dividing surface, which separates states $a$
and $b$. In the common case where the top of the energy barrier is a saddle
point this surface is represented by a single “activated state” $\ddagger$ as
shown in fig. 5. Transition rate theory assumes that states on the dividing
surface are at equilibrium with the reactants. It further assumes that once
states cross the dividing surface, they proceed to the formation of products
without “recrossing” the dividing surface. The reaction $a+b\rightarrow c+d$
can be written $a+b\rightleftharpoons\ddagger\rightarrow c+d$. The rate of
reaction is therefore $k_{forward}=[\ddagger]k_{\ddagger\rightarrow{\rm
prod}}$ where $k_{\ddagger\rightarrow{\rm prod}}$ is the rate at which
transition states become products and $[\ddagger]$ is the concentration of
transition rates species. The rate of the reaction is

$\displaystyle k_{\rm forward}=Ae^{-\frac{\Delta G^{\ddagger}}{k_{B}T}}$ (17)

where A is a constant depending on a reference volume, $V_{0}$ and $\Delta
G^{\ddagger}$ is the height of the energy barrier[44] from the bottom of
macrostate $a$. Given the difficulty (or perhaps impossibility) of creating a
reaction surface which is never recrossed, transition rate theory provides an
upper bound on the rate. Transition state theory is most useful in deciding
relative rates of similar processes, for which uncertainties in $A$ cancel.

##  VII Information Theory

The key parameter in information theory is the Shannon Entropy[63, 62], which
is defined as $H=\sum_{p_{i}}p_{i}\ln{p_{i}}$ where $p_{i}$ is the normalised
probability of an outcome of a random process. It is a measure of uncertainty
and is directly analogous to the thermodynamic entropy used above, different
only by a factor of $k_{B}$.

In general, when studying templated copying systems we are interested in
parameters that describe the sequences of the produced polymers and how they
compare to the sequences of template polymers. Both of these quantities are
random variables, because they are variables where the value depends on the
outcome of a process which itself contains randomness or stochasticity. We use
specific quantities which describe how the two sequences relate to each other.

The error in the sequence and the mutual information between copy and template
are both quantities which compare two random variables. The error $\epsilon$
is the proportion of mismatches incorporated into a chain and the mutual
information is $I(X;Y)=\sum_{X,Y}\frac{P(x,y)}{P(x)P(y)}$ where $X$ and $Y$
are random variables such as the sequence of a template polymer and a copy
polymer, gives the reduction in Shannon entropy about $Y$ having measured $X$
and vice versa[64, 65, 66]. Here $P(x,y)$ is the joint probability of random
variables $X$ and $Y$ and $P(x)$ and $P(y)$ are their marginal
probabilities[62]. These quantities are related to the distribution of the
quantities $x$ and $y$. Because the value of $X$ and $Y$ are dependent on the
outcome of a stochastic process, it can be useful to express these quantities
as a distribution, $P(x)$ is the probability that a randomly selected
trajectory would give outcome $X=x$[64], equally the joint probability
$P(x,y)$ is the probability that a randomly selected trajectory would give
outcome $X=x$ and $Y=y$ and the conditional probability $P(x|y)$ is the
probability that a randomly selected trajectory from the pool of trajectories
that give $Y=y$ will also give $X=x$[56].

In more detail, Shannon entropy and mutual information are related as follows;
$I(X;Y)=H(X)-H(X|Y)=H(Y)-H(Y|X)$[62, 64, 65, 66, 55, 56]. Fig. 8 shows a Venn
diagram representation of entropy and mutual information. We can see that the
mutual information is the intersection of the two circles and the joint
entropy is the the area of the two overlapping circles. It is therefore clear
that as the mutual information increases, then the joint entropy must
decrease. In our copying analogy if we define $X$ as the monomer type at
position $n$ in the copy polymer and $Y$ as the monomer type at position $n$
in the template, then the more accurate the copy, the more the mutual
information $I(X;Y)$ increases and the joint entropy H(X,Y) decreases.

The joint entropy can also be expanded to the entropy rate. The entropy rate
is the limit of the average of the joint entropy over all variables in the
system, ie
$H(Z)=\lim_{n\rightarrow\infty}\frac{1}{n}H(Z_{0},Z_{1},...,Z_{n})$, where
$Z_{i}$ is a random variable set by the system[62, 67, 56]. As an example, for
a polymer sequence, each monomer of which represents a random variable, the
entropy rate is capable of taking into account correlations between monomers
of any length.

Another useful quantity is Kullback-Leibler divergence $D(P||Q)$, which
quantifies the distance between two probability distributions $P(x)$ and
$Q(x)$ of random variable $X$[62, 56]. For example we might quantify This for
example could quantify the distance between the distribution of polymers in
the baths coupled to the system and the distribution of copy polymers produced
by the system.

Figure 8: A venn diagram showing the additive relationship between various
forms of entropy and mutual information. We can straightforwardly see from
here that $I(X;Y)=H(X)-H(X|Y)=H(Y)-H(Y|X)$ or $I(X;Y)=H(X,Y)-H(X|Y)-H(Y|X)$
etc.

##  VIII Thermodynamics of information

Having outlined some of the information theoretic quantities used in this
thesis, it is worth exploring the thermodynamics of information. Historically
the thermodynamics of information has centred around the study of a paradox
known as “Maxwell’s demon” and a hypothetical engine called a Szilard engine,
designed to provide a framework in which to exorcise the demon.

Figure 9: (a) The Maxwell’s demon paradox[3]. A box containing a uniform gas
at equilibrium is partitioned down the middle. That partition contains a door
which is operated by a “demon”. Without spending any energy, the demon can
selectively allow particles to cross the partition. It allows fast particles
from left to right, while blocking slow particles, and allows slow particles
from right to left, while blocking fast particles. This results in the system
now having a cold section and a hot section, which can hypothetically be used
to do work. Thus the system has gone from an equilibrium state, to an out-of-
equilibrium state without work being done on it, breaking the second law. (b)
The solution to the Maxwell’s demon paradox is best understood by simplifying
the system into a Szilard engine[4]. Here a box contains a single particle
gas. A partition is dropped into the box, confining the particle to one side
of the box. The particle then pushes the partition back to the edge of the
box. If a weight is coupled to the partition, then the particle can lift the
weight doing a maximum of $k_{B}T\ln{2}$ of work. This problem is identical to
that posed by Maxwell, fluctuations in an equilibrium gas are being harnessed
to do work, without a corresponding input of work. The key to exorcising the
demon comes from the observation that in order to extract the work, the
position of the pulley must be correlated with the position of the particle.
If it was always attached on e.g. the right hand side, then half the time the
particle would lift the weight, but half the time it would lower it, on
average doing no work. The minimum cost of correlating two bits of information
is $k_{B}T\ln{2}$, exactly the same as the maximum amount of work that can be
extracted from the system. Thus the second law is restored and the demon is
exorcised.

The Maxwell’s demon paradox was first outlined by James Clerk Maxwell in
1867[3]. He suggested that the second law of thermodynamics could be broken
using the following method. Illustrated in fig. 9(a) is a box containing a
uniform gas. This box is partitioned down the middle, and the partition
contains a door operated by a demon. Although the temperature of the box is
initially the same on both sides of the partition, it is in theory possible
for the demon to open the door and allow through any fast moving particle to
the left box, and to allow slow particle through to the right box, and block
all others. If we assume that the demon can act for free; which is reasonable
because it does no work against a force, then this would cause the left box to
heat relative to the right box without an input of free energy. Thus work can
be extracted from the box which was previously at equilibrium. This breaks the
second law of thermodynamics.

In 1929, Leo Szilard came up with a model system which illustrates this issue
without the invocation of a difficult to quantify “demon”. The Szilard
engine[4], illustrated in fig. 9(b), works as follows. Consider a gas made up
of a single particle in a box. If a partition is instantaneously dropped into
the box, the single particle will now be trapped on one side of the partition.
The particle will now expand against the partition, pushing it back to the
edge of the box. If the partition is coupled to a pulley and weight, then this
system is able to perform a maximum of $k_{B}T\ln{2}$ of work in raising the
weight. This framework allows us to identify that this system is in fact
exploiting fluctuations in the particles position in order to extract
work[68]. This is however absolutely forbidden by the second law.

The solution to this paradox has been much discussed[55, 53, 69, 68, 52, 70,
65, 61], but the simplest explanation comes from the observation that in order
to extract work from the system, it is necessary to correlate the position of
the particle with the position of the pulley and weight. If the pulley and
weight were always attached to the left hand side of the partition, or if the
attachment was assigned randomly, then 50$\%$ of the time the system would
raise the weight and 50$\%$ of the time the system would lower the weight; on
average doing no work. Therefore the particle and pulley must be correlated in
order for the engine to work as designed. Correlating two bits of information
costs a minimum of $k_{B}T\ln{2}$ of work, the same amount that can be
extracted by the engine. This can be understood by considering that
correlating the position of the the piston and the position of the particle is
increasing the mutual information to the extent that the joint entropy becomes
just the uncertainty in the position of the particle. Reducing the joint
entropy of the system requires an input of free energy. Thus the second law is
restored and the demon is exorcised.

It should be noted that if the pulley and the particle were correlated via a
direct physical interaction, such as an attractive force, then the system
would not be able to work as designed. In order for the engine to act as an as
in fig. 9 it is necessary for the pulley to remain coupled to the left hand
side, even as the particle moves into the right hand side in the process of
moving the partition. It is necessary to instead create a temporary coupling
via a complex mechanism, such as performing a measurement. This is in neat
analogy to the process of self assembly in which interaction have to be
predesigned and permanent, and the process of templated copying in which
interactions that generate correlations are via a template catalyst which the
copy must ultimately separate from.

A class of machines known collectively as information engines have been
proposed which can extract work from correlations, or spend work to generate
correlations. These often take the form of a tape of bits[68, 52, 70]
interacting with a heat bath at equilibrium. Through various mechanisms which
use correlations as fuel, the system is able to use this equilibrium heat bath
to do work, such as lift a weight. The flaw in many of these systems however
is that they leave the details of the interactions somewhat abstract,
effectively mystifying correlations as a source of fuel.

The paper “Biochemical Machines for the Interconversion of Mutual Information
and Work”[64] seeks to demystify information engines as a concept by proposing
a physically realisable system based around an enzyme, which when it is in its
active state, either extracts fuel by converting ATP into ADP while activating
a complex $X\rightarrow X^{*}$ or pumps fuel by converting ADP into ATP while
deactivating a complex $X^{*}\rightarrow X$. A pair of tapes $A$ and $B$ are
simultaneously pulled past the enzyme. Tapes $A$ and $B$ are correlated, and
tape $B$ dictates whether or not the enzyme can interact with the tape. The
system is set up so that tape $B$ allows tape $A$ to interact with the enzyme
only when tape $A$ contains an $X^{*}$. The system pumps fuel by converting
ADP into ATP while deactivating a complex $X^{*}\rightarrow X$. Thus by
exploiting the correlations between $A$ and $B$, with tape $B$ only exposing
the tape $A$ to the enzyme when $A$ contains an $X^{*}$, the system can pump
fuel. An explicit design for this system is then proposed using DNA origami.
This very concrete example of an information engine makes explicit many things
which are left abstract in more theoretical versions of information
engines[68, 52, 70].

##  IX Thermodynamics of copying

Having defined many of the tools needed to study simple molecular systems, we
now explicitly discuss the properties of templated copying systems. We start
by considering an isolated bath of polymers. We make the assumption that all
backbone bonds are identical. While this is not technically true for
biological polymers, it is true that some backbone bonds are stronger than
others and this could be leveraged if targeting a specific sequence (ie.
always copying the same polymer), this is not beneficial for generating
accuracy while copying a generic polymer.

This isolated bath of polymers has a macroscopic free energy, which is a
property of the whole bath;

$\displaystyle G(p({\bf s}))=U(p({\bf s}))-TS(p({\bf s})).$ (18)

Here $G$ is the free energy which is a function of the probability
distribution of the $n$ different types of polymer s. $U$ is the enthalpy, $S$
is the entropy and $T$ is a temperature. In general, for a system evolving
autonomously, the rate of change in free energy $\dot{G}<0$.

Thus far all quantities have been functions of the entire distribution of
polymers, but we can expand this to be functions of the microstates of the
individual polymers. For the enthalpy this is straightforwardly a weighted sum
of the enthalpies of the individual polymers;

$\displaystyle U(p({\bf s}))=\sum_{\bf s}p({\bf s})u({\bf s}).$ (19)

For the entropy this is slightly more complex as there is a component of the
individual polymer’s entropies, but there is also a macroscopic entropy which
is the entropy of the distribution of the polymers; ie its Shannon entropy.

$\displaystyle TS(p({\bf s}))=T\sum_{\bf s}p({\bf s})s({\bf
s})-k_{B}T\sum_{\bf s}p({\bf s})\ln{p({\bf
s})}\cite[cite]{[\@@bibref{Number}{ouldridge2018importance}{}{}]}.$ (20)

Substituting these two into equation 18 and observing that the microstate free
energy $g=u-Ts$ we get

$\displaystyle G=\sum_{\bf s}p({\bf s})g({\bf s})+k_{B}T\sum_{\bf s}p({\bf
s})\ln{p({\bf s})}.$ (21)

Thus the macrostate free energy is the sum of the microstate free energy and
the negative of the Shannon entropy. Given the assumption that all backbone
bonds are the same then the first term is independent of whether the polymers
in the bath are randomly distributed or all identical. Thus, just considering
the microstate free energy, there is no cost to creating an accurate copy,
your system could produce as many perfectly accurate copies as it likes and
not have to pay any extra free energy for accuracy. However it is clear that
the second term is much more positive for systems with all identical copies
than those with a random distribution of polymers[61]. It is the production of
a biased polymer bath which costs free energy. Thus the production of an
accurate copy is intrinsically the production of a high free energy state.
Copying systems are not relaxing to equilibrium, they are transducing free
energy from chemical free energy stored in the components and any fuel
molecules such as ATP, to information free energy stored in the distribution
of the polymer baths.

We therefore propose that accuracy can only be generated in an out-of-
equilibrium system. Previous work has considered either self-assembly[71, 72,
73] or templated self-assembly[5, 6, 7, 8, 9, 73, 10, 41, 42] in non-
equilibrium contexts. However, in these cases, the non-equilibrium driving
merely modulates a non-zero equilibrium specificity which relies on
stabilising copy/template or intra-component interactions. This work is the
first work which considers a fully autonomous templated copying system which
includes separation of copy and template.

##  X Aims and Outline

It is clear that the full process of templated copying is insufficiently
described by the existing theoretical literature. The separation of copy and
template fundamentally changes the underlying physics of a copying system.
This thesis attempts to fill in this gap.

In chapter one I solve and analyse the first model of an autonomous templated
copying process which includes the sequential separation of copy and template.
Here I analyse a copy growing on, and sequentially separating from an
infinitely long template, remaining attached only by its final monomer. Given
the observation that the creation of an accurate polymer requires an input of
free energy, I identify a general measure of the thermodynamic efficiency with
which these nonequilibrium states are created and analyze the accuracy and
efficiency of a family of dynamical models that produce persistent copies. For
the weakest chemical driving, when polymer growth occurs in equilibrium, both
the copy accuracy and, more surprisingly, the efficiency vanish. At higher
driving strengths, accuracy and efficiency both increase, with efficiency
showing one or more peaks at moderate driving. Correlations generated within
the copy sequence, as well as between template and copy, store additional free
energy in the copied polymer and limit the single-site accuracy for a given
chemical work input.

In chapter two I extend this model to finite length templates, explicitly
considering the final separation step in which the polymer must detach from
its template and mix with surrounding baths of polymers. In this work I split
the free-energy change of copy formation into informational and chemical
terms. I show that, surprisingly, copy accuracy plays no direct role in the
overall thermodynamics. Instead, finite-length templates function as highly-
selective engines that interconvert chemical and information-based free energy
stored in the environment; it is thermodynamically costly to produce outputs
that are more similar to the oligomers in the environment than sequences
obtained by randomly sampling monomers. By contrast, any excess free energy
stored in correlations between copy and template sequences is lost when the
copy fully detaches and mixes with the environment; these correlations
therefore do not feature in the overall thermodynamics. Previously-derived
constraints on copy accuracy therefore only manifest as kinetic barriers
experienced while the copy is template attached; these barriers are easily
surmounted by shorter oligomers.

In chapter three I then add in the destruction of templated oligomers, and ask
whether driven cycles of creation and destruction can contribute to accuracy
in the created polymers. I find regions in which a system can net create the
most accurate polymer and net destroy all other polymers. This chapter seeks
to question the validity of Bennett’s claim that if the system creates and
destroys a bit of information, that the system must disperse $\ln{4}$ of free-
energy in the form of entropy generation[74]. I calculate four constraints on
a creation and destruction system which suggest that this limit can be broken.
Having analysed a simple model of a system in which the template and destroyer
reach steady state with the oligomer baths, we find no system which beats the
limit, but do find a signature of systems which may beat the limit for longer
length oligomers.

## Nonequilibrium correlations in minimal dynamical models of polymer copying

This work is adapted from work published in January 2019 in Proceedings of the
National Academy of Science under the same title.

Living systems produce “persistent” copies of information-carrying polymers,
in which template and copy sequences remain correlated after physically
decoupling. We identify a general measure of the thermodynamic efficiency with
which these non-equilibrium states are created, and analyze the accuracy and
efficiency of a family of dynamical models that produce persistent copies. For
the weakest chemical driving, when polymer growth occurs in equilibrium, both
the copy accuracy and, more surprisingly, the efficiency vanish. At higher
driving strengths, accuracy and efficiency both increase, with efficiency
showing one or more peaks at moderate driving. Correlations generated within
the copy sequence, as well as between template and copy, store additional free
energy in the copied polymer and limit the single-site accuracy for a given
chemical work input. Our results provide insight in the design of natural
self-replicating systems and can aid the design of synthetic replicators.

Figure 1: Free-energy landscapes for simple examples of (a) templated self-
assembly, in which the monomers remain bound to the template during the copy
process; and (b) persistent copying, in which the monomers detach from the
template after they have been incorporated into the polymer. Both diagrams
show the addition of three monomers to a growing polymer, driven by a chemical
free energy of backbone polymerisation $\Delta G_{\rm pol}$. In each case, two
scenarios are considered: the addition of two incorrect monomers, followed by
a correct one (top), and the addition of three correct monomers (bottom).
Local minima in the landscape represent macrostates following complete
incorporation of monomers; intermediate configurations, illustrated
schematically for the first transition, are part of the effective barriers. In
templated self-assembly, the chemical free-energy cost of previously
incorporated mismatches is retained as the daughter grows [5, 6, 7, 8, 9, 10].
Thus in (a), each mismatch in the daughter increases the chemical free energy
by $\Delta G_{\rm D}$ relative to the perfect match. In persistent copying
(b), the chemical free-energy penalty for incorporating wrong momomers is only
temporary; it arises when the incorrect monomer is added to the growing
polymer, but is lost when that monomer subsequently detaches from the
template. As a result, the overall chemical free-energy change of creating an
incorrect polymer is the same as that for a correct one. Analyzing the
consequences of this constraint, which is a generic feature of copying but
does not arise in templated self assembly, is the essence of this work. The
figure also shows that in our specific model, incorporating a wrong monomer
after a correct one tends to reduce the chemical free-energy drop to $\Delta
G_{\rm pol}-\Delta G_{\rm TT}$, and incorporating a correct monomer after an
incorrect one tends to increase it to $\Delta G_{\rm pol}+\Delta G_{\rm TT}$;
however, adding a wrong monomer to a wrong one, and adding a correct monomer
to a correct one, does not change the free-energy drop $\Delta G_{\rm pol}$.

It is clear that a model of templated copying, which explicitly includes
separation, is necessary for a fuller understanding of this crucial mechanism.
Our first challenge is to analyse a family of model systems, which describe
the growth and sequential separation of an infinitely long polymer growing on
an infinitely long template. This allows us to consider a system which is
constantly attached to the template by a single bond between the final monomer
in the copy and it’s corresponding template site. Here we omit the final step,
where the copy polymer breaks it’s final bond with the template, and mixes
with surrounding baths of polymers and the first step, in which the first
monomer comes out of solution and binds with the first site on the template.

In this chapter we introduce a new metric for the thermodynamic efficiency of
copying, and investigate the accuracy and efficiency of our models. We
highlight the profound consequences of requiring separation, namely that
correlations between copy and template can only be generated by pushing the
system out of equilibrium. Previous work has considered self-assembly [71, 72,
73] or templated self-assembly [5, 6, 7, 8, 9, 73, 10, 41, 42] in non-
equilibrium contexts; in these cases, however, the non-equilibrium driving
merely modulates a non-zero equilibrium specificity. Alongside the effect on
copy-template interactions, we find that intra-copy-sequence correlations
arise naturally. These correlations store additional free energy in the copied
polymer, which do not contribute towards the accuracy of copying.

##  I Models and Methods

###  I.1 Model definition

We consider a copy polymer ${M}=M_{1},...,M_{l}$, made up of a series of sub-
units or monomers $M_{x}$, growing with respect to a template
${N}=N_{1},...,N_{L}$ ($l\leq L$). Inspired by transcription and translation,
we consider a copy that detaches from the template as it grows; fig. 1b, gives
a generic coarse grained model with a strictly enforced step order. In this
work we consider only the coarse grained steps in which a single monomer is
added or removed, encompassing many individual chemical sub-steps [9, 6].
After each step there is only a single inter-polymer bond at position $l$,
between $M_{l}$ and $N_{l}$. As a new monomer joins the copy at position
$l+1$, the bond position $l$ is broken, contrasting with previous models of
templated self assembly [5, 6, 7, 8, 9, 10] (fig. 1a). Importantly, as
explained below, each step now depends on both of the two leading monomers,
generating extra correlations within the copy sequence. It should be noted
that throughout this thesis, we assume that this step order is fixed, and do
not investigate the conditions needed to enforce the ordering.

Figure 2: Transitions of an arbitrary polymer $\&$. To relate the final chain
to the growing chain it is useful to consider fluxes through interfaces in
this transition diagram. Using the tip and combined probabilities, along with
relative propensities it is possible to describe the fluxes through interfaces
4-7 in terms of properties of the growing chain. Equally by considering errors
and conditional errors and taking fractions of the overall growth velocity, it
is possible to find the fluxes through interfaces 4-7 in terms of properties
of the final chain and growth velocity.

Following earlier work, we assume that both polymers are copolymers, and that
the two monomer types are symmetric [5, 6, 7, 8, 9, 10]. Thus the relevant
question is whether monomers $M_{l}$ and $N_{l}$ match; we therefore ignore
the specific sequence of $N$ and describe $M_{l}$ simply as right or wrong.
Thus $M_{l}\in r,w$; with example chain $M=rrwwrrrrrwrr$. An excess of $r$
indicates a correlation between template and copy sequences. Breaking this
symmetry would favour specific template sequences over others, disfavouring
the accurate copying of other templates and compromising the generality of the
process.

Given the model’s state space, we now consider state free energies (which must
be time-invariant for autonomy). We treat the environment as a bath of
monomers at constant chemical potential [5, 6, 7, 8, 9, 10]. By symmetry,
extending the polymer while leaving the copy-template interaction unchanged
involves a fixed polymerization free energy. We start by defining
concentrations of monomer types with respect to a standard reference
concentration. Here $[1]$ and $[2]$ are the monomer concentration of monomer
types 1 and 2 respectively. The free-energy change when a matching monomer
binds to the template (here both matching monomers are of type $1$ and the
template is type $1,1$) is $\Delta
G^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}_{r}-\ln[1]$ and a mismatching monomer
(here type $2$) is $\Delta
G^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}_{w}-\ln[2]$, in units of $k_{\rm B}T$.
The difference between the two at the reference concentration, $\Delta G_{\rm
TT}=\Delta G_{w}^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}-\Delta
G^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}_{r}$, is the discrimination free
energy, which biases the system towards accuracy. This bias can be describes
as “temporary thermodynamic” (TT) since it only lasts until that contact is
broken. $\Delta G^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}_{\rm dim}$ is the
free-energy change of creating a backbone bond at the reference concentration
and we further define $\Delta G_{\rm pol}=\ln{[m_{l}]}-\Delta
G^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}_{\rm dim}$ as the free energy change
of a monomer coming out of solution and forming a backbone bond (neglecting
copy/template interactions). Throughout this work $[m_{l}]=\frac{1}{2}$ for
$[m_{l}]=r,w$ so $\Delta G_{\rm pol}=-\ln{2}-\Delta
G^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}_{\rm dim}$. $\Delta
G^{{-\kern-4.0pt{\ominus}\kern-4.0pt-}}_{\rm dim}$ incorporates the difference
in entropy between a monomer in solution and as part of a polymer. If the
polymer is ideal, ie does not interact with itself then it’s free-energy will
grow linearly with length. We omit any polymer self interactions which may
cause the free energy to grow non-linearly with length. These could include
attractive interactions which encourage the polymer to form a “sticky ball”,
which would lead to longer polymers capable of coiling being more stable than
longer polymers without this interaction, with shorter polymers not able to
self stabilise.

Overall, each forward step makes and breaks one copy-template bond. There are
four possibilities: either adding $r$ or $w$ at position $l+1$ to a template
with $M_{l}=r$; or adding $r$ or $w$ in position $l+1$ to a template with
$M_{l}=w$. The first and last of these options make and break the same kind of
template bond, so the total free-energy change is $\Delta G_{\rm pol}$. For
the second case there is a $r$ bond broken and a $w$ bond added, implying a
free-energy change of $\Delta G_{\rm pol}+\Delta G_{\rm{TT}}$. Conversely, for
the third case, there is a $w$ bond broken and a $r$ bond added, giving a
free-energy change of $\Delta G_{\rm pol}-\Delta G_{\rm{TT}}$. These
constraints are shown in fig. 1b; the contribution of this work is to study
the consequences of these constraints. Models of templated self assembly (fig.
1a) of equivalent complexity can be constructed, but they are not bound by
these constraints and hence the underlying results and biophysical
interpretation are distinct.

Having specified model thermodynamics, we now parameterize kinetics. We assume
that there are no “futile cycles”, such as appear in kinetic proofreading
[23], each forward step must add either a correct or incorrect match, and each
reverse step must remove one. Reactions are thus tightly coupled: each step
requires a well defined input of free energy determined by $\Delta
G_{\rm{pol}}$ and $\pm\Delta G_{\rm{TT}}$ [75], and no free-energy release
occurs without a step.

A full kinetic treatment would be a continuous time Markov process
incorporating the intermediate states shown schematically in fig. 1b. To
identify sequence output, however, we need only consider the state space in
fig. 1b and the relative probabilities for transitions between these
explicitly modelled states, ignoring the complexity of non-exponential
transition waiting times [6]. We define propensities $\psi^{+}_{xy}$ as the
rate per unit time in which a system in state $\&x$ starts the process of
becoming $\&xy$ and $\psi^{-}_{xy}$ as the equivalent quantity in the reverse
direction ($\&$ is an unspecified polymer sequence). Our system has eight of
these propensities ($\psi^{\pm}_{rr}$, $\psi^{\pm}_{rw}$, $\psi^{\pm}_{wr}$
and $\psi^{\pm}_{ww}$); the simplest templated self assembly models require
four [5, 8, 9, 10, 6].

Prior literature on templated self assembly [8] has differentiated between
purely “kinetic” discrimination, in which $r$ and $w$ have an equal template-
binding free energy but different binding rates; and purely thermodynamic
discrimination in which $r$ and $w$ bind at the same rate, but $r$ is
stabilized in equilibrium by stronger binding interactions. Eventually, all
discrimination is “kinetic” for persistent copying, since there is no lasting
equilibrium bias (Fig. 1 (b)). However, by analogy with templated self
assembly, we do consider two distinct mechanisms for discrimination - a
kinetic one, in which $r$ is added faster than $w$ to the growing tip, and one
based on the temporary thermodynamic bias towards correct matches at the tip
of the growing polymer due to short-lived favourable interactions with the
template, quantified by $\Delta G_{\rm{TT}}>0$ (Fig. 1b). The kinetic
mechanism should not be conflated with fuel consuming “kinetic proofreading”
cycles that can also enhance accuracy, which are not considered.

We parameterize the propensities as follows. Assuming, for simplicity, that
the propensity for adding $r$ or $w$ is independent of the previous monomer,
we have: $\psi^{+}_{rr}=\psi^{+}_{wr}$ and $\psi^{+}_{rw}=\psi^{+}_{ww}=1$,
also defining the overall timescale. “Kinetic” discrimination is then
quantified by $\psi^{+}_{xr}/\psi^{+}_{xw}=\exp(\Delta G_{\rm{K}}/k_{B}T)$.
Forwards propensities are thus differentiated solely by $\Delta G_{\rm{K}}$;
backwards propensities are set by fixing the ratios
$\psi^{+}_{xy}/\psi^{-}_{xy}$ according to the free energy change of the
reaction, which follow from $\Delta G_{\rm{pol}}$ and $\Delta G_{\rm TT}$
(Fig. 1(b)). Thus, setting $k_{B}T=1$,

$\displaystyle\begin{split}&\psi^{+}_{rr}=e^{\Delta
G_{\rm{K}}},\,\psi^{-}_{rr}=e^{-\Delta G_{\rm pol}}e^{\Delta
G_{K}},\end{split}$ (1)
$\displaystyle\begin{split}&\psi^{+}_{rw}=1,\,\psi^{-}_{rw}=e^{-\Delta G_{\rm
pol}}e^{-\Delta G_{\rm{TT}}},\end{split}$ (2)
$\displaystyle\begin{split}&\psi^{+}_{wr}=e^{\Delta
G_{\rm{K}}},\,\psi^{-}_{wr}=e^{-\Delta G_{\rm pol}}e^{\Delta
G_{\rm{K}}}e^{-\Delta G_{\rm{TT}}},\end{split}$ (3)
$\displaystyle\begin{split}&\psi^{+}_{ww}=1,\,\psi^{-}_{ww}=e^{-\Delta G_{\rm
pol}}.\end{split}$ (4)

For a given $\Delta G_{\rm{K}}$, $-\Delta G_{\rm pol}$ and $\Delta
G_{\rm{\rm{TT}}}$, eqs. 1-4 describe a set of models with distinct
intermediate states that yield the same copy sequence distribution. We can
thus analyze the simplest model in each set, which is Markovian at the level
of the explicitly modelled states with $\psi^{\pm}_{xy}$ as rate constants.

###  I.2 Model analysis

####  I.2.1 Properties of the growing monomer

This section is adapted from the paper “Kinetics and thermodynamics of first-
order Markov chain copolymerization”[73], for the specific case of the system
being studied in this work.

In order to solve this system we use a method developed by Gaspard[73]; we
assume that the growth process of a polymer is a Markov process, the addition
of the monomer at site $l+1$ is dependent only on the monomer at site $l$. We
work in the limit where the system is sufficiently dilute, so that while there
are in fact multiple templates in solution, they are independent and the
system can be considered at the level of a single template. This is the
equivalent of considering a single stochastic trajectory, and so time
evolution is described in terms of the probability
$P_{t}(M_{1}...M_{l-1},M_{l})$ of the specific polymer
$M_{1}M_{2}...M_{l-1}M_{l}$ attached to the template, existing in solution.

In this approach, we set up the kinetic equations

$\displaystyle\frac{d}{dt}P_{t}(M_{1}...M_{l-1},M_{l})=$
$\displaystyle\psi^{+}_{M_{l-1}M_{1}}P_{t}(M_{1}...M_{l-1})$ (5)
$\displaystyle+$
$\displaystyle\psi^{-}_{M_{l}r}P_{t}(M_{1}...M_{l-1},M_{l},r)+\psi^{-}_{M_{l}w}P_{t}(M_{1}...M_{l-1},M_{l},w)$
$\displaystyle-$
$\displaystyle(\psi^{-}_{M_{l-1}M_{l}}-\psi^{+}_{M_{l}r}-\psi^{+}_{M_{l}w})P_{t}(M_{1}...M_{l-1},M_{1}),$

which form an infinite hierarchy of coupled ordinary differential equations
linking the probabilities all possible copies at different stages of growth.
In a dilute solution, the concentrations follow the same kinetic equations as
the probabilities.

It is possible to define a mean velocity of growth of the polymer, so that the
polymer length is distributed around an average length $\langle
l\rangle\approx vt$ with variance $\sigma\approx 2Dt$ where $D$ is the the
diffusivity. The sequence of the polymer measured back from the tip can be
assumed to become stationary over long time. This allows the solution of the
kinetic equations to be written as

$\displaystyle
P_{t}(M_{1}...M_{l-1},M_{l})\approx\mu(M_{1}...M_{l-1},M_{l})p_{t}(l),$ (6)

where $p_{t}(l)$ is the probability that a polymer has length $l$ at time $t$
found by marginalising over the probability of all possible polymers of length
$l$. $\mu(M_{1}...M_{l-1},M_{l})$ is the stationary probability of the copy
polymer having the specific sequence $M_{1}...M_{l-1},M_{l}$, given the
polymer is length $l$. This assumption that $\mu$ becomes stationary in the
long time limit is the key assumption of this derivation.

The paper now define tip-monomer identity probabilities and joint tip and
penultimate-monomer identity probabilities by marginalising over all earlier
monomers giving

$\displaystyle\mu(M_{1})\equiv\sum_{M_{1}...M_{l-1}}\mu(M_{1}...M_{l-1}M_{l}),$
(7)
$\displaystyle\mu(M_{l-1}M_{l})\equiv\sum_{M_{1}...M_{l-2}}\mu(M_{1}...M_{l-1}M_{l}).$
(8)

These can be inserted into the kinetic equations to give

$\displaystyle\frac{d}{dt}p_{t}(l)=ap_{t}(l-1)+bp_{t}(l+1)-(a+b)p_{t}(l),$ (9)

where

$\displaystyle a$
$\displaystyle=(\psi_{rr}^{+}+\psi_{rw}^{+})\mu(r)+(\psi_{wr}^{+}+\psi_{ww}^{+})\mu(w)$
(10) $\displaystyle b$
$\displaystyle=\psi_{rr}^{-}\mu(rr)+\psi_{rw}^{-}\mu(rw)+\psi_{wr}^{-}\mu(wr)+\psi_{ww}^{-}\mu(ww).$
(11)

We find the velocity by first postulating an ansatz,
$p_{t}(l)=e^{s_{q}t+iql}$, which we put into equation 9. Here $s_{q}=iqv-
Dq^{2}+O(q^{3})$ is an exponential rate and $q$ is a phase which can take
values from $-\pi$ to $\pi$. On doing this we find;

$\displaystyle s_{q}=$ $\displaystyle ae^{-iq}+be^{iq}-a+b$ (12)
$\displaystyle=$ $\displaystyle(a+b)(\cos{q}-1)-i(a-b)\sin{q}$ (13)
$\displaystyle=$
$\displaystyle(a+b)(\cancel{1}-\frac{q^{2}}{2}+\frac{q^{4}}{24}...-\cancel{1})-i(a-b)(q-\frac{q^{3}}{6}...).$
(14)

From here it is straightforward to observe that $v=a-b$ and $D=\frac{a+b}{2}$
giving

$\displaystyle
v=(\psi_{rr}^{+}+\psi_{rw}^{+})\mu(r)+(\psi_{wr}^{+}+\psi_{ww}^{+})\mu(w)$
(15)
$\displaystyle-\psi_{rr}^{-}\mu(rr)-\psi_{rw}^{-}\mu(rw)-\psi_{wr}^{-}\mu(wr)-\psi_{ww}^{-}\mu(ww).$

In order to find the stationary probabilities we return to the equation 9 in
the limit where
$P_{t}(m_{1},m_{2},...,m_{l})=\mu(m_{1},m_{2},...,m_{l})p_{t}(l)$. W first
marginalise over all monomers except the tip monomer to find a first equation,
and marginalise over all except the tip and penultimate monomer to find a
second equation, marginalise over all except the tip, penultimate and single
previous monomer for a third, carrying on until all monomers have been
marginalised out. We then subsitute $p_{t}(l)=e^{s_{q}t+iql}$ and take
$q\rightarrow 0$. This gives $l$ equations of which the first three are;

$\displaystyle 0=\sum_{M_{l-1}}$
$\displaystyle\psi^{+}_{M_{l-1}M_{l}}\mu(M_{l-1})+\psi^{-}_{M_{l}r}\mu(M_{l},r)+\psi^{-}_{M_{l}w}\mu(M_{l},w)$
(16) $\displaystyle-\sum_{M_{l-1}}$
$\displaystyle\psi^{-}_{M_{l-1}M_{l}}\mu(M_{l-1}M_{l})-(\psi^{+}_{M_{l}r}-\psi^{+}_{M_{l}w}))\mu(M_{l}),$
$\displaystyle 0=$
$\displaystyle\psi^{+}_{M_{l-1}M_{1}}\mu(M_{l-1})+\psi^{-}_{M_{l}r}\mu(M_{l-1},M_{l},r)+\psi^{-}_{M_{l}w}\mu(M_{l-1},M_{l},w)$
(17) $\displaystyle-$
$\displaystyle\psi^{-}_{M_{l-1}M_{l}}-\psi^{+}_{M_{l}r}-\psi^{+}_{M_{l}w})\mu(M_{l-1}M_{l}),$
$\displaystyle 0=$
$\displaystyle\psi^{+}_{M_{l-1}M_{1}}\mu(M_{l-2},M_{l-1})+\psi^{-}_{M_{l}r}\mu(M_{l-2},M_{l-1},M_{l},r)+\psi^{-}_{M_{l}w}\mu(M_{l-2},M_{l-1},M_{l},w)$
(18) $\displaystyle-$
$\displaystyle\psi^{-}_{M_{l-1}M_{l}}-\psi^{+}_{M_{l}r}-\psi^{+}_{M_{l}w})\mu(M_{l-2},M_{l-1}M_{l}).$

Examining the form of these equations allows us to observe that solutions for
the joint probabilities should be of the form;

$\displaystyle\mu(M_{l-1}M_{l})=\mu(M_{l-1}|M_{l})\mu(M_{l}),$ (19)
$\displaystyle\mu(M_{l-2}M_{l-1}M_{l})=\mu(M_{l-2}|M_{l-1})\mu(M_{l-1}|M_{l})\mu(M_{l}),$
(20)

and if these are replaced into equations 16-18 then the conditional
probabilities can be found as a pair of coupled nonlinear equations;

$\displaystyle\mu(M_{l-1}|M_{l})=\frac{\psi_{M_{l-1}M_{l}}^{+}\mu(M_{l-1})}{(\psi^{-}_{M_{l-1}M_{l}}+\psi^{+}_{M_{l}r}-\psi^{+}_{M_{l}w})\mu(M_{l})-\psi^{-}_{M_{l}r}\mu(M_{l}|r)\mu(r)-\psi^{-}_{M_{l}w}\mu(M_{l}|w)\mu(w)},$
(21)
$\displaystyle\mu(M_{l})=\frac{\sum_{M_{l-1}}\psi_{M_{l-1}M_{l}}^{+}\mu(m_{l-1})+\psi_{M_{l}r}^{-}\mu(M_{l}|r)\mu(r)+\psi_{M_{l}w}^{-}\mu(M_{l}|w)\mu(w)}{\psi_{M_{l}r}^{+}+\psi_{M_{l}w}^{+}+\sum_{M_{l-1}}\psi_{M_{l-1}M_{l}}^{-}\mu(M_{l-1}|M_{l})}.$
(22)

We can also express the mean velocity in terms of our tip and conditional
probabilities as

$\displaystyle v=$
$\displaystyle(\psi_{rr}^{+}+\psi_{rw}^{+})\mu(r)+(\psi_{wr}^{+}+\psi_{ww}^{+})\mu(w)$
(23)
$\displaystyle-\psi_{rr}^{-}\mu(r|r)\mu(r)-\psi_{rw}^{-}\mu(r|w)\mu(w)-\psi_{wr}^{-}\mu(w|r)\mu(r)-\psi_{ww}^{-}\mu(w|w)\mu(w).$

Equation 19 and 20 show that the stationary distribution of sequences is
described by a first order Markov chain running backwards from the tip of the
polymer

$\displaystyle\mu(M_{1}...M_{l-1}M_{l})=\mu(M_{1}|M_{2})\mu(M_{2}|M_{3})...\mu(M_{l-2}|M_{l-1})\mu(M_{l-1}|M_{l})\mu(M_{l}).$
(24)

We now define partial velocities, $v_{r}$ and $v_{w}$. The quantity
$v_{x}\mu(x)$ is the net rate at which monomers are added after an $x$,

$\displaystyle
v_{x}\equiv\psi^{+}_{xr}-\frac{\mu(x|r)\mu(r)}{\mu(x)}\psi^{-}_{xr}+\psi^{+}_{xw}-\frac{\mu(x|w)\mu(w)}{\mu(x)}\psi^{-}_{xw}.$
(25)

By inserting these velocities into equation 21 the conditional probabilities
can be rewritten as

$\displaystyle\mu(m|n)=\frac{\psi_{mn}^{+}\mu(m)}{(\psi_{mn}^{-}+v_{n})\mu(n)},$
(26)

which in turn can be reinserted into the partial velocities to find

$\displaystyle
v_{x}=\frac{\psi^{+}_{xr}v_{r}}{\psi^{-}_{xr}+v_{r}}+\frac{\psi^{+}_{xw}v_{w}}{\psi^{-}_{xw}+v_{w}}.$
(27)

Allowing $x=r,w$ gives a pair of simultaneous equations that can be solved via
the tip and conditional probabilities to find $v_{x}$ in terms of the
propensities. In turn, the velocities and propensities determine tip
probability $\mu(M_{l})$ via

$\displaystyle\mu(n)=\sum_{m}\frac{\psi^{+}_{mn}\mu(m)}{\psi^{-}_{mn}+v_{n}}$
(28)

which follows from multiplying equation 26 by $\mu(n)$, summing over $m$ and
normalising. Conditional probability follows straightforwardly from equation
26. It is therefore possible to solve for the steady state of the tip velocity
by identifying our propensities $\psi^{\pm}_{xy}$, and solving a pair of non-
linear simultanious equations.

####  I.2.2 Final state of the polymer

Gaspard’s method describes the chain while it is still growing through the
stochastic variables $M_{l}$ and $M_{l-1}$, with the index $l$ being the
current length of the polymer. We, however, are interested in the identity of
the monomer at position $n$ when $l\gg n$. We label this “final” state of the
monomer at position $n$ as $M_{n}^{\infty}$. $M_{n}^{\infty}$ is described by
the error probability $\epsilon$ and the conditional error probabilities
$\epsilon_{r}$ and $\epsilon_{w}$, defined as the probability that
$M^{\infty}_{n+1}=w$ given that $M^{\infty}_{n}=r$ or $M^{\infty}_{n}=w$,
respectively.

It might not be immediately obvious why the properties of the growing chain
described by $\mu(M_{l})$ and $\mu(M_{l},M_{l-1})$ should be different from
those of the final chain described by $\epsilon$, $\epsilon_{r}$ and
$\epsilon_{w}$ but the difference can be illustrated with a simple example.
Consider a system in which incorrect monomers could be added to the end of the
chain, but where nothing can be added after an incorrect match. In this case
while the tip probability for an incorrect match µ(w) would be finite, the
error of the final chain $\epsilon$ would be vanishingly small, as all
incorrect matches would have to be removed in order for the polymer to grow
further.

In order to demonstrate that $\epsilon$, $\epsilon_{r}$ and $\epsilon_{w}$ are
sufficient it is necessary to prove that the $M^{\infty}$ is a Markov chain of
$r$ and $w$ monomers. While the model has been designed with Markovian
dynamics, it is not necessarily the case that the produced sequence itself is