rna 
 molecules are not merely simple carriers of genetic information 
 but can assemble into complex tertiary structures and even catalyze 
 reactions .
in fact , the existence of catalytic rna molecules ( ribozymes ) 
 has led to the proposition of the rna world hypothesis . in modern cells ,
rna molecules catalyze just 
 two classes of chemical reactions : modifications of phosphodiester 
 bonds ( dna and rna cleavage , rna splicing ) and peptide bond formation .
artificially designed ribozymes , however , are 
 known to catalyze a wide range of chemical reactions . in some ribozymes , the slow opening and closing of 
 tertiary structure 
 ( rna breathing )
this latter process is hierarchical , first proceeding 
 on the secondary structure level via formation of fairly stable watson  crick 
 base pairs .
subsequently , secondary structure elements fold into a 
 compact , three - dimensional structure .
the folding of rna into 
 the native tertiary fold may proceed via 
 a complex sequence of secondary structures .
formation of compact 
 tertiary structures may require the presence of counterions , particularly 
 divalent cations such as mg , which screen the intrinsic 
 negative charges on the rna phosphate groups and , thereby , stabilize 
 certain tertiary motifs .
even small modifications of single 
 nucleotides may result in different tertiary structures and hence 
 different energy landscapes .
indeed , rna sequence , structure , and function interact in a complex , 
 not yet fully understood fashion , and 
 the characterization of rna folding kinetics , including the pathways 
 of secondary and tertiary structure changes , remains an intricate 
 problem . in this work ,
we have investigated 
 the conformational equilibrium 
 and the folding pathway of the 49mer single - stranded rna ribozyme 
 diels  alderase ( dase ) using a 
 novel hidden markov model ( hmm ) analysis of single - molecule fret data . 

dase catalyzes a diels  alder reaction , i.e. , the [ 4 + 2 ] cycloaddition reaction between anthracene dienes 
 and maleimide dienophiles .
dase is a true multiple - turnover catalyst 
 and shows remarkable enantioselectivity ( > 95% enantiomeric excess ) .
it has a well - defined folded structure , as revealed 
 by x - ray crystallography .
the folded state consists of three helices 
 arranged around a pseudoknot region , in which the catalytic pocket 
 of the ribozyme is located ( figure 1b ) .
a continuous 
 sequence of stacking interactions runs from the bottom of helix ii 
 to the top of helix iii and has been termed the  spine  
 of the folded structure
. the tertiary 
 fold is held together by a pseudoknot , in which the 5-g1-g2-a3-g4 
 segment bridges the unpaired strands of the asymmetric bulge ( figure 1a ) . the precise hydrogen - bond pattern in the pseudoknot 
 region is known to be crucial both for thermal stability of the overall 
 fold as well as for the shape of the catalytic pocket .
recent experimental and computational evidence 
 showed that cations specifically bind to certain sites that stabilize 
 the tertiary fold , without interfering with the catalytic reaction .
low mg concentrations were found to destabilize the 
 folded conformation and 
 to dramatically decrease the catalytic activity of the ribozyme . diels
solid lines , secondary structure 
 base pairs ; dotted lines , tertiary structure base pairs .
attachment 
 sites of the fret labels are marked by green ( donor dye cy3 at u6 
 in construct i and at the 5 end in construct ii ) and orange 
 ( acceptor dye cy5 at u42 in construct i and at u30 in construct ii ) 
 arrows .
color - coding 
 of the secondary structure elements as in panel a. attachment sites 
 of the fret labels are indicated by green ( cy3 , donor ) and red ( cy5 , 
 acceptor ) spheres .
( the figure has been adapted from figure 1 in ref ( 9 ) . )
single - molecule frster resonance energy transfer 
 ( smfret ) 
 is a powerful tool to follow conformational fluctuations of biomolecules 
 on length scales of a few nanometers in real time .
smfret measurements with surface - immobilized molecules revealed 
 that dase is highly dynamic and can exist in substantially different 
 conformations , which were found to interconvert on time scales of 
 hundreds of milliseconds .
the concentration 
 of mg influences the shape or population of the accessible 
 conformational states , as indicated by the mg dependence 
 of the fret efficiency histograms and the apparent folding rates .
consistent with conformational fluctuations , 
 a poor resolution of dase spectra was found in subsequent nmr studies .
the mg - dependent fret efficiency 
 histograms revealed at least two conformational ensembles : ( i ) a high 
 fret state , attributed to the folded conformation , whose population 
 increases with increasing mg concentration , and ( ii ) 
 a distribution of intermediate fret efficiencies , whose population 
 decreases with increasing mg concentration .
the intermediates 
 were observed to spread out over a wide range of fret efficiency values 
 and , presumably , comprise multiple conformations with different secondary 
 and tertiary structures . in practice , 

only two or three states with significantly different 
 fret efficiencies can be distinguished in a histogram - based analysis .
consequently , stochastic fluctuations of the number of photons within 
 a time bin ( shot noise ) significantly contribute to the widths of 
 the fret distributions and prevent the separation of states with similar 
 mean fret efficiencies .
it completely 
 neglects the time sequence of events in the single - molecule trajectories 
 and , thus , discards a substantial part of the available information . 

in contrast , hidden markov models can 
 distinguish states in the data by using both the differences in fret 
 efficiency and the time sequence of events , and , thus , can decompose 
 states with similar fret efficiencies but different kinetic properties . 

recent studies on single - molecule protein and rna data sets have demonstrated the power of hmms to resolve a multitude of states . 

hmm analysis has its intrinsic challenges , however , because ( i ) the 
 results depend on the number of states used , ( ii ) the hmm optimization 
 may get stuck in local minima , ( iii ) models with many states are difficult 
 to validate , and ( iv ) the quality of the model depends crucially on 
 the validity of the underlying likelihood function ( i.e. , the stochastic 
 model of the measured process ) . here
, we present an hmm analysis scheme 
 that addresses these problems . at the core of this scheme is 
 the idea that the number of states 
 required to describe the kinetics in a hierarchical energy landscape 
 is not fixed but depends on the time scales of interest ( figure 2 ) . directly estimating hmms with 
 a few states often yields wrong kinetics , as they tend to prefer models whose states have clearly different 
 fret efficiency .
however , in real data , distinct and slowly interconverting 
 conformations may have strongly overlapping fret efficiency distributions , 
 which are difficult to separate .
therefore , we construct an initial 
 hmm with many states ( corresponding to a fine discretization of conformational 
 space ) . the initial number of states is determined by a validation 
 scheme , which tests reproducibility and consistency of the model with 
 the underlying data set .
the initial states are subsequently coarse - grained 
 on the basis of their kinetics .
this approach allows 
 us to model ( coarse ) states even when they strongly overlap in their 
 fret efficiencies and have very irregular ( e.g. , non - gaussian ) fret 
 distributions .
our hmm uses a poissonian likelihood function to model 
 the physical process of photon emission .
this approach 
 is preferable over using gaussian likelihood functions of the fret 
 efficiency . for a detailed discussion , see the supporting 
 information .
in addition , we have developed an approach to 
 account for the trace - specific background noise .
( a ) hierarchical 
 free energy landscape with various minima ( conformations ) interconverting 
 on different time scales .
( b ) a fret efficiency versus distance curve , 
 with the five conformations in panel a assigned to certain fret efficiencies 
 ( distances ) .
conformations with suitably long lifetimes can be distinguished 
 by hmm analysis of fret traces but may have overlapping fret efficiencies 
 even when they are distinct .
( c ) probability density function of finding 
 the system at a certain value of the distance parameter .
( d ) the states 
 found in the hmm analysis are depicted as disks located in a two - dimensional 
 space of efficiency ( abcissa ) and lifetime ( ordinate ) .
( e , f ) some 
 states kinetically merge on longer observation time scales indicated 
 by the blue and red areas in panels e (  = 10 ms ) and f (  
 = 100 ms ) .
for example , state pairs ( i , ii ) and ( iv , v ) each merge 
 into a single apparent state for times longer than 10 ms .
independent hmm analyses were carried out on two 
 differently labeled 
 dase constructs , referred to as constructs i and ii . altogether ,
four 
 different data sets were analyzed ( dase construct i at mg concentrations of 0.0 , 5.0 , and 40.0 mm and dase construct ii at 
 mg concentration 5.0 mm ) , yielding hmms with seven to 
 nine conformational states . these hmms provide comprehensive models 
 of the dynamics on millisecond time scales .
we also determined relaxation 
 times , identified the associated conformational transitions by an 
 eigenvector / eigenvalue analysis of the transition matrix , and computed the ensemble of rna folding pathways . on the basis of their kinetics
, the original 
 states were lumped together to effective five - state ( on time scales 
 of tens of milliseconds ) and three- or four - state models ( on time 
 scales of hundreds of milliseconds ) .
most notably , we identified consistent , 
 characteristic features of the kinetic network of dase in all four 
 data sets . to the best of our knowledge ,
these results represent the 
 most detailed rna folding models obtained from single - molecule measurements 
 to date .
furthermore , they reveal that the transition rates in this landscape 
 change substantially as the mg concentration is varied , 
 while the general topology of the landscape ( position of minima , relative 
 height of energy barriers ) is not affected . at all mg concentrations , the observed kinetic processes can be attributed 
 to either secondary or tertiary structure rearrangements .
by using a combinatorial strategy , we had earlier 
 synthesized a set of nine dase fret constructs with dyes attached 
 at different nucleotide positions .
construct 
 i was chosen for in - depth studies because it showed the most pronounced 
 changes in its fret histogram with varying mg concentration . 

here we have also performed surface - immobilized measurements on a 
 second variant , construct ii , because ( 1 ) its fret histogram was multimodal , 
 suggesting that multiple states could be distinguished by the hmm , 
 and ( 2 ) it was not too different from construct i and , therefore , 
 could serve for validation ( see below ) .
single - molecule fluorescence 
 time traces of surface - immobilized dase were obtained for construct 
 i ( cy3 at u6 and cy5 at u42 ) at mg concentrations of 
 0 , 5 , and 40 mm and for construct ii ( cy3 at the 5 end and 
 cy5 at u30 ) at a mg concentration of 5 mm .
details on 
 the data , the experimental procedures , and the effects of surface 
 immobilization are included in the supporting 
 information , tables s1 and s2 and figures s2 and s3 .
for each 
 trace , the rates of the background noise , ka , bg an kd , bg , in the acceptor and donor 
 channel , respectively , as well as the amount of spectral crosstalk , 
  , from the donor into the acceptor channel were estimated , 
 as described in the supporting information .
we have developed an 
 hmm analysis and associated optimization algorithms for single - molecule 
 fret .
the hmm analysis scheme has the following features:the hmm works 
 with discrete photon 
 counts , which are assumed to obey poissonian statistics ( figure s4 , supporting information).background noise levels of measured 
 photon traces are taken into account explicitly by employing an appropriate 
 emission probability.the reproducibility of the hmms 
 is tested.the 
 number of states of the hmm 
 is maximized under a number of constraints , which ensures that the 
 model reproduces physically and chemically relevant quantities.the final hmm represents 
 a fine 
 discretization into states that , depending on the time scale , are 
 lumped into larger states according to kinetic proximity .
the hmm works 
 with discrete photon 
 counts , which are assumed to obey poissonian statistics ( figure s4 , supporting information ) .
background noise levels of measured 
 photon traces are taken into account explicitly by employing an appropriate 
 emission probability .
the 
 number of states of the hmm 
 is maximized under a number of constraints , which ensures that the 
 model reproduces physically and chemically relevant quantities .
the final hmm represents 
 a fine 
 discretization into states that , depending on the time scale , are 
 lumped into larger states according to kinetic proximity .
a workflow diagram of the hmm analysis scheme 
 is shown 
 in figure 3 .
the algorithms are described in 
 full detail in the supporting information , and the salient characteristics of the workflow are discussed in 
 the following sections .
workflow diagram used for our hmm analysis of 
 single - molecule fret 
 data .
consider the hypothetical energy landscape with five 
 minima in panel a in figure 2 .
each minimum 
 corresponds to a conformational state and is associated with a mean 
 fret efficiency , ei ( figure 2b ) , and a fractional population in equilibrium , 
 i ( figure 2c ) , where i denotes the number of the state . using 
 hmm analysis , these five states can be extracted from smfret traces 
 of a molecule diffusing in this free energy landscape .
we represent 
 the main characteristics of the states of the hmms by scatter plots 
 ( figure 2d ) : each state is marked by a disc , 
 the position of which encodes the mean fret efficiency of the state 
 and its lifetime , i .
the area of 
 the disc is proportional to the stationary probability i of the state , as computed from the hmm .
the hmm transition matrix has eigenvalues corresponding to time scales 
 of transitions and eigenvectors denoting states that interconvert 
 on these time scales .
, states i and ii interconvert on time scales of 10 ms ( figure 2a ) .
thus , when computing a fret histogram with an 
 averaging window much longer than 10 ms , these two states merge into 
 a single apparent state .
likewise , states iv and v kinetically merge 
 for time scales longer than 10 ms , as depicted by the red and blue 
 regions in figure 2e .
states i and ii kinetically 
 merge with state iii for time scales above 100 ms ( figure 2f ) .
complete equilibration occurs for times longer 
 than 1 s. in a high - dimensional energy landscape , kinetic merging 
 may not 
 necessarily involve only neighboring states along the fret efficiency 
 axis .
in fact , high fret efficiency states can merge kinetically with 
 low fret efficiency states even if there are states with intermediate 
 fret efficiencies in between .
hidden markov models 
 ( hmms ) are stochastic 
 models ,  = ( t , e ) , of the observed 
 ( measured ) trace , o = ( o1 , ... , on ) , with oi = ( na , i , nd , i ) containing the number of photons observed in the acceptor 
 and donor channels at each time step i. in the construction 
 of hmms , it is assumed that the observation is generated by a hidden 
 markov chain with transition matrix t , whose states represent 
 regions in the conformational space of the molecule . at every time 
 step in the markov chain , an additional stochastic process ,
the emission probability , , describes 
 the conditional probability 
 of observing the signal , oi , given that the molecule is currently in conformation ( hidden 
 state ) si .
one typically 
 chooses the same functional form of   for all 
 hidden states but uses a parameter ej to adapt it to a specific 
 hidden state .
the parameters ej form a vector e and are part of the model . 
 the hmm optimization problem maximizes the likelihood ( i.e. , the conditional 
 probability of observing the measured trace o , given 
 that the molecule is accurately described by the model  = ( t , e)):1over all values of ( t , e ) and all possible 
 hidden paths . for a given number of states , n , the 
 model  consists of an n  n transition matrix , t , and of a vector of 
 observation parameters e , of length n. hmm classes differ by the way that the hidden process and the measurement 
 process are modeled and by the way that the corresponding parameters 
 are optimized .
it is crucial to choose an 
 emission probability , , that 
 models the measurement process as 
 accurately as possible .
the arrival times of the photons are assumed to obey 
 poissonian statistics , which is validated in figure s4 ( supporting information ) .
n ) is a poisson distribution of variable n with rate 
 coefficient k. the acceptor and donor photon count 
 rates , ka and kd , are given as3where ei is the apparent fret efficiency 
 of the current hidden state si and kmol is the detection rate 
 of photons emitted by the labeled 
 molecule ( through either the donor or the acceptor ) .
a problem inherent in the experimental data is the presence 
 of trace - dependent background noise , which may cause identical conformational 
 states to display different apparent fret efficiencies in different 
 time traces .
the trace specific background rates , ka , bg and kd , bg , can be estimated 
 from the bleached phase of the measured photon traces . given these 
 rates , we derive a likelihood of observing ( na , nd ) photons during a time step , 
 t , in the acceptor and donor channels , respectively 
 ( see the supporting information ) .
the emission 
 probability has the functional form given in eq 2 , but the photon count rates are now given as4we assume that background noise may vary from 
 trace to trace but that all other measurement errors , including spectral 
 cross - talk and differences in the quantum yield of the chromophores , 
 depend on the conformational state but are identical for different 
 traces . then , e contains the apparent fret efficiencies 
 ( without background noise ) of the hidden states .
these apparent fret 
 efficiencies can be corrected for spectral cross - talk a posteriori 
 to obtain the true fret efficiencies ( see the supporting information ) .
hmm optimization is done by using 
 the expectation - maximization 
 algorithm , which finds a local maximum of   from an initial guess of the parameters 
 ( t , e ) . to facilitate finding the global 
 optimum ,
the hmms presented here are obtained by first running 100 
 explorations that optimize random starting values of ( t , e ) for a few steps only .
subsequently , the parameter 
 set with the largest likelihood is optimized to full convergence . 

nonetheless , the hmm algorithm might find different local maxima for 
 different initial parameters .
hence , for each mg concentration , 
 we compute 10 hmms in the described way to test for reproducibility . 

two hmms are accepted as identical if their log - likelihoods differ 
 by less than 1.0 . by a heuristic criterion
, an hmm optimization is 
 reproducible if identical maximum likelihood hmms are found in at 
 least 2 out of the 10 trials .
the number of states , n , is an input parameter for the hmm optimization algorithm . 

as argued in the supporting information , information - criteria - based choices of the number of states are 
 inadequate for the present data . to determine the number of hidden 
 states , we instead adopt a viewpoint for the construction of direct 
 markov models that is well established in the community : rather than finding the  ideal  
 number of states to statistically classify the data
consequently , the resulting 
 discretization of state space will be fine enough that the hmms can 
 reproduce the stationary and long - time kinetic behavior of the data . 

the resulting states can subsequently be grouped according to kinetic 
 connectivity given by t , as described in refs ( 31 and 32 ) and illustrated in figure 2 . following this approach , we build hmms for a varying 
 number of states , n = 2 , 3 , ... , and choose the largest 
 number of states for which hmms can be constructed reproducibly .
different tests were 
 used to check whether the hmms are consistent with the data set from 
 which they were parametrized , and whether the hidden paths obtained 
 from the hmms are consistent with markovian dynamics .
the consistency 
 of the hmm with the underlying data set was tested by comparing fret 
 efficiency histograms obtained from the data with histograms estimated 
 from the hmms . for this test , we used time windows between 10 and 
 100 ms .
as previously discussed , this 
 approach tests both the stationary and kinetic properties of the model . 

the data - based distributions were obtained using the 
 likelihood from eq 2 , as described in the supporting information .
the hmm - based distributions 
 were obtained by sampling hidden trajectories of the time window length 
 from an equilibrium distribution , and then generating artificial photon 
 counts using poisson statistics with the appropriate output rates 
 ( figure 4a and figures s6a , s7a , and s8a , supporting information ) .
the markovianity of 
 individual states was tested by inspecting their lifetime distributions , 
 which can be computed from the maximum - likelihood hidden paths , (t ) , of the hmm .
a single exponential 
 decay in these distributions is consistent with markovian dynamics 
 ( figure 4b ) .
states that failed this test were 
 split using a newly developed bayesian model selection algorithm ( supporting information ) .
the overall markovianity 
 of the hmms was tested using the implied time scales test that is frequently used for simulation - based 
 markov state models . to this end , the relaxation time scales , tihmm = t / ln ihmm , were computed , where ihmm are the eigenvalues of the hmm 
 transition matrix t. these are compared to the implied 
 time scales of a markov model t( ) constructed 
 from the maximum likelihood hidden paths , (t ) , for different lag times
. if the overall 
 dynamics is markovian , these time scales should be independent of 
 the lag time  used to compute them , hence yielding constant 
 functions in figure 4c . as an additional test , 
 they should agree with the hmm time scales , tihmm .
figure s9 ( supporting information ) shows 
 fret traces colored according to the hidden states in the final model .
( a ) dependence of the fret 
 efficiency histograms on the lengths of the time windows ( 10 , 50 , 
 and 100 ms ) . dashed colored lines , prediction from the hidden markov 
 model ; gray areas / dotted black lines , estimation from the smfret data 
 set ( bootstrapping mean/95% confidence interval ) .
( b ) lifetime distributions 
 of the individual states calculated from the maximum - likelihood paths . 

( c ) implied time scales , indicating that the long - time 
 kinetics of the hidden paths is markovian and converges to time scales 
 similar to those found in the hmm .
the divergence of the shortest 
 time scales at larger lag times is expected and due to numerical problems .
we analyzed 
 three sets of smfret traces of dase construct i ( chromophores attached 
 to residues 6 and 42 , see figure 1 ) , measured 
 at different mg concentrations , 0.0 , 5.0 , and 40.0 mm . 

background - corrected fret efficiency distributions were calculated 
 from these data sets by using the likelihood ( eq 2 ) and a bootstrapping procedure to estimate the uncertainty in the 
 data ( dotted gray lines and gray areas in figure 4a ) .
two ensembles of states can be 
 visually distinguished : a broad intermediate state in the fret efficiency 
 range 0.40.8 and a putative native state at efficiency values 
 of 0.91.0 . with increasing mg concentration , 
 the populations shift to states with high fret efficiency . in ref
( 9 ) , it was already hypothesized 
 that the broad ensemble at intermediate fret efficiencies may consist 
 of multiple conformational states with overlapping fret efficiency 
 distributions .
hmms were constructed for the smfret data 
 sets as described in 
 the materials and methods section .
the largest 
 number of states for which hmms could be reproducibly obtained was 
 eight ( 0 mm mg ) , eight ( 5 mm mg ) , and seven 
 ( 40 mm mg ) states , where we used the optimization protocol 
 described in the materials and methods section . 

the eight - state models for 0 and 5 mm mg passed the validation 
 test ( figure 4 ) .
a single , weakly populated 
 state with fret efficiency e  0 , which was 
 assigned to an acceptor blinking state , was removed from these models 
 a posteriori .
the seven - state model at 40 mm mg required 
 an intermediate step , in which non - markovian states were split and 
 regrouped according to kinetic proximity , yielding a nine - state model . 

( see the supporting information for a detailed 
 description of the protocol employed . ) to test whether the remaining 
 nonexponentiality came from an actual non - markovianity of the discrete 
 state dynamics or just from spurious transitions generated from the 
 estimation of the maximum likelihood , we conducted the implied time 
 scale test as described in the materials and methods section .
the results shown in figure 4c demonstrate 
 that the maximum likelihood hidden paths , (t ) , are non - markovian in all models at short time 
 scales but then converge to approximately constant time scale estimates 
 at lag times of 1030 ms .
the time scales agree with the time 
 scales estimated from the hmm transition matrix , indicating that the 
 kinetics of all three hmms are consistent with the data .
note 
 that the hmms for the three different mg concentrations 
 were constructed independently of each other .
therefore , when similar 
 or consistent features are found across all three mg concentrations , 
 this is a two - fold validation of an observation .
the scatter 
 plots in figure 5a ( upper row ) show the main 
 characteristics of the ( hidden ) states of the hmms : each state is 
 represented by a disc whose position indicates the mean fret efficiency 
 of the state and its lifetime i = t / ln tii , where t is the time step 
 of the hmm transition matrix and tii are the diagonal elements of this matrix .
the area of the 
 disc is proportional to the stationary probability i of the state as computed from the hmm .
the states 
 that consistently appear in construct i at different mg concentrations are depicted in the same color ( i.e. , black , blue , 
 red , and green states ) .
the purple state at 0.0 mm mg could not be matched to any state at higher mg concentrations . 

conformational states and subensembles found by the hmm analysis 
 of construct i and construct ii .
( a ) first row : state parameters of 
 the hidden markov models which are for each state i : the fret efficiency ei ( abcissa ) , the state lifetime i ( ordinate ) , and the equilibrium population i ( disc size ) .
second and third rows : state decomposition for 
 time scales of 10 and > 100 ms .
( b ) fret histograms of the subensembles 
 of the states shown in the second row of panel a. a feature found for all mg concentrations is 
 the black 
 high - fret efficiency state .
it has a relatively small stationary probability 
 but a long lifetime at all mg conditions .
the region 
 of intermediate fret efficiencies is populated mostly by short - lived 
 states ( blue , red ) , and a few long - lived states with low fret efficiencies 
 ( green ) .
remarkably , the states appearing at multiple mg concentrations 
 show only rather subtle changes .
there are two cooperative effects 
 upon mg increase : ( i ) all states shift to slightly higher 
 fret efficiencies , indicating that mg causes these conformations 
 to become more compact , ( ii ) the intermediate - efficiency purple state 
 is depopulated with increasing mg , while some substates 
 with higher fret efficiencies ( light red state , which is split into 
 an orange and a dark red state at 40 mm mg , as well as 
 the dark blue state ) become more populated at high mg concentrations .
the populations of the other red and blue states , 
 as well as the black state , show surprisingly little dependence on 
 the mg concentration , indicating that the associated 
 conformations do not experience stabilization by mg ions .
to better understand the nature of the conformational states of 
 the hmms , we have investigated their kinetics .
detailed information 
 is presented by the networks plotted in figures s10s13 ( supporting information ) .
alternatively , an eigenvector / eigenvalue 
 analysis of the transition matrix t allows conformational 
 states interconverting faster than the time scale of interest to be 
 grouped ( figures s10s13 , supporting information ) .
the second row of figure 5a shows a striking feature found independently for the hmms 
 at all mg concentrations : at a few tens of milliseconds , 
 the substates of the red subensemble as well as the substates of the 
 blue subensemble interconvert .
consequently , kinetic proximitiy and 
 proximity on the fret axis are , in general , unrelated properties . 

this finding is emphasized by the fret efficiency histograms of the 
 corresponding subsembles in figure 5b , which 
 were constructed by partitioning the photon traces according to the 
 associated hidden states .
the blue and red subensembles are doubly 
 peaked because they are composed of multiple hidden states .
in addition , 
 these subensembles overlap strongly , clearly showing why the present 
 single - molecule fret data were difficult to model kinetically , and 
 emphasizing the usefulness of a detailed hmm analysis for dissecting 
 them . for all mg concentrations ,
the high - efficiency 
 peak 
 in the fret histograms of the blue subensemble overlaps with the high - efficiency 
 black state , indicating that the high - fret - efficiency peak identified 
 in ref ( 9 ) consists 
 of two states , one of which rapidly interconverts with a state of 
 intermediate fret efficiency ( blue ) and is stabilized by mg , and a long - lived high - efficiency state ( black ) , which is insensitive 
 to mg . in figure 5a
, the third 
 row shows that , on time scales of a few hundred ms , the long - lived 
 state ( black ) interconverts with the blue subsemble .
the mixing time 
 for all subensembles is on the order of seconds ( see figure 6 ) .
these results indicate the presence of a hierarchical 
 energy landscape , with different processes occurring on very different 
 time scales , ranging from a few milliseconds to 1 s. free energy landscape 
 and folding pathways .
states are indicated 
 by bars or discs with the same colors used in figure 5 .
gray bullets indicate transition 
 states facilitating that states or sets of states kinetically merge 
 at longer time scales .
( b ) the complete ensemble of folding pathways from the least 
 compact states ( green / yellow ) to the most compact state ( black ) .
the 
 states are positioned depending on their mean fret efficiency ( abcissa ) 
 and the probability of folding ( committor , q , ordinate ) .
the thickness of an arrow is proportional to 
 the probability that a green / yellow state will fold along this pathway . on the basis of the processes 
 depicted in figure 5 , we find fast interconversion 
 between the  open  
 ( e  0.5 ) and  closed  ( e > 0.7 ) states within the blue and red subensembles , 
 while 
 the exchange dynamics between these subensembles happens much slower . 

we propose that the states within each subensemble ( with a given color 
 in figure 5 ) have similar secondary structures 
 yet different tertiary structures , interconverting rapidly without 
 breaking large strands of watson  crick base pairs .
this proposition 
 is supported by the fact that , at high mg concentrations , 
 the compact parts of the red and blue subensemble are stabilized . 

different subensembles are proposed to correspond to different secondary 
 structures because they are long - lived , suggesting that the stable 
 watson 
crick base pairs need to be broken in order to transit 
 to another subensemble .
the connectivity between different subensembles 
 ( and , thus , presumably different secondary structures ) is similar 
 at all mg concentrations .
the high - efficiency ( black ) 
 state is connected to the blue subensemble in the presence of mg ( 5 and 40 mm ) directly and , at 0 mm mg , via 
 the purple intermediate .
figure 6a illustrates this connectivity , 
 and the free energies of these conformations as well as the transition 
 states ( see the materials and methods section ) .
this connectivity suggests an ordering of subensembles from the 
 least compact ( lowest fret efficiencies ) to the most compact ( highest 
 fret efficiencies ) which can be found at all mg concentrations : 
 ( 1 ) green , ( 2 ) red , ( 3 ) blue , and ( 4 ) black .
the fact that they have high 
 lifetimes and fret efficiencies that are much greater than zero suggests 
 that they still have some secondary structure , although probably not 
 the native one . they are therefore called  misfolded  .
this ordering suggests to study the transition pathways from the 
 misfolded states ( green ) to the most compact state ( black ) .
transition 
 path theory provides the basis for calculating 
 the pathways between two subensembles .
we use the protocol and equations 
 described in ref ( 36 ) employing the implementation in the emma software .
a transition pathway is defined as a series of transitions 
 that lead from the misfolded to the native state without returning 
 to the misfolded state .
figure 6b locates the 
 states by their fret efficiency , and by the committor value ( vertical 
 axis ) , i.e. , the probability of the system , when being in this state , 
 to move  forward  and fold toward the black state , rather 
 than misfold back to the green state .
the committor value q = 0.5 designates states in which the molecule 
 is equally likely to go either way .
note that there is a continuous 
 shift of these transition states with increasing mg concentration . 

at 0 mm mg , the transition state lies between the green 
 and red subensembles .
once a molecule has reached the red subensemble , 
 it is likely to continue folding to the black state . with increasing 
 mg concentration ,
the red and blue subensembles become 
 more and more kinetic intermediates , and lie at committor values around 
 0.5 for 40 mm mg .
figure 6b shows the probability fluxes of 
 transition pathways from misfolded states to the folded state .
the 
 size of the arrows indicates the probability flux , which is related 
 to the folding rate . without mg ,
the folding rate kab is about 0.09 s , and increases to 0.28 s for 5 mm and 0.17 s for 40 mm mg .
the strong increase in 
 folding rate from 0 to 5 mm mg is mainly due to a lowering 
 of the transition state energy , while the decrease in folding rate 
 from 5 to 40 mm mg is mainly due to an increased stability 
 of the dark blue intermediate state ( compare figure 6a and b ) .
moreover , it is apparent that addition of mg increases the number of accessible pathways , making the 
 folding process more parallel .
two main mechanisms are observed at 
 all mg concentrations : a compact folding mechanism , in 
 which the green misfolded state refolds via the higher fret efficiency 
 substates of red and blue toward the black state , and an  close  open  close  
 mechanism , in which the green state folds via the open substates , 
 or via successive closing , opening , and closing , i.e. , involving tertiary 
 unfolded states .
both types of pathways have similar weights , with 
 some preference for close  open 
close pathways at low 
 mg concentrations and a slight preference for compact 
 pathways at high mg concentrations . to further confirm 
 our findings
, we performed a fourth independent 
 measurement on a dase ( construct ii ) with a different set of label 
 positions .
if they do not introduce major energetic conflicts , 
 the state probabilities , time scales , and the kinetic connectivity 
 should remain comparable .
single - molecule fret data were recorded , 
 and an hmm was computed using the same approach as above .
a seven - state 
 model was found to pass the validation test ( see figure s14a and s14b , supporting information ) .
open  states at efficiencies 
 of 0.40.6 and high - fret ,  closed  states at 
 efficiencies above 0.8 .
as for construct i , two pairs of rapidly interconverting 
 states , each with a low- and a high - fret state , were found . additionally , 
 a single stable state with high efficiency was also identified .
consequently , 
 the red , blue , and black subensembles of construct ii match the corresponding 
 subensembles in construct i and , thus , can be identified in all experimental 
 data with high confidence ( see figure 5a ) .
moreover , the time scales found in constructs i and ii are in qualitative 
 agreement ( see figure s14c in the supporting information ) . open and closed states of the red and blue subensembles interconvert 
 at time scales of a few milliseconds ( 10 ms in construct i , 
 3 ms in construct ii ) . at time scales of 100 ms to seconds , ( i ) the 
 blue ensemble merges with the black state and ( ii ) the red and blue 
 ensembles kinetically merge . at low mg concentrations , 
 the blue
 black interconversion is several 100 ms faster than 
 the blue  red interconversion , while , at 40 mm mg , the two processes happen at about the same time scales ( figure 
 s14c , supporting information ) .
the 
 gray states in construct ii and the green / yellow states in 
 construct i do not have clear corresponding states in the other construct . 

for example , the presence 
 of a label in a particular position may prevent certain structures 
 from forming . in the following discussion
, we will thus concentrate 
 on those states that can be safely matched across all data sets ( red , 
 blue , and black ) .
note that , due to the reduced state lifetimes 
 in construct ii , 
 the partitioning of the photon traces resulted in subtraces which 
 were too short for a histogram analysis .
hence , the subensemble fret 
 histograms could not be generated ( see figure 5b ) .
a kinetic pattern is found 
 consistently for different mg ion concentrations and 
 for different attachment points of the chromophores : 
 ( i ) a long - lived , high - fret - efficiency state ( black ) , ( ii ) two ensembles 
 of states ( red , blue ) comprising rapidly interconverting open and 
 closed states , the ratio of which depends on mg , and 
 ( iii ) a linear connection between the three subensembles ( red , blue , 
 black ) .
the long interconversion times along this linear connection 
 suggest that these transitions involve breaking and reforming of watson 
crick 
 base pairs . to investigate whether there are secondary structures 
 consistent 
 with this pattern
, minimum energy secondary structures of the dase 
 were calculated using the vienna rna webserver ( see 
 the supporting information ) .
the algorithm 
 correctly identified the secondary structure of the known folded state 
 ( excluding the pseudoknot connectivity ) as the lowest free - energy 
 structure .
two alternative secondary structures with low free energies 
 ( g < 1.4 kj / mol above native , i.e. , accessible 
 at room temperature ) were also identified .
these structures ( labeled 
 2 and 3 ) are shown along with the secondary structure of the folded 
 state ( labeled 1 ) in figure 7 .
although they 
 are very close in energy and structurally very similar to each other , 
 structures 2 and 3 differ from structure 1 in that helix ii is broken 
 and helix i is prolonged by two base pairs .
all other secondary structures 
 identified by the algorithm had estimated free energy differences 
 of g > 9.5 kj / mol with respect to structure 
 1 . proposed folding mechanism .
the red set of states 
 has a non - native secondary structure but includes both open ( low - fret ) 
 tertiary structures and compact ( high - fret ) tertiary structures .
the 
 blue set of states has the native secondary structure but also includes 
 both open and compact tertiary structures .
in contrast to the compact 
 blue state , it is additionally stabilized by the tertiary watson 
crick 
 pairs that form the pseudoknot . in the absence of stabilizing tertiary interactions , secondary 
 structures 1 , 2 , and 3
facilitate transitions between open and compact 
 states , associated with large fluctuations in the donor 
therefore , they have properties matching 
 those found for the blue and red subensembles in the hmm analysis . 
 the black state displays exclusively high fret efficiencies in all 
 constructs under all conditions and
its long lifetime and the 
 fact that its population does not vary strongly with the mg concentration suggest that it is stabilized by base - pairing rather 
 than mg ions . therefore , we propose that the black state 
 represents the tertiary folded structure including the pseudoknot 
 topology .
the pseudoknot base pairs ( g1c26 , g2c25 , 
 a3u45 , g4c44 ; see figure 1 ) 
 are consistent with stable interactions that do not depend on mg .
their formation stabilizes an already compact structure 
 with the correct secondary fold so as to acquire a well - defined tertiary 
 structure .
this proposal is supported by computer simulations which 
 show that the active site of dase is distorted if mg is 
 removed ( explaining the loss in catalytic activity ) but the overall 
 lambda - shaped tertiary structure stays intact . since the blue subensemble acts as a precursor to 
 the black tertiary 
 folded structure ( linearly connected folding path , figure 6 ) , it is only logical to match the blue state with 
 the secondary structure of the folded state ( structure 1 ) .
the native 
 secondary structure still facilitates extended and compact states . 
 like the fully native black state ,
the high - efficiency blue states 
 are compact and possess the correct native secondary structure , but 
 in contrast to the black state , they lack the pseudoknot base pairs , 
 which stabilize the native tertiary fold .
consistently , the probability 
 of extended versus compact blue states depends on the concentration 
 of mg ions that are required to stabilize the compact 
 state in the absence of tertiary base pairs .
consequently , the 
 red ensemble contains structures 2 and/or 3 , 
 i.e. , extended and compact states with non - native secondary structure . 

the proposed assignment is 
 consistent not only with the kinetic 
 connectivity and the mg - dependent equilibrium populations 
 but also with the observed time scales .
the fluctuation between open 
 and compact conformations within the blue and red ensembles involves 
 no or little secondary structure change , consistent with relatively 
 short transition time scales ( figures 5 and 6 ) .
in contrast , a transition from the red to the 
 blue subensemble involves rupture of watson  crick pairs , which 
 is consistent with slower transition time scales of hundreds of milliseconds 
 ( figures 5 and 6 ) .
likewise , 
 the change of tertiary base - pairing is consistent with long transition 
 time scales between the blue and black states and the long lifetime 
 of the black native state .
the kinetic model found here and 
 our proposed folding mechanism 
 exhibits a number of features consistent with previous findings or 
 hypotheses for other rna systems . in particular , secondary and tertiary 
 structure formation has been proposed to be kinetically decoupled , 
 such that secondary structure elements can exist without further stabilization 
 by specific tertiary interactions . for 
 the tetrahymena thermophila ribosome , metastable 
 structures with a partially misfolded secondary structure
have been 
 described , lending credibility to the present assignment of the red 
 subensemble to structures 2 and/or 3 .
in addition , other rnas 
 have been proposed to fold via multiple parallel pathways . to the best of our knowledge
, 
 we have presented the most detailed 
 experimentally derived model of an rna folding mechanism , providing 
 a kinetic model connecting different secondary and tertiary stabilized 
 structures , and showing how they are orchestrated during the folding 
 pathways .
the multitude of time scales found in the data provide direct 
 evidence that the rna folding landscape is hierarchical and that secondary 
 and tertiary structure formation occur on different time scales .
the 
 techniques described here also facilitate detailed kinetic models 
 to be derived for other macromolecular systems . as yet , the 
 field is still lacking an experiment that could simultaneously 
 resolve kinetics and the structures of the individual states in detail . 
 unfortunately , computational approaches can not step in here . with 
 folding times on the order of seconds ,
the dynamics are as yet out 
 of reach for direct molecular dynamics ( md ) simulation . over time , 
 however , enhanced sampling strategies may help access these processes
. however , molecular modeling and md simulation 
 may be useful for exploring the local dynamics within individual states , 
 and by using new biophysical techniques , the distribution of measurable 
 fret values can be computed and compared to the subensemble distributions 
 shown in figure 5b . on the experimental side , using multicolor - fret or the systematic reconciliation of multiple dual - color - fret 
 experiments may provide distance constraints 
 to resolve the structures in more detail .
finally , the combination 
 of fret and site - specific fluorescence quenching may also be employed 
 to disentangle the tertiary dynamics from secondary structure formation .
 we 
 have developed a hidden markov model and optimization procedure 
 for photon - based single - molecule fret data , which takes into account 
 the trace - dependent background intensities . 
 this analysis technique 
 reveals an unprecedented amount of detail in the folding kinetics 
 of the diels  alderase ribozyme . 
 we find a multitude of extended 
 ( low - fret ) and compact ( high - fret ) states . 
 five states were consistently 
 and independently identified in two fret constructs and at three mg2 + concentrations . structures generally tend to become more 
 compact upon addition of mg2 + . 
 some compact structures 
 are observed to significantly depend on mg2 + concentration , 
 suggesting a tertiary fold stabilized by mg2 + ions . 
 one 
 compact structure was observed to be mg2 + -independent , 
 consistent with stabilization by tertiary watson  
 crick base 
 pairing found in the folded diels  
 alderase structure . 
 a hierarchy 
 of time scales was discovered , including dynamics of 10 ms or faster , 
 likely due to tertiary structure fluctuations , and slow dynamics on 
 the seconds time scale , presumably associated with significant changes 
 in secondary structure . 
 the folding pathways proceed through a series 
 of intermediate secondary structures . there 
 exist both compact pathways 
 and more complex ones , which display tertiary unfolding , then secondary 
 refolding , and , subsequently , again tertiary refolding . 
Introduction
Materials and Methods
Results and Discussion
Discussion