title: Easier way to create time inhomogeneous Markov Chain
body: <p>I have a series of observations of a machine that can be in different states. I
  want to create a multi state model where the survivability of each state is modelled
  with a Weibull distribution. My real data set has 119 states and about 30K
  observations. I do not want to create the transition matrix that is required
  for mstate manually so found a way to derive it from the data. </p>
  
  <p>Two questions:</p>
  
  <ol>
  <li>Does anyone know whether there is a better way to set up the
  transition matrix or is an easier way to achieve my goal than the code below?</li>
  <li>What is the purpose of the <code>shape()</code> function in the <code>flexsurvreg</code> command?</li>
  </ol>
  
  <h1>Data</h1>
  
  <pre><code>require(mstate)
  require(data.table)
  test_data &lt;- data.table(id = c(1, 1, 1, 2, 2, 2, 3, 3),
                          state = c("A", "B", "End", "A", "A", "End", "B", "End"),
                          time = c(10, 30, 100, 10, 30, 102, 5, 50))
  </code></pre>
  
  <h1>Derive the Transition matrix</h1>
  
  <p>(any advise how to shorten this code?)</p>
  
  <pre><code>require(markovchain)
  mc_fit &lt;- markovchainFit(test_data[, state])
  
  states &lt;- apply(mc_fit$estimate@transitionMatrix, 1, function(x) {
    list(which(x &gt; 0))
  })
  
  
  # Unpack by one layer.
  states &lt;- lapply(states, function(x) {
    return(x[[1]])
  })
  
  # Store names temporarily.
  states_names &lt;- names(states)
  require(stringr)
  # Make all END states absorbing.
  states &lt;- lapply(names(states), function(x){
    if (str_detect(x, "End")) {
      return(c())
    } else {
      return(states[[x]])
    }
  })
  # Assing the names back to make reading easier.
  names(states) &lt;- states_names
  tmat &lt;- transMat(states, names = mc_fit$estimate@states)
  
  tmat_dt &lt;- as.data.table(tmat)
  tmat_dt[, event := rownames(tmat)]
  trans_mapping &lt;- melt(tmat_dt, id.var = "event")
  trans_mapping[, trans_key := paste(event, variable)][, c("event", "variable")
                                                       := NULL]
  setnames(trans_mapping, "value", "trans")
  
  test_data[, c("next_event",
                "tstop") := 
                list(Hmisc::Lag(state, shift = -1),
                     Hmisc::Lag(time, shift = -1)),
  by = id]
  
  test_data &lt;- test_data[state != "End"]
  
  test_data[, trans_key := paste(state, next_event)]
  
  test_data &lt;- merge(test_data, trans_mapping, by = "trans_key")
  
  test_data[, trans_key := NULL]
  setorder(test_data, id, time)
  
  test_data[, event := TRUE][next_event == "End", event := FALSE]
  </code></pre>
  
  <h1>Create the time inhomogenous Markov chain</h1>
  
  <pre><code>require(flexsurv)
  
  # Build a clock forward time inhomogeneous Markov chain.
  cfweib &lt;- flexsurvreg(Surv(time, tstop, event) ~ trans
                        + shape(trans), data = test_data,
                        dist = "weibull")
  tgrid &lt;- seq(0, 100, by = 0.1)
  mfwei &lt;- msfit.flexsurvreg(cfweib, t = tgrid, trans = tmat)
  </code></pre>