title: Moving beyond R's optim function
body: <p>I am trying to use R to estimate a multinomial logit model with a manual specification. I have found a few packages that allow you to estimate MNL models <a href="http://hosho.ees.hokudai.ac.jp/~kubo/Rdoc/library/VGAM/html/multinomial.html" rel="noreferrer">here</a> or <a href="http://cran.r-project.org/web/packages/mlogit/index.html" rel="noreferrer">here</a>.  </p>
  
  <p>I've found some other writings on "rolling" your own MLE function <a href="http://www.mayin.org/ajayshah/KB/R/documents/mle/mle.html" rel="noreferrer">here</a>. However, from my digging around - all of these functions and packages rely on the internal <code>optim</code> function. </p>
  
  <p>In my benchmark tests, <code>optim</code> is the bottleneck. Using a simulated dataset with ~16000 observations and 7 parameters, R takes around 90 seconds on my machine. The equivalent model in <a href="http://transp-or.epfl.ch/page63023.html" rel="noreferrer">Biogeme</a> takes ~10 seconds. A colleague who writes his own code in <a href="http://www.doornik.com/" rel="noreferrer">Ox</a> reports around 4 seconds for this same model.</p>
  
  <p>Does anyone have experience with writing their own MLE function or can point me in the direction of something that is optimized beyond the default <code>optim</code> function (no pun intended)? </p>
  
  <p>If anyone wants the R code to recreate the model, let me know - I'll glady provide it. I haven't provided it since it isn't directly relevant to the problem of optimizing the <code>optim</code> function and to preserve space...</p>
  
  <p><em>EDIT: Thanks to everyone for your thoughts. Based on a myriad of comments below, we were able to get R in the same ballpark as Biogeme for more complicated models, and R was actually faster for several smaller / simpler models that we ran. I think the long term solution to this problem is going to involve writing a separate maximization function that relies on a fortran or C library, but am certainly open to other approaches.</em></p>