title: Efficient way to process pandas DataFrame timeseries with Numba
body: <p>I have a DataFrame with 1,500,000 rows. It's one-minute level stock market data that I bought from QuantQuote.com. (Open, High, Low, Close, Volume). I'm trying to run some home-made backtests of stockmarket trading strategies. Straight python code to process the transactions is too slow and I wanted to try to use numba to speed things up. The trouble is that <a href="https://groups.google.com/a/continuum.io/forum/#!topic/numba-users/ZNfJtXeH2Lc" rel="noreferrer">numba doesn't seem to work with pandas functions</a>.</p>
  
  <p>Google searches uncover a surprising lack of information about using numba with pandas. Which makes me wonder if I'm making a mistake by considering it.</p>
  
  <p>My setup is Numba 0.13.0-1, Pandas 0.13.1-1. Windows 7, MS VS2013 with PTVS, Python 2.7, Enthought Canopy</p>
  
  <p>My existing Python+Pandas innerloop has the following general structure</p>
  
  <ul>
  <li>Compute "indicator" columns, (with pd.ewma, pd.rolling_max, pd.rolling_min etc.)</li>
  <li>Compute "event" columns for predetermined events such as moving average crosses, new highs etc. </li>
  </ul>
  
  <p>I then use DataFrame.iterrows to process the DataFrame.</p>
  
  <p>I've tried various optimizations but it's still not as fast as I would like. And the optimizations are causing bugs.</p>
  
  <p>I want to use numba to process the rows.  Are there preferred methods of approaching this?</p>
  
  <p>Because my DataFrame is really just a rectangle of floats, I was considering using something like DataFrame.values to get access to the data and then write a series of functions that use numba to access the rows. But that removes all the timestamps and I don't think it is a reversible operation. I'm not sure if the values matrix that I get from DataFrame.values is guaranteed to not be a copy of the data. </p>
  
  <p>Any help is greatly appreciated.</p>