title: Can I use statsmodel ARIMA to interpolate a time series?
body: <p>In Python 3.7, I have a time series represented by a Pandas dataframe in which the index is a DateTimeIndex and the single value column is stock price:</p>
  
  <p><a href="https://i.stack.imgur.com/29fN0.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/29fN0.png" alt="enter image description here"></a></p>
  
  <p>The gaps correspond to NaN "price" values, and there are 126 non-NaN values and 20 NaN values.  What I'm trying to do is to interpolate the non-NaN values to predict the values that are NaN.  I tried several interpolation methods (linear, cubic spline) but they're not sufficiently accurate, and looking at the plot above, it appears there is a significant upward trend and also some traces of weekly periodicity, so I decided to use statsmodel ARIMA.  Here is my code:</p>
  
  <pre><code>def fill_in_dataframe_ARIMA(  df ):
      price_is_not_NaN = df[ 'price' ].notnull()
      price_is_NaN = np.logical_not( price_is_not_NaN )
      # Convert the datetimes of the index into milliseconds:
      datetime_ms = df.index.map( to_ms )
      # Train the ARIMA model:
      train_datetime_ms = datetime_ms[ price_is_not_NaN ]
      train_price = df.price[ price_is_not_NaN ]
      arima_model = ARIMA( train_price, ( 5, 1, 2 ), train_datetime_ms ).fit()
      # Use model to predict the missing prices:
      missing_datetime_ms = datetime_ms[ price_is_NaN ]
      missing_price = arima_model.predict( exog = missing_datetime_ms )
      return missing_price
  </code></pre>
  
  <p>What I'm expecting is that <code>missing_price</code> ends up being an array-like object of twenty entries, like <code>missing_datetime_ms</code>.  Instead, <code>missing_price</code> has 125 entries, one fewer than the number of samples in <code>train_datetime_ms:train_price</code>.</p>
  
  <p>Clearly I am not understanding what's meant by <em>endogenous</em> and <em>exogenous</em> (not to mention <em>interpolate</em> vs. <em>extrapolate</em>).  Can someone please explain how I can get the intended result of 20 predicted entries?</p>