title: How to create a lagged data structure using pandas dataframe
body: <p>Example</p>
  
  <pre><code>s=pd.Series([5,4,3,2,1], index=[1,2,3,4,5])
  print s 
  1    5
  2    4
  3    3
  4    2
  5    1
  </code></pre>
  
  <p>Is there an efficient way to create a series. e.g. containing in each row the lagged values (in this example up to lag 2)</p>
  
  <pre><code>3    [3, 4, 5]
  4    [2, 3, 4]
  5    [1, 2, 3]
  </code></pre>
  
  <p>This corresponds to <em>s=pd.Series([[3,4,5],[2,3,4],[1,2,3]], index=[3,4,5])</em></p>
  
  <p>How can this be done in an efficient way for dataframes with a lot of timeseries which are very long?</p>
  
  <p>Thanks</p>
  
  <p><strong>Edited after seeing the answers</strong></p>
  
  <p>ok, at the end I implemented this function:</p>
  
  <pre><code>def buildLaggedFeatures(s,lag=2,dropna=True):
  '''
  Builds a new DataFrame to facilitate regressing over all possible lagged features
  '''
  if type(s) is pd.DataFrame:
      new_dict={}
      for col_name in s:
          new_dict[col_name]=s[col_name]
          # create lagged Series
          for l in range(1,lag+1):
              new_dict['%s_lag%d' %(col_name,l)]=s[col_name].shift(l)
      res=pd.DataFrame(new_dict,index=s.index)
  
  elif type(s) is pd.Series:
      the_range=range(lag+1)
      res=pd.concat([s.shift(i) for i in the_range],axis=1)
      res.columns=['lag_%d' %i for i in the_range]
  else:
      print 'Only works for DataFrame or Series'
      return None
  if dropna:
      return res.dropna()
  else:
      return res 
  </code></pre>
  
  <p>it produces the wished outputs and manages the naming of columns in the resulting DataFrame.</p>
  
  <p>For a Series as input:</p>
  
  <pre><code>s=pd.Series([5,4,3,2,1], index=[1,2,3,4,5])
  res=buildLaggedFeatures(s,lag=2,dropna=False)
     lag_0  lag_1  lag_2
  1      5    NaN    NaN
  2      4      5    NaN
  3      3      4      5
  4      2      3      4
  5      1      2      3
  </code></pre>
  
  <p>and for a DataFrame as input:</p>
  
  <pre><code>s2=s=pd.DataFrame({'a':[5,4,3,2,1], 'b':[50,40,30,20,10]},index=[1,2,3,4,5])
  res2=buildLaggedFeatures(s2,lag=2,dropna=True)
  
     a  a_lag1  a_lag2   b  b_lag1  b_lag2
  3  3       4       5  30      40      50
  4  2       3       4  20      30      40
  5  1       2       3  10      20      30
  </code></pre>