title: Pandas: add columns through shifting on MultiIndexed DataFrame
body: <p>For a text classification problem, I have the following Pandas MultiIndexed DataFrame:</p>
  <pre><code>| *Date* | *Group* | Titles | Label |
  | --- | --- | --- | --- |
  |2009-08-17 |6.0|   [67, 18, 84, 9, 97, 570, 206, 487, 0, 0, 0, 0,...|0.0|
  |2009-08-17|7.0|    [518, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0...|0.0|
  |2009-08-17|8.0|    [430, 196, 639, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...|0.0|
  |2009-08-17|9.0|    [535, 779, 861, 602, 19, 0, 0, 0, 0, 0, 0, 0, ...|0.0|
  |2009-08-18|0.0|    [779, 125, 205, 13, 255, 272, 642, 83, 570, 93...|1.0|
  |2009-08-18|1.0|    [125, 851, 20, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0...|1.0|
  |2009-08-18|2.0|    [18, 205, 29, 726, 643, 99, 642, 4, 3, 369, 8,...|1.0|
  |2009-08-18|3.0|    [10, 3, 545, 13, 331, 4, 903, 0, 0, 0, 0, 0, 0...|1.0|
  |2009-08-19 |0.0|   [579, 917, 947, 214, 168, 0, 0, 0, 0, 0, 0, 0,...|1.0|
  |2009-08-21 |0.0|   [239, 237, 46, 70, 634, 0, 0, 0, 0, 0, 0, 0, 0...|1.0|
  </code></pre>
  <p>The &quot;Titles&quot; column represent collections of tokenized and indexed news published on the respective date. I'm supposed to add K columns containing the news published on the K previous days. I used <code>df.shift(i,freq=&quot;D&quot;)</code> but the size of the data would overload RAM when training my network due to the amount of articles published on certain days.</p>
  <p>I ended up splitting the news in smaller groups but if day <code>X-1</code> has 2 groups <code>[a],[b]</code> and day <code>X-2</code> has 3 groups <code>[1],[2],[3]</code>, I need to generate 6 rows for day X with each combination of groups, as to get:</p>
  <pre><code>| *Date* | *Group* | Titles | Titles - 1 | Titles - 2 | Label |
  | --- | --- | --- | --- | --- | --- |
  | X | 0.0 | [...] | [a] | [1] | 1 |
  | X | 0.0 | [...] | [a] | [2] | 1 |
  | X | 0.0 | [...] | [a] | [3] | 1 |
  | X | 0.0 | [...] | [b] | [1] | 1 |
  | X | 0.0 | [...] | [b] | [2] | 1 |
  | X | 0.0 | [...] | [b] | [3] | 1 |
  </code></pre>
  <p>Can anyone help?</p>