title: Merge two dataframe to reduce memory consumption
body: <p>I am trying to explode a list in my dataframe column and merge it back to the df, but i get a memory error while merging the flatten column with the initial dataframe. I would like to know if i can merge it in chunks, so that i can overcome the memory issue.</p>
  <pre><code>def flatten_colum_with_list(df, column, reset_index=False):
          column_to_flatten = pd.DataFrame([[i, x] for i, y in df[column].apply(list).iteritems() for x in y], columns=['I', column])
          column_to_flatten = column_to_flatten.set_index('I')
          df = df.drop(column, axis=1)
          df = df.merge(column_to_flatten, left_index=True, right_index=True)
          if reset_index:
              df = df.reset_index(drop=True)
          return df
  </code></pre>
  <p>I would appreciate any support.</p>