title: Python Memory Error with a large Pandas Dataframe
body: <p>Could you please help me with the memory?</p>
  
  <p>I got below MemoryError.</p>
  
  <p>File "pandas_libs\algos_common_helper.pxi", line 361, in pandas._libs.algos.ensure_int64
  MemoryError</p>
  
  <p>Then, I output all the memory of all variables using the below codes:</p>
  
  <pre><code>def sizeof_fmt(num, suffix='B'):
      ''' by Fred Cirera,  https://stackoverflow.com/a/1094933/1870254, modified'''
      for unit in ['','Ki','Mi','Gi','Ti','Pi','Ei','Zi']:
          if abs(num) &lt; 1024.0:
              return "%3.1f %s%s" % (num, unit, suffix)
          num /= 1024.0
      return "%.1f %s%s" % (num, 'Yi', suffix)
  
  print('Memory size of each Varaible:')
  for name, size in sorted(((name, sys.getsizeof(value)) for name, value in locals().items()),
                           key= lambda x: -x[1])[:10]:
      print("{:&gt;30}: {:&gt;8}".format(name, sizeof_fmt(size)))
  
  Memory size of each Varaible:
                        df_baker: 572.6 MiB
                         df_hall: 37.5 MiB
                   df_WSGT_baker: 12.1 KiB
                    df_B12_baker: 12.1 KiB
                    df_WSGT_hall:  7.7 KiB
                     df_B12_hall:  7.7 KiB
                        __file__:  178.0 B
                 __annotations__:  136.0 B
                           MyWho:   72.0 B
                      sizeof_fmt:   72.0 B
  </code></pre>
  
  <p>The largest memory is only 570MB for the Pandas dataframe df_baker. And I have 5GB memory. so why I have a Memory Error?   Thanks for your help. I appreciate it.</p>