title: Is there a maximum size for python's shelve module?
body: <p>I'm getting this exception when trying to open shelve persisted files over a certain size which is actually pretty small (&lt; 1MB) but I'm not sure where the exactly number is. Now, I know pickle is sort of the bastard child of python and shelve isn't thought of as a particularly robust solution, but it happens to solve my problem wonderfully (in theory) and I haven't been able to find a reason for this exception.</p>
  
  <pre><code>Traceback (most recent call last):
    File "test_shelve.py", line 27, in &lt;module&gt;
      print len(f.keys())
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shelve.py", line 101, in keys
      return self.dict.keys()
  SystemError: Negative size passed to PyString_FromStringAndSize
  </code></pre>
  
  <p>I can reproduce it consistently, but I haven't found much on google. Here's a script that will reproduce.</p>
  
  <pre><code>import shelve
  import random
  import string
  import pprint
  
  f = shelve.open('test')
  # f = {}
  
  def rand_list(list_size=20, str_size=40):
      return [''.join([random.choice(string.ascii_uppercase + string.digits) for j in range(str_size)]) for i in range(list_size)]
  
  def recursive_dict(depth=3):
      if depth==0:
          return rand_list()
      else:
          d = {}
          for k in rand_list():
              d[k] = recursive_dict(depth-1)
          return d
  
  for k,v in recursive_dict(2).iteritems():
      f[k] = v
  
  f.close()
  
  f = shelve.open('test')
  print len(f.keys())
  </code></pre>