title: Pickling large NumPy array
body: <p>I have a large 3d numpy array that I'd like to preserve. My first approach is simply to use pickle, but this seems to lead to a poorly explained error. </p>
  
  <pre><code>test_rand = np.random.random((100000,200,50))
  with open('models/test.pkl', 'wb') as save_file:
      pickle.dump(test_rand, save_file, -1)
  
  ---------------------------------------------------------------------------
  error                                     Traceback (most recent call last)
  &lt;ipython-input-18-511e30b08440&gt; in &lt;module&gt;()
        1 with open('models/test.pkl', 'wb') as save_file:
  ----&gt; 2         pickle.dump(test_rand, save_file, -1)
        3 
  
  C:\Users\g1dak02\AppData\Local\Continuum\Anaconda\lib\pickle.pyc in dump(obj, file, protocol)
     1368 
     1369 def dump(obj, file, protocol=None):
  -&gt; 1370     Pickler(file, protocol).dump(obj)
     1371 
     1372 def dumps(obj, protocol=None):
  
  C:\Users\g1dak02\AppData\Local\Continuum\Anaconda\lib\pickle.pyc in dump(self, obj)
      222         if self.proto &gt;= 2:
      223             self.write(PROTO + chr(self.proto))
  --&gt; 224         self.save(obj)
      225         self.write(STOP)
      226 
  
  C:\Users\g1dak02\AppData\Local\Continuum\Anaconda\lib\pickle.pyc in save(self, obj)
      329 
      330         # Save the reduce() output and finally memoize the object
  --&gt; 331         self.save_reduce(obj=obj, *rv)
      332 
      333     def persistent_id(self, obj):
  
  C:\Users\g1dak02\AppData\Local\Continuum\Anaconda\lib\pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
      417 
      418         if state is not None:
  --&gt; 419             save(state)
      420             write(BUILD)
      421 
  
  C:\Users\g1dak02\AppData\Local\Continuum\Anaconda\lib\pickle.pyc in save(self, obj)
      284         f = self.dispatch.get(t)
      285         if f:
  --&gt; 286             f(self, obj) # Call unbound method with explicit self
      287             return
      288 
  
  C:\Users\g1dak02\AppData\Local\Continuum\Anaconda\lib\pickle.pyc in save_tuple(self, obj)
      560         write(MARK)
      561         for element in obj:
  --&gt; 562             save(element)
      563 
      564         if id(obj) in memo:
  
  C:\Users\g1dak02\AppData\Local\Continuum\Anaconda\lib\pickle.pyc in save(self, obj)
      284         f = self.dispatch.get(t)
      285         if f:
  --&gt; 286             f(self, obj) # Call unbound method with explicit self
      287             return
      288 
  
  C:\Users\g1dak02\AppData\Local\Continuum\Anaconda\lib\pickle.pyc in save_string(self, obj, pack)
      484                 self.write(SHORT_BINSTRING + chr(n) + obj)
      485             else:
  --&gt; 486                 self.write(BINSTRING + pack("&lt;i", n) + obj)
      487         else:
      488             self.write(STRING + repr(obj) + '\n')
  
  error: integer out of range for 'i' format code
  </code></pre>
  
  <p>So the two questions I have are as follows:</p>
  
  <ul>
  <li>What is actually going on in this error?</li>
  <li>How should I go about saving the array to disk? </li>
  </ul>
  
  <p>I am using Python 2.7.8 and NumPy 1.9.0. </p>