title: Why is this numpy array too big to load?
body: <p>I have a 3.374Gb npz file, <code>myfile.npz</code>.</p>
  
  <p>I can read it in and view the filenames:</p>
  
  <pre><code>a = np.load('myfile.npz')
  a.files
  </code></pre>
  
  <p>gives</p>
  
  <pre><code>['arr_1','arr_0']
  </code></pre>
  
  <p>I can read in 'arr_1' ok</p>
  
  <pre><code>a1=a['arr_1']
  </code></pre>
  
  <p>However, I cannot load in <code>arr_0</code>, or read its shape:</p>
  
  <pre><code>a1=a['arr_0']
  a['arr_0'].shape
  </code></pre>
  
  <p>both above operations give the following error:</p>
  
  <pre><code>ValueError: array is too big
  </code></pre>
  
  <p>I have 16Gb RAM of which 8.370Gb is available. So the problem doesn't seem related to memory. My questions are:</p>
  
  <ol>
  <li><p>Should I be able to read this file in?</p></li>
  <li><p>Can anyone explain this error?</p></li>
  <li><p>I have been looking at using <code>np.memmap</code> to get around this - is this a reasonable approach?</p></li>
  <li><p>What debugging approach should I use?</p></li>
  </ol>
  
  <p><strong>EDIT:</strong></p>
  
  <p>I got access to a computer with more RAM (48GB) and it loaded. The <code>dtype</code> was in fact <code>complex128</code> and the uncompressed memory of <code>a['arr_0']</code> was 5750784000 bytes. It seems that a RAM overhead may be required. Either that or my predicted amount of available RAM was wrong (I used windows sysinternals RAMmap). </p>