title: MemoryError vs "ValueError: array is too big" when allocating a large ndarray on different machines
body: <p>I am running Python on two separate devices—MacBook Air mid-2013 (Laptop 1) and ThinkPad X1 Yoga 3G (Laptop 2)—and creating numpy arrays on both. Despite the two laptops having relatively similar memory:</p>
  
  <p>Laptop 1: Memory 4 GB 1600 MHz DDR3</p>
  
  <p>Laptop 2: Installed RAM 16.0 GB (15.8 GB usable)</p>
  
  <p>I am finding wildly different threshold array sizes on the two laptops before observing a <code>MemoryError</code>. For example:</p>
  
  <p><strong>Laptop 1</strong></p>
  
  <pre><code>&gt;&gt;&gt; import numpy as np
  &gt;&gt;&gt; np.zeros(int(5.*10.**12))
  array([0., 0., 0., ..., 0., 0., 0.])
  &gt;&gt;&gt; np.zeros(int(6.*10.**12))
  Python(6138,0x7fffda9413c0) malloc: *** mach_vm_map(size=48000000000000) failed (error code=3)
  *** error: can't allocate region
  *** set a breakpoint in malloc_error_break to debug
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  MemoryError
  </code></pre>
  
  <p><strong>Laptop 2</strong></p>
  
  <pre><code>&gt;&gt;&gt; import numpy as np
  &gt;&gt;&gt; np.zeros(int(1.*10.**8.))
  array([0., 0., 0., ..., 0., 0., 0.])
  &gt;&gt;&gt; np.zeros(int(5.*10.**8.))
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  ValueError: array is too big; `arr.size * arr.dtype.itemsize` is larger than the maximum possible size.
  &gt;&gt;&gt; np.zeros(int(5.*10.**9.))
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  ValueError: Maximum allowed dimension exceeded
  &gt;&gt;&gt; import numpy as np
  &gt;&gt;&gt; np.zeros(int(1.*10.**8.))
  array([0., 0., 0., ..., 0., 0., 0.])
  &gt;&gt;&gt; np.zeros(int(5.*10.**8.))
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  ValueError: array is too big; `arr.size * arr.dtype.itemsize` is larger than the maximum possible size.
  &gt;&gt;&gt; np.zeros(int(5.*10.**9.))
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  ValueError: Maximum allowed dimension exceeded
  </code></pre>
  
  <p><strong>Linux:</strong></p>
  
  <pre><code>Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44) 
  [GCC 7.3.0] on linux
  Type "help", "copyright", "credits" or "license" for more information.
  &gt;&gt;&gt; import numpy as np
  &gt;&gt;&gt; np.zeros(int(5.*10.**8))
  array([0., 0., 0., ..., 0., 0., 0.])
  &gt;&gt;&gt; np.zeros(int(5.*10.**9))
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  MemoryError
  </code></pre>
  
  <p>Therefore an array of length (approximately) 5 trillion appears to be Laptop 1's limit before receiving a <code>MemoryError</code>, while Laptop 2's threshold is closer to about 0.5 billion. This is essentially a disparity of 4 orders of magnitude, yet the available memory on each laptop is not nearly that different. I also tried creating similar arrays on two Linux machines with about 4 GB of RAM each and observed a length of approximately 1.0 billion to be the limit. </p>
  
  <p>I am clearly missing something perhaps obvious. Any clarity into the situation of why this disparity is observed would be greatly appreciated. </p>