title: How to deal with MemoryError in numpy with keras?
body: <p>I am trying to process data and use it later for training a model in keras. However, I am running out of memory while returning 4 arrays at the same time to the main function. The code looks like this:</p>
  
  <pre class="lang-py prettyprint-override"><code>def prepare_data:
      all_3d_data = []
      for j in range(all_3d_data):
          ...
          X_train.append(all_3d_data)
      del all_3d_data
  </code></pre>
  
  <p>There are four similar loops in each of which one array is filled with some data. These 4 arrays are returned to the main function at once:</p>
  
  <pre class="lang-py prettyprint-override"><code>    return X_train, Y_train, X_test, Y_test
  </code></pre>
  
  <p>In the main function when it is called:</p>
  
  <pre class="lang-py prettyprint-override"><code>if __name__ == "__main__":
      X_train, Y_train, X_test,Y_test = prepare_data()
  </code></pre>
  
  <p>An error raises:</p>
  
  <blockquote>
    <p>File "C:...\tensorflow\lib\site-packages\numpy\core\numeric.py", line 501, in asarray
        return array(a, dtype, copy=False, order=order)</p>
    
    <p>MemoryError</p>
  </blockquote>
  
  <p>This error doesn't happen when I am using  smaller data. How can I fix this? Should I use batch in Keras?</p>