title: How to solve memory error while filtering data between 2 lists
body: <p>I wrote a simple code that would filter between 2 lists and combine dictionaries by id from both lists and then write that into another file. But whatever I do I keep getting Memory error.</p>
  <p>My code:</p>
  <pre><code>import ast
  
  with open('fileMain.js') as f:
      content = f.read()
      fileMain = ast.literal_eval(content) 
  
  with open('fileItems.js') as f:
      content = f.read()
      fileItems = ast.literal_eval(content) 
  
  
  for x in fileMain:
      for y in fileItems:
          if x.get('id') == y.get('id'):
              x['new'] = y 
  
  
  with open(&quot;allData.js&quot;, 'w') as f:
      f.write(str(fileMain))
  </code></pre>
  <p>fileMain and fileItems equal to 2.5GB on disc and I am using 64-bit version.</p>
  <p>Data structure:</p>
  <p>fileMain.js</p>
  <pre><code>[
  //Objects with the same ID are not necessarily next to one another.
    {
      &quot;ID&quot;: 30574062,
      &quot;anotherNumber&quot;: &quot;52,3&quot;,
      &quot;color&quot;: &quot;red&quot;
    },
    {
      &quot;ID&quot;: 30574062,
      &quot;anotherNumber&quot;: &quot;13&quot;,
      &quot;color&quot;: &quot;yellow&quot;
    },
    {
      &quot;ID&quot;: 30574063,
      &quot;anotherNumber&quot;: &quot;60,6&quot;,
      &quot;color&quot;: &quot;blue&quot;
    },
    ...
  ]
  </code></pre>
  <p>fileItems.js</p>
  <pre><code>[
  //Objects with the same ID are not necessarily next to one another.
    {
      ID: 30574062,
      number: 28234702,
      place: London,
    },
    {
      ID: 30574063,
      number: 45232502,
      place: Paris,
    },
    ...
  ]
  </code></pre>
  <p>Expected result:</p>
  <pre><code>[
    {
      ID: 30574062,
      number: 28234702,
      place: London,
      data: [
        {
         &quot;anotherNumber&quot;: &quot;52,3&quot;,
         &quot;color&quot;: &quot;red&quot;
        },
        {
         &quot;anotherNumber&quot;: &quot;13&quot;,
         &quot;color&quot;: &quot;yellow&quot;
        }
      ]
    },
    {
      ID: 30574063,
      number: 45232502,
      place: Paris,
      data: [
        {
          &quot;anotherNumber&quot;: &quot;60,6&quot;,
          &quot;color&quot;: &quot;blue&quot;
        },
      ]
    },
    ...
  ]
  </code></pre>