title: Getting large AJAX responses in Chrome without crashing
body: <p>I am working on a project which is aimed at the Chrome browser.  Our goal which we would like to accomplish is to get a one million record array into the browser to work with the data.  When I generated a test file that contained a million records it was a bit more than one gigabyte.</p>
  
  <p>For reasons I will explain, I believe we can accomplish the goal if can get the browser to collect the garbage when necessary.  I believe the browser holds the text of the AJAX responses when it doesn't need to and crashes for that reason.</p>
  
  <p>Now, I can generate a million records within the browser and manipulate it as I need to.  However, I have trouble sending the AJAX to the browser without crashing it.</p>
  
  <p>Since sending one million crashes it, I tried sending batches of one hundred thousand.  I can get two such batches across and parse the JSON.  If I do not have a onreadystatechange on my AJAX call, I can make the call a number of times.  Also, if I receive a hundred thousand records, I can go over it ten times and make the full array.</p>
  
  <p>Because I seem to be able to actually hold one million records, I believe that, as I said, holding the response texts is overwhelming the browsers.</p>
  
  <p>In order to try to get better memory management, I have pushed the AJAX resquests and parsing into a web worker.  When the webworker gets the AJAX and makes the hundred thousand record array, it pushes it to the DOM thread.  When the DOM thread has taken the data it has the web worker do another AJAX.</p>
  
  <p>However, it still crashes.</p>
  
  <p>I am open to using websockets or something else, if that would help somehow.</p>
  
  <p>Here is the code in the DOM thread:</p>
  
  <pre><code>var iterations=3;
  var url='hunthou.json';      
  var worker=new Worker('src/worker.js');
  var count=0;
      worker.addEventListener('message',function(e){
        alert('count: '+count);
        //bigArr=bigArr.concat(e.data);
        console.log('e.data length: '+e.data.length);
        bigArr[count]=e.data;
        console.log('bigArr length: '+bigArr.length);
        if(count&lt;(iterations-1)){
          worker.postMessage(url);  
        } else{
          alert('done');
          console.log('done');
          worker.terminate();
          console.log('bye');
        }
        count++;
      });
  worker.postMessage(url);
  </code></pre>
  
  <p>Here is the webworker:</p>
  
  <pre><code>var arr=[];
  var request = new XMLHttpRequest();
  request.onreadystatechange = function () {
      var DONE = this.DONE || 4;
      if (this.readyState === DONE){          
          arr=JSON.parse(request.responseText);
          self.postMessage(arr);          
          arr.length=0;
          request.responseText.length=0;
          console.log('okay');
      }
  };
  
  self.addEventListener('message', function(e) {
      var url=e.data;
      console.log('url: '+url);
  
      request.open("GET",'../'+url,true);
      request.send(null);
  }, false);
  </code></pre>