title: Node.js and Express.js create a large CSV file (1GB) and download it from ajax client
body: <p>The UI allows users to select a couple query parameters and submit via AJAX.
  The server will query mongoDB with the parameters and create a very large CSV file (30 columns/1.5MM rows/~1GB) on the server.
  Once the file is created, the server responds to the AJAX call with the URL to download the file.
  A second call is made to the server to download the CSV file.</p>
  
  <p>My problem is scaling this solution...  Node.js runs out of memory when trying to retrieve more than 50,0000 records from mongoDB.  System is running with 4GB memory.</p>
  
  <p>AJAX call to create and download the file:</p>
  
  <pre><code> handleExport: function (){
          NProgress.done();
          NProgress.start();
          var data = {spMle: this.state.spMle, custMle: this.state.custMle};
          $.ajax({
              type: 'POST',
              url: '/api/aps/export',
              data: data,
              success: function (filename){
                  NProgress.done();
                  var url = '/api/aps/export/download/' + filename
                  $('body').append("&lt;iframe src='" + url + "' style='display: none;' &gt;&lt;/iframe&gt;");
                  //window.open(url);
              }
          });
      },
  </code></pre>
  
  <p>Server request to create the CSV file from the MongoDB query, and send the URL to download the file back to the client:</p>
  
  <pre><code>var spMle       = req.body.spMle;
  var custMle     = req.body.custMle;
  var limit       = 50000;
  var filename    = 'export.csv';
  var path        = './public/downloads/' + filename;
  var query       = ApsModel.find();
  if (spMle !== "0"){query.where('SP_MLE_ID').equals(spMle);}
  if (custMle !== "0"){query.where('Cust_MLE_ID').equals(custMle);}
  if (limit){query.limit(limit);}
  
  var writeStream = fs.createWriteStream(path);
  writeStream.on('finish', function () {
      console.log('CSV file completed!');
      res.send(filename);
  });
  var readStream = query.stream({ transform: JSON.stringify});
  var json2CsvStream = new Json2CsvStream();
  json2CsvStream.on('header', function(data) {
    // console.log(' ++ yeah header found ++');
    // return console.log(data);
  })
  json2CsvStream.on('line', function(data) {
    console.log('row streamed');
    // return console.log(data);
  });
  
  readStream.pipe(json2CsvStream).pipe(writeStream);
  </code></pre>
  
  <p>Server request to download the file:</p>
  
  <pre><code>router.get('/aps/export/download/:filename', function (req, res){
      var path = './public/downloads/' + req.params.filename;
      res.download(path, "export.csv")
  });
  </code></pre>