title: NodeJS Sharp Node Package Memory Consumption Issue
body: <p>I am working on a NodeJS project that is using <code>sharp node package</code> for resizing JPEG/JPG images.</p>
  
  <p>The problem is Node process keeps adding the processed file in memory and never releases it.</p>
  
  <p>As a result, amount of memory consumed increases with every request &amp; it never gets released.</p>
  
  <p>After debugging the application, I realised that <a href="https://sharp.pixelplumbing.com/en/stable/api-output/#tobuffer" rel="noreferrer">sharp toBuffer API</a> is causing the issue.</p>
  
  <p>I tried using alternative solution by creating a <code>custom writable stream</code> and piping it with <code>sharp Duplex stream</code> but it ends up with the same problem.</p>
  
  <p>I don't understand if I'm missing anything here or it's a bug.</p>
  
  <p>Sharing the code below (I have removed unwanted code to make it compact) - </p>
  
  <pre><code>const { Writable } = require("stream");
  const { createServer } = require("http");
  const { readFileSync } = require("fs");
  const sharp = require("sharp");
  
  async function resizeJpeg(input_buffer) {
      // initialise file, the response object
      const file = { normalisedImage: null, originalImage: null };
      // initialise sharp instance using original image buffer
      let image = sharp(input_buffer);
      // set the original image metadata
      file.originalImage = await image.metadata();
      file.originalImage.quality = 85;
  
      // generate buffer using sharp resize API with default quality &amp; dimensions.
      // ############### THIS IS WHERE MEMORY CONSUMPTION OCCURS ###############
  
      // APPROACH 1 (SHARP toBuffer API)
      const buffer = await image.resize(2000, 798).jpeg({ quality: 85 }).toBuffer();
      // APPROACH 1 ENDS 
  
      // APPROACH 2 (CUSTOM WRITABLE STREAM)
      // const buffer = await sharpToBuffer(image.resize(2000, 798).jpeg({ quality: 85 }));
      // APPROACH 2 ENDS
  
      // set resized image metadata
      file.normalisedImage = await sharp(buffer).metadata();
      file.normalisedImage.quality = 85;
      return file;
  }
  
  // converts sharp readable stream to buffer using custom writable buffer
  async function sharpToBuffer(readable) {
      return new Promise((resolve, reject) =&gt; {
          const writable = new WriteStream().on("finish", () =&gt; resolve(Buffer.concat(writable.bufferChunks)));
          readable.pipe(writable);
      });
  }
  
  // simple writable stream
  class WriteStream extends Writable {
      constructor() { super(); this.bufferChunks = [] }
      _write(chunk, encoding, next) { this.bufferChunks.push(chunk); next(); }
  }
  
  createServer(async (request, response) =&gt; {
      // ignore favicon calls
      if (request.url.indexOf("favicon.ico") &gt; -1) { return response.end(); }
      // trigger resize call and pass buffer of original image file
      const { normalisedImage, originalImage } = await resizeJpeg(readFileSync(`${__dirname}/30mb.jpg`));
      // respond stringified json
      response.end(
          JSON.stringify({
              normalisedImage: { size: normalisedImage.size, width: normalisedImage.width, height: normalisedImage.height, quality: normalisedImage.quality },
              originalImage: { size: originalImage.size, width: originalImage.width, height: originalImage.height, quality: originalImage.quality }
          }, null, 4));
  }).listen(3000, () =&gt; console.log("server started"));
  </code></pre>
  
  <p>As you can see, <code>resizeJpeg function</code> has both the approaches implemented.</p>
  
  <p>To make this work you just need to make sure <code>30mb.jpg</code> file exists in same directory.</p>
  
  <p>The image I used is available <a href="https://drive.google.com/file/d/1muJ8UfgP_wsV49DT4p9siXTUc3dmI-5u/view?usp=sharing" rel="noreferrer">here</a></p>
  
  <p>In case you're using linux, following <code>top command</code> can be used to monitor the memory usage assuming file name is <code>so.js</code> -</p>
  
  <p>top <code>ps -ef | grep 'so.js' | awk '{print $2}' | sed 's/.*/-p &amp;/' | xargs echo -c</code></p>