title: JavaScript heap out of memory when calling async function on an event
body: <p>I am writing an Node.js application that reads avro encoded messages from very busy kafka topics (millions of messages per day), decompresses them, decodes them and writes them to PostgreSQL database. The problem is that the program runs for a minute or two and then crashes on JavaScript heap out of memory error. I've investigated where the issue might be and it seems that it happens when I am writing decoded message to database. If I just log decoded messages, it works fine. Or If I write messages to database without decoding them, it works fine. But when I call my writeRow() function on decoder.on('data') event, it runs out of memory very soon. Am I not handling events emited by the decoder correctly?</p>
  
  <p>I'm using following modules:</p>
  
  <ul>
  <li>kafkajs, kafkajs-snappy, avsc for consuming, decompressing and decoding messages from kafka topics</li>
  <li>node-postgres for writing messages to database</li>
  </ul>
  
  <p>Here's the relevant code:</p>
  
  <p>Consumer:</p>
  
  <pre><code>const run = async () =&gt; {
    await consumer.connect();
    await topics.forEach(topic =&gt; {
      consumer.subscribe({ topic: topic, fromBeginning: false });
    });
  
    await consumer.run({
      eachMessage: async ({ topic, message }) =&gt; {
        try {
          const buf = await Buffer.from(message.value, "binary");
          const decoder = new avro.streams.BlockDecoder();
          await decoder.on("data", async msg =&gt; {
            console.log(`Message received: ${msg}`);
            await writeRow(topic, new Date().toISOString(), msg);
          });
          await decoder.end(buf);
        } catch (err) {
          console.log(err);
        }
      }
    });
  };
  </code></pre>
  
  <p>writeRow() function:</p>
  
  <pre><code>const writeRow = async (topic, timestamp, data) =&gt; {
    const client = await pool.connect();
    try {
      const res = await client.query(
        `INSERT INTO ${topic.replace(
          /\./g,
          "_"
        )}(topic, timestamp, data) VALUES('${topic}', '${timestamp}', '${JSON.stringify(
          data
        )}')`
      );
      console.log(`${res.rowCount} row written to database.`);
    } catch (err) {
      console.log(err);
    } finally {
      await client.release();
    }
  };
  </code></pre>