title: Node js TCP server, socket.on('data') - data buffer contains garbage data on high load
body: <p>i use the net server of node js and use the socket.on('data') function to receive data. To parse the TCP messages I use the parse buffer method. This uses the first 4 bytes as the length of the TCP message so that I can read from the TCP stream and form individual commands.In summary what happens at high load is that there is some garbage data returned as part of the TCP stream which causes problems.</p>
  
  <pre><code>function onConnect(client) {
  var accumulatingBuffer = new Buffer(0);
  var totalPacketLen = -1;
  var accumulatingLen = 0;
  var recvedThisTimeLen = 0;
  
  client.on('data', function (data) {
      parseBuffer(client, data, accumulatingBuffer, totalPacketLen, accumulatingLen, recvedThisTimeLen);
  });
  }
  </code></pre>
  
  <p>and here is the parsebuffer method.</p>
  
  <pre><code>function parseBuffer(client, data, accumulatingBuffer, totalPacketLen, accumulatingLen, recvedThisTimeLen) {
  recvedThisTimeLen = Buffer.byteLength(data);
  var tmpBuffer = new Buffer(accumulatingLen + recvedThisTimeLen);
  accumulatingBuffer.copy(tmpBuffer);
  data.copy(tmpBuffer, accumulatingLen); // offset for accumulating
  accumulatingBuffer = tmpBuffer;
  tmpBuffer = null;
  accumulatingLen = accumulatingLen + recvedThisTimeLen;
  
  if (accumulatingLen &lt; PACKETHEADERLEN) { 
      return;
  } else if (accumulatingLen === PACKETHEADERLEN) { 
      packetHeaderLen
      return;
  } else {
      //a packet info is available..
      if (totalPacketLen &lt; 0) {
          totalPacketLen = accumulatingBuffer.readUInt32BE(0);
      }
  }
  while (accumulatingLen &gt;= totalPacketLen + PACKETHEADERLEN) {
      var aPacketBufExceptHeader = new Buffer(totalPacketLen); // a whole packet is available...
      accumulatingBuffer.copy(aPacketBufExceptHeader, 0, PACKETHEADERLEN, PACKETHEADERLEN + totalPacketLen);
  
      ////////////////////////////////////////////////////////////////////
      //process packet data
      var stringData = aPacketBufExceptHeader.toString();
      try {
          var JSONObject = JSON.parse(stringData);
          handler(client, JSONObject);
  
  
          var newBufRebuild = new Buffer(accumulatingBuffer.length - (totalPacketLen + PACKETHEADERLEN)); // we can reduce size of allocatin
          accumulatingBuffer.copy(newBufRebuild, 0, totalPacketLen + PACKETHEADERLEN, accumulatingBuffer.length);
  
          //init      
          accumulatingLen = accumulatingLen - (totalPacketLen + PACKETHEADERLEN); //totalPacketLen+4
          accumulatingBuffer = newBufRebuild;
          newBufRebuild = null;
          totalPacketLen = -1;
  
          //For a case in which multiple packets are transmitted at once.
          if (accumulatingLen &lt;= PACKETHEADERLEN) {
              //need to get more data -&gt; wait..
              return;
          } else {
              totalPacketLen = accumulatingBuffer.readUInt32BE(0);
          }
      } catch (ex) {
          console.log(ex + ' unable to process data');
          return;
      }
  }
  }
  </code></pre>
  
  <p>All is well until there is high simulated load using a bunch of clients sending messages fast. At that point of time inside the ParseBuffer method the first line  "data.length"returns more than the length of the TCP data. This leads into the code reading garbage as UInt32BE which cause a very high value in totalpacketlength(which tells the next packets length). This leads to lost messages. Am I missing something. Please help.</p>