title: Sending byte array of approximately fixed size everytime to another method
body: <p>I have a method which takes a parameter which is <code>Partition</code> enum. This method will be called by multiple background threads (15 max) around same time period by passing different value of <code>partition</code>. Here <code>dataHoldersByPartition</code> is a <code>ImmutableMap</code> of <code>Partition</code> and <code>ConcurrentLinkedQueue&lt;DataHolder&gt;</code>.</p>
  
  <pre><code>  private final ImmutableMap&lt;Partition, ConcurrentLinkedQueue&lt;DataHolder&gt;&gt; dataHoldersByPartition;
  
    //... some code to populate entry in `dataHoldersByPartition` map
  
    private void validateAndSend(final Partition partition) {  
      ConcurrentLinkedQueue&lt;DataHolder&gt; dataHolders = dataHoldersByPartition.get(partition);
      Map&lt;byte[], byte[]&gt; clientKeyBytesAndProcessBytesHolder = new HashMap&lt;&gt;();
      int totalSize = 0;      
      DataHolder dataHolder;
      while ((dataHolder = dataHolders.poll())  != null) {      
        byte[] clientKeyBytes = dataHolder.getClientKey().getBytes(StandardCharsets.UTF_8);
        if (clientKeyBytes.length &gt; 255)
          continue;
  
        byte[] processBytes = dataHolder.getProcessBytes();
        int clientKeyLength = clientKeyBytes.length;
        int processBytesLength = processBytes.length;
  
        int additionalLength = clientKeyLength + processBytesLength;
        if (totalSize + additionalLength &gt; 50000) {
          Message message = new Message(clientKeyBytesAndProcessBytesHolder, partition);
          // here size of `message.serialize()` byte array should always be less than 50k at all cost
          sendToDatabase(message.getAddress(), message.serialize());
          clientKeyBytesAndProcessBytesHolder = new HashMap&lt;&gt;();
          totalSize = 0;
        }
        clientKeyBytesAndProcessBytesHolder.put(clientKeyBytes, processBytes);
        totalSize += additionalLength;
      }
      // calling again with remaining values only if clientKeyBytesAndProcessBytesHolder is not empty
      if(!clientKeyBytesAndProcessBytesHolder.isEmpty()) {
          Message message = new Message(partition, clientKeyBytesAndProcessBytesHolder);
          // here size of `message.serialize()` byte array should always be less than 50k at all cost
          sendToDatabase(message.getAddress(), message.serialize());      
      }
    }
  </code></pre>
  
  <p>And below is my <code>Message</code> class:</p>
  
  <pre><code>public final class Message {
    private final byte dataCenter;
    private final byte recordVersion;
    private final Map&lt;byte[], byte[]&gt; clientKeyBytesAndProcessBytesHolder;
    private final long address;
    private final long addressFrom;
    private final long addressOrigin;
    private final byte recordsPartition;
    private final byte replicated;
  
    public Message(Map&lt;byte[], byte[]&gt; clientKeyBytesAndProcessBytesHolder, Partition recordPartition) {
      this.clientKeyBytesAndProcessBytesHolder = clientKeyBytesAndProcessBytesHolder;
      this.recordsPartition = (byte) recordPartition.getPartition();
      this.dataCenter = Utils.CURRENT_LOCATION.get().datacenter();
      this.recordVersion = 1;
      this.replicated = 0;
      long packedAddress = new Data().packAddress();
      this.address = packedAddress;
      this.addressFrom = 0L;
      this.addressOrigin = packedAddress;
    }
  
    // Output of this method should always be less than 50k always
    public byte[] serialize() {
      // 36 + dataSize + 1 + 1 + keyLength + 8 + 2;
      int bufferCapacity = getBufferCapacity(clientKeyBytesAndProcessBytesHolder);
  
      ByteBuffer byteBuffer = ByteBuffer.allocate(bufferCapacity).order(ByteOrder.BIG_ENDIAN);
      // header layout
      byteBuffer.put(dataCenter).put(recordVersion).putInt(clientKeyBytesAndProcessBytesHolder.size())
          .putInt(bufferCapacity).putLong(address).putLong(addressFrom).putLong(addressOrigin)
          .put(recordsPartition).put(replicated);
  
      // data layout
      for (Map.Entry&lt;byte[], byte[]&gt; entry : clientKeyBytesAndProcessBytesHolder.entrySet()) {
        byte keyType = 0;
        byte[] key = entry.getKey();
        byte[] value = entry.getValue();
        byte keyLength = (byte) key.length;
        short valueLength = (short) value.length;
  
        ByteBuffer dataBuffer = ByteBuffer.wrap(value);
        long timestamp = valueLength &gt; 10 ? dataBuffer.getLong(2) : System.currentTimeMillis();
  
        byteBuffer.put(keyType).put(keyLength).put(key).putLong(timestamp).putShort(valueLength)
            .put(value);
      }
      return byteBuffer.array();
    }
  
    private int getBufferCapacity(Map&lt;byte[], byte[]&gt; clientKeyBytesAndProcessBytesHolder) {
      int size = 36;
      for (Entry&lt;byte[], byte[]&gt; entry : clientKeyBytesAndProcessBytesHolder.entrySet()) {
        size += 1 + 1 + 8 + 2;
        size += entry.getKey().length;
        size += entry.getValue().length;
      }
      return size;
    }
  
      // getters and to string method here
  }
  </code></pre>
  
  <p>Basically, what I have to make sure is whenever the <code>sendToDatabase</code> method is called in <code>validateAndSend</code> method, size of <code>message.serialize()</code> byte array should always be less than 50k at all cost. My <code>sendToDatabase</code> method sends byte array coming out from <code>serialize</code> method. So for example if I have 60k records in <code>dataHolders</code> CLQ then I will send in two chunks in <code>validateAndSend</code> method:</p>
  
  <ul>
  <li>First, I will make a byte array  which is approx less than 50k (meaning byte array coming out from <code>message.serialize()</code> is less than 50k) and call <code>sendToDatabase</code> method on it.</li>
  <li>Second, I will call <code>sendToDatabase</code> method for remianing part.</li>
  </ul>
  
  <p>And to accomplish above things, I was having <code>totalSize</code> variable in <code>validateAndSend</code> method which is trying to measure 50k size but looks like my approach might not be right and I maybe dropping some records or sending more than 50k everytime I guess? </p>
  
  <p>Looks like my <code>Message</code> class knows about <code>clientKeyBytesAndProcessBytesHolder</code> map and I can use this map to accurately define the size by calling <code>getBufferCapacity</code> method and if is approx less than 50k then call <code>sendToDatabase</code> method?</p>