title: Huge amounts of lag when using kafka mirrormaker
body: <p>I have set up a MirrorMaker cluster that consumes topics from clusters around the world.</p>
  
  <p>The issue is the latency between the mirror cluster (located in the EU) and a source cluster (located in the US) creates a massive spike in offset lag.</p>
  
  <p>The mirrors are consuming 9 topics, each one consisting of 24 partitions.</p>
  
  <h1>Mirror Settings</h1>
  
  <pre><code>/opt/kafka/bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config /opt/kafka/config/us1.consumer.properties --num.streams 48 --producer.config /opt/kafka/config/glb.producer.properties --whitelist="a,b,c,d,e,f,g,h,i"
  </code></pre>
  
  <h1>The <code>producer.properties</code> Settings</h1>
  
  <pre><code>bootstrap.servers=localhost:9092
  acks=-1
  retries=2147483647
  client.id=us.mm.producer
  batch.size=200
  linger.ms=10
  </code></pre>
  
  <p>I am playing around with increasing the batch size and introducing linger to try and increase throughput, but so far my efforts have failed.</p>
  
  <p>What is the best way to work the ideal batch size, and is introducing a <code>linger.ms</code> going to be useful with such high latency rates?</p>
  
  <p>Apologies if this seems vague, but I am new to Kafka, and tuning a server is proving difficult.</p>