title: kafka: what do 'soTimeout', 'bufferSize' and 'minBytes' mean for SimpleConsumer?
body: <p>I'm using Kafka
  0.8.2.1 SimpleConsumer . Could somebody clarify the meaning of a few config parameters for the SimpleConsumer and FetchRequestBuilder?  Short of reading KAfka's source code, I could not find any docs on then. (I tried posting this question to the kafka user group - but no luck):</p>
  
  <p>-- Q1: in the signature of the SimpleConsumer constructor I see the Int '<strong>soTimeout'</strong> parameter -
  what is the meaning of this timeout? Is this a timeout to connect to the Kafka broker? Timeout on getting a response from any [or specific??] request to Kafka (like FetchRequest)? Something else?</p>
  
  <pre><code>kafka.javaapi.consumer.SimpleConsumer
      (val host: String,
       val port: Int,
       val soTimeout: Int,
       val bufferSize: Int,
       val clientId: String)
  </code></pre>
  
  <p>-- Q2: also, SimpleConsumer constructor takes Int 'bufferSize' parameter. What is the meaning of it? Is this how many bytes SimpleConsumer will read when a fetchRequest is issued? Or is it the max number of bytes read per one fetch from Kafka - and multiple fetches will happen if more data is available?</p>
  
  <p>-- When building FetchRequest via FetchRequestBuilder (see below), I also need to specify '<strong>fetchSize</strong>':</p>
  
  <pre><code>FetchRequest req= newFetchRequestBuilder ()
    .clientId(kafkaGroupId)
    .addFetch(topic, partition, offset, fetchSizeInBytes)
    .build();
  </code></pre>
  
  <p>Looking at the source code of the FetchRequestBuilder , I think (I'm not a Scala pro) those calls
  translate into the below method calls - and there the final parameter passed into the FetchRequest is called '<strong>minBytes</strong>', hinting that this is not the exact fetch size, possibly? . Does it mean it won't even fetch anything unless the is at least 'minBytes' of data available?</p>
  
  <pre><code>class FetchRequestBuilder():
      def addFetch(topic: String, partition: Int, offset: Long, fetchSize: Int)
  
      def build() = {
        val fetchRequest= FetchRequest(versionId, correlationId.getAndIncrement, clientId, replicaId, maxWait, minBytes, requestMap.toMap)
  
  FetchRequest(versionId: Short = FetchRequest.CurrentVersion,
      correlationId: Int = FetchRequest.DefaultCorrelationId,
      clientId: String = ConsumerConfig.DefaultClientId,
      replicaId: Int = Request.OrdinaryConsumerId,
      maxWait: Int = FetchRequest.DefaultMaxWait,
      **minBytes: Int = FetchRequest.DefaultMinBytes**,
  ...)
  </code></pre>
  
  <p>So, my final question is:</p>
  
  <p>-- Q3: how do '<strong>bufferSize</strong>' and '<strong>fetchSize/minBytes</strong>' relate? What exactly do they define? Do I have to make
  sure one is smaller or grater than the other?</p>
  
  <p>Thanks,</p>
  
  <p>Marina</p>