title: Kafka/Confluent: ProducerConfig change default max.request.size using docker images
body: <p>I'm using kafka/confluent (3.2.0) to retrive change on a Mongodb instances we have.</p>
  
  <p>The source process is managed by <a href="http://debezium.io/docs/connectors/mongodb/" rel="nofollow noreferrer">Debezium source connector</a> who uses Source Connect  Api and is deployed on our systems using Mesos (DC/OS) expanding the Confluent Connect docker image.
  Kafka itself is deployed on the same DC/OS using the framework version.</p>
  
  <p>Since we've some message bigger than default size I've changed these kafka installation parameters:</p>
  
  <p>•   replica.fetch.max.bytes<br>
  •   message.max.bytes </p>
  
  <p>both to 4MB.</p>
  
  <p>Then I start the Connector Docker image using this </p>
  
  <pre><code>docker run -d --rm --net=bridge --name=kafka-connect-mongodb -e CONNECT_BOOTSTRAP_SERVERS=${KAFKA_BOOTSTRAP_SERVERS} -e CONNECT_REST_PORT=${CONNECT_REST_PORT}  -e CONNECT_GROUP_ID="mongo-to-kafka-source-connector" -e CONNECT_CONFIG_STORAGE_TOPIC="${CONFIG.TOPIC}"  -e CONNECT_OFFSET_STORAGE_TOPIC="${OFFSETS.TOPIC}"   -e CONNECT_STATUS_STORAGE_TOPIC="${STATUS.TOPIC}"   -e CONNECT_KEY_CONVERTER="io.confluent.connect.avro.AvroConverter"   -e CONNECT_VALUE_CONVERTER="io.confluent.connect.avro.AvroConverter"   -e CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL="${SCHEMA_REGISTRY_LISTENERS}"   -e CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL="${SCHEMA_REGISTRY_LISTENERS}"   -e CONNECT_INTERNAL_KEY_CONVERTER="org.apache.kafka.connect.json.JsonConverter"   -e CONNECT_INTERNAL_VALUE_CONVERTER="org.apache.kafka.connect.json.JsonConverter"   -e CONNECT_REST_ADVERTISED_HOST_NAME="${CONNECTOR_HOME}"   -e CONNECT_LOG4J_ROOT_LOGLEVEL=INFO  -e CONNECT_MAX_REQUEST_SIZE=4194304 -e KAFKA_MAX_REQUEST_SIZE=4194304  mongodb-source-connector:1.1
  </code></pre>
  
  <p>I changed the default max.request.size producer value passing both KAFKA_MAX_REQUEST_SIZE and CONNECT_MAX_REQUEST_SIZE and the log was properly changed to 4MB.</p>
  
  <p>The issue arises when I start the extraction from Mongodb. 
  To do so i run this POST</p>
  
  <pre><code>curl -X POST \  
  http://hostname:8083/connectors \
    -d '{ 
    "name": "source_connector",  
    "config": {
      "tasks.max":"1",
      "connector.class": "io.debezium.connector.mongodb.MongoDbConnector", 
      "mongodb.hosts": "mongodbhost:27017", 
      "mongodb.name": "replica", 
      "collection.whitelist": "db[.]table",
      "max.request.size": "4194304"
    }
  }'
  </code></pre>
  
  <p>but then the log says</p>
  
  <pre><code>[2017-10-09 12:22:56,036] INFO ProducerConfig values: 
      acks = all
      batch.size = 16384
      block.on.buffer.full = false
      bootstrap.servers = [PLAINTEXT://172.17.0.3:9093]
      buffer.memory = 33554432
      client.id = 
      compression.type = none
      connections.max.idle.ms = 540000
      interceptor.classes = null
      key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
      linger.ms = 0
      max.block.ms = 9223372036854775807
      max.in.flight.requests.per.connection = 1
      max.request.size = 1048576
      metadata.fetch.timeout.ms = 60000
      metadata.max.age.ms = 300000
      metric.reporters = []
      metrics.num.samples = 2
      metrics.sample.window.ms = 30000
      partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
      receive.buffer.bytes = 32768
      reconnect.backoff.ms = 50
      request.timeout.ms = 2147483647
      retries = 2147483647
      retry.backoff.ms = 100
      sasl.jaas.config = null
      sasl.kerberos.kinit.cmd = /usr/bin/kinit
      sasl.kerberos.min.time.before.relogin = 60000
      sasl.kerberos.service.name = null
      sasl.kerberos.ticket.renew.jitter = 0.05
      sasl.kerberos.ticket.renew.window.factor = 0.8
      sasl.mechanism = GSSAPI
      security.protocol = PLAINTEXT
      send.buffer.bytes = 131072
      ssl.cipher.suites = null
      ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
      ssl.endpoint.identification.algorithm = null
      ssl.key.password = null
      ssl.keymanager.algorithm = SunX509
      ssl.keystore.location = null
      ssl.keystore.password = null
      ssl.keystore.type = JKS
      ssl.protocol = TLS
      ssl.provider = null
      ssl.secure.random.implementation = null
      ssl.trustmanager.algorithm = PKIX
      ssl.truststore.location = null
      ssl.truststore.password = null
      ssl.truststore.type = JKS
      timeout.ms = 30000
      value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
  </code></pre>
  
  <p>So when i start the source process, the default value for max.request.size is used.</p>
  
  <p><a href="https://www.dropbox.com/s/dfszid94ot8pfud/log.log?dl=0" rel="nofollow noreferrer">Here</a> the full log.  </p>
  
  <p>I don't undestand what i miss.</p>