{"action":"created","issue":{"url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/84","repository_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python","labels_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/84/labels{/name}","comments_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/84/comments","events_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/84/events","html_url":"https://github.com/confluentinc/confluent-kafka-python/issues/84","id":191471304,"number":84,"title":"Producer memory leak","user":{"login":"fillest","id":103396,"avatar_url":"https://avatars.githubusercontent.com/u/103396?v=3","gravatar_id":"","url":"https://api.github.com/users/fillest","html_url":"https://github.com/fillest","followers_url":"https://api.github.com/users/fillest/followers","following_url":"https://api.github.com/users/fillest/following{/other_user}","gists_url":"https://api.github.com/users/fillest/gists{/gist_id}","starred_url":"https://api.github.com/users/fillest/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fillest/subscriptions","organizations_url":"https://api.github.com/users/fillest/orgs","repos_url":"https://api.github.com/users/fillest/repos","events_url":"https://api.github.com/users/fillest/events{/privacy}","received_events_url":"https://api.github.com/users/fillest/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-11-24T09:32:04Z","updated_at":"2016-11-26T09:20:45Z","closed_at":null,"body":"confluent-kafka==0.9.2, librdkafka 0.9.2, Python 2.7.6, Ubuntu 14.04\r\n\r\nI run a web app in 16 gunicorn workers (processes). Each has a producer, which works continually until the process is restarted (normally by deploy system). This instance serves ~100-250 req/s (some part of which produce to kafka) and has ~3.8gb or memory. After switching to confluent-kafka and running it for some time in production I'm observing worker RSS memory only growing and growing (it was not the case before with kafka-python). Here is a screenshot of memory monitoring graph:\r\n![mem monitoring](https://cloud.githubusercontent.com/assets/103396/20592202/45f3b818-b23d-11e6-9948-f7931d26c5da.png)\r\nThe code is roughly like this (so I `poll(0)` after each `produce`):\r\n```python\r\ndef error_cb (err):\r\n    logger.error('kafka producer error_cb: {}: %s'.format(err.name()), err.str())\r\ndef on_delivery (err, msg):\r\n    assert err\r\n    logger.error('kafka producer on_delivery {}'.format(err.name()))\r\n_producer = confluent_kafka.Producer(**{\r\n    'client.id': 'qwe_producer',\r\n    'bootstrap.servers': bootstrap_servers, #3 brokers\r\n    'log.connection.close': False,\r\n    'api.version.request': True,\r\n\r\n    'queue.buffering.max.messages': 100000,\r\n    'queue.buffering.max.kbytes': 4000000,\r\n    'queue.buffering.max.ms': 1000,\r\n    'message.send.max.retries': 9000,\r\n    'batch.num.messages': 10000,\r\n    'delivery.report.only.error': True,\r\n\r\n    'default.topic.config': {\r\n        'request.required.acks': -1,\r\n    },\r\n    'error_cb': error_cb,\r\n    'on_delivery': on_delivery,\r\n})\r\natexit.register(_producer.flush)\r\n\r\ndef kafka_async_send (topic, value):\r\n    _producer.produce(topic, cPickle.dumps(value, 2))\r\n    _producer.poll(0)\r\n\r\ndef some_request_handler (request):\r\n    #...\r\n    kafka_async_send('foo', ('some', 'stuff'))\r\n    #...\r\n```\r\nWhat should I do now? Do you probably have a suggestion how to debug it? (preferably right in production - it is kinda testing production so any overhead is currently affordable)"},"comment":{"url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/comments/263053313","html_url":"https://github.com/confluentinc/confluent-kafka-python/issues/84#issuecomment-263053313","issue_url":"https://api.github.com/repos/confluentinc/confluent-kafka-python/issues/84","id":263053313,"user":{"login":"edenhill","id":524990,"avatar_url":"https://avatars.githubusercontent.com/u/524990?v=3","gravatar_id":"","url":"https://api.github.com/users/edenhill","html_url":"https://github.com/edenhill","followers_url":"https://api.github.com/users/edenhill/followers","following_url":"https://api.github.com/users/edenhill/following{/other_user}","gists_url":"https://api.github.com/users/edenhill/gists{/gist_id}","starred_url":"https://api.github.com/users/edenhill/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/edenhill/subscriptions","organizations_url":"https://api.github.com/users/edenhill/orgs","repos_url":"https://api.github.com/users/edenhill/repos","events_url":"https://api.github.com/users/edenhill/events{/privacy}","received_events_url":"https://api.github.com/users/edenhill/received_events","type":"User","site_admin":false},"created_at":"2016-11-26T09:20:45Z","updated_at":"2016-11-26T09:20:45Z","body":"Could you monitor _producer.Len() to see if messages are stuck in the underlying C library?"}}
{"actor":{"display_login":"edenhill"}}