{"action":"created","issue":{"url":"https://api.github.com/repos/logstash-plugins/logstash-input-kafka/issues/68","labels_url":"https://api.github.com/repos/logstash-plugins/logstash-input-kafka/issues/68/labels{/name}","comments_url":"https://api.github.com/repos/logstash-plugins/logstash-input-kafka/issues/68/comments","events_url":"https://api.github.com/repos/logstash-plugins/logstash-input-kafka/issues/68/events","html_url":"https://github.com/logstash-plugins/logstash-input-kafka/issues/68","id":129039036,"number":68,"title":"Kafka logstash input do not continue from where it left off on restart","user":{"login":"r-tock","id":11711723,"avatar_url":"https://avatars.githubusercontent.com/u/11711723?v=3","gravatar_id":"","url":"https://api.github.com/users/r-tock","html_url":"https://github.com/r-tock","followers_url":"https://api.github.com/users/r-tock/followers","following_url":"https://api.github.com/users/r-tock/following{/other_user}","gists_url":"https://api.github.com/users/r-tock/gists{/gist_id}","starred_url":"https://api.github.com/users/r-tock/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/r-tock/subscriptions","organizations_url":"https://api.github.com/users/r-tock/orgs","repos_url":"https://api.github.com/users/r-tock/repos","events_url":"https://api.github.com/users/r-tock/events{/privacy}","received_events_url":"https://api.github.com/users/r-tock/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/logstash-plugins/logstash-input-kafka/labels/bug","name":"bug","color":"e11d21"},{"url":"https://api.github.com/repos/logstash-plugins/logstash-input-kafka/labels/needs%20details","name":"needs details","color":"cc317c"}],"state":"open","locked":false,"assignee":{"login":"suyograo","id":1595958,"avatar_url":"https://avatars.githubusercontent.com/u/1595958?v=3","gravatar_id":"","url":"https://api.github.com/users/suyograo","html_url":"https://github.com/suyograo","followers_url":"https://api.github.com/users/suyograo/followers","following_url":"https://api.github.com/users/suyograo/following{/other_user}","gists_url":"https://api.github.com/users/suyograo/gists{/gist_id}","starred_url":"https://api.github.com/users/suyograo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/suyograo/subscriptions","organizations_url":"https://api.github.com/users/suyograo/orgs","repos_url":"https://api.github.com/users/suyograo/repos","events_url":"https://api.github.com/users/suyograo/events{/privacy}","received_events_url":"https://api.github.com/users/suyograo/received_events","type":"User","site_admin":false},"milestone":null,"comments":6,"created_at":"2016-01-27T05:45:40Z","updated_at":"2016-01-27T12:41:57Z","closed_at":null,"body":"This is highly undesirable, the reason we are publishing data to kafka is to ensure the consumers can be taken down and can come back up in an asynchronous fashion. Is this a limitation of this plugin? or simply a configuration issue. \r\n\r\nAlso where does the kafka input save the kafka position per partition ?\r\n\r\nI am using the latest version of the plugin with the following kafka config\r\n\r\n```\r\ninput {\r\n    kafka {\r\n        zk_connect => \"zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181\"\r\n        topic_id => \"logs\"\r\n        consumer_threads => 1\r\n        consumer_restart_on_error => true\r\n        consumer_restart_sleep_ms => 100\r\n        decorate_events => true\r\n\ttype => \"logs\"\r\n    }\r\n}\r\n```"},"comment":{"url":"https://api.github.com/repos/logstash-plugins/logstash-input-kafka/issues/comments/175599575","html_url":"https://github.com/logstash-plugins/logstash-input-kafka/issues/68#issuecomment-175599575","issue_url":"https://api.github.com/repos/logstash-plugins/logstash-input-kafka/issues/68","id":175599575,"user":{"login":"joekiller","id":1022919,"avatar_url":"https://avatars.githubusercontent.com/u/1022919?v=3","gravatar_id":"","url":"https://api.github.com/users/joekiller","html_url":"https://github.com/joekiller","followers_url":"https://api.github.com/users/joekiller/followers","following_url":"https://api.github.com/users/joekiller/following{/other_user}","gists_url":"https://api.github.com/users/joekiller/gists{/gist_id}","starred_url":"https://api.github.com/users/joekiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joekiller/subscriptions","organizations_url":"https://api.github.com/users/joekiller/orgs","repos_url":"https://api.github.com/users/joekiller/repos","events_url":"https://api.github.com/users/joekiller/events{/privacy}","received_events_url":"https://api.github.com/users/joekiller/received_events","type":"User","site_admin":false},"created_at":"2016-01-27T12:41:57Z","updated_at":"2016-01-27T12:41:57Z","body":"This discussion is probably better for the logstash discussion group vs an issue here as issues generally have steps to reproduce a problem and I'm unsure if this is really a bug."}}