{"action":"created","issue":{"url":"https://api.github.com/repos/dpkp/kafka-python/issues/343","labels_url":"https://api.github.com/repos/dpkp/kafka-python/issues/343/labels{/name}","comments_url":"https://api.github.com/repos/dpkp/kafka-python/issues/343/comments","events_url":"https://api.github.com/repos/dpkp/kafka-python/issues/343/events","html_url":"https://github.com/dpkp/kafka-python/issues/343","id":60435905,"number":343,"title":"KafkaConsumer always get some commited data after restart","user":{"login":"halolk","id":4044257,"avatar_url":"https://avatars.githubusercontent.com/u/4044257?v=3","gravatar_id":"","url":"https://api.github.com/users/halolk","html_url":"https://github.com/halolk","followers_url":"https://api.github.com/users/halolk/followers","following_url":"https://api.github.com/users/halolk/following{/other_user}","gists_url":"https://api.github.com/users/halolk/gists{/gist_id}","starred_url":"https://api.github.com/users/halolk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/halolk/subscriptions","organizations_url":"https://api.github.com/users/halolk/orgs","repos_url":"https://api.github.com/users/halolk/repos","events_url":"https://api.github.com/users/halolk/events{/privacy}","received_events_url":"https://api.github.com/users/halolk/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dpkp/kafka-python/labels/consumer","name":"consumer","color":"d4c5f9"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-03-10T01:08:19Z","updated_at":"2016-01-08T18:45:59Z","closed_at":"2016-01-08T18:45:59Z","body":"Hi all,\r\n\r\nI'm using SimpleProducer and KafkaConsumer to process some web logs. A strange thing is that although the Consumer has read all the messages and Producer is not running any more, if I terminate the processes using KafkaConsumer and rerun them, they will still read some old messages again. Since my script enabled auto_commit (and even explicitly called commit() for every message later), and the Consumer called set_topic_partitions(), I expected the Consumer will noitce the offset and get nothing. But no matter how many times I restart the processes, they will always read a random number of commited old messages (not all commited messages).\r\n\r\nCan you point out any idea about this problem? Thanks a lot!"},"comment":{"url":"https://api.github.com/repos/dpkp/kafka-python/issues/comments/170087663","html_url":"https://github.com/dpkp/kafka-python/issues/343#issuecomment-170087663","issue_url":"https://api.github.com/repos/dpkp/kafka-python/issues/343","id":170087663,"user":{"login":"dpkp","id":843444,"avatar_url":"https://avatars.githubusercontent.com/u/843444?v=3","gravatar_id":"","url":"https://api.github.com/users/dpkp","html_url":"https://github.com/dpkp","followers_url":"https://api.github.com/users/dpkp/followers","following_url":"https://api.github.com/users/dpkp/following{/other_user}","gists_url":"https://api.github.com/users/dpkp/gists{/gist_id}","starred_url":"https://api.github.com/users/dpkp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dpkp/subscriptions","organizations_url":"https://api.github.com/users/dpkp/orgs","repos_url":"https://api.github.com/users/dpkp/repos","events_url":"https://api.github.com/users/dpkp/events{/privacy}","received_events_url":"https://api.github.com/users/dpkp/received_events","type":"User","site_admin":false},"created_at":"2016-01-08T18:45:59Z","updated_at":"2016-01-08T18:45:59Z","body":"Updated KafkaConsumer to expose the official java api. Use KafkaConsumer.close() to commit any pending offsets that have not yet auto-committed. Also note that the default auto-commit-interval has been reduced to 5 seconds."}}