{"action":"created","issue":{"url":"https://api.github.com/repos/fluent/fluentd/issues/1087","repository_url":"https://api.github.com/repos/fluent/fluentd","labels_url":"https://api.github.com/repos/fluent/fluentd/issues/1087/labels{/name}","comments_url":"https://api.github.com/repos/fluent/fluentd/issues/1087/comments","events_url":"https://api.github.com/repos/fluent/fluentd/issues/1087/events","html_url":"https://github.com/fluent/fluentd/issues/1087","id":163815787,"number":1087,"title":"Losing logs when restarting Fluentd and Elastichsearch is stopped ","user":{"login":"fahimeh2010","id":7750151,"avatar_url":"https://avatars.githubusercontent.com/u/7750151?v=3","gravatar_id":"","url":"https://api.github.com/users/fahimeh2010","html_url":"https://github.com/fahimeh2010","followers_url":"https://api.github.com/users/fahimeh2010/followers","following_url":"https://api.github.com/users/fahimeh2010/following{/other_user}","gists_url":"https://api.github.com/users/fahimeh2010/gists{/gist_id}","starred_url":"https://api.github.com/users/fahimeh2010/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fahimeh2010/subscriptions","organizations_url":"https://api.github.com/users/fahimeh2010/orgs","repos_url":"https://api.github.com/users/fahimeh2010/repos","events_url":"https://api.github.com/users/fahimeh2010/events{/privacy}","received_events_url":"https://api.github.com/users/fahimeh2010/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-07-05T10:10:14Z","updated_at":"2016-07-09T17:48:00Z","closed_at":null,"body":"Hello All\r\nFirst thanks for your open source project.\r\n\r\nApplication versions:\r\n\r\nOS ubuntu 14.04\r\nElasticsearch 2.3.3\r\nKibana 4.5\r\ntd-agent 2\r\nKafka 0.10.0\r\n\r\nI use EFK (4 containers : Elasticsearch, Fluentd, Kibana , kafka) for managing logs. Actually this works fine.\r\nBut I see a problem, when elasticsearch container is down, if there are some new logs that are put in kafka, If I restart td-aganet, these logs after staring elasticsearch is lost and will not be shown in Kibana.\r\nI also can see the missing logs in stdout.\r\n\r\nYou can see the Fluentd configuration below:\r\n\r\n```aconf\r\n<match kafka.test2>\r\n    type copy\r\n    <store>\r\n      type stdout\r\n    </store>\r\n    <store>\r\n      @type elasticsearch\r\n      num_threads 8\r\n      buffer_type memory\r\n      buffer_chunk_limit 500k\r\n      buffer_queue_limit 1000\r\n      flush_interval 15s\r\n      host elasticsearch\r\n      port 9200\r\n      logstash_format true\r\n      logstash_prefix fluentd-nginx\r\n      logstash_dateformat %Y.%m.%d\r\n      time_key_format %Y-%m-%dT%H:%M:%S.%N%z\r\n      flush_interval 10s\r\n      index_name fluentd-nginx.${Time.at(time).getutc.strftime(@logstash_dateformat)}\r\n      type_name nginx-access\r\n    </store>\r\n</match>\r\n```\r\n\r\nThanks a lot\r\n"},"comment":{"url":"https://api.github.com/repos/fluent/fluentd/issues/comments/231546587","html_url":"https://github.com/fluent/fluentd/issues/1087#issuecomment-231546587","issue_url":"https://api.github.com/repos/fluent/fluentd/issues/1087","id":231546587,"user":{"login":"repeatedly","id":16928,"avatar_url":"https://avatars.githubusercontent.com/u/16928?v=3","gravatar_id":"","url":"https://api.github.com/users/repeatedly","html_url":"https://github.com/repeatedly","followers_url":"https://api.github.com/users/repeatedly/followers","following_url":"https://api.github.com/users/repeatedly/following{/other_user}","gists_url":"https://api.github.com/users/repeatedly/gists{/gist_id}","starred_url":"https://api.github.com/users/repeatedly/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/repeatedly/subscriptions","organizations_url":"https://api.github.com/users/repeatedly/orgs","repos_url":"https://api.github.com/users/repeatedly/repos","events_url":"https://api.github.com/users/repeatedly/events{/privacy}","received_events_url":"https://api.github.com/users/repeatedly/received_events","type":"User","site_admin":false},"created_at":"2016-07-09T17:48:00Z","updated_at":"2016-07-09T17:48:00Z","body":"Is Buffer kept on buffer_path directory?"}}
{"actor":{"display_login":"repeatedly"}}