{"action":"created","issue":{"url":"https://api.github.com/repos/fluent/fluent-plugin-s3/issues/223","repository_url":"https://api.github.com/repos/fluent/fluent-plugin-s3","labels_url":"https://api.github.com/repos/fluent/fluent-plugin-s3/issues/223/labels{/name}","comments_url":"https://api.github.com/repos/fluent/fluent-plugin-s3/issues/223/comments","events_url":"https://api.github.com/repos/fluent/fluent-plugin-s3/issues/223/events","html_url":"https://github.com/fluent/fluent-plugin-s3/issues/223","id":294272432,"number":223,"title":"fluent-plugin-s3 sometimes causes Zlib::DataError","user":{"login":"mochizuki-masao","id":6082538,"avatar_url":"https://avatars1.githubusercontent.com/u/6082538?v=4","gravatar_id":"","url":"https://api.github.com/users/mochizuki-masao","html_url":"https://github.com/mochizuki-masao","followers_url":"https://api.github.com/users/mochizuki-masao/followers","following_url":"https://api.github.com/users/mochizuki-masao/following{/other_user}","gists_url":"https://api.github.com/users/mochizuki-masao/gists{/gist_id}","starred_url":"https://api.github.com/users/mochizuki-masao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mochizuki-masao/subscriptions","organizations_url":"https://api.github.com/users/mochizuki-masao/orgs","repos_url":"https://api.github.com/users/mochizuki-masao/repos","events_url":"https://api.github.com/users/mochizuki-masao/events{/privacy}","received_events_url":"https://api.github.com/users/mochizuki-masao/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"repeatedly","id":16928,"avatar_url":"https://avatars3.githubusercontent.com/u/16928?v=4","gravatar_id":"","url":"https://api.github.com/users/repeatedly","html_url":"https://github.com/repeatedly","followers_url":"https://api.github.com/users/repeatedly/followers","following_url":"https://api.github.com/users/repeatedly/following{/other_user}","gists_url":"https://api.github.com/users/repeatedly/gists{/gist_id}","starred_url":"https://api.github.com/users/repeatedly/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/repeatedly/subscriptions","organizations_url":"https://api.github.com/users/repeatedly/orgs","repos_url":"https://api.github.com/users/repeatedly/repos","events_url":"https://api.github.com/users/repeatedly/events{/privacy}","received_events_url":"https://api.github.com/users/repeatedly/received_events","type":"User","site_admin":false},"assignees":[{"login":"repeatedly","id":16928,"avatar_url":"https://avatars3.githubusercontent.com/u/16928?v=4","gravatar_id":"","url":"https://api.github.com/users/repeatedly","html_url":"https://github.com/repeatedly","followers_url":"https://api.github.com/users/repeatedly/followers","following_url":"https://api.github.com/users/repeatedly/following{/other_user}","gists_url":"https://api.github.com/users/repeatedly/gists{/gist_id}","starred_url":"https://api.github.com/users/repeatedly/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/repeatedly/subscriptions","organizations_url":"https://api.github.com/users/repeatedly/orgs","repos_url":"https://api.github.com/users/repeatedly/repos","events_url":"https://api.github.com/users/repeatedly/events{/privacy}","received_events_url":"https://api.github.com/users/repeatedly/received_events","type":"User","site_admin":false}],"milestone":null,"comments":6,"created_at":"2018-02-05T04:44:46Z","updated_at":"2018-05-21T08:17:12Z","closed_at":null,"author_association":"NONE","body":"Hi,\r\n\r\nfluent-plugin-s3 start to emit warning occasionally since I have updated fluentd from v0.12 to v1. I'm using `v1.1.0-debian` docker image provided in DockerHub.\r\n\r\n```\r\nfailed to flush the buffer. retry_time=0 next_retry_seconds=2018-01-30 06:16:02 +0000 chunk=\"563f8368970c1de01cd924eacce3a69e\" error_class=Zlib::DataError error=\"data error\"\r\n\r\n/var/lib/gems/2.3.0/gems/fluent-plugin-s3-1.1.1/lib/fluent/plugin/out_s3.rb:498:in `finish'\r\n/var/lib/gems/2.3.0/gems/fluent-plugin-s3-1.1.1/lib/fluent/plugin/out_s3.rb:498:in `compress'\r\n/var/lib/gems/2.3.0/gems/fluent-plugin-s3-1.1.1/lib/fluent/plugin/out_s3.rb:288:in `write'\r\n/var/lib/gems/2.3.0/gems/fluentd-1.1.0/lib/fluent/plugin/output.rb:1094:in `try_flush'\r\n/var/lib/gems/2.3.0/gems/fluentd-1.1.0/lib/fluent/plugin/output.rb:1319:in `flush_thread_run'\r\n/var/lib/gems/2.3.0/gems/fluentd-1.1.0/lib/fluent/plugin/output.rb:439:in `block (2 levels) in start'\r\n/var/lib/gems/2.3.0/gems/fluentd-1.1.0/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'\r\n\r\nretry succeeded. chunk_id=\"563f8368970c1de01cd924eacce3a69e\"\r\n```\r\n\r\nI found:\r\n\r\n1. I can find this warnings especially in high load server.\r\n2. retry just after this warning always succeeds.\r\n\r\nIs this a bug for GzipCompressor or a problem of my configuration?\r\n\r\n(partial) configuration is below:\r\n\r\n```\r\n<match **>\r\n  @type s3\r\n  s3_bucket \"#{ENV['LOG_BUCKET_NAME']}\"\r\n  s3_region \"#{ENV['AWS_REGION']}\"\r\n  format json\r\n  path applications/${service}/${log_type}/\r\n  s3_object_key_format %{path}%{time_slice}_${container_id}_%{index}.%{file_extension}\r\n  time_slice_format %Y/%m/%d/%H/%M\r\n  utc\r\n  <buffer time,service,log_type,container_id>\r\n    @type memory\r\n    timekey 5m\r\n    timekey_wait 1m\r\n  </buffer>\r\n</match>\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/fluent/fluent-plugin-s3/issues/comments/390585944","html_url":"https://github.com/fluent/fluent-plugin-s3/issues/223#issuecomment-390585944","issue_url":"https://api.github.com/repos/fluent/fluent-plugin-s3/issues/223","id":390585944,"user":{"login":"mochizuki-masao","id":6082538,"avatar_url":"https://avatars1.githubusercontent.com/u/6082538?v=4","gravatar_id":"","url":"https://api.github.com/users/mochizuki-masao","html_url":"https://github.com/mochizuki-masao","followers_url":"https://api.github.com/users/mochizuki-masao/followers","following_url":"https://api.github.com/users/mochizuki-masao/following{/other_user}","gists_url":"https://api.github.com/users/mochizuki-masao/gists{/gist_id}","starred_url":"https://api.github.com/users/mochizuki-masao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mochizuki-masao/subscriptions","organizations_url":"https://api.github.com/users/mochizuki-masao/orgs","repos_url":"https://api.github.com/users/mochizuki-masao/repos","events_url":"https://api.github.com/users/mochizuki-masao/events{/privacy}","received_events_url":"https://api.github.com/users/mochizuki-masao/received_events","type":"User","site_admin":false},"created_at":"2018-05-21T08:17:12Z","updated_at":"2018-05-21T08:17:12Z","author_association":"NONE","body":"Thanks! I will try this and report if `gzip_command` works."}}
{"actor":{"display_login":"mochizuki-masao"}}