{"action":"created","issue":{"url":"https://api.github.com/repos/fluent/fluentd/issues/869","repository_url":"https://api.github.com/repos/fluent/fluentd","labels_url":"https://api.github.com/repos/fluent/fluentd/issues/869/labels{/name}","comments_url":"https://api.github.com/repos/fluent/fluentd/issues/869/comments","events_url":"https://api.github.com/repos/fluent/fluentd/issues/869/events","html_url":"https://github.com/fluent/fluentd/issues/869","id":143444651,"number":869,"title":"out_file + logrotate error","user":{"login":"perrin4869","id":5774716,"avatar_url":"https://avatars.githubusercontent.com/u/5774716?v=3","gravatar_id":"","url":"https://api.github.com/users/perrin4869","html_url":"https://github.com/perrin4869","followers_url":"https://api.github.com/users/perrin4869/followers","following_url":"https://api.github.com/users/perrin4869/following{/other_user}","gists_url":"https://api.github.com/users/perrin4869/gists{/gist_id}","starred_url":"https://api.github.com/users/perrin4869/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/perrin4869/subscriptions","organizations_url":"https://api.github.com/users/perrin4869/orgs","repos_url":"https://api.github.com/users/perrin4869/repos","events_url":"https://api.github.com/users/perrin4869/events{/privacy}","received_events_url":"https://api.github.com/users/perrin4869/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-03-25T07:28:41Z","updated_at":"2016-03-28T09:07:10Z","closed_at":null,"body":"I have the following setup:\r\n\r\n```\r\n<match foo>\r\n  type file\r\n  path /var/log/td-agent/foo\r\n  format csv\r\n  time_slice_format %H\r\n  append true # append to already existing file instead of creating a new file, ie, log.20140608_0.log, log.20140608_1.log, etc...\r\n  flush_interval 1m\r\n</match>\r\n```\r\n\r\nIt works exactly as expected as long as I don't point `logrotate` to it. However, if I use `logrotate`, for example, as follows:\r\n\r\n```\r\n/var/log/td-agent/foo.*.log {\r\n    size 1000k\r\n    create 700 username groupname\r\n    rotate 4\r\n}\r\n```\r\n\r\nIn that case, at around 3AM in the morning I'll most likely find the following kind of log in `/var/log/td-agent/td-agent.log`:\r\n\r\n```\r\n2016-03-17 03:15:28 +0900 [warn]: temporarily failed to flush the buffer. next_retry=2016-03-17 03:15:29 +0900 error_class=\"Errno::EACCES\" error=\"Permission denied @ rb_sysopen - /var/log/foo.03.log\" plugin_id=\"object:3ffed140e254\"\r\n  2016-03-17 03:15:28 +0900 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.20/lib/fluent/plugin/out_file.rb:91:in `initialize'\r\n  2016-03-17 03:15:28 +0900 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.20/lib/fluent/plugin/out_file.rb:91:in `open'\r\n  2016-03-17 03:15:28 +0900 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.20/lib/fluent/plugin/out_file.rb:91:in `write'\r\n  2016-03-17 03:15:28 +0900 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.20/lib/fluent/buffer.rb:345:in `write_chunk'\r\n  2016-03-17 03:15:28 +0900 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.20/lib/fluent/buffer.rb:324:in `pop'\r\n  2016-03-17 03:15:28 +0900 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.20/lib/fluent/output.rb:329:in `try_flush'\r\n  2016-03-17 03:15:28 +0900 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.20/lib/fluent/output.rb:140:in `run'\r\n2016-03-17 03:15:29 +0900 [warn]: temporarily failed to flush the buffer. next_retry=2016-03-17 03:15:31 +0900 error_class=\"Errno::EACCES\" error=\"Permission denied @ rb_sysopen - /var/log/td-agent/foo.03.log\" plugin_id=\"object:3ffed140e254\"\r\n  2016-03-17 03:15:29 +0900 [warn]: suppressed same stacktrace\r\n2016-03-17 03:15:31 +0900 [warn]: temporarily failed to flush the buffer. next_retry=2016-03-17 03:15:35 +0900 error_class=\"Errno::EACCES\" error=\"Permission denied @ rb_sysopen - /var/log/td-agent/foo.03.log\" plugin_id=\"object:3ffed140e254\"\r\n  2016-03-17 03:15:31 +0900 [warn]: suppressed same stacktrace\r\n\r\n...\r\n```\r\nAfter that, all consecutive flushing fails even for logs at 4AM (`/var/log/td-agent/foo.04.log`),\r\n5AM, etc., while the buffer files keep getting written.\r\n\r\nUsing td-agent 0.12.20 on CentOS 6.4\r\n\r\nPS: In a slightly unrelated note, is there a plan to support variables in `out_file` paths, based on tag parts or record fields? Would you be open to accepting a pull request adding such feature?\r\n\r\nEdit: I removed the `logrotate` configuration for 2 days and the problems have gone, so it's safe to assume this is some incompatibility between `fluentd` and `logrotate`"},"comment":{"url":"https://api.github.com/repos/fluent/fluentd/issues/comments/202313893","html_url":"https://github.com/fluent/fluentd/issues/869#issuecomment-202313893","issue_url":"https://api.github.com/repos/fluent/fluentd/issues/869","id":202313893,"user":{"login":"perrin4869","id":5774716,"avatar_url":"https://avatars.githubusercontent.com/u/5774716?v=3","gravatar_id":"","url":"https://api.github.com/users/perrin4869","html_url":"https://github.com/perrin4869","followers_url":"https://api.github.com/users/perrin4869/followers","following_url":"https://api.github.com/users/perrin4869/following{/other_user}","gists_url":"https://api.github.com/users/perrin4869/gists{/gist_id}","starred_url":"https://api.github.com/users/perrin4869/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/perrin4869/subscriptions","organizations_url":"https://api.github.com/users/perrin4869/orgs","repos_url":"https://api.github.com/users/perrin4869/repos","events_url":"https://api.github.com/users/perrin4869/events{/privacy}","received_events_url":"https://api.github.com/users/perrin4869/received_events","type":"User","site_admin":false},"created_at":"2016-03-28T09:07:10Z","updated_at":"2016-03-28T09:07:10Z","body":"Any ideas as to how to troubleshoot this problem? I will re-enable logrotate and try to look at the change of permissions today"}}