{"action":"opened","issue":{"url":"https://api.github.com/repos/logstash-plugins/logstash-output-s3/issues/107","repository_url":"https://api.github.com/repos/logstash-plugins/logstash-output-s3","labels_url":"https://api.github.com/repos/logstash-plugins/logstash-output-s3/issues/107/labels{/name}","comments_url":"https://api.github.com/repos/logstash-plugins/logstash-output-s3/issues/107/comments","events_url":"https://api.github.com/repos/logstash-plugins/logstash-output-s3/issues/107/events","html_url":"https://github.com/logstash-plugins/logstash-output-s3/issues/107","id":194916216,"number":107,"title":"S3 output can’t find files it generates","user":{"login":"vad","id":29362,"avatar_url":"https://avatars.githubusercontent.com/u/29362?v=3","gravatar_id":"","url":"https://api.github.com/users/vad","html_url":"https://github.com/vad","followers_url":"https://api.github.com/users/vad/followers","following_url":"https://api.github.com/users/vad/following{/other_user}","gists_url":"https://api.github.com/users/vad/gists{/gist_id}","starred_url":"https://api.github.com/users/vad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vad/subscriptions","organizations_url":"https://api.github.com/users/vad/orgs","repos_url":"https://api.github.com/users/vad/repos","events_url":"https://api.github.com/users/vad/events{/privacy}","received_events_url":"https://api.github.com/users/vad/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-12-12T08:33:13Z","updated_at":"2016-12-12T08:33:13Z","closed_at":null,"body":"Cross posted on: https://discuss.elastic.co/t/s3-output-cant-find-files-it-generates/68546\r\n\r\nI'm using Logstash 5.0.2 (and .0.1 before) and I need to add an s3 output but I can't because randomly one of the file logstash generates cannot be found:\r\n\r\n```\r\n[2016-12-07T14:37:39,809][ERROR][logstash.outputs.s3      ] failed to upload, will re-enqueue /tmp/logstash/ls.s3.ip-172-31-45-248.2016-12-07T14.31.part0.txt for upload {:ex=>#<Errno::ENOENT: No such file or directory - /tmp/logstash/ls.s3.ip-172-31-45-248.2016-12-07T14.31.part0.txt>, :backtrace=>[\"org/jruby/RubyFile.java:370:in `initialize'\", \"org/jruby/RubyIO.java:1197:in `open'\", \"/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-s3-3.2.0/lib/logstash/outputs/s3.rb:175:in `write_on_bucket'\", \"/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-s3-3.2.0/lib/logstash/outputs/s3.rb:289:in `move_file_to_bucket'\", \"/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-s3-3.2.0/lib/logstash/outputs/s3.rb:454:in `upload_worker'\", \"/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-s3-3.2.0/lib/logstash/outputs/s3.rb:436:in `configure_upload_workers'\", \"org/jruby/RubyProc.java:281:in `call'\", \"/usr/share/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:24:in `initialize'\"]}\r\n```\r\nAs it says Logstash \"will re-enqueue\" the file over and over again filling up the disk with log messages. This happens randomly: Logstash works for some time (minutes or hours) and then suddenly this happens.\r\n\r\nThis is my output configuration:\r\n\r\n```ruby\r\nif [type] == \"my-log\" {\r\n  s3 {\r\n    bucket => \"mybucket\"\r\n    prefix => \"logstash/bla/\"\r\n    size_file => 10485760  # rotate every 10M\r\n    codec => line {\r\n      format => \"%{message}\"\r\n    }\r\n  }\r\n}\r\n```\r\n\r\nI'm on ubuntu 14.04. Oracle JVM:\r\n\r\n```\r\n# java -version\r\njava version \"1.8.0_111\"\r\nJava(TM) SE Runtime Environment (build 1.8.0_111-b14)\r\nJava HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)\r\n```\r\n"}}
{"actor":{"display_login":"vad"}}