{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/logstash/issues/4931","repository_url":"https://api.github.com/repos/elastic/logstash","labels_url":"https://api.github.com/repos/elastic/logstash/issues/4931/labels{/name}","comments_url":"https://api.github.com/repos/elastic/logstash/issues/4931/comments","events_url":"https://api.github.com/repos/elastic/logstash/issues/4931/events","html_url":"https://github.com/elastic/logstash/issues/4931","id":144654739,"number":4931,"title":"Asynchronous start of input plugins","user":{"login":"andrewvc","id":131427,"avatar_url":"https://avatars.githubusercontent.com/u/131427?v=3","gravatar_id":"","url":"https://api.github.com/users/andrewvc","html_url":"https://github.com/andrewvc","followers_url":"https://api.github.com/users/andrewvc/followers","following_url":"https://api.github.com/users/andrewvc/following{/other_user}","gists_url":"https://api.github.com/users/andrewvc/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewvc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewvc/subscriptions","organizations_url":"https://api.github.com/users/andrewvc/orgs","repos_url":"https://api.github.com/users/andrewvc/repos","events_url":"https://api.github.com/users/andrewvc/events{/privacy}","received_events_url":"https://api.github.com/users/andrewvc/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/logstash/labels/design","name":"design","color":"eb6420"},{"url":"https://api.github.com/repos/elastic/logstash/labels/enhancement","name":"enhancement","color":"009800"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-03-30T17:06:08Z","updated_at":"2016-03-30T21:58:12Z","closed_at":null,"body":"This line of inquiry was inspired by the question here: https://github.com/logstash-plugins/logstash-input-rabbitmq/issues/58\r\n\r\nThis is how it works today, which is great, but has a problem I will describe below.\r\n\r\n1. Logstash connects to rabbitmq server starts ingesting logs\r\n1. RabbitMQ Goes down, logstash repeatedly retries connecting\r\n1. Connection attempt succeeds, logstash succeeds in processing logs.\r\n\r\nThe above is great, however, if in step 2. Logstash is restarted, ALL inputs are blocked until the one RabbitMQ input connected to the down server is connected since its `register` method blocks waiting for an RMQ server.\r\n\r\nI propose the following:\r\n\r\n1. Input plugins should have their register method invoked asynchronously\r\n1. Input plugins should have an internal state machine with the states `:starting, :running, :stopped, :permanent_error, :transient_error`. These states could be accessible via API, and state transitions would be logged.\r\n\r\nThese state machines could be really useful from an operator standpoint if exposed from an API or used in a centralized config mgmt system."},"comment":{"url":"https://api.github.com/repos/elastic/logstash/issues/comments/203655502","html_url":"https://github.com/elastic/logstash/issues/4931#issuecomment-203655502","issue_url":"https://api.github.com/repos/elastic/logstash/issues/4931","id":203655502,"user":{"login":"talevy","id":388837,"avatar_url":"https://avatars.githubusercontent.com/u/388837?v=3","gravatar_id":"","url":"https://api.github.com/users/talevy","html_url":"https://github.com/talevy","followers_url":"https://api.github.com/users/talevy/followers","following_url":"https://api.github.com/users/talevy/following{/other_user}","gists_url":"https://api.github.com/users/talevy/gists{/gist_id}","starred_url":"https://api.github.com/users/talevy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/talevy/subscriptions","organizations_url":"https://api.github.com/users/talevy/orgs","repos_url":"https://api.github.com/users/talevy/repos","events_url":"https://api.github.com/users/talevy/events{/privacy}","received_events_url":"https://api.github.com/users/talevy/received_events","type":"User","site_admin":false},"created_at":"2016-03-30T21:58:12Z","updated_at":"2016-03-30T21:58:12Z","body":"as in:\r\n\r\nchange this existing snippet[[1](https://github.com/elastic/logstash/blob/master/logstash-core/lib/logstash/pipeline.rb#L378-L386)] into something like:\r\n\r\n```\r\n...\r\n@inputs.each do |input|\r\n    input_thread = Thread.new do\r\n        input.register\r\n        inputworker(plugin)\r\n    end\r\n    @input_threads << input_thread\r\nend\r\n```"}}