{"action":"opened","issue":{"url":"https://api.github.com/repos/logstash-plugins/logstash-output-elasticsearch/issues/520","repository_url":"https://api.github.com/repos/logstash-plugins/logstash-output-elasticsearch","labels_url":"https://api.github.com/repos/logstash-plugins/logstash-output-elasticsearch/issues/520/labels{/name}","comments_url":"https://api.github.com/repos/logstash-plugins/logstash-output-elasticsearch/issues/520/comments","events_url":"https://api.github.com/repos/logstash-plugins/logstash-output-elasticsearch/issues/520/events","html_url":"https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/520","id":193680391,"number":520,"title":"ES hosts array: Support for parsing Environment Variables","user":{"login":"berglh","id":12455338,"avatar_url":"https://avatars.githubusercontent.com/u/12455338?v=3","gravatar_id":"","url":"https://api.github.com/users/berglh","html_url":"https://github.com/berglh","followers_url":"https://api.github.com/users/berglh/followers","following_url":"https://api.github.com/users/berglh/following{/other_user}","gists_url":"https://api.github.com/users/berglh/gists{/gist_id}","starred_url":"https://api.github.com/users/berglh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/berglh/subscriptions","organizations_url":"https://api.github.com/users/berglh/orgs","repos_url":"https://api.github.com/users/berglh/repos","events_url":"https://api.github.com/users/berglh/events{/privacy}","received_events_url":"https://api.github.com/users/berglh/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-12-06T04:26:39Z","updated_at":"2016-12-06T04:26:39Z","closed_at":null,"body":"I am trying to use Environment Variables to configure the hosts item in the elasticsearch plugin. This is as per the document [Using Environment Variables in the Configuration](https://www.elastic.co/guide/en/logstash/current/environment-variables.html#_setting_the_tcp_port).\r\n\r\nI'm guessing this behaviour is by design. I haven't tried this on any of the other parameters in the output plugin, I'm assuming it was decided that not reading environments would be something most people would not want to do. \r\n\r\nI am using containers and planning on scaling, so I'm designing my config to scale with simple changes to environment only. This would be useful to me; perhaps also to other architectures where the IP or hostnames of nodes might arbitrarily change and are read from a service discovery key value store on startup; like etcd.\r\n\r\n- Version: 2.4.1 (Inside a Docker container)\r\n- Operating System: OEL 7.2\r\n- Config File (if you have sensitive info, please remove it):\r\n```\r\noutput {\r\n  if \"[@metadata][ls-dev-mode]\" == \"false\" {\r\n    elasticsearch {\r\n      hosts => [ \"${ES_HOSTS}\" ]\r\n      index => \"%{index}\"\r\n      flush_size => 1000\r\n      workers => 16\r\n      idle_flush_time => 30\r\n    }\r\n  }\r\n}\r\n```\r\n - Debug Log Output:\r\n```\r\n{:timestamp=>\"2016-12-06T04:11:30.967000+0000\", :message=>\"Pipeline aborted due to error\", :exception=>\"LogStash::ConfigurationError\", :error=>\"Host '${ES_HOSTS}' was specified, but is not valid! Use either a full URL or a hostname:port string!\", :backtrace=>[\"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:170:in `host_to_url'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:109:in `build_client'\", \"org/jruby/RubyArray.java:2414:in `map'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:109:in `build_client'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:20:in `initialize'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:44:in `build'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch.rb:134:in `build_client'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.1-java/lib/logstash/outputs/elasticsearch/common.rb:14:in `register'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/output_delegator.rb:75:in `register'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:181:in `start_workers'\", \"org/jruby/RubyArray.java:1613:in `each'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:181:in `start_workers'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:136:in `run'\", \"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/agent.rb:491:in `start_pipeline'\"], :level=>:error, :file=>\"logstash/agent.rb\", :line=>\"493\", :method=>\"start_pipeline\"}\r\n```\r\n\r\nI also tried setting hosts to: `%{[@metadata][es-hosts]}` after adding a metadata field from the env; but I realised this would only populate with a JSON payload parsing through the pipeline which only starts after the ES plugin has been initialised; it was never going to work."}}
{"actor":{"display_login":"berglh"}}