{"action":"created","issue":{"url":"https://api.github.com/repos/mozilla-services/heka/issues/1863","repository_url":"https://api.github.com/repos/mozilla-services/heka","labels_url":"https://api.github.com/repos/mozilla-services/heka/issues/1863/labels{/name}","comments_url":"https://api.github.com/repos/mozilla-services/heka/issues/1863/comments","events_url":"https://api.github.com/repos/mozilla-services/heka/issues/1863/events","html_url":"https://github.com/mozilla-services/heka/pull/1863","id":135820400,"number":1863,"title":"sandbox_filter: allow user to override Hostname","user":{"login":"nathanleiby","id":102242,"avatar_url":"https://avatars.githubusercontent.com/u/102242?v=3","gravatar_id":"","url":"https://api.github.com/users/nathanleiby","html_url":"https://github.com/nathanleiby","followers_url":"https://api.github.com/users/nathanleiby/followers","following_url":"https://api.github.com/users/nathanleiby/following{/other_user}","gists_url":"https://api.github.com/users/nathanleiby/gists{/gist_id}","starred_url":"https://api.github.com/users/nathanleiby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nathanleiby/subscriptions","organizations_url":"https://api.github.com/users/nathanleiby/orgs","repos_url":"https://api.github.com/users/nathanleiby/repos","events_url":"https://api.github.com/users/nathanleiby/events{/privacy}","received_events_url":"https://api.github.com/users/nathanleiby/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-02-23T18:20:44Z","updated_at":"2016-02-26T21:43:33Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/mozilla-services/heka/pulls/1863","html_url":"https://github.com/mozilla-services/heka/pull/1863","diff_url":"https://github.com/mozilla-services/heka/pull/1863.diff","patch_url":"https://github.com/mozilla-services/heka/pull/1863.patch"},"body":"This change allows the user to set a message's Hostname in a Lua sandbox filter. It maintains the existing behavior, unless the Hostname is explicitly overridden.\r\n\r\nThe behavior we'd like is to be able to filter messages (sometimes: enriching/modifying fields) and the re-inject nearly the same message after passing through the filter. We need to preserve the Hostname since its value is used directly in downstream Outputs. Ultimately, we need the hostname of the original log in order to properly do metrics/alerts/search/etc. \r\n\r\nWe're running Heka in a container and if you `inject_payload` to create a new message, you can't keep the original message's Hostname. Instead, the new message's hostname is Heka's container ID.\r\n\r\n--\r\n\r\nIn our pipeline, we use Hostname to mean \"the host the log originated from\".\r\n\r\nIt looks like the expected usage is [\"Hostname that generated the message\"](http://hekad.readthedocs.org/en/latest/message/#message-variables) -- perhaps you can illuminate if we are using it improperly. It seems like e.g. with an [RsyslogDecoder](http://hekad.readthedocs.org/en/latest/config/decoders/rsyslog.html#rsyslog-decoder), the `preserve_hostname` option means that Hostname should align with the host that created the log... that's exactly how we're using it.\r\n\r\nSome core plugins rely on the Hostname of the Heka message, for example the [ElasticSearch plugin pulls directly from this field](https://github.com/mozilla-services/heka/blob/149c483a52f0f4c9ea50b73805feebed1b7c45c3/plugins/elasticsearch/encoders.go#L369-L370).\r\n\r\nWe also use it in many of our custom plugins. ([Example](https://github.com/Clever/heka-clever-plugins/blob/master/lua/filters/signalfxbatch.lua#L40-L50))"},"comment":{"url":"https://api.github.com/repos/mozilla-services/heka/issues/comments/189492769","html_url":"https://github.com/mozilla-services/heka/pull/1863#issuecomment-189492769","issue_url":"https://api.github.com/repos/mozilla-services/heka/issues/1863","id":189492769,"user":{"login":"nathanleiby","id":102242,"avatar_url":"https://avatars.githubusercontent.com/u/102242?v=3","gravatar_id":"","url":"https://api.github.com/users/nathanleiby","html_url":"https://github.com/nathanleiby","followers_url":"https://api.github.com/users/nathanleiby/followers","following_url":"https://api.github.com/users/nathanleiby/following{/other_user}","gists_url":"https://api.github.com/users/nathanleiby/gists{/gist_id}","starred_url":"https://api.github.com/users/nathanleiby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nathanleiby/subscriptions","organizations_url":"https://api.github.com/users/nathanleiby/orgs","repos_url":"https://api.github.com/users/nathanleiby/repos","events_url":"https://api.github.com/users/nathanleiby/events{/privacy}","received_events_url":"https://api.github.com/users/nathanleiby/received_events","type":"User","site_admin":false},"created_at":"2016-02-26T21:43:33Z","updated_at":"2016-02-26T21:43:33Z","body":"Pushed an empty commit, and it fixed the build failure."}}