{"action":"created","issue":{"url":"https://api.github.com/repos/apache/incubator-metron/issues/363","repository_url":"https://api.github.com/repos/apache/incubator-metron","labels_url":"https://api.github.com/repos/apache/incubator-metron/issues/363/labels{/name}","comments_url":"https://api.github.com/repos/apache/incubator-metron/issues/363/comments","events_url":"https://api.github.com/repos/apache/incubator-metron/issues/363/events","html_url":"https://github.com/apache/incubator-metron/pull/363","id":190696847,"number":363,"title":"METRON-578: Missing error handling bolts for enrichment and threat intel","user":{"login":"justinleet","id":5077341,"avatar_url":"https://avatars.githubusercontent.com/u/5077341?v=3","gravatar_id":"","url":"https://api.github.com/users/justinleet","html_url":"https://github.com/justinleet","followers_url":"https://api.github.com/users/justinleet/followers","following_url":"https://api.github.com/users/justinleet/following{/other_user}","gists_url":"https://api.github.com/users/justinleet/gists{/gist_id}","starred_url":"https://api.github.com/users/justinleet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinleet/subscriptions","organizations_url":"https://api.github.com/users/justinleet/orgs","repos_url":"https://api.github.com/users/justinleet/repos","events_url":"https://api.github.com/users/justinleet/events{/privacy}","received_events_url":"https://api.github.com/users/justinleet/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":17,"created_at":"2016-11-21T12:01:41Z","updated_at":"2016-11-23T16:18:19Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/incubator-metron/pulls/363","html_url":"https://github.com/apache/incubator-metron/pull/363","diff_url":"https://github.com/apache/incubator-metron/pull/363.diff","patch_url":"https://github.com/apache/incubator-metron/pull/363.patch"},"body":"In the flux files, added outputs for the various `GenericEnrichmentBolt` used throughout, to tie their error output streams to Kafka topics.  Specifically, enrichment is tied to enrichments_error and threat intel is tied to threatintel_error.\r\n\r\nQuick-dev and the Ambari mpack are updated to ensure the topics are created, so that no errors occur on usage.  Both were run up to ensure that the topics were created and the flux files don't give errors.\r\n\r\nTo test, an adjustment was made to the `EnrichmentIntegrationTest` to have an enrichment bolt that always throws errors.  This bolt is derived from `GenericEnrichmentBolt` (which has been slightly refactored) to ensure that it uses the same error logic, but ignores the actual logic that leads to errors.  This was done because I couldn't find input that resulted in the error stream code path in actual practice.  Other errors can result, but don't go to the error stream (e.g. JSON decoding errors).\r\n\r\nAs a note, the `ErrorEnrichmentBolt` invariably ends up in the stack traces being printed (which is not handled by `ErrorEnrichmentBolt` or `GenericEnrichmentBolt`).\r\n\r\nGiven that the test explicitly calls `UnitTestHelper.verboseLogging()`, I'm not sure how to suppress that without killing actual errors at the same time is.  Open to suggestions if someone knows a more appropriate way to handle it (if we care).\r\n"},"comment":{"url":"https://api.github.com/repos/apache/incubator-metron/issues/comments/262560769","html_url":"https://github.com/apache/incubator-metron/pull/363#issuecomment-262560769","issue_url":"https://api.github.com/repos/apache/incubator-metron/issues/363","id":262560769,"user":{"login":"ottobackwards","id":1111551,"avatar_url":"https://avatars.githubusercontent.com/u/1111551?v=3","gravatar_id":"","url":"https://api.github.com/users/ottobackwards","html_url":"https://github.com/ottobackwards","followers_url":"https://api.github.com/users/ottobackwards/followers","following_url":"https://api.github.com/users/ottobackwards/following{/other_user}","gists_url":"https://api.github.com/users/ottobackwards/gists{/gist_id}","starred_url":"https://api.github.com/users/ottobackwards/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ottobackwards/subscriptions","organizations_url":"https://api.github.com/users/ottobackwards/orgs","repos_url":"https://api.github.com/users/ottobackwards/repos","events_url":"https://api.github.com/users/ottobackwards/events{/privacy}","received_events_url":"https://api.github.com/users/ottobackwards/received_events","type":"User","site_admin":false},"created_at":"2016-11-23T16:18:19Z","updated_at":"2016-11-23T16:18:19Z","body":"sorry, my mistake\r\ni'll run this right away"}}
{"actor":{"display_login":"ottobackwards"}}