{"action":"created","comment":{"url":"https://api.github.com/repos/apache/incubator-metron/pulls/comments/83432385","pull_request_review_id":4278189,"id":83432385,"diff_hunk":"@@ -34,15 +34,15 @@\n    * @param rawMessage\n    * @return If null is returned, this is treated as an empty list.\n    */\n-  List<T> parse(byte[] rawMessage);\n+  List<T> parse(byte[] rawMessage, SensorParserConfig sensorParserConfig);","path":"metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/interfaces/MessageParser.java","position":17,"original_position":17,"commit_id":"d82866cca698922ec0cede51f11d89943c867bd5","original_commit_id":"b914fbc96d0e240137175312afe0474a48baad4e","user":{"login":"cestella","id":540359,"avatar_url":"https://avatars.githubusercontent.com/u/540359?v=3","gravatar_id":"","url":"https://api.github.com/users/cestella","html_url":"https://github.com/cestella","followers_url":"https://api.github.com/users/cestella/followers","following_url":"https://api.github.com/users/cestella/following{/other_user}","gists_url":"https://api.github.com/users/cestella/gists{/gist_id}","starred_url":"https://api.github.com/users/cestella/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cestella/subscriptions","organizations_url":"https://api.github.com/users/cestella/orgs","repos_url":"https://api.github.com/users/cestella/repos","events_url":"https://api.github.com/users/cestella/events{/privacy}","received_events_url":"https://api.github.com/users/cestella/received_events","type":"User","site_admin":false},"body":"Now that I think about it, you could actually do the following:\r\n* Provide a `configurationChanged` method in `MessageParser` with a default noop implementation\r\n* Change the argument to `parseOptionally` to take a Config and in the default implementation of `parseOptionally` call `configurationChanged` upon configuration change.  \r\n\r\nThis would leave `parse` alone and provide the capability to reconfigure on configuration changes seamlessly to parsers going forward.","created_at":"2016-10-14T14:26:37Z","updated_at":"2016-10-14T14:26:38Z","html_url":"https://github.com/apache/incubator-metron/pull/308#discussion_r83432385","pull_request_url":"https://api.github.com/repos/apache/incubator-metron/pulls/308","_links":{"self":{"href":"https://api.github.com/repos/apache/incubator-metron/pulls/comments/83432385"},"html":{"href":"https://github.com/apache/incubator-metron/pull/308#discussion_r83432385"},"pull_request":{"href":"https://api.github.com/repos/apache/incubator-metron/pulls/308"}}},"pull_request":{"url":"https://api.github.com/repos/apache/incubator-metron/pulls/308","id":89103111,"html_url":"https://github.com/apache/incubator-metron/pull/308","diff_url":"https://github.com/apache/incubator-metron/pull/308.diff","patch_url":"https://github.com/apache/incubator-metron/pull/308.patch","issue_url":"https://api.github.com/repos/apache/incubator-metron/issues/308","number":308,"state":"open","locked":false,"title":"Metron-498 Grok patterns are now read from zookeeper parser config property \"grokPattern\"","user":{"login":"merrimanr","id":311928,"avatar_url":"https://avatars.githubusercontent.com/u/311928?v=3","gravatar_id":"","url":"https://api.github.com/users/merrimanr","html_url":"https://github.com/merrimanr","followers_url":"https://api.github.com/users/merrimanr/followers","following_url":"https://api.github.com/users/merrimanr/following{/other_user}","gists_url":"https://api.github.com/users/merrimanr/gists{/gist_id}","starred_url":"https://api.github.com/users/merrimanr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/merrimanr/subscriptions","organizations_url":"https://api.github.com/users/merrimanr/orgs","repos_url":"https://api.github.com/users/merrimanr/repos","events_url":"https://api.github.com/users/merrimanr/events{/privacy}","received_events_url":"https://api.github.com/users/merrimanr/received_events","type":"User","site_admin":false},"body":"This PR moves Grok patterns from HDFS to a property in Zookeeper (\"SensorParserConfig.parserConfig.grokPattern\").  Most of the important changes were made to the GrokParser class and related tests.  There were 2 primary challenges with implementing this:\r\n\r\n- A change to a Grok pattern in Zookeeper should take effect immediately without a topology restart.  Currently the MessageParser does not have access to the Zookeeper config.\r\n- Grok patterns must now comply with JSON formatting standards (files had less restrictions)\r\n\r\nThe first challenge was solved by changing the MessageParser.parse method to include an additional \"SensorParserConfig\" parameter that the GrokParser can use to determine if the grok pattern has changed and reinit if necessary.  The ParserBolt now passes in the most recent SensorParserConfig to the MessageParser for every message processed.  This is similar to how the writer interfaces work.  I believe this approach is simpler than having the ParserBolt listen for changes and reconfigure/reinit the GrokParser.  Changing this interface makes this PR look much bigger than it actually is because all the parsers and parser tests had to be adjusted.  Most of files included are 1 line changes.\r\n\r\nThe second challenge was overcome by using '\\n' to separate lines inside the grok pattern JSON property.  An extra '\\' also had to be added to '\\' escape characters in the grok pattern files.  The advantage of this is that no special serialization/deserialization is needed inside of the GrokParser code.  The disadvantage is that the patterns are not as human-readable.\r\n\r\nAll unit tests and integration tests are passing.  I added an extra unit test in SampleGrokParserTest to simulate and test a config being updated.  I also tested yaf and squid sensors in quick-dev by updating the grokPattern properties and verifying the new patterns were applied. ","created_at":"2016-10-12T22:07:52Z","updated_at":"2016-10-14T14:26:38Z","closed_at":null,"merged_at":null,"merge_commit_sha":"5509f873e8f24797cd0836d2694c66c385e8de76","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/incubator-metron/pulls/308/commits","review_comments_url":"https://api.github.com/repos/apache/incubator-metron/pulls/308/comments","review_comment_url":"https://api.github.com/repos/apache/incubator-metron/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/incubator-metron/issues/308/comments","statuses_url":"https://api.github.com/repos/apache/incubator-metron/statuses/d82866cca698922ec0cede51f11d89943c867bd5","head":{"label":"merrimanr:METRON-498","ref":"METRON-498","sha":"d82866cca698922ec0cede51f11d89943c867bd5","user":{"login":"merrimanr","id":311928,"avatar_url":"https://avatars.githubusercontent.com/u/311928?v=3","gravatar_id":"","url":"https://api.github.com/users/merrimanr","html_url":"https://github.com/merrimanr","followers_url":"https://api.github.com/users/merrimanr/followers","following_url":"https://api.github.com/users/merrimanr/following{/other_user}","gists_url":"https://api.github.com/users/merrimanr/gists{/gist_id}","starred_url":"https://api.github.com/users/merrimanr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/merrimanr/subscriptions","organizations_url":"https://api.github.com/users/merrimanr/orgs","repos_url":"https://api.github.com/users/merrimanr/repos","events_url":"https://api.github.com/users/merrimanr/events{/privacy}","received_events_url":"https://api.github.com/users/merrimanr/received_events","type":"User","site_admin":false},"repo":{"id":62420865,"name":"incubator-metron","full_name":"merrimanr/incubator-metron","owner":{"login":"merrimanr","id":311928,"avatar_url":"https://avatars.githubusercontent.com/u/311928?v=3","gravatar_id":"","url":"https://api.github.com/users/merrimanr","html_url":"https://github.com/merrimanr","followers_url":"https://api.github.com/users/merrimanr/followers","following_url":"https://api.github.com/users/merrimanr/following{/other_user}","gists_url":"https://api.github.com/users/merrimanr/gists{/gist_id}","starred_url":"https://api.github.com/users/merrimanr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/merrimanr/subscriptions","organizations_url":"https://api.github.com/users/merrimanr/orgs","repos_url":"https://api.github.com/users/merrimanr/repos","events_url":"https://api.github.com/users/merrimanr/events{/privacy}","received_events_url":"https://api.github.com/users/merrimanr/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/merrimanr/incubator-metron","description":"Mirror of Apache Metron (Incubating)","fork":true,"url":"https://api.github.com/repos/merrimanr/incubator-metron","forks_url":"https://api.github.com/repos/merrimanr/incubator-metron/forks","keys_url":"https://api.github.com/repos/merrimanr/incubator-metron/keys{/key_id}","collaborators_url":"https://api.github.com/repos/merrimanr/incubator-metron/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/merrimanr/incubator-metron/teams","hooks_url":"https://api.github.com/repos/merrimanr/incubator-metron/hooks","issue_events_url":"https://api.github.com/repos/merrimanr/incubator-metron/issues/events{/number}","events_url":"https://api.github.com/repos/merrimanr/incubator-metron/events","assignees_url":"https://api.github.com/repos/merrimanr/incubator-metron/assignees{/user}","branches_url":"https://api.github.com/repos/merrimanr/incubator-metron/branches{/branch}","tags_url":"https://api.github.com/repos/merrimanr/incubator-metron/tags","blobs_url":"https://api.github.com/repos/merrimanr/incubator-metron/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/merrimanr/incubator-metron/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/merrimanr/incubator-metron/git/refs{/sha}","trees_url":"https://api.github.com/repos/merrimanr/incubator-metron/git/trees{/sha}","statuses_url":"https://api.github.com/repos/merrimanr/incubator-metron/statuses/{sha}","languages_url":"https://api.github.com/repos/merrimanr/incubator-metron/languages","stargazers_url":"https://api.github.com/repos/merrimanr/incubator-metron/stargazers","contributors_url":"https://api.github.com/repos/merrimanr/incubator-metron/contributors","subscribers_url":"https://api.github.com/repos/merrimanr/incubator-metron/subscribers","subscription_url":"https://api.github.com/repos/merrimanr/incubator-metron/subscription","commits_url":"https://api.github.com/repos/merrimanr/incubator-metron/commits{/sha}","git_commits_url":"https://api.github.com/repos/merrimanr/incubator-metron/git/commits{/sha}","comments_url":"https://api.github.com/repos/merrimanr/incubator-metron/comments{/number}","issue_comment_url":"https://api.github.com/repos/merrimanr/incubator-metron/issues/comments{/number}","contents_url":"https://api.github.com/repos/merrimanr/incubator-metron/contents/{+path}","compare_url":"https://api.github.com/repos/merrimanr/incubator-metron/compare/{base}...{head}","merges_url":"https://api.github.com/repos/merrimanr/incubator-metron/merges","archive_url":"https://api.github.com/repos/merrimanr/incubator-metron/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/merrimanr/incubator-metron/downloads","issues_url":"https://api.github.com/repos/merrimanr/incubator-metron/issues{/number}","pulls_url":"https://api.github.com/repos/merrimanr/incubator-metron/pulls{/number}","milestones_url":"https://api.github.com/repos/merrimanr/incubator-metron/milestones{/number}","notifications_url":"https://api.github.com/repos/merrimanr/incubator-metron/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/merrimanr/incubator-metron/labels{/name}","releases_url":"https://api.github.com/repos/merrimanr/incubator-metron/releases{/id}","deployments_url":"https://api.github.com/repos/merrimanr/incubator-metron/deployments","created_at":"2016-07-01T21:33:17Z","updated_at":"2016-07-01T21:33:20Z","pushed_at":"2016-10-13T19:09:04Z","git_url":"git://github.com/merrimanr/incubator-metron.git","ssh_url":"git@github.com:merrimanr/incubator-metron.git","clone_url":"https://github.com/merrimanr/incubator-metron.git","svn_url":"https://github.com/merrimanr/incubator-metron","homepage":null,"size":46687,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"3a4023382b24d3fc3b874820c9bd81317ee00656","user":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":47607561,"name":"incubator-metron","full_name":"apache/incubator-metron","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/incubator-metron","description":"Mirror of Apache Metron (Incubating)","fork":false,"url":"https://api.github.com/repos/apache/incubator-metron","forks_url":"https://api.github.com/repos/apache/incubator-metron/forks","keys_url":"https://api.github.com/repos/apache/incubator-metron/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/incubator-metron/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/incubator-metron/teams","hooks_url":"https://api.github.com/repos/apache/incubator-metron/hooks","issue_events_url":"https://api.github.com/repos/apache/incubator-metron/issues/events{/number}","events_url":"https://api.github.com/repos/apache/incubator-metron/events","assignees_url":"https://api.github.com/repos/apache/incubator-metron/assignees{/user}","branches_url":"https://api.github.com/repos/apache/incubator-metron/branches{/branch}","tags_url":"https://api.github.com/repos/apache/incubator-metron/tags","blobs_url":"https://api.github.com/repos/apache/incubator-metron/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/incubator-metron/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/incubator-metron/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/incubator-metron/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/incubator-metron/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/incubator-metron/languages","stargazers_url":"https://api.github.com/repos/apache/incubator-metron/stargazers","contributors_url":"https://api.github.com/repos/apache/incubator-metron/contributors","subscribers_url":"https://api.github.com/repos/apache/incubator-metron/subscribers","subscription_url":"https://api.github.com/repos/apache/incubator-metron/subscription","commits_url":"https://api.github.com/repos/apache/incubator-metron/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/incubator-metron/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/incubator-metron/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/incubator-metron/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/incubator-metron/contents/{+path}","compare_url":"https://api.github.com/repos/apache/incubator-metron/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/incubator-metron/merges","archive_url":"https://api.github.com/repos/apache/incubator-metron/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/incubator-metron/downloads","issues_url":"https://api.github.com/repos/apache/incubator-metron/issues{/number}","pulls_url":"https://api.github.com/repos/apache/incubator-metron/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/incubator-metron/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/incubator-metron/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/incubator-metron/labels{/name}","releases_url":"https://api.github.com/repos/apache/incubator-metron/releases{/id}","deployments_url":"https://api.github.com/repos/apache/incubator-metron/deployments","created_at":"2015-12-08T08:00:05Z","updated_at":"2016-10-13T14:18:23Z","pushed_at":"2016-10-14T14:15:56Z","git_url":"git://github.com/apache/incubator-metron.git","ssh_url":"git@github.com:apache/incubator-metron.git","clone_url":"https://github.com/apache/incubator-metron.git","svn_url":"https://github.com/apache/incubator-metron","homepage":null,"size":72050,"stargazers_count":119,"watchers_count":119,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":122,"mirror_url":"git://git.apache.org/incubator-metron.git","open_issues_count":10,"forks":122,"open_issues":10,"watchers":119,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/incubator-metron/pulls/308"},"html":{"href":"https://github.com/apache/incubator-metron/pull/308"},"issue":{"href":"https://api.github.com/repos/apache/incubator-metron/issues/308"},"comments":{"href":"https://api.github.com/repos/apache/incubator-metron/issues/308/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/incubator-metron/pulls/308/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/incubator-metron/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/incubator-metron/pulls/308/commits"},"statuses":{"href":"https://api.github.com/repos/apache/incubator-metron/statuses/d82866cca698922ec0cede51f11d89943c867bd5"}}}}
{"actor":{"display_login":"cestella"}}