{"action":"created","comment":{"url":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/comments/64438215","id":64438215,"diff_hunk":"@@ -25,6 +25,7 @@\n \n import azkaban.utils.Props;\n \n+\n public abstract class HadoopSecurityManager {\n \n   public static final String ENABLE_PROXYING = \"azkaban.should.proxy\"; // boolean","path":"plugins/hadoopsecuritymanager-common/src/azkaban/security/commons/HadoopSecurityManager.java","position":7,"original_position":7,"commit_id":"511a32c5ab380159555321a46a44aac75e2a9cf8","original_commit_id":"511a32c5ab380159555321a46a44aac75e2a9cf8","user":{"login":"logiclord","id":770329,"avatar_url":"https://avatars.githubusercontent.com/u/770329?v=3","gravatar_id":"","url":"https://api.github.com/users/logiclord","html_url":"https://github.com/logiclord","followers_url":"https://api.github.com/users/logiclord/followers","following_url":"https://api.github.com/users/logiclord/following{/other_user}","gists_url":"https://api.github.com/users/logiclord/gists{/gist_id}","starred_url":"https://api.github.com/users/logiclord/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/logiclord/subscriptions","organizations_url":"https://api.github.com/users/logiclord/orgs","repos_url":"https://api.github.com/users/logiclord/repos","events_url":"https://api.github.com/users/logiclord/events{/privacy}","received_events_url":"https://api.github.com/users/logiclord/received_events","type":"User","site_admin":false},"body":"Please verify that you are not affecting hadoop file formatting.","created_at":"2016-05-24T17:32:04Z","updated_at":"2016-05-24T17:32:04Z","html_url":"https://github.com/azkaban/azkaban-plugins/pull/222#discussion_r64438215","pull_request_url":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/222","_links":{"self":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/comments/64438215"},"html":{"href":"https://github.com/azkaban/azkaban-plugins/pull/222#discussion_r64438215"},"pull_request":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/222"}}},"pull_request":{"url":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/222","id":70305493,"html_url":"https://github.com/azkaban/azkaban-plugins/pull/222","diff_url":"https://github.com/azkaban/azkaban-plugins/pull/222.diff","patch_url":"https://github.com/azkaban/azkaban-plugins/pull/222.patch","issue_url":"https://api.github.com/repos/azkaban/azkaban-plugins/issues/222","number":222,"state":"open","locked":false,"title":"Add support to hadoopJavaJob to fetch tokens from multi hcat locations.","user":{"login":"evlstyle","id":7330414,"avatar_url":"https://avatars.githubusercontent.com/u/7330414?v=3","gravatar_id":"","url":"https://api.github.com/users/evlstyle","html_url":"https://github.com/evlstyle","followers_url":"https://api.github.com/users/evlstyle/followers","following_url":"https://api.github.com/users/evlstyle/following{/other_user}","gists_url":"https://api.github.com/users/evlstyle/gists{/gist_id}","starred_url":"https://api.github.com/users/evlstyle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/evlstyle/subscriptions","organizations_url":"https://api.github.com/users/evlstyle/orgs","repos_url":"https://api.github.com/users/evlstyle/repos","events_url":"https://api.github.com/users/evlstyle/events{/privacy}","received_events_url":"https://api.github.com/users/evlstyle/received_events","type":"User","site_admin":false},"body":"Add support to hadoopJavaJob to fetch tokens from multi hcat locations. The feature will get the extra hcat thrift uris from the user input , fetch the hcat token and then store them in the token file . \r\n\r\nto specify and use the extra hcat location - \r\n1.  in the submitted job , specify the following job property - \r\n     extra.hcat.location=[ThriftUri1],[ThriftUri2] .... \r\nmultiple Thrift uris supported,  separate them by comma. \r\n2.  in the java code of the job, as each hiveClient instance will be bound to a single hcat location therefore user need to spin up separate hiveClients with corresponding settings, aka the thriftUri and the token signature to find the token an communicate with the server.  See the code blow for example - \r\n\r\nHiveConf hiveConf = new HiveConf();\r\nhiveConf.set(HiveConf.ConfVars.METASTOREURIS.varname, [ThriftUrl]);\r\nhiveConf.set(\"hive.metastore.token.signature\", [ThriftUrl]);\r\nHiveMetaStoreClient hiveClient = new HiveMetaStoreClient(hiveConf); \r\n\r\n....\r\n\r\nPlease note - \r\n1. If user doesn't specify the property for multi hcat location, the system will run with the default behavior, it will still fetch the token from the default hcat location defined in the hiveClient installed on the server and job will run as the usual. \r\n2. Job will fail - \r\na. user specified an invalid location . \r\nb.  The token fetch failed. \r\n","created_at":"2016-05-17T06:26:12Z","updated_at":"2016-05-24T17:32:04Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a2df2a36badad7a4346bacf9e33109837b66e341","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/222/commits","review_comments_url":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/222/comments","review_comment_url":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/comments{/number}","comments_url":"https://api.github.com/repos/azkaban/azkaban-plugins/issues/222/comments","statuses_url":"https://api.github.com/repos/azkaban/azkaban-plugins/statuses/511a32c5ab380159555321a46a44aac75e2a9cf8","head":{"label":"evlstyle:master","ref":"master","sha":"511a32c5ab380159555321a46a44aac75e2a9cf8","user":{"login":"evlstyle","id":7330414,"avatar_url":"https://avatars.githubusercontent.com/u/7330414?v=3","gravatar_id":"","url":"https://api.github.com/users/evlstyle","html_url":"https://github.com/evlstyle","followers_url":"https://api.github.com/users/evlstyle/followers","following_url":"https://api.github.com/users/evlstyle/following{/other_user}","gists_url":"https://api.github.com/users/evlstyle/gists{/gist_id}","starred_url":"https://api.github.com/users/evlstyle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/evlstyle/subscriptions","organizations_url":"https://api.github.com/users/evlstyle/orgs","repos_url":"https://api.github.com/users/evlstyle/repos","events_url":"https://api.github.com/users/evlstyle/events{/privacy}","received_events_url":"https://api.github.com/users/evlstyle/received_events","type":"User","site_admin":false},"repo":{"id":40509854,"name":"azkaban-plugins","full_name":"evlstyle/azkaban-plugins","owner":{"login":"evlstyle","id":7330414,"avatar_url":"https://avatars.githubusercontent.com/u/7330414?v=3","gravatar_id":"","url":"https://api.github.com/users/evlstyle","html_url":"https://github.com/evlstyle","followers_url":"https://api.github.com/users/evlstyle/followers","following_url":"https://api.github.com/users/evlstyle/following{/other_user}","gists_url":"https://api.github.com/users/evlstyle/gists{/gist_id}","starred_url":"https://api.github.com/users/evlstyle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/evlstyle/subscriptions","organizations_url":"https://api.github.com/users/evlstyle/orgs","repos_url":"https://api.github.com/users/evlstyle/repos","events_url":"https://api.github.com/users/evlstyle/events{/privacy}","received_events_url":"https://api.github.com/users/evlstyle/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/evlstyle/azkaban-plugins","description":"Plugins for Azkaban.","fork":true,"url":"https://api.github.com/repos/evlstyle/azkaban-plugins","forks_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/forks","keys_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/keys{/key_id}","collaborators_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/teams","hooks_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/hooks","issue_events_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/issues/events{/number}","events_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/events","assignees_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/assignees{/user}","branches_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/branches{/branch}","tags_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/tags","blobs_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/git/refs{/sha}","trees_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/git/trees{/sha}","statuses_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/statuses/{sha}","languages_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/languages","stargazers_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/stargazers","contributors_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/contributors","subscribers_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/subscribers","subscription_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/subscription","commits_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/commits{/sha}","git_commits_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/git/commits{/sha}","comments_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/comments{/number}","issue_comment_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/issues/comments{/number}","contents_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/contents/{+path}","compare_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/compare/{base}...{head}","merges_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/merges","archive_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/downloads","issues_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/issues{/number}","pulls_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/pulls{/number}","milestones_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/milestones{/number}","notifications_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/labels{/name}","releases_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/releases{/id}","deployments_url":"https://api.github.com/repos/evlstyle/azkaban-plugins/deployments","created_at":"2015-08-10T22:49:40Z","updated_at":"2016-05-11T23:36:20Z","pushed_at":"2016-05-20T23:52:15Z","git_url":"git://github.com/evlstyle/azkaban-plugins.git","ssh_url":"git@github.com:evlstyle/azkaban-plugins.git","clone_url":"https://github.com/evlstyle/azkaban-plugins.git","svn_url":"https://github.com/evlstyle/azkaban-plugins","homepage":"https://azkaban.github.io","size":257637,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"azkaban:master","ref":"master","sha":"e4347dde9f1da13fce0bf3db44a18e59b4146cbd","user":{"login":"azkaban","id":213618,"avatar_url":"https://avatars.githubusercontent.com/u/213618?v=3","gravatar_id":"","url":"https://api.github.com/users/azkaban","html_url":"https://github.com/azkaban","followers_url":"https://api.github.com/users/azkaban/followers","following_url":"https://api.github.com/users/azkaban/following{/other_user}","gists_url":"https://api.github.com/users/azkaban/gists{/gist_id}","starred_url":"https://api.github.com/users/azkaban/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/azkaban/subscriptions","organizations_url":"https://api.github.com/users/azkaban/orgs","repos_url":"https://api.github.com/users/azkaban/repos","events_url":"https://api.github.com/users/azkaban/events{/privacy}","received_events_url":"https://api.github.com/users/azkaban/received_events","type":"Organization","site_admin":false},"repo":{"id":7566859,"name":"azkaban-plugins","full_name":"azkaban/azkaban-plugins","owner":{"login":"azkaban","id":213618,"avatar_url":"https://avatars.githubusercontent.com/u/213618?v=3","gravatar_id":"","url":"https://api.github.com/users/azkaban","html_url":"https://github.com/azkaban","followers_url":"https://api.github.com/users/azkaban/followers","following_url":"https://api.github.com/users/azkaban/following{/other_user}","gists_url":"https://api.github.com/users/azkaban/gists{/gist_id}","starred_url":"https://api.github.com/users/azkaban/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/azkaban/subscriptions","organizations_url":"https://api.github.com/users/azkaban/orgs","repos_url":"https://api.github.com/users/azkaban/repos","events_url":"https://api.github.com/users/azkaban/events{/privacy}","received_events_url":"https://api.github.com/users/azkaban/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/azkaban/azkaban-plugins","description":"Plugins for Azkaban.","fork":false,"url":"https://api.github.com/repos/azkaban/azkaban-plugins","forks_url":"https://api.github.com/repos/azkaban/azkaban-plugins/forks","keys_url":"https://api.github.com/repos/azkaban/azkaban-plugins/keys{/key_id}","collaborators_url":"https://api.github.com/repos/azkaban/azkaban-plugins/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/azkaban/azkaban-plugins/teams","hooks_url":"https://api.github.com/repos/azkaban/azkaban-plugins/hooks","issue_events_url":"https://api.github.com/repos/azkaban/azkaban-plugins/issues/events{/number}","events_url":"https://api.github.com/repos/azkaban/azkaban-plugins/events","assignees_url":"https://api.github.com/repos/azkaban/azkaban-plugins/assignees{/user}","branches_url":"https://api.github.com/repos/azkaban/azkaban-plugins/branches{/branch}","tags_url":"https://api.github.com/repos/azkaban/azkaban-plugins/tags","blobs_url":"https://api.github.com/repos/azkaban/azkaban-plugins/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/azkaban/azkaban-plugins/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/azkaban/azkaban-plugins/git/refs{/sha}","trees_url":"https://api.github.com/repos/azkaban/azkaban-plugins/git/trees{/sha}","statuses_url":"https://api.github.com/repos/azkaban/azkaban-plugins/statuses/{sha}","languages_url":"https://api.github.com/repos/azkaban/azkaban-plugins/languages","stargazers_url":"https://api.github.com/repos/azkaban/azkaban-plugins/stargazers","contributors_url":"https://api.github.com/repos/azkaban/azkaban-plugins/contributors","subscribers_url":"https://api.github.com/repos/azkaban/azkaban-plugins/subscribers","subscription_url":"https://api.github.com/repos/azkaban/azkaban-plugins/subscription","commits_url":"https://api.github.com/repos/azkaban/azkaban-plugins/commits{/sha}","git_commits_url":"https://api.github.com/repos/azkaban/azkaban-plugins/git/commits{/sha}","comments_url":"https://api.github.com/repos/azkaban/azkaban-plugins/comments{/number}","issue_comment_url":"https://api.github.com/repos/azkaban/azkaban-plugins/issues/comments{/number}","contents_url":"https://api.github.com/repos/azkaban/azkaban-plugins/contents/{+path}","compare_url":"https://api.github.com/repos/azkaban/azkaban-plugins/compare/{base}...{head}","merges_url":"https://api.github.com/repos/azkaban/azkaban-plugins/merges","archive_url":"https://api.github.com/repos/azkaban/azkaban-plugins/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/azkaban/azkaban-plugins/downloads","issues_url":"https://api.github.com/repos/azkaban/azkaban-plugins/issues{/number}","pulls_url":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls{/number}","milestones_url":"https://api.github.com/repos/azkaban/azkaban-plugins/milestones{/number}","notifications_url":"https://api.github.com/repos/azkaban/azkaban-plugins/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/azkaban/azkaban-plugins/labels{/name}","releases_url":"https://api.github.com/repos/azkaban/azkaban-plugins/releases{/id}","deployments_url":"https://api.github.com/repos/azkaban/azkaban-plugins/deployments","created_at":"2013-01-11T20:53:36Z","updated_at":"2016-05-19T00:30:43Z","pushed_at":"2016-05-23T21:58:16Z","git_url":"git://github.com/azkaban/azkaban-plugins.git","ssh_url":"git@github.com:azkaban/azkaban-plugins.git","clone_url":"https://github.com/azkaban/azkaban-plugins.git","svn_url":"https://github.com/azkaban/azkaban-plugins","homepage":"https://azkaban.github.io","size":358523,"stargazers_count":45,"watchers_count":45,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":84,"mirror_url":null,"open_issues_count":28,"forks":84,"open_issues":28,"watchers":45,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/222"},"html":{"href":"https://github.com/azkaban/azkaban-plugins/pull/222"},"issue":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/issues/222"},"comments":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/issues/222/comments"},"review_comments":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/222/comments"},"review_comment":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/pulls/222/commits"},"statuses":{"href":"https://api.github.com/repos/azkaban/azkaban-plugins/statuses/511a32c5ab380159555321a46a44aac75e2a9cf8"}}}}
{"actor":{"display_login":"logiclord"}}