{"action":"created","issue":{"url":"https://api.github.com/repos/apache/ambari/issues/951","repository_url":"https://api.github.com/repos/apache/ambari","labels_url":"https://api.github.com/repos/apache/ambari/issues/951/labels{/name}","comments_url":"https://api.github.com/repos/apache/ambari/issues/951/comments","events_url":"https://api.github.com/repos/apache/ambari/issues/951/events","html_url":"https://github.com/apache/ambari/pull/951","id":312911840,"number":951,"title":"[AMBARI-23526] Make sure that we update topology cache on agent side when Kerberizing the cluster","user":{"login":"smolnar82","id":34065904,"avatar_url":"https://avatars3.githubusercontent.com/u/34065904?v=4","gravatar_id":"","url":"https://api.github.com/users/smolnar82","html_url":"https://github.com/smolnar82","followers_url":"https://api.github.com/users/smolnar82/followers","following_url":"https://api.github.com/users/smolnar82/following{/other_user}","gists_url":"https://api.github.com/users/smolnar82/gists{/gist_id}","starred_url":"https://api.github.com/users/smolnar82/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smolnar82/subscriptions","organizations_url":"https://api.github.com/users/smolnar82/orgs","repos_url":"https://api.github.com/users/smolnar82/repos","events_url":"https://api.github.com/users/smolnar82/events{/privacy}","received_events_url":"https://api.github.com/users/smolnar82/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"smolnar82","id":34065904,"avatar_url":"https://avatars3.githubusercontent.com/u/34065904?v=4","gravatar_id":"","url":"https://api.github.com/users/smolnar82","html_url":"https://github.com/smolnar82","followers_url":"https://api.github.com/users/smolnar82/followers","following_url":"https://api.github.com/users/smolnar82/following{/other_user}","gists_url":"https://api.github.com/users/smolnar82/gists{/gist_id}","starred_url":"https://api.github.com/users/smolnar82/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smolnar82/subscriptions","organizations_url":"https://api.github.com/users/smolnar82/orgs","repos_url":"https://api.github.com/users/smolnar82/repos","events_url":"https://api.github.com/users/smolnar82/events{/privacy}","received_events_url":"https://api.github.com/users/smolnar82/received_events","type":"User","site_admin":false},"assignees":[{"login":"smolnar82","id":34065904,"avatar_url":"https://avatars3.githubusercontent.com/u/34065904?v=4","gravatar_id":"","url":"https://api.github.com/users/smolnar82","html_url":"https://github.com/smolnar82","followers_url":"https://api.github.com/users/smolnar82/followers","following_url":"https://api.github.com/users/smolnar82/following{/other_user}","gists_url":"https://api.github.com/users/smolnar82/gists{/gist_id}","starred_url":"https://api.github.com/users/smolnar82/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smolnar82/subscriptions","organizations_url":"https://api.github.com/users/smolnar82/orgs","repos_url":"https://api.github.com/users/smolnar82/repos","events_url":"https://api.github.com/users/smolnar82/events{/privacy}","received_events_url":"https://api.github.com/users/smolnar82/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2018-04-10T13:00:20Z","updated_at":"2018-04-11T08:39:14Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/ambari/pulls/951","html_url":"https://github.com/apache/ambari/pull/951","diff_url":"https://github.com/apache/ambari/pull/951.diff","patch_url":"https://github.com/apache/ambari/pull/951.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThere is a cache (in fact there are multiple caches) holding properties on different levels (host, component, etc...) in `ambari-agent` to support better performance. The problem was that this cache was not updated when we enabled Kerberos so that `ambari-agent` did not realize that unlimited key JCE policy should have been installed.\r\nTo fix this issue `ambari-server` should send an event to the agent upon cluster Kerberization to update `topology cache` (this is where `componentLevelParams` are being stored).\r\n\r\n## How was this patch tested?\r\n\r\nRunning Junit tests and added a new one to test the new class. Latest results in `ambari-server`:\r\n```\r\nambari-server smolnar$ mvn clean install\r\n...\r\n[INFO] ------------------------------------------------------------------------\r\n[INFO] BUILD SUCCESS\r\n[INFO] ------------------------------------------------------------------------\r\n[INFO] Total time: 27:00 min\r\n[INFO] Finished at: 2018-04-10T21:28:48+02:00\r\n[INFO] Final Memory: 209M/836M\r\n[INFO] ------------------------------------------------------------------------\r\n```\r\n\r\nBesides unit testing the following E2E test was executed:\r\n\r\n1. had a cluster with HDFS only; created a snapshot here\r\n2. Kerberized the cluster and confirmed that on c7402 - where only `ambari-agent` was running - the unlimited key JCE policy was **_not_** installed\r\n3. built a new `ambari-server.jar` with my changes\r\n4. went back to the snapshot (I created at step 1) and replaced the original `ambari-server.jar` with the new one\r\n5. changed the log level in `shared_initialization.py` to `INFO`\r\n6. restarted the server and the agents\r\n7. Kerberized the cluster: the new stage (`Triggering Topology Update Event`) appeared and unlimited key JCE policy was installed as part of the `DataNode` start step\r\n\r\n```\r\n[root@c7402 data]# /usr/jdk64/jdk1.8.0_112/bin/java -jar /var/lib/ambari-agent/tools/jcepolicyinfo.jar -tu\r\nUnlimited Key JCE Policy: true\r\n```\r\n\r\n<img width=\"1606\" alt=\"screen shot 2018-04-10 at 9 41 17 pm\" src=\"https://user-images.githubusercontent.com/34065904/38579900-b9cc0d6a-3d08-11e8-9d12-3f1619ff0173.png\">\r\n\r\n<img width=\"1606\" alt=\"screen shot 2018-04-10 at 9 43 01 pm\" src=\"https://user-images.githubusercontent.com/34065904/38579902-b9e6745c-3d08-11e8-8f2a-2075af6e6d7a.png\">\r\n"},"comment":{"url":"https://api.github.com/repos/apache/ambari/issues/comments/380374079","html_url":"https://github.com/apache/ambari/pull/951#issuecomment-380374079","issue_url":"https://api.github.com/repos/apache/ambari/issues/951","id":380374079,"user":{"login":"asfgit","id":1341245,"avatar_url":"https://avatars1.githubusercontent.com/u/1341245?v=4","gravatar_id":"","url":"https://api.github.com/users/asfgit","html_url":"https://github.com/asfgit","followers_url":"https://api.github.com/users/asfgit/followers","following_url":"https://api.github.com/users/asfgit/following{/other_user}","gists_url":"https://api.github.com/users/asfgit/gists{/gist_id}","starred_url":"https://api.github.com/users/asfgit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asfgit/subscriptions","organizations_url":"https://api.github.com/users/asfgit/orgs","repos_url":"https://api.github.com/users/asfgit/repos","events_url":"https://api.github.com/users/asfgit/events{/privacy}","received_events_url":"https://api.github.com/users/asfgit/received_events","type":"User","site_admin":false},"created_at":"2018-04-11T08:39:14Z","updated_at":"2018-04-11T08:39:14Z","author_association":"NONE","body":"\nRefer to this link for build results (access rights to CI server needed): \nhttps://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1764/\nTest PASSed.\n"}}
{"actor":{"display_login":"asfgit"}}