{"action":"created","issue":{"url":"https://api.github.com/repos/apache/ambari/issues/2397","repository_url":"https://api.github.com/repos/apache/ambari","labels_url":"https://api.github.com/repos/apache/ambari/issues/2397/labels{/name}","comments_url":"https://api.github.com/repos/apache/ambari/issues/2397/comments","events_url":"https://api.github.com/repos/apache/ambari/issues/2397/events","html_url":"https://github.com/apache/ambari/pull/2397","id":364860030,"node_id":"MDExOlB1bGxSZXF1ZXN0MjE4OTQ1MzQ2","number":2397,"title":"AMBARI-24708. Automation script for upgrade old style isilon cluster to the new mpack based structure (amagyar)","user":{"login":"zeroflag","id":641365,"node_id":"MDQ6VXNlcjY0MTM2NQ==","avatar_url":"https://avatars2.githubusercontent.com/u/641365?v=4","gravatar_id":"","url":"https://api.github.com/users/zeroflag","html_url":"https://github.com/zeroflag","followers_url":"https://api.github.com/users/zeroflag/followers","following_url":"https://api.github.com/users/zeroflag/following{/other_user}","gists_url":"https://api.github.com/users/zeroflag/gists{/gist_id}","starred_url":"https://api.github.com/users/zeroflag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zeroflag/subscriptions","organizations_url":"https://api.github.com/users/zeroflag/orgs","repos_url":"https://api.github.com/users/zeroflag/repos","events_url":"https://api.github.com/users/zeroflag/events{/privacy}","received_events_url":"https://api.github.com/users/zeroflag/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"zeroflag","id":641365,"node_id":"MDQ6VXNlcjY0MTM2NQ==","avatar_url":"https://avatars2.githubusercontent.com/u/641365?v=4","gravatar_id":"","url":"https://api.github.com/users/zeroflag","html_url":"https://github.com/zeroflag","followers_url":"https://api.github.com/users/zeroflag/followers","following_url":"https://api.github.com/users/zeroflag/following{/other_user}","gists_url":"https://api.github.com/users/zeroflag/gists{/gist_id}","starred_url":"https://api.github.com/users/zeroflag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zeroflag/subscriptions","organizations_url":"https://api.github.com/users/zeroflag/orgs","repos_url":"https://api.github.com/users/zeroflag/repos","events_url":"https://api.github.com/users/zeroflag/events{/privacy}","received_events_url":"https://api.github.com/users/zeroflag/received_events","type":"User","site_admin":false},"assignees":[{"login":"zeroflag","id":641365,"node_id":"MDQ6VXNlcjY0MTM2NQ==","avatar_url":"https://avatars2.githubusercontent.com/u/641365?v=4","gravatar_id":"","url":"https://api.github.com/users/zeroflag","html_url":"https://github.com/zeroflag","followers_url":"https://api.github.com/users/zeroflag/followers","following_url":"https://api.github.com/users/zeroflag/following{/other_user}","gists_url":"https://api.github.com/users/zeroflag/gists{/gist_id}","starred_url":"https://api.github.com/users/zeroflag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zeroflag/subscriptions","organizations_url":"https://api.github.com/users/zeroflag/orgs","repos_url":"https://api.github.com/users/zeroflag/repos","events_url":"https://api.github.com/users/zeroflag/events{/privacy}","received_events_url":"https://api.github.com/users/zeroflag/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-09-28T12:01:24Z","updated_at":"2018-10-01T11:04:20Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/ambari/pulls/2397","html_url":"https://github.com/apache/ambari/pull/2397","diff_url":"https://github.com/apache/ambari/pull/2397.diff","patch_url":"https://github.com/apache/ambari/pull/2397.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nAn automation script that can be used to convert old style isilon clusters with HDFS service to the new mpack based structure. The script replaces HDFS service to ONEFS and all of the HDFS clients to ONEFS_CLIENT. Config types like core-site, hdfs-site, hadoop-env should be preserved. The script uses the ambari REST api to carry out the conversion.\r\n\r\nAll of the operations in the script are supposed to be idempotent so that the user can rerun the script multiple times in case of a failure.\r\n\r\nThese are the main steps (see Upgrade>>perform())\r\n* Checks prerequisites (stack version, existence of ONEFS service in the stack)\r\n* Collects all of the hosts where an HDFS_CLIENT is installed (and saves this into a file)\r\n* Stops all services\r\n* Downloads core-site, hdfs-site, hadoop-env (and saves these into a file)\r\n* Delete HDFS service\r\n* Add ONEFS service and ONEFS_CLIENT service component\r\n* Adds core-site, hdfs-site, hadoop-env to ONEFS and sets the smart connect zone address\r\n* Adds ONEFS_CLIENTs to all hosts where an HDFS_CLIENT was installed\r\n* Starts all services\r\n\r\nIf the script fails after HDFS was deleted then the stored configs and hosts will be used when rerunning the script.\r\n\r\n## How was this patch tested?\r\n\r\n* Created a HDP 3.0 cluster with HDP and ZOOKEEPER\r\n* Ran the script\r\n* Checked that HDFS was deleted and all of the HDFS_CLIENTs were replaced to ONEFS_CLIENTs\r\n* Checked core-site, hadoop-env, hdfs-site, and onefs/smartconnectzone address\r\n\r\n```txt\r\n$ python hdfs_to_onefs_upgrade.py --host=c7401 --port=8080 --protocol=http --cluster=cc --user=admin --password=admin\r\nThis script will replace the HDFS service to ONEFS\r\nThe following prerequisites are required:\r\n  * ONEFS management package must be installed\r\n  * Ambari must be upgraded to >=v2.7.0\r\n  * Stack must be upgraded to HDP-3.0\r\n  * Is highly recommended to backup ambari database before you proceed.\r\nChecking Cluster: cc (http://c7401:8080/api/v1/clusters/cc)\r\nFound stack HDP-3.0\r\nPlease, confirm you have made backup of the Ambari db [y/n] (n)? y\r\nCollecting hosts with HDFS_CLIENT\r\nFound hosts [u'c7401.ambari.apache.org', u'c7402.ambari.apache.org', u'c7403.ambari.apache.org']\r\nStopping all services..\r\nDownloading core-site..\r\nDownloading hdfs-site..\r\nDownloading hadoop-env..\r\nDeleting HDFS..\r\nAdding ONEFS..\r\nAdding ONEFS config..\r\nAdding core-site\r\nAdding hdfs-site\r\nAdding hadoop-env-site\r\nAdding ONEFS_CLIENT to hosts: [u'c7401.ambari.apache.org', u'c7402.ambari.apache.org', u'c7403.ambari.apache.org']\r\nStarting all services..\r\n```\r\n\r\ncc: @ro-ket "},"comment":{"url":"https://api.github.com/repos/apache/ambari/issues/comments/425868929","html_url":"https://github.com/apache/ambari/pull/2397#issuecomment-425868929","issue_url":"https://api.github.com/repos/apache/ambari/issues/2397","id":425868929,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNTg2ODkyOQ==","user":{"login":"zeroflag","id":641365,"node_id":"MDQ6VXNlcjY0MTM2NQ==","avatar_url":"https://avatars2.githubusercontent.com/u/641365?v=4","gravatar_id":"","url":"https://api.github.com/users/zeroflag","html_url":"https://github.com/zeroflag","followers_url":"https://api.github.com/users/zeroflag/followers","following_url":"https://api.github.com/users/zeroflag/following{/other_user}","gists_url":"https://api.github.com/users/zeroflag/gists{/gist_id}","starred_url":"https://api.github.com/users/zeroflag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zeroflag/subscriptions","organizations_url":"https://api.github.com/users/zeroflag/orgs","repos_url":"https://api.github.com/users/zeroflag/repos","events_url":"https://api.github.com/users/zeroflag/events{/privacy}","received_events_url":"https://api.github.com/users/zeroflag/received_events","type":"User","site_admin":false},"created_at":"2018-10-01T11:04:20Z","updated_at":"2018-10-01T11:04:20Z","author_association":"CONTRIBUTOR","body":"retest this please\r\n"}}
{"actor":{"display_login":"zeroflag"}}