{"action":"created","issue":{"url":"https://api.github.com/repos/ClusterHQ/flocker/issues/2885","repository_url":"https://api.github.com/repos/ClusterHQ/flocker","labels_url":"https://api.github.com/repos/ClusterHQ/flocker/issues/2885/labels{/name}","comments_url":"https://api.github.com/repos/ClusterHQ/flocker/issues/2885/comments","events_url":"https://api.github.com/repos/ClusterHQ/flocker/issues/2885/events","html_url":"https://github.com/ClusterHQ/flocker/issues/2885","id":171167937,"number":2885,"title":" Unit flocker-dataset-agent.service entered failed state.","user":{"login":"akamalov","id":7885257,"avatar_url":"https://avatars.githubusercontent.com/u/7885257?v=3","gravatar_id":"","url":"https://api.github.com/users/akamalov","html_url":"https://github.com/akamalov","followers_url":"https://api.github.com/users/akamalov/followers","following_url":"https://api.github.com/users/akamalov/following{/other_user}","gists_url":"https://api.github.com/users/akamalov/gists{/gist_id}","starred_url":"https://api.github.com/users/akamalov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akamalov/subscriptions","organizations_url":"https://api.github.com/users/akamalov/orgs","repos_url":"https://api.github.com/users/akamalov/repos","events_url":"https://api.github.com/users/akamalov/events{/privacy}","received_events_url":"https://api.github.com/users/akamalov/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ClusterHQ/flocker/labels/kind:%20installing-flocker","name":"kind: installing-flocker","color":"006b75"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":45,"created_at":"2016-08-15T13:17:20Z","updated_at":"2016-08-22T18:19:15Z","closed_at":null,"body":"Greetings,\r\n\r\nI am having a problem with trying to start flocker-dataset-agent service.  Brief background: the environment consists of RHEL7.2 attempting to access PureStorage backend via iSCSI.  PureStorage drivers have been installed and test (which I will go further below). Let me first display environmentals:\r\n\r\n**Operating System:**\r\n```\r\nNAME=\"Red Hat Enterprise Linux Server\"\r\nVERSION=\"7.2 (Maipo)\"\r\nID=\"rhel\"\r\nID_LIKE=\"fedora\"\r\nVERSION_ID=\"7.2\"\r\nPRETTY_NAME=\"Red Hat Enterprise Linux Server 7.2 (Maipo)\"\r\nANSI_COLOR=\"0;31\"\r\nCPE_NAME=\"cpe:/o:redhat:enterprise_linux:7.2:GA:server\"\r\nHOME_URL=\"https://www.redhat.com/\"\r\nBUG_REPORT_URL=\"https://bugzilla.redhat.com/\"\r\n\r\nREDHAT_BUGZILLA_PRODUCT=\"Red Hat Enterprise Linux 7\"\r\nREDHAT_BUGZILLA_PRODUCT_VERSION=7.2\r\nREDHAT_SUPPORT_PRODUCT=\"Red Hat Enterprise Linux\"\r\nREDHAT_SUPPORT_PRODUCT_VERSION=\"7.2\"\r\n```\r\n\r\n**Flocker environment:**\r\n\r\n```\r\nclusterhq-python-flocker-1.13.0-1.x86_64\r\nclusterhq-flocker-docker-plugin-1.13.0-1.noarch\r\nclusterhq-flocker-node-1.13.0-1.noarch\r\n```\r\n\r\n**/etc/flocker/agent.yml**\r\n\r\n```\r\n\"version\": 1\r\n\"control-service\": \r\n  \"hostname\": 192.168.120.156 \r\n  \"port\": 4523\r\n\"dataset\": \r\n    \"backend\": \"purestorage_flasharray_flocker_driver\"\r\n    \"pure_ip\": 192.168.128.157 \r\n    \"pure_api_token\": \"50cac451-fefe-b635-9692-5870aada9c49\"\r\n    \"pure_chap_host_user\": \"server15\"\r\n    \"pure_chap_host_password\": \"XXXXXX\"\r\n```\r\n\r\n**PureStorage Driver:**\r\n\r\nPureStorage Driver GitHub link - https://github.com/PureStorage-OpenConnect/purestorage-flocker-driver\r\n\r\nRestart Flocker services, display current status:\r\n\r\n```\r\n##########################\r\n● flocker-container-agent.service - Flocker Container Agent\r\n   Loaded: loaded (/usr/lib/systemd/system/flocker-container-agent.service; enabled; vendor preset: disabled)\r\n   Active: active (running) since Mon 2016-08-15 08:58:04 EDT; 5min ago\r\n Main PID: 29760 (flocker-contain)\r\n   Memory: 63.4M\r\n   CGroup: /system.slice/flocker-container-agent.service\r\n           └─29760 /opt/flocker/bin/python /usr/sbin/flocker-container-agent --journald\r\n\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"task_uuid\": \"f1cb436f-5845-4152-b500-3bc59c636f4b\", \"error\": false, \"timestamp\": 1471266234.010053, \"message\": \"AgentAMP connection established (HOST:IPv4Address(TCP, '192.168.120.165', 52992) PEER:IPv4Address(TCP, '192.168.120.156', 4523))\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"fsm_identifier\": \"<flocker.node._loop.ClusterStatus object at 0x45f4110>\", \"fsm_input\": \"<ClusterStatusInputs=CONNECTED_TO_CONTROL_SERVICE>\", \"timestamp\": 1471266234.01245, \"fsm_rich_input\": \"<_ConnectedToControlService>\", \"action_status\": \"started\", \"task_uuid\": \"789f2609-d355-49d7-9e76-4cfb1db67acc\", \"action_type\": \"fsm:transition\", \"fsm_state\": \"<ClusterStatusStates=DISCONNECTED>\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"fsm_next_state\": \"<ClusterStatusStates=IGNORANT>\", \"task_level\": [2], \"action_type\": \"fsm:transition\", \"timestamp\": 1471266234.013306, \"fsm_output\": [\"<ClusterStatusOutputs=STORE_CLIENT>\"], \"task_uuid\": \"789f2609-d355-49d7-9e76-4cfb1db67acc\", \"action_status\": \"succeeded\"}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"task_uuid\": \"d11cc085-6f78-4523-9246-2149a23ae968\", \"error\": false, \"timestamp\": 1471266234.116931, \"message\": \"AgentAMP connection lost (HOST:IPv4Address(TCP, '192.168.120.165', 52992) PEER:IPv4Address(TCP, '192.168.120.156', 4523))\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"exception\": \"OpenSSL.SSL.Error\", \"reason\": \"[('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert certificate unknown'), ('SSL routines', 'SSL3_WRITE_BYTES', 'ssl handshake failure')]\", \"timestamp\": 1471266234.118531, \"traceback\": \"Traceback: <class 'OpenSSL.SSL.Error'>: [('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert certificate unknown'), ('SSL routines', 'SSL3_WRITE_BYTES', 'ssl handshake failure')]\\n/opt/flocker/lib/python2.7/site-packages/twisted/internet/posixbase.py:597:_doReadOrWrite\\n/opt/flocker/lib/python2.7/site-packages/twisted/internet/tcp.py:209:doRead\\n/opt/flocker/lib/python2.7/site-packages/twisted/internet/tcp.py:215:_dataReceived\\n/opt/flocker/lib/python2.7/site-packages/twisted/protocols/tls.py:421:dataReceived\\n--- <exception caught here> ---\\n/opt/flocker/lib/python2.7/site-packages/twisted/protocols/tls.py:569:_write\\n/opt/flocker/lib/python2.7/site-packages/OpenSSL/SSL.py:1271:send\\n/opt/flocker/lib/python2.7/site-packages/OpenSSL/SSL.py:1191:_raise_ssl_error\\n/opt/flocker/lib/python2.7/site-packages/OpenSSL/_util.py:48:exception_from_error_queue\\n\", \"message_type\": \"eliot:traceback\", \"task_uuid\": \"a49e6ac7-0aa4-43b4-a129-111b66af8f31\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"fsm_identifier\": \"<flocker.node._loop.ClusterStatus object at 0x45f4110>\", \"fsm_input\": \"<ClusterStatusInputs=DISCONNECTED_FROM_CONTROL_SERVICE>\", \"timestamp\": 1471266234.12051, \"fsm_rich_input\": null, \"action_status\": \"started\", \"task_uuid\": \"b670b5d0-86d3-4899-9f09-89bc2a8b0fd0\", \"action_type\": \"fsm:transition\", \"fsm_state\": \"<ClusterStatusStates=IGNORANT>\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"fsm_next_state\": \"<ClusterStatusStates=DISCONNECTED>\", \"task_level\": [2], \"action_type\": \"fsm:transition\", \"timestamp\": 1471266234.121409, \"fsm_output\": [], \"task_uuid\": \"b670b5d0-86d3-4899-9f09-89bc2a8b0fd0\", \"action_status\": \"succeeded\"}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"task_uuid\": \"6793744e-5b23-434b-a18e-2752789c0acc\", \"error\": false, \"timestamp\": 1471266234.122539, \"message\": \"<twisted.internet.tcp.Connector instance at 0x45ef7e8> will retry in 2 seconds\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"task_uuid\": \"31b1fe22-151f-419c-9b2f-7dce6c19772f\", \"error\": false, \"timestamp\": 1471266234.123821, \"message\": \"Stopping factory <twisted.internet.protocol.ReconnectingClientFactory instance at 0x45f6518>\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-container-agent[29760]: {\"task_uuid\": \"6d217799-8e1f-4b2f-8319-5ce55e26b5f0\", \"error\": true, \"timestamp\": 1471266234.176803, \"message\": \"Unhandled Error\\nTraceback (most recent call last):\\n  File \\\"/opt/flocker/lib/python2.7/site-packages/flocker/common/script.py\\\", line 295, in main\\n    self._react(run_and_log, [], _reactor=self._reactor)\\n  File \\\"/opt/flocker/lib/python2.7/site-packages/twisted/internet/task.py\\\", line 936, in react\\n    _reactor.run()\\n  File \\\"/opt/flocker/lib/python2.7/site-packages/twisted/internet/base.py\\\", line 1194, in run\\n    self.mainLoop()\\n  File \\\"/opt/flocker/lib/python2.7/site-packages/twisted/internet/base.py\\\", line 1203, in mainLoop\\n    self.runUntilCurrent()\\n--- <exception caught here> ---\\n  File \\\"/opt/flocker/lib/python2.7/site-packages/twisted/internet/base.py\\\", line 825, in runUntilCurrent\\n    call.func(*call.args, **call.kw)\\n  File \\\"/opt/flocker/lib/python2.7/site-packages/flocker/control/_protocol.py\\\", line 455, in <lambda>\\n    lambda: protocol.transport.abortConnection())\\nexceptions.AttributeError: 'NoneType' object has no attribute 'abortConnection'\\n\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\n\r\n##############################\r\n\r\n● flocker-dataset-agent.service - Flocker Dataset Agent\r\n   Loaded: loaded (/usr/lib/systemd/system/flocker-dataset-agent.service; enabled; vendor preset: disabled)\r\n   Active: failed (Result: start-limit) since Mon 2016-08-15 08:58:11 EDT; 5min ago\r\n  Process: 29869 ExecStart=/usr/sbin/flocker-dataset-agent --journald (code=exited, status=1/FAILURE)\r\n Main PID: 29869 (code=exited, status=1/FAILURE)\r\n\r\nAug 15 08:58:11 server15 systemd[1]: Unit flocker-dataset-agent.service entered failed state.\r\nAug 15 08:58:11 server15 systemd[1]: flocker-dataset-agent.service failed.\r\nAug 15 08:58:11 server15 systemd[1]: flocker-dataset-agent.service holdoff time over, scheduling restart.\r\nAug 15 08:58:11 server15 systemd[1]: start request repeated too quickly for flocker-dataset-agent.service\r\nAug 15 08:58:11 server15 systemd[1]: Failed to start Flocker Dataset Agent.\r\nAug 15 08:58:11 server15 systemd[1]: Unit flocker-dataset-agent.service entered failed state.\r\nAug 15 08:58:11 server15 systemd[1]: flocker-dataset-agent.service failed.\r\n\r\n##############################\r\n\r\n● flocker-docker-plugin.service - Flocker Docker Plugin\r\n   Loaded: loaded (/usr/lib/systemd/system/flocker-docker-plugin.service; enabled; vendor preset: disabled)\r\n   Active: active (running) since Mon 2016-08-15 08:58:04 EDT; 5min ago\r\n Main PID: 29781 (flocker-docker-)\r\n   Memory: 62.4M\r\n   CGroup: /system.slice/flocker-docker-plugin.service\r\n           └─29781 /opt/flocker/bin/python /usr/sbin/flocker-docker-plugin --journald\r\n\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"exception\": \"flocker.apiclient._client.NotFound\", \"task_level\": [1592, 4], \"action_type\": \"flocker:apiclient:http_request\", \"reason\": \"{\\\"description\\\": \\\"No node found with given era.\\\"}\", \"timestamp\": 1471266234.313487, \"task_uuid\": \"2dd0b28b-cdc9-434a-9dff-a592c304b836\", \"action_status\": \"failed\"}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"task_uuid\": \"c731df94-e1a3-4d88-a911-c3c1d85d971e\", \"error\": false, \"timestamp\": 1471266234.316437, \"message\": \"Stopping factory <twisted.web.client._HTTP11ClientFactory instance at 0x5b134d0>\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"request_body\": null, \"url\": \"https://mmaster1:4523/v1/state/nodes/by_era/61181a3e-978a-4e8c-a2b1-7f5e4ae1bf54\", \"timestamp\": 1471266234.415257, \"action_status\": \"started\", \"task_uuid\": \"2dd0b28b-cdc9-434a-9dff-a592c304b836\", \"action_type\": \"flocker:apiclient:http_request\", \"method\": \"GET\", \"task_level\": [1593, 1]}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"task_uuid\": \"2dd0b28b-cdc9-434a-9dff-a592c304b836\", \"error\": false, \"timestamp\": 1471266234.428725, \"message\": \"Starting factory <twisted.web.client._HTTP11ClientFactory instance at 0x5a2e128>\", \"message_type\": \"twisted:log\", \"task_level\": [1593, 3]}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"exception\": \"flocker.apiclient._client.NotFound\", \"task_level\": [1593, 4], \"action_type\": \"flocker:apiclient:http_request\", \"reason\": \"{\\\"description\\\": \\\"No node found with given era.\\\"}\", \"timestamp\": 1471266234.531891, \"task_uuid\": \"2dd0b28b-cdc9-434a-9dff-a592c304b836\", \"action_status\": \"failed\"}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"task_uuid\": \"37ffc3fe-c4ee-4bf0-b30a-ef0b1a900942\", \"error\": false, \"timestamp\": 1471266234.534984, \"message\": \"Stopping factory <twisted.web.client._HTTP11ClientFactory instance at 0x5a2e128>\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"request_body\": null, \"url\": \"https://mmaster1:4523/v1/state/nodes/by_era/61181a3e-978a-4e8c-a2b1-7f5e4ae1bf54\", \"timestamp\": 1471266234.633406, \"action_status\": \"started\", \"task_uuid\": \"2dd0b28b-cdc9-434a-9dff-a592c304b836\", \"action_type\": \"flocker:apiclient:http_request\", \"method\": \"GET\", \"task_level\": [1594, 1]}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"task_uuid\": \"2dd0b28b-cdc9-434a-9dff-a592c304b836\", \"error\": false, \"timestamp\": 1471266234.644697, \"message\": \"Starting factory <twisted.web.client._HTTP11ClientFactory instance at 0x5b114d0>\", \"message_type\": \"twisted:log\", \"task_level\": [1594, 3]}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"exception\": \"flocker.apiclient._client.NotFound\", \"task_level\": [1594, 4], \"action_type\": \"flocker:apiclient:http_request\", \"reason\": \"{\\\"description\\\": \\\"No node found with given era.\\\"}\", \"timestamp\": 1471266234.732458, \"task_uuid\": \"2dd0b28b-cdc9-434a-9dff-a592c304b836\", \"action_status\": \"failed\"}\r\nAug 15 09:03:54 server15 flocker-docker-plugin[29781]: {\"task_uuid\": \"4dceda0b-7b13-47d3-b17f-1a7fbeed1a50\", \"error\": false, \"timestamp\": 1471266234.733952, \"message\": \"Stopping factory <twisted.web.client._HTTP11ClientFactory instance at 0x5b114d0>\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\n\r\n```\r\nTrying to get more information on **flocker-dataset-agent service**:\r\n\r\n```\r\n[root@server1 ~]# journalctl -u flocker-dataset-agent.service -l -r\r\n-- Logs begin at Sat 2016-08-13 23:16:11 EDT, end at Mon 2016-08-15 09:06:33 EDT. --\r\nAug 15 08:58:11 server1 systemd[1]: flocker-dataset-agent.service failed.\r\nAug 15 08:58:11 server1 systemd[1]: Unit flocker-dataset-agent.service entered failed state.\r\nAug 15 08:58:11 server1 systemd[1]: Failed to start Flocker Dataset Agent.\r\nAug 15 08:58:11 server1 systemd[1]: start request repeated too quickly for flocker-dataset-agent.service\r\nAug 15 08:58:11 server1 systemd[1]: flocker-dataset-agent.service holdoff time over, scheduling restart.\r\nAug 15 08:58:11 server1 systemd[1]: flocker-dataset-agent.service failed.\r\nAug 15 08:58:11 server1 systemd[1]: Unit flocker-dataset-agent.service entered failed state.\r\nAug 15 08:58:11 server1 systemd[1]: flocker-dataset-agent.service: main process exited, code=exited, status=1/FAILURE\r\nAug 15 08:58:11 server1 flocker-dataset-agent[29869]: {\"task_uuid\": \"d10519ee-b417-4ccd-a90c-7c780a4d5b14\", \"error\": false, \"timestamp\": 1471265891.600116, \"message\": \"Main loop terminated.\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\nAug 15 08:58:11 server1 flocker-dataset-agent[29869]: {\"task_uuid\": \"8a2f9dff-07d1-4aab-b8c4-717536adf44e\", \"error\": true, \"timestamp\": 1471265891.599722, \"message\": \"main function encountered error\\nTraceback (most recent call last):\\n  File \\\"/opt/floc\r\nAug 15 08:58:11 server1 flocker-dataset-agent[29869]: {\"task_uuid\": \"f7b0c1cf-eefd-4007-8742-156177a62b0a\", \"error\": true, \"timestamp\": 1471265891.598659, \"message\": \"Unhandled Error\\nTraceback (most recent call last):\\n  File \\\"/opt/flocker/lib/python2.\r\nAug 15 08:58:11 server1 flocker-dataset-agent[29869]: {\"task_uuid\": \"1a59b3b6-386d-4c61-a938-ba687b38f8a0\", \"error\": false, \"timestamp\": 1471265891.587703, \"message\": \"Log opened.\", \"message_type\": \"twisted:log\", \"task_level\": [1]}\r\nAug 15 08:58:10 server1 systemd[1]: Starting Flocker Dataset Agent...\r\nAug 15 08:58:10 server1 systemd[1]: Started Flocker Dataset Agent.\r\n```\r\n\r\nTested PureStorage driver manually:\r\n\r\n```\r\n[root@server1 purestorage-flocker-driver]# python\r\nPython 3.4.2 (default, Apr 19 2016, 08:30:31) \r\n[GCC 4.8.4] on linux\r\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\r\n>>> import purestorage\r\n>>> fa=purestorage.FlashArray('192.168.128.157',api_token='50cac451-fefe-b635-9692-5870aada9c49')\r\n/opt/mesosphere/packages/python--e3169ded66609d3cb4055a3f9f8f0b1113a557a6/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\r\n  InsecureRequestWarning)\r\n/opt/mesosphere/packages/python--e3169ded66609d3cb4055a3f9f8f0b1113a557a6/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\r\n  InsecureRequestWarning)\r\n>>> fa.get()\r\n/opt/mesosphere/packages/python--e3169ded66609d3cb4055a3f9f8f0b1113a557a6/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\r\n  InsecureRequestWarning)\r\n{'revision': '201605132143+b186ed4', 'id': '7e4d75d0-ffc1-4e7c-880a-c044849e1993', 'version': '4.6.8', 'array_name': 'PURTPC0027'}\r\n>>> \r\n\r\n```\r\nAs you can see above, connectivity is there. We were able to connect, authenticate using a token and were able to retrieve PureStorage basic info (i.e., version, name, etc)..\r\n\r\nVerified the sanity of **/etc/flocker/node.crt:**\r\n\r\n```\r\n[root@server1 ~]# openssl x509 -text -in /etc/flocker/node.crt \r\nCertificate:\r\n    Data:\r\n        Version: 1 (0x0)\r\n        Serial Number:\r\n            fb:69:42:2e:a2:2e:52:1d:28:b9:59:d7:1d:06:21:13\r\n    Signature Algorithm: sha256WithRSAEncryption\r\n        Issuer: OU=06dc4747-5c8d-4878-8943-b7d34cbf08c4, CN=flocker\r\n        Validity\r\n            Not Before: Jun 29 15:15:36 2016 GMT\r\n            Not After : Jun 24 15:15:36 2036 GMT\r\n        Subject: OU=06dc4747-5c8d-4878-8943-b7d34cbf08c4, CN=node-f810470c-7140-4017-b61d-4d9d326e4a3f\r\n        Subject Public Key Info:\r\n            Public Key Algorithm: rsaEncryption\r\n                Public-Key: (4096 bit)\r\n                Modulus:\r\n                    00:98:f7:e4:d0:4d:84:a8:43:a7:97:f4:95:09:19:\r\n                    6f:d0:71:0b:76:56:3e:ca:14:86:3b:6f:03:be:ad:\r\n                    4a:fb:1a:54:5c:51:e7:80:67:3d:e1:1a:92:8a:f5:\r\n                    13:87:b1:e5:bd:16:72:de:53:85:31:f7:d3:c5:63:\r\n                    dd:51:90:bb:03:68:b4:9a:ed:bc:82:44:18:6c:4a:\r\n                    9d:21:32:c0:d1:bf:51:84:81:d4:ba:d6:ee:8d:6a:\r\n                    f2:07:f6:8c:ac:87:61:6e:92:8e:2f:62:6a:5a:18:\r\n                    38:ed:40:14:7f:87:e7:f0:cf:af:af:a7:94:54:b5:\r\n                    83:c8:4c:38:6f:72:27:13:ee:50:53:1d:f2:04:0d:\r\n                    70:d3:7f:a0:06:53:19:b9:dd:d7:85:4b:56:18:c7:\r\n                    f9:29:2a:23:7e:0a:fb:9f:8c:f0:4a:b9:fe:c8:1f:\r\n                    c6:97:3c:d9:a4:19:3a:5b:95:4b:87:d8:2a:71:55:\r\n                    01:92:f3:ed:85:b9:12:8b:b1:4c:ab:14:51:1f:48:\r\n                    47:bf:b2:36:ca:fd:4a:64:12:2f:c5:eb:bb:49:9f:\r\n                    12:03:84:d4:02:dc:90:13:6a:c7:f8:7e:ff:03:2c:\r\n                    16:c8:06:3d:a0:6a:59:e2:00:7b:1a:07:9f:9a:b8:\r\n                    12:95:5a:a2:28:ce:84:2d:d7:e8:03:ae:c6:9b:5c:\r\n                    60:c4:2c:c6:8b:31:7c:e7:02:aa:9d:85:1e:f5:b0:\r\n                    64:de:d9:78:29:f9:35:d2:db:64:28:01:6a:c4:7e:\r\n                    21:63:7d:09:0d:44:50:02:7c:23:ff:32:23:4e:5d:\r\n                    51:be:28:05:b2:8c:7c:26:d0:01:9e:4f:ca:bc:69:\r\n                    c1:a0:31:43:75:1a:f8:1e:7b:b7:f8:c0:88:2d:9b:\r\n                    2f:09:00:8f:9e:95:42:39:a6:21:3e:cf:f8:0a:27:\r\n                    f9:85:0d:fd:9a:14:3c:6e:2e:35:57:bc:b5:fa:ec:\r\n                    06:04:c3:cb:5a:e4:a1:cf:ca:05:d8:67:5a:70:27:\r\n                    39:0a:6d:9a:70:67:6a:a7:50:a2:25:01:d2:85:d5:\r\n                    40:ce:12:29:75:d2:98:c1:88:14:56:1e:6a:a6:26:\r\n                    f8:a2:3a:cc:36:57:e1:75:d5:00:48:cf:a4:bb:16:\r\n                    41:59:a6:39:79:0d:52:63:cf:9f:6a:cc:bf:fd:90:\r\n                    56:1c:de:eb:55:e8:a4:ed:62:4d:af:08:97:2b:a9:\r\n                    cc:14:53:02:51:84:8a:bb:c7:8b:cb:c9:32:b2:cc:\r\n                    7c:5d:8a:83:0c:97:06:28:bf:9f:c7:62:97:7b:e1:\r\n                    c5:9f:24:a6:c8:73:f4:a5:34:2a:3b:7c:a2:b1:6b:\r\n                    af:15:5a:99:72:27:19:49:b8:85:15:c8:39:48:72:\r\n                    ba:05:1b\r\n                Exponent: 65537 (0x10001)\r\n    Signature Algorithm: sha256WithRSAEncryption\r\n         9d:bc:37:b9:43:cd:89:ac:ad:50:c0:4e:83:03:26:0e:ce:42:\r\n         ed:1f:1f:d5:3d:06:63:b3:6c:d4:d0:7d:f8:26:99:32:11:4f:\r\n         ce:27:9f:ac:80:06:90:71:72:ef:7e:5f:a8:0a:86:ed:4d:65:\r\n         d5:4f:55:5a:26:eb:94:27:0b:23:62:39:4f:0a:b3:07:a4:c8:\r\n         dd:7c:b6:27:77:62:8c:f7:28:61:c0:56:bc:9c:b2:5e:08:36:\r\n         78:52:34:a5:9f:25:54:e0:0d:0c:76:5b:27:a9:fc:5c:72:82:\r\n         6d:82:65:11:fd:43:fa:29:d5:be:85:37:37:1b:4e:87:db:f5:\r\n         d0:5e:92:48:ba:c0:ef:8f:b1:d4:04:11:00:53:4d:b4:e8:74:\r\n         41:2e:6a:49:8a:3b:6a:b9:74:98:96:1a:74:f6:2f:de:5c:7c:\r\n         98:4b:7a:3a:1a:55:98:99:23:d8:8e:c7:1f:32:c1:10:4c:4b:\r\n         34:01:9c:bc:44:8b:eb:59:0e:c7:1a:0f:17:81:35:42:38:42:\r\n         72:4a:06:d9:f4:9c:94:9b:66:f9:fb:ae:04:b2:32:cc:3c:d6:\r\n         48:06:5b:c1:83:f6:ea:36:66:73:fc:1b:d0:46:91:8a:bf:12:\r\n         d5:0e:71:6a:e9:18:5e:37:45:97:d5:d3:8d:3d:28:33:ed:83:\r\n         9e:ca:79:d0:b2:b4:98:a7:ab:f9:da:48:b9:98:85:f7:5b:97:\r\n         09:af:08:04:ef:f9:33:fb:31:d1:b5:8f:e4:69:d8:a5:12:53:\r\n         92:cd:4f:17:d1:2d:ce:c7:c3:14:db:ac:dc:6c:4f:4c:79:2b:\r\n         1b:ed:a0:c9:40:29:bd:93:48:3e:2d:96:4b:ab:6c:09:fe:b3:\r\n         08:cd:4e:87:01:72:fb:d4:db:85:ed:5c:60:83:20:c2:c1:0f:\r\n         c4:ce:fb:bd:aa:80:bc:eb:49:7d:30:ee:49:f1:11:7c:6b:8d:\r\n         5c:aa:b0:9f:5d:ad:12:fa:fa:7f:aa:00:88:52:a9:a7:41:fc:\r\n         79:5d:9f:2e:12:81:95:07:f3:64:e5:b7:22:49:95:42:4e:50:\r\n         03:72:c4:ae:11:cf:ae:99:b5:47:5a:2c:a0:09:1e:98:0f:c7:\r\n         a0:8a:64:e3:4b:e0:49:70:db:6b:4e:bd:e3:14:3a:ca:1f:c7:\r\n         59:29:bc:28:41:fd:45:ea:be:fc:ed:c5:2e:ff:4a:57:ec:59:\r\n         42:44:08:33:18:50:fc:78:00:99:a7:50:2a:73:c6:3a:1c:63:\r\n         08:ce:53:45:0d:97:5a:50:08:ed:07:7f:90:35:56:a0:c4:c1:\r\n         5b:28:96:d0:1c:7a:7a:54:b6:98:d6:e4:ed:08:a7:94:91:a7:\r\n         86:e3:6f:36:e7:7f:6b:99\r\n-----BEGIN CERTIFICATE-----\r\nMIIFLDCCAxQCEQD7aUIuoi5SHSi5WdcdBiETMA0GCSqGSIb3DQEBCwUAMEExLTAr\r\nBgNVBAsMJDA2ZGM0NzQ3LTVjOGQtNDg3OC04OTQzLWI3ZDM0Y2JmMDhjNDEQMA4G\r\nA1UEAwwHZmxvY2tlcjAiGA8yMDE2MDYyOTE1MTUzNloYDzIwMzYwNjI0MTUxNTM2\r\nWjBjMS0wKwYDVQQLDCQwNmRjNDc0Ny01YzhkLTQ4NzgtODk0My1iN2QzNGNiZjA4\r\nYzQxMjAwBgNVBAMMKW5vZGUtZjgxMDQ3MGMtNzE0MC00MDE3LWI2MWQtNGQ5ZDMy\r\nNmU0YTNmMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAmPfk0E2EqEOn\r\nl/SVCRlv0HELdlY+yhSGO28Dvq1K+xpUXFHngGc94RqSivUTh7HlvRZy3lOFMffT\r\nxWPdUZC7A2i0mu28gkQYbEqdITLA0b9RhIHUutbujWryB/aMrIdhbpKOL2JqWhg4\r\n7UAUf4fn8M+vr6eUVLWDyEw4b3InE+5QUx3yBA1w03+gBlMZud3XhUtWGMf5KSoj\r\nfgr7n4zwSrn+yB/GlzzZpBk6W5VLh9gqcVUBkvPthbkSi7FMqxRRH0hHv7I2yv1K\r\nZBIvxeu7SZ8SA4TUAtyQE2rH+H7/AywWyAY9oGpZ4gB7GgefmrgSlVqiKM6ELdfo\r\nA67Gm1xgxCzGizF85wKqnYUe9bBk3tl4Kfk10ttkKAFqxH4hY30JDURQAnwj/zIj\r\nTl1RvigFsox8JtABnk/KvGnBoDFDdRr4Hnu3+MCILZsvCQCPnpVCOaYhPs/4Cif5\r\nhQ39mhQ8bi41V7y1+uwGBMPLWuShz8oF2GdacCc5Cm2acGdqp1CiJQHShdVAzhIp\r\nddKYwYgUVh5qpib4ojrMNlfhddUASM+kuxZBWaY5eQ1SY8+fasy//ZBWHN7rVeik\r\n7WJNrwiXK6nMFFMCUYSKu8eLy8kyssx8XYqDDJcGKL+fx2KXe+HFnySmyHP0pTQq\r\nO3yisWuvFVqZcicZSbiFFcg5SHK6BRsCAwEAATANBgkqhkiG9w0BAQsFAAOCAgEA\r\nnbw3uUPNiaytUMBOgwMmDs5C7R8f1T0GY7Ns1NB9+CaZMhFPziefrIAGkHFy735f\r\nqAqG7U1l1U9VWibrlCcLI2I5TwqzB6TI3Xy2J3dijPcoYcBWvJyyXgg2eFI0pZ8l\r\nVOANDHZbJ6n8XHKCbYJlEf1D+inVvoU3NxtOh9v10F6SSLrA74+x1AQRAFNNtOh0\r\nQS5qSYo7arl0mJYadPYv3lx8mEt6OhpVmJkj2I7HHzLBEExLNAGcvESL61kOxxoP\r\nF4E1QjhCckoG2fSclJtm+fuuBLIyzDzWSAZbwYP26jZmc/wb0EaRir8S1Q5xaukY\r\nXjdFl9XTjT0oM+2Dnsp50LK0mKer+dpIuZiF91uXCa8IBO/5M/sx0bWP5GnYpRJT\r\nks1PF9EtzsfDFNus3GxPTHkrG+2gyUApvZNIPi2WS6tsCf6zCM1OhwFy+9Tbhe1c\r\nYIMgwsEPxM77vaqAvOtJfTDuSfERfGuNXKqwn12tEvr6f6oAiFKpp0H8eV2fLhKB\r\nlQfzZOW3IkmVQk5QA3LErhHPrpm1R1osoAkemA/HoIpk40vgSXDba0694xQ6yh/H\r\nWSm8KEH9Req+/O3FLv9KV+xZQkQIMxhQ/HgAmadQKnPGOhxjCM5TRQ2XWlAI7Qd/\r\nkDVWoMTBWyiW0Bx6elS2mNbk7QinlJGnhuNvNud/a5k=\r\n-----END CERTIFICATE-----\r\n[root@server1 ~]# \r\n```\r\n\r\nTried to run ```flocker-diagnostics```, but it exited:\r\n```\r\n[root@server1 ~]# flocker-diagnostics \r\nTraceback (most recent call last):\r\n  File \"/usr/sbin/flocker-diagnostics\", line 7, in <module>\r\n    from flocker.node.script import flocker_diagnostics_main\r\n  File \"/opt/flocker/lib/python2.7/site-packages/flocker/node/__init__.py\", line 20, in <module>\r\n    from .script import DeployerType\r\n  File \"/opt/flocker/lib/python2.7/site-packages/flocker/node/script.py\", line 16, in <module>\r\n    import yaml\r\n  File \"/opt/mesosphere/lib/python3.4/site-packages/yaml/__init__.py\", line 284\r\n    class YAMLObject(metaclass=YAMLObjectMetaclass):\r\n                              ^\r\nSyntaxError: invalid syntax\r\n[root@server1 ~]# \r\n\r\n```\r\n\r\nI am trying at least configure one node correctly. Once configured and visible from ```flockerctl``` command, I will proceed with the rest of the nodes (i.e., using node 1 as a template).  Any pointers or suggestions ?\r\n\r\nThanks so much!!"},"comment":{"url":"https://api.github.com/repos/ClusterHQ/flocker/issues/comments/241502530","html_url":"https://github.com/ClusterHQ/flocker/issues/2885#issuecomment-241502530","issue_url":"https://api.github.com/repos/ClusterHQ/flocker/issues/2885","id":241502530,"user":{"login":"akamalov","id":7885257,"avatar_url":"https://avatars.githubusercontent.com/u/7885257?v=3","gravatar_id":"","url":"https://api.github.com/users/akamalov","html_url":"https://github.com/akamalov","followers_url":"https://api.github.com/users/akamalov/followers","following_url":"https://api.github.com/users/akamalov/following{/other_user}","gists_url":"https://api.github.com/users/akamalov/gists{/gist_id}","starred_url":"https://api.github.com/users/akamalov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akamalov/subscriptions","organizations_url":"https://api.github.com/users/akamalov/orgs","repos_url":"https://api.github.com/users/akamalov/repos","events_url":"https://api.github.com/users/akamalov/events{/privacy}","received_events_url":"https://api.github.com/users/akamalov/received_events","type":"User","site_admin":false},"created_at":"2016-08-22T18:19:15Z","updated_at":"2016-08-22T18:19:15Z","body":"Thanks for editing it out @wallrj "}}
{"actor":{"display_login":"akamalov"}}