{"action":"created","issue":{"url":"https://api.github.com/repos/CiscoCloud/mantl/issues/1293","repository_url":"https://api.github.com/repos/CiscoCloud/mantl","labels_url":"https://api.github.com/repos/CiscoCloud/mantl/issues/1293/labels{/name}","comments_url":"https://api.github.com/repos/CiscoCloud/mantl/issues/1293/comments","events_url":"https://api.github.com/repos/CiscoCloud/mantl/issues/1293/events","html_url":"https://github.com/CiscoCloud/mantl/issues/1293","id":143249408,"number":1293,"title":"new deployment failing due to 'logstash -> distributive' dependency","user":{"login":"pfrybar","id":1022924,"avatar_url":"https://avatars.githubusercontent.com/u/1022924?v=3","gravatar_id":"","url":"https://api.github.com/users/pfrybar","html_url":"https://github.com/pfrybar","followers_url":"https://api.github.com/users/pfrybar/followers","following_url":"https://api.github.com/users/pfrybar/following{/other_user}","gists_url":"https://api.github.com/users/pfrybar/gists{/gist_id}","starred_url":"https://api.github.com/users/pfrybar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pfrybar/subscriptions","organizations_url":"https://api.github.com/users/pfrybar/orgs","repos_url":"https://api.github.com/users/pfrybar/repos","events_url":"https://api.github.com/users/pfrybar/events{/privacy}","received_events_url":"https://api.github.com/users/pfrybar/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-03-24T13:44:30Z","updated_at":"2016-03-24T22:08:56Z","closed_at":"2016-03-24T22:08:55Z","body":"With the latest code from master (9e93ccf511cc10c88dcb96e3a7c578bdb84cca20), a new deployment to openstack is failing:\r\n\r\n```\r\nNOTIFIED: [handlers | reload consul] ****************************************** \r\nfailed: [esi-etl-test-edge-01] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.020814\", \"end\": \"2016-03-23 21:56:21.932798\", \"rc\": 6, \"start\": \"2016-03-23 21:56:21.911984\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-edge-02] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.023280\", \"end\": \"2016-03-23 21:56:22.013492\", \"rc\": 6, \"start\": \"2016-03-23 21:56:21.990212\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-control-02] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.017571\", \"end\": \"2016-03-23 21:56:21.871630\", \"rc\": 6, \"start\": \"2016-03-23 21:56:21.854059\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-worker-004] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.016349\", \"end\": \"2016-03-23 21:56:21.478459\", \"rc\": 6, \"start\": \"2016-03-23 21:56:21.462110\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-worker-005] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.016488\", \"end\": \"2016-03-23 21:56:22.086435\", \"rc\": 6, \"start\": \"2016-03-23 21:56:22.069947\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-worker-003] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.017721\", \"end\": \"2016-03-23 21:56:22.008096\", \"rc\": 6, \"start\": \"2016-03-23 21:56:21.990375\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-control-03] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.023206\", \"end\": \"2016-03-23 21:56:22.164690\", \"rc\": 6, \"start\": \"2016-03-23 21:56:22.141484\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-worker-002] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.031444\", \"end\": \"2016-03-23 21:56:21.666524\", \"rc\": 6, \"start\": \"2016-03-23 21:56:21.635080\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-control-01] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.027544\", \"end\": \"2016-03-23 21:56:21.165563\", \"rc\": 6, \"start\": \"2016-03-23 21:56:21.138019\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\nfailed: [esi-etl-test-worker-001] => {\"changed\": true, \"cmd\": [\"systemctl\", \"reload\", \"consul\"], \"delta\": \"0:00:00.031886\", \"end\": \"2016-03-23 21:56:22.024889\", \"rc\": 6, \"start\": \"2016-03-23 21:56:21.993003\", \"warnings\": []}\r\nstderr: Failed to reload consul.service: Unit consul.service failed to load: No such file or directory.\r\n\r\nFATAL: all hosts have already failed -- aborting\r\n```\r\n\r\nI believe it's due to this line in the new distribute packaging which was merged recently: https://github.com/CiscoCloud/mantl/pull/1268/files#diff-02148fc5e171ddae37cf62ba6ce1df4aR33\r\n\r\nLogstash now depends on distributive, which has a `reload consul` notification. Consul doesn't get installed (according to `sample.yml`) until after logstash, which is why the notification fails the build."},"comment":{"url":"https://api.github.com/repos/CiscoCloud/mantl/issues/comments/201045519","html_url":"https://github.com/CiscoCloud/mantl/issues/1293#issuecomment-201045519","issue_url":"https://api.github.com/repos/CiscoCloud/mantl/issues/1293","id":201045519,"user":{"login":"pfrybar","id":1022924,"avatar_url":"https://avatars.githubusercontent.com/u/1022924?v=3","gravatar_id":"","url":"https://api.github.com/users/pfrybar","html_url":"https://github.com/pfrybar","followers_url":"https://api.github.com/users/pfrybar/followers","following_url":"https://api.github.com/users/pfrybar/following{/other_user}","gists_url":"https://api.github.com/users/pfrybar/gists{/gist_id}","starred_url":"https://api.github.com/users/pfrybar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pfrybar/subscriptions","organizations_url":"https://api.github.com/users/pfrybar/orgs","repos_url":"https://api.github.com/users/pfrybar/repos","events_url":"https://api.github.com/users/pfrybar/events{/privacy}","received_events_url":"https://api.github.com/users/pfrybar/received_events","type":"User","site_admin":false},"created_at":"2016-03-24T22:08:55Z","updated_at":"2016-03-24T22:08:55Z","body":"Alright, so this turns out to be an edge case. My ansible run failed trying to download the logstash docker image (network issues, unrelated), after the distributive checks had been installed. It never got to the point of installing consul. When I re-ran ansible it saw the distributive files, removed them, and triggered the handler to reload consul. On a fresh deploy (where nothing goes wrong), this worked as expected."}}