{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/fleet/issues/1000","repository_url":"https://api.github.com/repos/coreos/fleet","labels_url":"https://api.github.com/repos/coreos/fleet/issues/1000/labels{/name}","comments_url":"https://api.github.com/repos/coreos/fleet/issues/1000/comments","events_url":"https://api.github.com/repos/coreos/fleet/issues/1000/events","html_url":"https://github.com/coreos/fleet/issues/1000","id":46752010,"number":1000,"title":"ExecStop / ExecStopPost not executed when fleetctl destroy is used","user":{"login":"flantel","id":3835944,"avatar_url":"https://avatars.githubusercontent.com/u/3835944?v=3","gravatar_id":"","url":"https://api.github.com/users/flantel","html_url":"https://github.com/flantel","followers_url":"https://api.github.com/users/flantel/followers","following_url":"https://api.github.com/users/flantel/following{/other_user}","gists_url":"https://api.github.com/users/flantel/gists{/gist_id}","starred_url":"https://api.github.com/users/flantel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flantel/subscriptions","organizations_url":"https://api.github.com/users/flantel/orgs","repos_url":"https://api.github.com/users/flantel/repos","events_url":"https://api.github.com/users/flantel/events{/privacy}","received_events_url":"https://api.github.com/users/flantel/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/coreos/fleet/labels/kind/bug","name":"kind/bug","color":"ff0000"}],"state":"open","locked":false,"assignee":{"login":"kayrus","id":4047483,"avatar_url":"https://avatars.githubusercontent.com/u/4047483?v=3","gravatar_id":"","url":"https://api.github.com/users/kayrus","html_url":"https://github.com/kayrus","followers_url":"https://api.github.com/users/kayrus/followers","following_url":"https://api.github.com/users/kayrus/following{/other_user}","gists_url":"https://api.github.com/users/kayrus/gists{/gist_id}","starred_url":"https://api.github.com/users/kayrus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kayrus/subscriptions","organizations_url":"https://api.github.com/users/kayrus/orgs","repos_url":"https://api.github.com/users/kayrus/repos","events_url":"https://api.github.com/users/kayrus/events{/privacy}","received_events_url":"https://api.github.com/users/kayrus/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/coreos/fleet/milestones/28","html_url":"https://github.com/coreos/fleet/milestones/v0.12.0","labels_url":"https://api.github.com/repos/coreos/fleet/milestones/28/labels","id":1515018,"number":28,"title":"v0.12.0","description":"","creator":{"login":"jonboulle","id":2601015,"avatar_url":"https://avatars.githubusercontent.com/u/2601015?v=3","gravatar_id":"","url":"https://api.github.com/users/jonboulle","html_url":"https://github.com/jonboulle","followers_url":"https://api.github.com/users/jonboulle/followers","following_url":"https://api.github.com/users/jonboulle/following{/other_user}","gists_url":"https://api.github.com/users/jonboulle/gists{/gist_id}","starred_url":"https://api.github.com/users/jonboulle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jonboulle/subscriptions","organizations_url":"https://api.github.com/users/jonboulle/orgs","repos_url":"https://api.github.com/users/jonboulle/repos","events_url":"https://api.github.com/users/jonboulle/events{/privacy}","received_events_url":"https://api.github.com/users/jonboulle/received_events","type":"User","site_admin":false},"open_issues":19,"closed_issues":7,"state":"open","created_at":"2016-01-19T11:57:39Z","updated_at":"2016-02-08T20:36:16Z","due_on":"2016-02-16T23:00:00Z","closed_at":null},"comments":22,"created_at":"2014-10-24T15:41:02Z","updated_at":"2016-02-09T09:52:52Z","closed_at":null,"body":"Hi\r\n\r\nUsing fleetctl version 0.8.3 and CoreOS stable, if I stop a unit with 'fleetctl destroy unit.service' the ExecStop and ExecStopPost are not executed. If I use 'fleetctl stop ...' they are.\r\n\r\nIs this expected behaviour? I would have expected (and I think this was the case with earlier versions) that destroy would execute StopExec and StopExecPost.\r\n\r\nThanks.\r\n\r\n-Barry Flanagan\r\n"},"comment":{"url":"https://api.github.com/repos/coreos/fleet/issues/comments/181787638","html_url":"https://github.com/coreos/fleet/issues/1000#issuecomment-181787638","issue_url":"https://api.github.com/repos/coreos/fleet/issues/1000","id":181787638,"user":{"login":"tixxdz","id":1549291,"avatar_url":"https://avatars.githubusercontent.com/u/1549291?v=3","gravatar_id":"","url":"https://api.github.com/users/tixxdz","html_url":"https://github.com/tixxdz","followers_url":"https://api.github.com/users/tixxdz/followers","following_url":"https://api.github.com/users/tixxdz/following{/other_user}","gists_url":"https://api.github.com/users/tixxdz/gists{/gist_id}","starred_url":"https://api.github.com/users/tixxdz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tixxdz/subscriptions","organizations_url":"https://api.github.com/users/tixxdz/orgs","repos_url":"https://api.github.com/users/tixxdz/repos","events_url":"https://api.github.com/users/tixxdz/events{/privacy}","received_events_url":"https://api.github.com/users/tixxdz/received_events","type":"User","site_admin":false},"created_at":"2016-02-09T09:52:52Z","updated_at":"2016-02-09T09:52:52Z","body":"Updates:\r\n\r\nWe are working on this right now, basically this is a bug in both fleet and systemd. In systemd this is a well known issue. In systemd when doing a \"daemon-reload\" some \"Exec*\" may just be skipped since units are replaced in memory and previous commands are not there any more. Please see below.\r\n\r\nSummary:\r\n\r\n1.  fleet\r\n\r\nOn fleet side when doing a destroy we should do our best to wait for at least a stop, but currently I'm new to fleet so my understanding and from reading this documentation https://coreos.com/fleet/docs/latest/using-the-client.html\r\n\"The destroy command does two things: Instruct systemd on the host machine to stop the unit, deferring to systemd completely for any custom stop directives (i.e. ExecStop option in the unit file).\r\nRemove the unit file from the cluster, making it impossible to start again until it has been re-submitted.\r\nOnce a unit is destroyed, state will continue to be reported for it in fleetctl list-units. Only once the unit has stopped will its state be removed.\"\r\n\r\nSo either we wait for units to stop then do the destroy, or we could add \"--block\" or  \"--no-block\" switch to destroy command. With \"--block\" we will be backward compatible but I don't like it. With \"--no-block\" we will have to change the behaviour and make destory block by default unless you set the \"--no-block\" switch which is also the same in systemd when you do \"systemctl --no-block daemon-reload\" and of course use it are your own risk but we have to add some debug messages and make fleet and systemd a bit smarter. However even with this change there will still be bugs here and there, the main issue is when X (unrelated unit, user or whatever) triggers a \"daemon-reload\" in systemd, if you happen to have changed unit Y and didn't want to do a \"daemon-reload\" cause you know that some \"Exec*\" directives will be skipped... then you are also out of luck! yeh X just messed up with Y...\r\n\r\n\r\n2.  systemd\r\n\r\nThe related issue is here: https://github.com/systemd/systemd/issues/518\r\n\r\nSo it's not only related to fleet destroy it may happen at anytime, we can improve fleet as noted above but we have to add at least some debug logs and make systemd smarter.\r\n\r\nSo we could check in systemd what section did change or save the line numbers and continue... we are trying to figure out the best solution, at least doing systemctl status changed.service should also tell you that not only you need a \"daemon-reload\" but perhaps you also need to do a stop unit before that.. and also make it log every time daemon-reload is triggered and where some \"Exec*\" where lost not executed etc.\r\n\r\nAnyway just to say that we are working on this. Thanks! "}}