{"action":"created","issue":{"url":"https://api.github.com/repos/saltstack/salt/issues/31575","repository_url":"https://api.github.com/repos/saltstack/salt","labels_url":"https://api.github.com/repos/saltstack/salt/issues/31575/labels{/name}","comments_url":"https://api.github.com/repos/saltstack/salt/issues/31575/comments","events_url":"https://api.github.com/repos/saltstack/salt/issues/31575/events","html_url":"https://github.com/saltstack/salt/issues/31575","id":137554516,"number":31575,"title":"event_return prevents salt-master from properly shutting down","user":{"login":"anlutro","id":163093,"avatar_url":"https://avatars.githubusercontent.com/u/163093?v=3","gravatar_id":"","url":"https://api.github.com/users/anlutro","html_url":"https://github.com/anlutro","followers_url":"https://api.github.com/users/anlutro/followers","following_url":"https://api.github.com/users/anlutro/following{/other_user}","gists_url":"https://api.github.com/users/anlutro/gists{/gist_id}","starred_url":"https://api.github.com/users/anlutro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anlutro/subscriptions","organizations_url":"https://api.github.com/users/anlutro/orgs","repos_url":"https://api.github.com/users/anlutro/repos","events_url":"https://api.github.com/users/anlutro/events{/privacy}","received_events_url":"https://api.github.com/users/anlutro/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/saltstack/salt/labels/Bug","name":"Bug","color":"e11d21"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Cannot%20Reproduce","name":"Cannot Reproduce","color":"c7def8"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Core","name":"Core","color":"fef2c0"},{"url":"https://api.github.com/repos/saltstack/salt/labels/High%20Severity","name":"High Severity","color":"ff9143"},{"url":"https://api.github.com/repos/saltstack/salt/labels/P2","name":"P2","color":"155fb5"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Returners","name":"Returners","color":"006b75"}],"state":"open","locked":false,"assignee":{"login":"cachedout","id":111616,"avatar_url":"https://avatars.githubusercontent.com/u/111616?v=3","gravatar_id":"","url":"https://api.github.com/users/cachedout","html_url":"https://github.com/cachedout","followers_url":"https://api.github.com/users/cachedout/followers","following_url":"https://api.github.com/users/cachedout/following{/other_user}","gists_url":"https://api.github.com/users/cachedout/gists{/gist_id}","starred_url":"https://api.github.com/users/cachedout/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cachedout/subscriptions","organizations_url":"https://api.github.com/users/cachedout/orgs","repos_url":"https://api.github.com/users/cachedout/repos","events_url":"https://api.github.com/users/cachedout/events{/privacy}","received_events_url":"https://api.github.com/users/cachedout/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/saltstack/salt/milestones/13","html_url":"https://github.com/saltstack/salt/milestones/Approved","labels_url":"https://api.github.com/repos/saltstack/salt/milestones/13/labels","id":9265,"number":13,"title":"Approved","description":"All issues that are ready to be worked on, both bugs and features.","creator":{"login":"thatch45","id":507599,"avatar_url":"https://avatars.githubusercontent.com/u/507599?v=3","gravatar_id":"","url":"https://api.github.com/users/thatch45","html_url":"https://github.com/thatch45","followers_url":"https://api.github.com/users/thatch45/followers","following_url":"https://api.github.com/users/thatch45/following{/other_user}","gists_url":"https://api.github.com/users/thatch45/gists{/gist_id}","starred_url":"https://api.github.com/users/thatch45/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thatch45/subscriptions","organizations_url":"https://api.github.com/users/thatch45/orgs","repos_url":"https://api.github.com/users/thatch45/repos","events_url":"https://api.github.com/users/thatch45/events{/privacy}","received_events_url":"https://api.github.com/users/thatch45/received_events","type":"User","site_admin":false},"open_issues":2916,"closed_issues":3395,"state":"open","created_at":"2011-05-14T04:00:56Z","updated_at":"2016-03-02T16:55:59Z","due_on":null,"closed_at":null},"comments":3,"created_at":"2016-03-01T12:01:50Z","updated_at":"2016-03-02T17:08:22Z","closed_at":null,"body":"On Debian 8, when I run `service salt-master stop`, it hangs for a long while, and even when it's done, there's a defunct process still running. Through strace/lsof I figured out that it's trying to talk to an anonymous inode, which it keeps polling every 5 seconds.\r\n\r\n```\r\nvagrant@orch:~$ ps aux | grep salt-master\r\nroot     11740  0.0  5.3 271016 40808 ?        Sl   12:57   0:00 /usr/bin/python /usr/bin/salt-master\r\nvagrant  12479  0.0  0.2  12728  2268 pts/2    S+   12:59   0:00 grep salt-master\r\n\r\nvagrant@orch:~$ sudo strace -p 11740\r\nProcess 11740 attached\r\nrestart_syscall(<... resuming interrupted call ...>) = 0\r\npoll([{fd=12, events=POLLIN}], 1, 0)    = 0 (Timeout)\r\npoll([{fd=12, events=POLLIN}], 1, 0)    = 0 (Timeout)\r\npoll([{fd=12, events=POLLIN}], 1, 0)    = 0 (Timeout)\r\npoll([{fd=12, events=POLLIN}], 1, 5000) = 0 (Timeout)\r\npoll([{fd=12, events=POLLIN}], 1, 0)    = 0 (Timeout)\r\npoll([{fd=12, events=POLLIN}], 1, 0)    = 0 (Timeout)\r\npoll([{fd=12, events=POLLIN}], 1, 0)    = 0 (Timeout)\r\npoll([{fd=12, events=POLLIN}], 1, 5000^CProcess 11740 detached\r\n <detached ...>\r\n\r\nvagrant@orch:~$ sudo lsof -p 11740 | grep 12u\r\nsalt-mast 11740 root   12u  0000                0,9        0    5589 anon_inode\r\n```\r\n\r\nI have an event returner on the master, defined in the master config as such:\r\n\r\n```\r\nevent_return: slack_failure_only\r\n```\r\n\r\nRemoving this line from my configuration fixes the problem.\r\n\r\nWhat the event returner does, doesn't seem to matter, I tried making it an empty function, the end result is still the same.\r\n\r\n```\r\nSalt Version:\r\n           Salt: 2015.8.7\r\n \r\nDependency Versions:\r\n         Jinja2: 2.7.3\r\n       M2Crypto: Not Installed\r\n           Mako: 1.0.0\r\n         PyYAML: 3.11\r\n          PyZMQ: 14.4.0\r\n         Python: 2.7.9 (default, Mar  1 2015, 12:57:24)\r\n           RAET: Not Installed\r\n        Tornado: 4.2.1\r\n            ZMQ: 4.0.5\r\n           cffi: 0.8.6\r\n       cherrypy: 3.2.3\r\n       dateutil: 2.2\r\n          gitdb: 0.5.4\r\n      gitpython: 0.3.2 RC1\r\n          ioflo: Not Installed\r\n        libgit2: 0.23.4\r\n        libnacl: Not Installed\r\n   msgpack-pure: Not Installed\r\n msgpack-python: 0.4.2\r\n   mysql-python: 1.2.3\r\n      pycparser: 2.10\r\n       pycrypto: 2.6.1\r\n         pygit2: 0.23.1\r\n   python-gnupg: Not Installed\r\n          smmap: 0.8.2\r\n        timelib: Not Installed\r\n \r\nSystem Versions:\r\n           dist: debian 8.3 \r\n        machine: x86_64\r\n        release: 3.16.0-4-amd64\r\n         system: debian 8.3\r\n```"},"comment":{"url":"https://api.github.com/repos/saltstack/salt/issues/comments/191328314","html_url":"https://github.com/saltstack/salt/issues/31575#issuecomment-191328314","issue_url":"https://api.github.com/repos/saltstack/salt/issues/31575","id":191328314,"user":{"login":"anlutro","id":163093,"avatar_url":"https://avatars.githubusercontent.com/u/163093?v=3","gravatar_id":"","url":"https://api.github.com/users/anlutro","html_url":"https://github.com/anlutro","followers_url":"https://api.github.com/users/anlutro/followers","following_url":"https://api.github.com/users/anlutro/following{/other_user}","gists_url":"https://api.github.com/users/anlutro/gists{/gist_id}","starred_url":"https://api.github.com/users/anlutro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anlutro/subscriptions","organizations_url":"https://api.github.com/users/anlutro/orgs","repos_url":"https://api.github.com/users/anlutro/repos","events_url":"https://api.github.com/users/anlutro/events{/privacy}","received_events_url":"https://api.github.com/users/anlutro/received_events","type":"User","site_admin":false},"created_at":"2016-03-02T17:08:21Z","updated_at":"2016-03-02T17:08:21Z","body":"I haven't tried running the master in the foreground so I can't comment on\nthat. Is returner behaviour very different if it's a custom returner as\nopposed to a built-in one?\n\nI will try running in the foreground and using the local returner tomorrow.\n\nOn Wed, 2 Mar 2016 18:04 Mike Place, <notifications@github.com> wrote:\n\n> I cannot replicate this problem at all using the v2015.8.7 tag.\n>\n> I set event_return: local in my master config, started it up and\n> connected a minion to it. Then I pinged the minion ten times and attempted\n> to ctl-c the master. It stopped immediately.\n>\n> @anlutro <https://github.com/anlutro> Does my test case produce the\n> problem for you as well? The only real difference here is your using the\n> init system to shut the daemon down instead of running it in the foreground\n> like I was, but I'm at a loss to explain why that would make a difference\n> here, since the master is shutting down cleanly for me regardless. Any\n> ideas?\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/saltstack/salt/issues/31575#issuecomment-191326958>.\n>\n"}}