{"action":"created","issue":{"url":"https://api.github.com/repos/Supervisor/supervisor/issues/584","repository_url":"https://api.github.com/repos/Supervisor/supervisor","labels_url":"https://api.github.com/repos/Supervisor/supervisor/issues/584/labels{/name}","comments_url":"https://api.github.com/repos/Supervisor/supervisor/issues/584/comments","events_url":"https://api.github.com/repos/Supervisor/supervisor/issues/584/events","html_url":"https://github.com/Supervisor/supervisor/issues/584","id":59366503,"number":584,"title":"Better way to determine if process has started","user":{"login":"msabramo","id":305268,"avatar_url":"https://avatars.githubusercontent.com/u/305268?v=3","gravatar_id":"","url":"https://api.github.com/users/msabramo","html_url":"https://github.com/msabramo","followers_url":"https://api.github.com/users/msabramo/followers","following_url":"https://api.github.com/users/msabramo/following{/other_user}","gists_url":"https://api.github.com/users/msabramo/gists{/gist_id}","starred_url":"https://api.github.com/users/msabramo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/msabramo/subscriptions","organizations_url":"https://api.github.com/users/msabramo/orgs","repos_url":"https://api.github.com/users/msabramo/repos","events_url":"https://api.github.com/users/msabramo/events{/privacy}","received_events_url":"https://api.github.com/users/msabramo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":14,"created_at":"2015-02-28T23:43:51Z","updated_at":"2016-07-27T08:10:08Z","closed_at":null,"body":"My understanding is that supervisor's only way of knowing if a process has started is if it `startsecs` seconds have elapsed and the process hasn't died. Correct me if I'm wrong. \r\n\r\nWe routinely run into problems where folks have specified, say, `startsecs = 5` and then the system is a bit slow so that it takes the process longer than usual to initialize. So then it is initializing for say 10 seconds and then it dies because it can't get a connection to a database or something. Because 5 seconds have elapsed, supervisor considers this to be that the process started successfully and then died, so it restarts the process. This repeats ad-infinitum, with the result that the process goes up and down and CPU usage on the box becomes very high and now that causes other processes to be slow, possibly resulting in cascading failures. \r\n\r\nIt is difficult to pick a `startsecs` high enough to prevent this and undesirable because you have to wait a long time to get confirmation that the process has started and even then you have no guarantee that it's not going to die a couple of seconds later. \r\n\r\nI wonder if supervisor could have a `healthcheck_command` option for processes so that you can tell supervisor a better way to determine whether the process has started successfully. For example this could be set to check if the process is listening on a port or it could do a simple HTTP request. \r\n\r\nThis would take precedence over `startsecs`. As soon as the command goes from failing to succeeding, the process is considered to have started. \r\n\r\nThoughts?"},"comment":{"url":"https://api.github.com/repos/Supervisor/supervisor/issues/comments/235516234","html_url":"https://github.com/Supervisor/supervisor/issues/584#issuecomment-235516234","issue_url":"https://api.github.com/repos/Supervisor/supervisor/issues/584","id":235516234,"user":{"login":"mikeyg123","id":20541399,"avatar_url":"https://avatars.githubusercontent.com/u/20541399?v=3","gravatar_id":"","url":"https://api.github.com/users/mikeyg123","html_url":"https://github.com/mikeyg123","followers_url":"https://api.github.com/users/mikeyg123/followers","following_url":"https://api.github.com/users/mikeyg123/following{/other_user}","gists_url":"https://api.github.com/users/mikeyg123/gists{/gist_id}","starred_url":"https://api.github.com/users/mikeyg123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikeyg123/subscriptions","organizations_url":"https://api.github.com/users/mikeyg123/orgs","repos_url":"https://api.github.com/users/mikeyg123/repos","events_url":"https://api.github.com/users/mikeyg123/events{/privacy}","received_events_url":"https://api.github.com/users/mikeyg123/received_events","type":"User","site_admin":false},"created_at":"2016-07-27T08:10:08Z","updated_at":"2016-07-27T08:10:08Z","body":"Specifying a regex or even a static string to match in the stdout to indicate that a process has started would really help us a great deal.\r\nSo far we've migrated nearly 100 different processes to our supervisor based environment but we have a few tricky ones remaining that can take a widely variable amount of time to restart.\r\nWe also have a rolling restart script that restarts a batch of processes, waits for them to all start before moving on to the next batch.  Having a reliable way to know that processes are properly up is crucial here to prevent service outages."}}
{"actor":{"display_login":"mikeyg123"}}