{"action":"created","comment":{"url":"https://api.github.com/repos/flynn/flynn/pulls/comments/60849079","id":60849079,"diff_hunk":"@@ -39,6 +39,7 @@ func JobConfig(f *ct.ExpandedFormation, name, hostID string, uuid string) *host.\n \tmetadata[\"flynn-controller.app\"] = f.App.ID\n \tmetadata[\"flynn-controller.app_name\"] = f.App.Name\n \tmetadata[\"flynn-controller.release\"] = f.Release.ID\n+\tmetadata[\"flynn-controller.formation\"] = f.App.ID + \":\" + f.Release.ID","path":"controller/utils/utils.go","position":4,"original_position":4,"commit_id":"fa474e3cfe32c9878457ed34f91db42a0313dd43","original_commit_id":"fa474e3cfe32c9878457ed34f91db42a0313dd43","user":{"login":"lmars","id":488515,"avatar_url":"https://avatars.githubusercontent.com/u/488515?v=3","gravatar_id":"","url":"https://api.github.com/users/lmars","html_url":"https://github.com/lmars","followers_url":"https://api.github.com/users/lmars/followers","following_url":"https://api.github.com/users/lmars/following{/other_user}","gists_url":"https://api.github.com/users/lmars/gists{/gist_id}","starred_url":"https://api.github.com/users/lmars/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lmars/subscriptions","organizations_url":"https://api.github.com/users/lmars/orgs","repos_url":"https://api.github.com/users/lmars/repos","events_url":"https://api.github.com/users/lmars/events{/privacy}","received_events_url":"https://api.github.com/users/lmars/received_events","type":"User","site_admin":false},"body":"`JobConfig` takes a formation argument, so it's always used when starting a job for a formation (there are currently three uses: [scheduler](https://github.com/flynn/flynn/blob/v20160423.0/controller/scheduler/scheduler.go#L1335-L1337), [bootstrapper](https://github.com/flynn/flynn/blob/v20160423.0/bootstrap/run_app_action.go#L109-L120) and [host/fixer](https://github.com/flynn/flynn/blob/v20160423.0/host/fixer/fixer.go#L335-L343)), but the controller `RunJob` method [also sets the AppID and ReleaseID](https://github.com/flynn/flynn/blob/master/controller/jobs.go#L286-L288) so that they are associated with the app and release, but these are not formation jobs.\r\n\r\nSo whilst the actual formation ID of `AppID:ReleaseID` is redundant, the extra information serves to differentiate one-off jobs from formation jobs. We could just make this `flynn-controller.formation = true` if you prefer?","created_at":"2016-04-24T18:17:43Z","updated_at":"2016-04-24T18:17:43Z","html_url":"https://github.com/flynn/flynn/pull/2733#discussion_r60849079","pull_request_url":"https://api.github.com/repos/flynn/flynn/pulls/2733","_links":{"self":{"href":"https://api.github.com/repos/flynn/flynn/pulls/comments/60849079"},"html":{"href":"https://github.com/flynn/flynn/pull/2733#discussion_r60849079"},"pull_request":{"href":"https://api.github.com/repos/flynn/flynn/pulls/2733"}}},"pull_request":{"url":"https://api.github.com/repos/flynn/flynn/pulls/2733","id":67640293,"html_url":"https://github.com/flynn/flynn/pull/2733","diff_url":"https://github.com/flynn/flynn/pull/2733.diff","patch_url":"https://github.com/flynn/flynn/pull/2733.patch","issue_url":"https://api.github.com/repos/flynn/flynn/issues/2733","number":2733,"state":"open","locked":false,"title":"scheduler: Formation stream fixes","user":{"login":"lmars","id":488515,"avatar_url":"https://avatars.githubusercontent.com/u/488515?v=3","gravatar_id":"","url":"https://api.github.com/users/lmars","html_url":"https://github.com/lmars","followers_url":"https://api.github.com/users/lmars/followers","following_url":"https://api.github.com/users/lmars/following{/other_user}","gists_url":"https://api.github.com/users/lmars/gists{/gist_id}","starred_url":"https://api.github.com/users/lmars/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lmars/subscriptions","organizations_url":"https://api.github.com/users/lmars/orgs","repos_url":"https://api.github.com/users/lmars/repos","events_url":"https://api.github.com/users/lmars/events{/privacy}","received_events_url":"https://api.github.com/users/lmars/received_events","type":"User","site_admin":false},"body":"In [this CI failure](https://ci.flynn.io/builds/20160423202834-61474f73), the cluster fell over during a controller deployment because new schedulers were started, old ones were killed but the new schedulers all crashed with:\r\n\r\n```\r\n20:47:41.968254 scheduler.go:159: timed out waiting for current formation list\r\n```\r\n\r\nWhilst I am unsure why the scheduler was not getting the current event from the formation stream, I've made two patches which will improve the situation:\r\n\r\n* Only register in service discovery once the current formation event has been received. This means that if there is the above error, the old schedulers will still be around and the deployment will just fail (rather than the old schedulers already being killed, and then the new schedulers also crashing, leaving no running schedulers)\r\n* Return an error to the client if `sendUpdatedSince` encounters an error (and therefore does not send the current event)\r\n\r\nI've also added a slightly unrelated patch so that the scheduler explicitly identifies jobs for which it needs to look up a formation, because I was seeing lots of errors looking up formations for one-off jobs.","created_at":"2016-04-24T15:21:49Z","updated_at":"2016-04-24T18:17:43Z","closed_at":null,"merged_at":null,"merge_commit_sha":"2cf2a9d460ab0f7b002144af9e2ceb1852d18d8d","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/flynn/flynn/pulls/2733/commits","review_comments_url":"https://api.github.com/repos/flynn/flynn/pulls/2733/comments","review_comment_url":"https://api.github.com/repos/flynn/flynn/pulls/comments{/number}","comments_url":"https://api.github.com/repos/flynn/flynn/issues/2733/comments","statuses_url":"https://api.github.com/repos/flynn/flynn/statuses/fa474e3cfe32c9878457ed34f91db42a0313dd43","head":{"label":"flynn:scheduler-fixes","ref":"scheduler-fixes","sha":"fa474e3cfe32c9878457ed34f91db42a0313dd43","user":{"login":"flynn","id":4973613,"avatar_url":"https://avatars.githubusercontent.com/u/4973613?v=3","gravatar_id":"","url":"https://api.github.com/users/flynn","html_url":"https://github.com/flynn","followers_url":"https://api.github.com/users/flynn/followers","following_url":"https://api.github.com/users/flynn/following{/other_user}","gists_url":"https://api.github.com/users/flynn/gists{/gist_id}","starred_url":"https://api.github.com/users/flynn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flynn/subscriptions","organizations_url":"https://api.github.com/users/flynn/orgs","repos_url":"https://api.github.com/users/flynn/repos","events_url":"https://api.github.com/users/flynn/events{/privacy}","received_events_url":"https://api.github.com/users/flynn/received_events","type":"Organization","site_admin":false},"repo":{"id":11290232,"name":"flynn","full_name":"flynn/flynn","owner":{"login":"flynn","id":4973613,"avatar_url":"https://avatars.githubusercontent.com/u/4973613?v=3","gravatar_id":"","url":"https://api.github.com/users/flynn","html_url":"https://github.com/flynn","followers_url":"https://api.github.com/users/flynn/followers","following_url":"https://api.github.com/users/flynn/following{/other_user}","gists_url":"https://api.github.com/users/flynn/gists{/gist_id}","starred_url":"https://api.github.com/users/flynn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flynn/subscriptions","organizations_url":"https://api.github.com/users/flynn/orgs","repos_url":"https://api.github.com/users/flynn/repos","events_url":"https://api.github.com/users/flynn/events{/privacy}","received_events_url":"https://api.github.com/users/flynn/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/flynn/flynn","description":"A next generation open source platform as a service (PaaS)","fork":false,"url":"https://api.github.com/repos/flynn/flynn","forks_url":"https://api.github.com/repos/flynn/flynn/forks","keys_url":"https://api.github.com/repos/flynn/flynn/keys{/key_id}","collaborators_url":"https://api.github.com/repos/flynn/flynn/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/flynn/flynn/teams","hooks_url":"https://api.github.com/repos/flynn/flynn/hooks","issue_events_url":"https://api.github.com/repos/flynn/flynn/issues/events{/number}","events_url":"https://api.github.com/repos/flynn/flynn/events","assignees_url":"https://api.github.com/repos/flynn/flynn/assignees{/user}","branches_url":"https://api.github.com/repos/flynn/flynn/branches{/branch}","tags_url":"https://api.github.com/repos/flynn/flynn/tags","blobs_url":"https://api.github.com/repos/flynn/flynn/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/flynn/flynn/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/flynn/flynn/git/refs{/sha}","trees_url":"https://api.github.com/repos/flynn/flynn/git/trees{/sha}","statuses_url":"https://api.github.com/repos/flynn/flynn/statuses/{sha}","languages_url":"https://api.github.com/repos/flynn/flynn/languages","stargazers_url":"https://api.github.com/repos/flynn/flynn/stargazers","contributors_url":"https://api.github.com/repos/flynn/flynn/contributors","subscribers_url":"https://api.github.com/repos/flynn/flynn/subscribers","subscription_url":"https://api.github.com/repos/flynn/flynn/subscription","commits_url":"https://api.github.com/repos/flynn/flynn/commits{/sha}","git_commits_url":"https://api.github.com/repos/flynn/flynn/git/commits{/sha}","comments_url":"https://api.github.com/repos/flynn/flynn/comments{/number}","issue_comment_url":"https://api.github.com/repos/flynn/flynn/issues/comments{/number}","contents_url":"https://api.github.com/repos/flynn/flynn/contents/{+path}","compare_url":"https://api.github.com/repos/flynn/flynn/compare/{base}...{head}","merges_url":"https://api.github.com/repos/flynn/flynn/merges","archive_url":"https://api.github.com/repos/flynn/flynn/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/flynn/flynn/downloads","issues_url":"https://api.github.com/repos/flynn/flynn/issues{/number}","pulls_url":"https://api.github.com/repos/flynn/flynn/pulls{/number}","milestones_url":"https://api.github.com/repos/flynn/flynn/milestones{/number}","notifications_url":"https://api.github.com/repos/flynn/flynn/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/flynn/flynn/labels{/name}","releases_url":"https://api.github.com/repos/flynn/flynn/releases{/id}","deployments_url":"https://api.github.com/repos/flynn/flynn/deployments","created_at":"2013-07-09T18:22:06Z","updated_at":"2016-04-24T16:43:35Z","pushed_at":"2016-04-24T15:21:49Z","git_url":"git://github.com/flynn/flynn.git","ssh_url":"git@github.com:flynn/flynn.git","clone_url":"https://github.com/flynn/flynn.git","svn_url":"https://github.com/flynn/flynn","homepage":"https://flynn.io","size":28234,"stargazers_count":4432,"watchers_count":4432,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":332,"mirror_url":null,"open_issues_count":332,"forks":332,"open_issues":332,"watchers":4432,"default_branch":"master"}},"base":{"label":"flynn:master","ref":"master","sha":"78dac4dc53686f2c753faceb26cd7eb95b98dcbc","user":{"login":"flynn","id":4973613,"avatar_url":"https://avatars.githubusercontent.com/u/4973613?v=3","gravatar_id":"","url":"https://api.github.com/users/flynn","html_url":"https://github.com/flynn","followers_url":"https://api.github.com/users/flynn/followers","following_url":"https://api.github.com/users/flynn/following{/other_user}","gists_url":"https://api.github.com/users/flynn/gists{/gist_id}","starred_url":"https://api.github.com/users/flynn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flynn/subscriptions","organizations_url":"https://api.github.com/users/flynn/orgs","repos_url":"https://api.github.com/users/flynn/repos","events_url":"https://api.github.com/users/flynn/events{/privacy}","received_events_url":"https://api.github.com/users/flynn/received_events","type":"Organization","site_admin":false},"repo":{"id":11290232,"name":"flynn","full_name":"flynn/flynn","owner":{"login":"flynn","id":4973613,"avatar_url":"https://avatars.githubusercontent.com/u/4973613?v=3","gravatar_id":"","url":"https://api.github.com/users/flynn","html_url":"https://github.com/flynn","followers_url":"https://api.github.com/users/flynn/followers","following_url":"https://api.github.com/users/flynn/following{/other_user}","gists_url":"https://api.github.com/users/flynn/gists{/gist_id}","starred_url":"https://api.github.com/users/flynn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flynn/subscriptions","organizations_url":"https://api.github.com/users/flynn/orgs","repos_url":"https://api.github.com/users/flynn/repos","events_url":"https://api.github.com/users/flynn/events{/privacy}","received_events_url":"https://api.github.com/users/flynn/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/flynn/flynn","description":"A next generation open source platform as a service (PaaS)","fork":false,"url":"https://api.github.com/repos/flynn/flynn","forks_url":"https://api.github.com/repos/flynn/flynn/forks","keys_url":"https://api.github.com/repos/flynn/flynn/keys{/key_id}","collaborators_url":"https://api.github.com/repos/flynn/flynn/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/flynn/flynn/teams","hooks_url":"https://api.github.com/repos/flynn/flynn/hooks","issue_events_url":"https://api.github.com/repos/flynn/flynn/issues/events{/number}","events_url":"https://api.github.com/repos/flynn/flynn/events","assignees_url":"https://api.github.com/repos/flynn/flynn/assignees{/user}","branches_url":"https://api.github.com/repos/flynn/flynn/branches{/branch}","tags_url":"https://api.github.com/repos/flynn/flynn/tags","blobs_url":"https://api.github.com/repos/flynn/flynn/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/flynn/flynn/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/flynn/flynn/git/refs{/sha}","trees_url":"https://api.github.com/repos/flynn/flynn/git/trees{/sha}","statuses_url":"https://api.github.com/repos/flynn/flynn/statuses/{sha}","languages_url":"https://api.github.com/repos/flynn/flynn/languages","stargazers_url":"https://api.github.com/repos/flynn/flynn/stargazers","contributors_url":"https://api.github.com/repos/flynn/flynn/contributors","subscribers_url":"https://api.github.com/repos/flynn/flynn/subscribers","subscription_url":"https://api.github.com/repos/flynn/flynn/subscription","commits_url":"https://api.github.com/repos/flynn/flynn/commits{/sha}","git_commits_url":"https://api.github.com/repos/flynn/flynn/git/commits{/sha}","comments_url":"https://api.github.com/repos/flynn/flynn/comments{/number}","issue_comment_url":"https://api.github.com/repos/flynn/flynn/issues/comments{/number}","contents_url":"https://api.github.com/repos/flynn/flynn/contents/{+path}","compare_url":"https://api.github.com/repos/flynn/flynn/compare/{base}...{head}","merges_url":"https://api.github.com/repos/flynn/flynn/merges","archive_url":"https://api.github.com/repos/flynn/flynn/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/flynn/flynn/downloads","issues_url":"https://api.github.com/repos/flynn/flynn/issues{/number}","pulls_url":"https://api.github.com/repos/flynn/flynn/pulls{/number}","milestones_url":"https://api.github.com/repos/flynn/flynn/milestones{/number}","notifications_url":"https://api.github.com/repos/flynn/flynn/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/flynn/flynn/labels{/name}","releases_url":"https://api.github.com/repos/flynn/flynn/releases{/id}","deployments_url":"https://api.github.com/repos/flynn/flynn/deployments","created_at":"2013-07-09T18:22:06Z","updated_at":"2016-04-24T16:43:35Z","pushed_at":"2016-04-24T15:21:49Z","git_url":"git://github.com/flynn/flynn.git","ssh_url":"git@github.com:flynn/flynn.git","clone_url":"https://github.com/flynn/flynn.git","svn_url":"https://github.com/flynn/flynn","homepage":"https://flynn.io","size":28234,"stargazers_count":4432,"watchers_count":4432,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":332,"mirror_url":null,"open_issues_count":332,"forks":332,"open_issues":332,"watchers":4432,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/flynn/flynn/pulls/2733"},"html":{"href":"https://github.com/flynn/flynn/pull/2733"},"issue":{"href":"https://api.github.com/repos/flynn/flynn/issues/2733"},"comments":{"href":"https://api.github.com/repos/flynn/flynn/issues/2733/comments"},"review_comments":{"href":"https://api.github.com/repos/flynn/flynn/pulls/2733/comments"},"review_comment":{"href":"https://api.github.com/repos/flynn/flynn/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/flynn/flynn/pulls/2733/commits"},"statuses":{"href":"https://api.github.com/repos/flynn/flynn/statuses/fa474e3cfe32c9878457ed34f91db42a0313dd43"}}}}