{"action":"created","issue":{"url":"https://api.github.com/repos/concourse/concourse/issues/2588","repository_url":"https://api.github.com/repos/concourse/concourse","labels_url":"https://api.github.com/repos/concourse/concourse/issues/2588/labels{/name}","comments_url":"https://api.github.com/repos/concourse/concourse/issues/2588/comments","events_url":"https://api.github.com/repos/concourse/concourse/issues/2588/events","html_url":"https://github.com/concourse/concourse/issues/2588","id":360041185,"node_id":"MDU6SXNzdWUzNjAwNDExODU=","number":2588,"title":"GC containers & volumes from the DB if a worker reports that it no longer has them","user":{"login":"topherbullock","id":1895900,"node_id":"MDQ6VXNlcjE4OTU5MDA=","avatar_url":"https://avatars3.githubusercontent.com/u/1895900?v=4","gravatar_id":"","url":"https://api.github.com/users/topherbullock","html_url":"https://github.com/topherbullock","followers_url":"https://api.github.com/users/topherbullock/followers","following_url":"https://api.github.com/users/topherbullock/following{/other_user}","gists_url":"https://api.github.com/users/topherbullock/gists{/gist_id}","starred_url":"https://api.github.com/users/topherbullock/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/topherbullock/subscriptions","organizations_url":"https://api.github.com/users/topherbullock/orgs","repos_url":"https://api.github.com/users/topherbullock/repos","events_url":"https://api.github.com/users/topherbullock/events{/privacy}","received_events_url":"https://api.github.com/users/topherbullock/received_events","type":"User","site_admin":false},"labels":[{"id":573132128,"node_id":"MDU6TGFiZWw1NzMxMzIxMjg=","url":"https://api.github.com/repos/concourse/concourse/labels/resiliency","name":"resiliency","color":"201940","default":false},{"id":995593735,"node_id":"MDU6TGFiZWw5OTU1OTM3MzU=","url":"https://api.github.com/repos/concourse/concourse/labels/size/medium","name":"size/medium","color":"bbbbbb","default":false}],"state":"open","locked":false,"assignee":{"login":"mhuangpivotal","id":1919762,"node_id":"MDQ6VXNlcjE5MTk3NjI=","avatar_url":"https://avatars2.githubusercontent.com/u/1919762?v=4","gravatar_id":"","url":"https://api.github.com/users/mhuangpivotal","html_url":"https://github.com/mhuangpivotal","followers_url":"https://api.github.com/users/mhuangpivotal/followers","following_url":"https://api.github.com/users/mhuangpivotal/following{/other_user}","gists_url":"https://api.github.com/users/mhuangpivotal/gists{/gist_id}","starred_url":"https://api.github.com/users/mhuangpivotal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mhuangpivotal/subscriptions","organizations_url":"https://api.github.com/users/mhuangpivotal/orgs","repos_url":"https://api.github.com/users/mhuangpivotal/repos","events_url":"https://api.github.com/users/mhuangpivotal/events{/privacy}","received_events_url":"https://api.github.com/users/mhuangpivotal/received_events","type":"User","site_admin":false},"assignees":[{"login":"mhuangpivotal","id":1919762,"node_id":"MDQ6VXNlcjE5MTk3NjI=","avatar_url":"https://avatars2.githubusercontent.com/u/1919762?v=4","gravatar_id":"","url":"https://api.github.com/users/mhuangpivotal","html_url":"https://github.com/mhuangpivotal","followers_url":"https://api.github.com/users/mhuangpivotal/followers","following_url":"https://api.github.com/users/mhuangpivotal/following{/other_user}","gists_url":"https://api.github.com/users/mhuangpivotal/gists{/gist_id}","starred_url":"https://api.github.com/users/mhuangpivotal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mhuangpivotal/subscriptions","organizations_url":"https://api.github.com/users/mhuangpivotal/orgs","repos_url":"https://api.github.com/users/mhuangpivotal/repos","events_url":"https://api.github.com/users/mhuangpivotal/events{/privacy}","received_events_url":"https://api.github.com/users/mhuangpivotal/received_events","type":"User","site_admin":false},{"login":"cirocosta","id":3574444,"node_id":"MDQ6VXNlcjM1NzQ0NDQ=","avatar_url":"https://avatars0.githubusercontent.com/u/3574444?v=4","gravatar_id":"","url":"https://api.github.com/users/cirocosta","html_url":"https://github.com/cirocosta","followers_url":"https://api.github.com/users/cirocosta/followers","following_url":"https://api.github.com/users/cirocosta/following{/other_user}","gists_url":"https://api.github.com/users/cirocosta/gists{/gist_id}","starred_url":"https://api.github.com/users/cirocosta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cirocosta/subscriptions","organizations_url":"https://api.github.com/users/cirocosta/orgs","repos_url":"https://api.github.com/users/cirocosta/repos","events_url":"https://api.github.com/users/cirocosta/events{/privacy}","received_events_url":"https://api.github.com/users/cirocosta/received_events","type":"User","site_admin":false}],"milestone":null,"comments":5,"created_at":"2018-09-13T19:50:07Z","updated_at":"2018-09-24T14:50:26Z","closed_at":null,"author_association":"MEMBER","body":"when a worker reports a list of existing containers/volumes back to the ATCs, we should diff that set against what the DB thinks should be there and remove the ones the worker didn't report having\r\n\r\nThis will help with some of the issues when a worker goes away and comes back with different state:\r\n- Task fails with \"config file not found\" after restarting Docker service #1796\r\n- failed to find created volume in baggageclaim #1550\r\n- non-BOSH worker operation lifecycle #1457\r\n\r\nGotcha ( potential refactoring / documentation ) [gc.Destroyer's methods](https://github.com/concourse/atc/blob/master/gc/destroyer.go#L16-L17) take in a list of _reported_ handles, not the list of handles to destroy :)"},"comment":{"url":"https://api.github.com/repos/concourse/concourse/issues/comments/424002349","html_url":"https://github.com/concourse/concourse/issues/2588#issuecomment-424002349","issue_url":"https://api.github.com/repos/concourse/concourse/issues/2588","id":424002349,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNDAwMjM0OQ==","user":{"login":"vito","id":1880,"node_id":"MDQ6VXNlcjE4ODA=","avatar_url":"https://avatars1.githubusercontent.com/u/1880?v=4","gravatar_id":"","url":"https://api.github.com/users/vito","html_url":"https://github.com/vito","followers_url":"https://api.github.com/users/vito/followers","following_url":"https://api.github.com/users/vito/following{/other_user}","gists_url":"https://api.github.com/users/vito/gists{/gist_id}","starred_url":"https://api.github.com/users/vito/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vito/subscriptions","organizations_url":"https://api.github.com/users/vito/orgs","repos_url":"https://api.github.com/users/vito/repos","events_url":"https://api.github.com/users/vito/events{/privacy}","received_events_url":"https://api.github.com/users/vito/received_events","type":"User","site_admin":false},"created_at":"2018-09-24T14:50:26Z","updated_at":"2018-09-24T14:50:26Z","author_association":"MEMBER","body":"@cirocosta I don't think it'll end up being the same number of writes. You should only have to clear out `missing_since` for containers that both a) have it set and b) ended up being in a subsequent report of the containers. Something like:\r\n\r\n```sql\r\nUPDATE containers\r\nSET missing_since = NULL\r\nWHERE missing_since IS NOT NULL\r\nAND handle IN (...)\r\n```\r\n\r\nWe've had similar cases in the past where making the write conditional leads to a significant performance improvement. One example: https://github.com/concourse/concourse/commit/8aa4aca9af1f2e8333c31df056460e0e6f87acd8\r\n\r\nSo with `last_seen` you have a guaranteed write for all containers on every report. With `missing_since` you have a write for each missing container, plus a handful of writes for containers that were created on the time boundary and need their `missing_since` cleared out on the next report. The total writes should end up being much lower since they're both tied to emergent cases (containers disappearing) or edge cases (container created close to time of report)."}}
{"actor":{"display_login":"vito"}}