{"action":"created","issue":{"url":"https://api.github.com/repos/mika/jenkins-debian-glue/issues/158","repository_url":"https://api.github.com/repos/mika/jenkins-debian-glue","labels_url":"https://api.github.com/repos/mika/jenkins-debian-glue/issues/158/labels{/name}","comments_url":"https://api.github.com/repos/mika/jenkins-debian-glue/issues/158/comments","events_url":"https://api.github.com/repos/mika/jenkins-debian-glue/issues/158/events","html_url":"https://github.com/mika/jenkins-debian-glue/issues/158","id":160923283,"number":158,"title":"build-and-provide does not update cowbuilder environment","user":{"login":"mark0n","id":4429621,"avatar_url":"https://avatars.githubusercontent.com/u/4429621?v=3","gravatar_id":"","url":"https://api.github.com/users/mark0n","html_url":"https://github.com/mark0n","followers_url":"https://api.github.com/users/mark0n/followers","following_url":"https://api.github.com/users/mark0n/following{/other_user}","gists_url":"https://api.github.com/users/mark0n/gists{/gist_id}","starred_url":"https://api.github.com/users/mark0n/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mark0n/subscriptions","organizations_url":"https://api.github.com/users/mark0n/orgs","repos_url":"https://api.github.com/users/mark0n/repos","events_url":"https://api.github.com/users/mark0n/events{/privacy}","received_events_url":"https://api.github.com/users/mark0n/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-06-17T16:11:20Z","updated_at":"2016-06-20T16:43:35Z","closed_at":null,"body":"I'm running into issues with build-and-provide not updating the cowbuilder environment. This seems to be caused by a lock file that must have been left behind on an earlier run:\r\n```\r\n<jenkins5:~ >ls /var/run/lock/jessie-amd64.update -l\r\n-rw-r--r-- 1 jenkins-slave jenkins-slave 0 Jun  1 10:20 /var/run/lock/jessie-amd64.update                                                                                          \r\n```\r\nI'm not sure _why_ these lock files are left behind but this has happened multiple times on one of our build slaves. But I guess the _why_ doesn't matter too much (it could just be the result of a crashed executor). Anyway, the issue is detected by build-and-provide correctly\r\n```\r\n15:45:30 + echo '*** Update run already taking place, skipping ***'\r\n15:45:30 *** Update run already taking place, skipping ***\r\n```\r\nUnfortunately it seems to handle this case in the wrong way. It just prints the warning, skips the cowbuilder update and proceeds. As a consequence I end up with a cowbuilder environment that has not seen an `apt-get update` run for weeks causing builds against old versions of my libraries.\r\n\r\n@mika: I'm still a bit confused by some details of the locking code (the `9>` part in particular) but overall it makes sense to me - except for the following line: https://github.com/mika/jenkins-debian-glue/commit/9c37df2e10b0649672ea09fd07760f248dcce46d#diff-8d65555d96716579f851f18d113e9d79R331 Can you please elaborate a bit on your intentions? I believe we should at least sleep until the lock file goes away. We definitely shouldn't proceed with the build if we aren't 100% sure that our cowbuilder environment has actually successfully (and completely!) been updated. Even if another `cowbuilder --update` run is underway we should wait until it's done and then start our own update run. This would make sure we never miss the latest version of our build dependencies."},"comment":{"url":"https://api.github.com/repos/mika/jenkins-debian-glue/issues/comments/227198613","html_url":"https://github.com/mika/jenkins-debian-glue/issues/158#issuecomment-227198613","issue_url":"https://api.github.com/repos/mika/jenkins-debian-glue/issues/158","id":227198613,"user":{"login":"mark0n","id":4429621,"avatar_url":"https://avatars.githubusercontent.com/u/4429621?v=3","gravatar_id":"","url":"https://api.github.com/users/mark0n","html_url":"https://github.com/mark0n","followers_url":"https://api.github.com/users/mark0n/followers","following_url":"https://api.github.com/users/mark0n/following{/other_user}","gists_url":"https://api.github.com/users/mark0n/gists{/gist_id}","starred_url":"https://api.github.com/users/mark0n/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mark0n/subscriptions","organizations_url":"https://api.github.com/users/mark0n/orgs","repos_url":"https://api.github.com/users/mark0n/repos","events_url":"https://api.github.com/users/mark0n/events{/privacy}","received_events_url":"https://api.github.com/users/mark0n/received_events","type":"User","site_admin":false},"created_at":"2016-06-20T16:43:34Z","updated_at":"2016-06-20T16:43:34Z","body":"Ok, I think I understand the problem a bit better:\r\n\r\nThe lock _file_ is created by\r\n```\r\n(\r\n# code that uses the lock\r\n) 9>\"${update_lockfile}\"\r\n```\r\nThe first line inside this block is acquiring the _lock_ on the open file with file descriptor 9 from the kernel. The lock is automatically released as soon as the file is closed. This should happen even in case the process dies pretty badly. But the file might be left behind!\r\nIt seems like the problem is the test here: It is not testing for the _lock_ but for the file. I guess something like\r\n```\r\n# remove the whole \"Update run already taking place, skipping\" block\r\n(\r\nflock --wait 600 9 || exit 1\r\n) 9>@${update_lockfile}\"\r\n```\r\nwould be more appropriate.\r\n\r\nWe are currently testing a fix. I'll open a PR as soon as I feel confident about it."}}
{"actor":{"display_login":"mark0n"}}