{"action":"created","issue":{"url":"https://api.github.com/repos/jenkinsci/docker-plugin/issues/658","repository_url":"https://api.github.com/repos/jenkinsci/docker-plugin","labels_url":"https://api.github.com/repos/jenkinsci/docker-plugin/issues/658/labels{/name}","comments_url":"https://api.github.com/repos/jenkinsci/docker-plugin/issues/658/comments","events_url":"https://api.github.com/repos/jenkinsci/docker-plugin/issues/658/events","html_url":"https://github.com/jenkinsci/docker-plugin/pull/658","id":328779423,"node_id":"MDExOlB1bGxSZXF1ZXN0MTkyMjE5MTcy","number":658,"title":"[FEATURE] Docker Container&Slave Watchdog (new implementation)","user":{"login":"eaglerainbow","id":12811079,"node_id":"MDQ6VXNlcjEyODExMDc5","avatar_url":"https://avatars2.githubusercontent.com/u/12811079?v=4","gravatar_id":"","url":"https://api.github.com/users/eaglerainbow","html_url":"https://github.com/eaglerainbow","followers_url":"https://api.github.com/users/eaglerainbow/followers","following_url":"https://api.github.com/users/eaglerainbow/following{/other_user}","gists_url":"https://api.github.com/users/eaglerainbow/gists{/gist_id}","starred_url":"https://api.github.com/users/eaglerainbow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eaglerainbow/subscriptions","organizations_url":"https://api.github.com/users/eaglerainbow/orgs","repos_url":"https://api.github.com/users/eaglerainbow/repos","events_url":"https://api.github.com/users/eaglerainbow/events{/privacy}","received_events_url":"https://api.github.com/users/eaglerainbow/received_events","type":"User","site_admin":false},"labels":[{"id":74322758,"node_id":"MDU6TGFiZWw3NDMyMjc1OA==","url":"https://api.github.com/repos/jenkinsci/docker-plugin/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":{"login":"pjdarton","id":984537,"node_id":"MDQ6VXNlcjk4NDUzNw==","avatar_url":"https://avatars2.githubusercontent.com/u/984537?v=4","gravatar_id":"","url":"https://api.github.com/users/pjdarton","html_url":"https://github.com/pjdarton","followers_url":"https://api.github.com/users/pjdarton/followers","following_url":"https://api.github.com/users/pjdarton/following{/other_user}","gists_url":"https://api.github.com/users/pjdarton/gists{/gist_id}","starred_url":"https://api.github.com/users/pjdarton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pjdarton/subscriptions","organizations_url":"https://api.github.com/users/pjdarton/orgs","repos_url":"https://api.github.com/users/pjdarton/repos","events_url":"https://api.github.com/users/pjdarton/events{/privacy}","received_events_url":"https://api.github.com/users/pjdarton/received_events","type":"User","site_admin":false},"assignees":[{"login":"pjdarton","id":984537,"node_id":"MDQ6VXNlcjk4NDUzNw==","avatar_url":"https://avatars2.githubusercontent.com/u/984537?v=4","gravatar_id":"","url":"https://api.github.com/users/pjdarton","html_url":"https://github.com/pjdarton","followers_url":"https://api.github.com/users/pjdarton/followers","following_url":"https://api.github.com/users/pjdarton/following{/other_user}","gists_url":"https://api.github.com/users/pjdarton/gists{/gist_id}","starred_url":"https://api.github.com/users/pjdarton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pjdarton/subscriptions","organizations_url":"https://api.github.com/users/pjdarton/orgs","repos_url":"https://api.github.com/users/pjdarton/repos","events_url":"https://api.github.com/users/pjdarton/events{/privacy}","received_events_url":"https://api.github.com/users/pjdarton/received_events","type":"User","site_admin":false}],"milestone":null,"comments":34,"created_at":"2018-06-02T21:43:10Z","updated_at":"2018-07-19T13:35:27Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/jenkinsci/docker-plugin/pulls/658","html_url":"https://github.com/jenkinsci/docker-plugin/pull/658","diff_url":"https://github.com/jenkinsci/docker-plugin/pull/658.diff","patch_url":"https://github.com/jenkinsci/docker-plugin/pull/658.patch"},"body":"This shall replace https://github.com/jenkinsci/docker-plugin/pull/426.\r\n\r\nDue to massive refactoring and the fact that https://github.com/jenkinsci/docker-plugin/pull/426 has not been merged before, this PR proposes a greenfield-new implementation of a cleanup watchdog. Mainly, it provides two features:\r\n* It automatically detects orphaned containers on any of the DockerClouds attached to the current Jenkins setup. \r\n* It automatically detects offline slaves created by Docker Plugin, which do not have any association to a container, which would be running on a DockerCloud.\r\n\r\nFurthermore, this PR has the following properties:\r\n\r\n* Only containers generated by this Jenkins instance will be considered for comparison (i.e. multiple Jenkins instances can share the same docker host without risking the watchdogs to interfere with each other). \r\n* Container-to-Node mapping (and vice versa) are being performed based on the container's labels (and the metadata stored with the transient nodes)\r\n* Containers have a grace period of 60 seconds to exist on the docker host, before they are considered for potential cleanup (that is mainly to ensure that containers are not pruned when they created).\r\n* Termination of orphaned containers are attempted to be terminated using the same logic as in   `DockerTemplate` ensuring that also the follow-up activity after cleanup (the `removeVolumes` flag) are being made. If this does not work, a fallback to a more unfriendly but generic \"forced remove container\" call is performed.\r\n* The watchdog is automatically triggered every 5 minutes and runs as an asynchronous job in background of Jenkins.\r\n* It has been implemented to be minimal-invasive to the rest of the code.\r\n* This PR also contains a significant amount of unit tests to ensure test coverage.\r\n* It is configuration-free.\r\n* Activities are logged with INFO priority.\r\n* By setting system property `com.nirima.jenkins.plugins.docker.DockerContainerWatchdog.enabled=false` you may disable the watchdog.\r\n* On each execution of the watchdog, statistical key figures about number of containers removed, number of nodes removed, execution times, etc. are logged.\r\n* There is an inherent protection against runtime overspilling to prevent dominating the Jenkins host with cleanup activities.\r\n* In case a `DockerCloud` is disabled, then also the removal of containers (by the watchdog) is suppressed.\r\n* If the containers of at least one `DockerCloud`s could not be read properly, then no cleanup of nodes takes place (as the internal list for comparison is not complete then). "},"comment":{"url":"https://api.github.com/repos/jenkinsci/docker-plugin/issues/comments/406278214","html_url":"https://github.com/jenkinsci/docker-plugin/pull/658#issuecomment-406278214","issue_url":"https://api.github.com/repos/jenkinsci/docker-plugin/issues/658","id":406278214,"node_id":"MDEyOklzc3VlQ29tbWVudDQwNjI3ODIxNA==","user":{"login":"pjdarton","id":984537,"node_id":"MDQ6VXNlcjk4NDUzNw==","avatar_url":"https://avatars2.githubusercontent.com/u/984537?v=4","gravatar_id":"","url":"https://api.github.com/users/pjdarton","html_url":"https://github.com/pjdarton","followers_url":"https://api.github.com/users/pjdarton/followers","following_url":"https://api.github.com/users/pjdarton/following{/other_user}","gists_url":"https://api.github.com/users/pjdarton/gists{/gist_id}","starred_url":"https://api.github.com/users/pjdarton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pjdarton/subscriptions","organizations_url":"https://api.github.com/users/pjdarton/orgs","repos_url":"https://api.github.com/users/pjdarton/repos","events_url":"https://api.github.com/users/pjdarton/events{/privacy}","received_events_url":"https://api.github.com/users/pjdarton/received_events","type":"User","site_admin":false},"created_at":"2018-07-19T13:35:27Z","updated_at":"2018-07-19T13:35:27Z","author_association":"CONTRIBUTOR","body":"Sorry for the (lengthy) absence: work issues + vacation + work issues piling up while I was on vacation = long delay before I had a chance to look at this again.\r\nHowever, I've now had a chance to look (again) as these and, overall, I think we're good.\r\nI've added a few minor whitespace/javadoc changes (and I can see that the jenkinsci system exploded for no good reason while building one of them) ... but when that's the most serious thing I could think of changing then it tells us that I've run out of important issues to raise :-)\r\n\r\nI think we're ready to test this on our own Jenkins systems and see how it fares in a \"real world\" environment.\r\nI know that you previously mentioned that your admins were planning a \"major maintenance activity\" - has that passed now?"}}
{"actor":{"display_login":"pjdarton"}}