{"action":"opened","issue":{"url":"https://api.github.com/repos/sul-dlss/preservation_catalog/issues/703","repository_url":"https://api.github.com/repos/sul-dlss/preservation_catalog","labels_url":"https://api.github.com/repos/sul-dlss/preservation_catalog/issues/703/labels{/name}","comments_url":"https://api.github.com/repos/sul-dlss/preservation_catalog/issues/703/comments","events_url":"https://api.github.com/repos/sul-dlss/preservation_catalog/issues/703/events","html_url":"https://github.com/sul-dlss/preservation_catalog/issues/703","id":317752897,"number":703,"title":"(CV, M2C, C2M) wire up checks to queues","user":{"login":"jmartin-sul","id":7741604,"avatar_url":"https://avatars1.githubusercontent.com/u/7741604?v=4","gravatar_id":"","url":"https://api.github.com/users/jmartin-sul","html_url":"https://github.com/jmartin-sul","followers_url":"https://api.github.com/users/jmartin-sul/followers","following_url":"https://api.github.com/users/jmartin-sul/following{/other_user}","gists_url":"https://api.github.com/users/jmartin-sul/gists{/gist_id}","starred_url":"https://api.github.com/users/jmartin-sul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jmartin-sul/subscriptions","organizations_url":"https://api.github.com/users/jmartin-sul/orgs","repos_url":"https://api.github.com/users/jmartin-sul/repos","events_url":"https://api.github.com/users/jmartin-sul/events{/privacy}","received_events_url":"https://api.github.com/users/jmartin-sul/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-04-25T18:57:37Z","updated_at":"2018-04-25T18:57:37Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"as a \"phase two\" improvement, we've discussed an alternate approach to continuously running C2M, M2C, and CV:\n- keep the query that determines what to check\n- instead of looping over the things to check when the query runs and checking them at that time, just get the list of druids and put them on a queue of things to CV/M2C/C2M check.\n- expose the logic that does the checking as a workers that listen to the queues (a CV worker, an M2C worker, a C2M worker).\n\nbenefits of this switch:\n- makes it easier to monitor progress and backlog on these checks.\n- keeps you from scheduling a long running rake task on cron:  the cron'ed task would be the query to populate the queue, which should run quickly, and the workers can chip away at the queue as needed.\n- makes it easy to have multiple machines doing a given check (e.g. CV) without having to segregate machines to checking specific endpoints (the queue will de-duplicate requests for a given druid, and one worker will pick up a given druid at a time).  so slightly easier to scale than the \"rake task does everything\" approach."}}
{"actor":{"display_login":"jmartin-sul"}}