{"action":"created","issue":{"url":"https://api.github.com/repos/Automattic/kue/issues/930","repository_url":"https://api.github.com/repos/Automattic/kue","labels_url":"https://api.github.com/repos/Automattic/kue/issues/930/labels{/name}","comments_url":"https://api.github.com/repos/Automattic/kue/issues/930/comments","events_url":"https://api.github.com/repos/Automattic/kue/issues/930/events","html_url":"https://github.com/Automattic/kue/issues/930","id":170095995,"number":930,"title":"How to processing multiple jobs in a batch?","user":{"login":"twang2218","id":6299096,"avatar_url":"https://avatars.githubusercontent.com/u/6299096?v=3","gravatar_id":"","url":"https://api.github.com/users/twang2218","html_url":"https://github.com/twang2218","followers_url":"https://api.github.com/users/twang2218/followers","following_url":"https://api.github.com/users/twang2218/following{/other_user}","gists_url":"https://api.github.com/users/twang2218/gists{/gist_id}","starred_url":"https://api.github.com/users/twang2218/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twang2218/subscriptions","organizations_url":"https://api.github.com/users/twang2218/orgs","repos_url":"https://api.github.com/users/twang2218/repos","events_url":"https://api.github.com/users/twang2218/events{/privacy}","received_events_url":"https://api.github.com/users/twang2218/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Automattic/kue/labels/Question","name":"Question","color":"bfe5bf"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-08-09T07:03:32Z","updated_at":"2016-08-24T17:28:06Z","closed_at":null,"body":"Assuming we have many jobs of insert data to a database in the queue, instead of process each jobs one by one, which is not very efficient, I expect to be able to grab multiple jobs, and insert those data in one go by using db's bulk insert command. How can I do that?\r\n\r\nIs there anyway to fetch all jobs(or a given number of jobs) from queue, process them and then mark them as done? Thanks."},"comment":{"url":"https://api.github.com/repos/Automattic/kue/issues/comments/242145252","html_url":"https://github.com/Automattic/kue/issues/930#issuecomment-242145252","issue_url":"https://api.github.com/repos/Automattic/kue/issues/930","id":242145252,"user":{"login":"kirps","id":698605,"avatar_url":"https://avatars.githubusercontent.com/u/698605?v=3","gravatar_id":"","url":"https://api.github.com/users/kirps","html_url":"https://github.com/kirps","followers_url":"https://api.github.com/users/kirps/followers","following_url":"https://api.github.com/users/kirps/following{/other_user}","gists_url":"https://api.github.com/users/kirps/gists{/gist_id}","starred_url":"https://api.github.com/users/kirps/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kirps/subscriptions","organizations_url":"https://api.github.com/users/kirps/orgs","repos_url":"https://api.github.com/users/kirps/repos","events_url":"https://api.github.com/users/kirps/events{/privacy}","received_events_url":"https://api.github.com/users/kirps/received_events","type":"User","site_admin":false},"created_at":"2016-08-24T17:28:06Z","updated_at":"2016-08-24T17:28:06Z","body":"Sorry @wiresurfer I wasn't monitoring this issue. \r\n\r\nMy use case is using Kue to create and ultimately execute weekly email jobs. These are summaries of activity in our system over the past week per customer. We send them to each customer stakeholder individually with Mandrill. I'm using Heroku and thus kue instead of something like a cron job.\r\n\r\nI originally tried to create a \"batch\" job per company, which would maintain the appropriate day for the company to get their emails and sit on the queue and track progress as it scheduled and completed individual jobs. However, I had a lot of trouble maintaining membership in the batch with the available tools. But the biggest issue was that I could not ensure job uniqueness. Given that it's an email to a person, I don't want it getting sent multiple times if the done function doesn't work or if Heroku happens to restart my dyno at the wrong time. \r\n\r\nI ended up just creating each email job individually, bypassing the batch and any additional interesting (but unneccessary) information I might get from that level of hierarchy. \r\n\r\nI did have to implement my own job map to (mostly) ensure uniqueness. There's a small chance that this could still fail if the job executed and heroku restarted at exactly the wrong time, but so far that has not proved to be an issue. You can see some discussion here: #856 "}}
{"actor":{"display_login":"kirps"}}