{"action":"created","issue":{"url":"https://api.github.com/repos/bwlewis/doRedis/issues/14","repository_url":"https://api.github.com/repos/bwlewis/doRedis","labels_url":"https://api.github.com/repos/bwlewis/doRedis/issues/14/labels{/name}","comments_url":"https://api.github.com/repos/bwlewis/doRedis/issues/14/comments","events_url":"https://api.github.com/repos/bwlewis/doRedis/issues/14/events","html_url":"https://github.com/bwlewis/doRedis/issues/14","id":43308605,"number":14,"title":"Task gets resubmitted when worker is just working on it; last task was not processed at all","user":{"login":"tomastelensky","id":1380190,"avatar_url":"https://avatars.githubusercontent.com/u/1380190?v=3","gravatar_id":"","url":"https://api.github.com/users/tomastelensky","html_url":"https://github.com/tomastelensky","followers_url":"https://api.github.com/users/tomastelensky/followers","following_url":"https://api.github.com/users/tomastelensky/following{/other_user}","gists_url":"https://api.github.com/users/tomastelensky/gists{/gist_id}","starred_url":"https://api.github.com/users/tomastelensky/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tomastelensky/subscriptions","organizations_url":"https://api.github.com/users/tomastelensky/orgs","repos_url":"https://api.github.com/users/tomastelensky/repos","events_url":"https://api.github.com/users/tomastelensky/events{/privacy}","received_events_url":"https://api.github.com/users/tomastelensky/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2014-09-20T11:49:04Z","updated_at":"2016-05-03T12:02:20Z","closed_at":null,"body":"It happened to me that the first task was re-submitted while worker was working on it. \r\n\r\nLooking at the console of a single worker (and also the master), the last task (of the total 4) was not processed at all, while the first task was processed twice. Foreach returned list where the last task was missing (list of length only 3). After the foreach job was done, the worker kept writing this on the console:\r\n\r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\nProcessing job  1  from queue  jobs \r\n...\r\n\r\nOn the master, the code is approximatelly sketched below. I always clean the queue by using special trick (see http://stackoverflow.com/q/25947991/684229)\r\n\r\n```R\r\n    removeQueue('jobs')\r\n    registerDoRedis('jobs', redis_server)\r\n    r <- foreach (i = c(226, 229, 230, 246), .errorhandling = 'pass', .verbose = TRUE) %dopar% {\r\n       ... really long code (running JAGS using runjags package)\r\n    }\r\n    removeQueue('jobs') # to clean up the queue\r\n    registerDoRedis('jobs', redis_server) # so that worker doesn't stop looping\r\n```\r\n\r\nThis is the console output of the verbose foreach loop:\r\n\r\nnumValues: 4, numResults: 0, stopped: TRUE\r\nautomatically exporting the following objects from the local environment:\r\n  bird_dataset_nick, counts, envir, formulas, pop_model, read_poi, species, var_nick \r\nWarning in e$fun(obj, substitute(ex), parent.frame(), e$data) :\r\n  Worker fault, resubmitting task 1.\r\ngot results for task 1\r\nnumValues: 4, numResults: 1, stopped: TRUE\r\nreturning status FALSE\r\ngot results for task 1\r\nnumValues: 4, numResults: 2, stopped: TRUE\r\nreturning status FALSE\r\ngot results for task 2\r\nnumValues: 4, numResults: 3, stopped: TRUE\r\nreturning status FALSE\r\ngot results for task 3\r\nnumValues: 4, numResults: 4, stopped: TRUE\r\ncalling combine function\r\nevaluating call object to combine results:\r\n  fun(accum, result.1, result.2, result.3)\r\nreturning status TRUE\r\n\r\nI use R 3.1.0, doRedis 1.1.1, rredis 1.6.9 and redis server 2.6.12, all on a single host Windows XP. \r\n\r\nI don't know how to reproduce the issue - it happens only sometimes.\r\n\r\n**EDIT: another case:**\r\n\r\nOutput of `foreach` on the master:\r\n\r\nnumValues: 4, numResults: 0, stopped: TRUE\r\nautomatically exporting the following objects from the local environment:\r\n  bird_dataset_nick, counts, data, env_formula, envir, euring, f_ind, formula, formulas, no_sites, poi, pop_model, read_poi, sciname, species, species.use, var_nick \r\nWarning in e$fun(obj, substitute(ex), parent.frame(), e$data) :\r\n  Worker fault, resubmitting task 2.\r\ngot results for task 2\r\nnumValues: 4, numResults: 1, stopped: TRUE\r\nreturning status FALSE\r\ngot results for task 1\r\nnumValues: 4, numResults: 2, stopped: TRUE\r\nreturning status FALSE\r\ngot results for task 3\r\nnumValues: 4, numResults: 3, stopped: TRUE\r\nreturning status FALSE\r\ngot results for task 2\r\nnumValues: 4, numResults: 4, stopped: TRUE\r\ncalling combine function\r\nevaluating call object to combine results:\r\n  fun(accum, result.1, result.2, result.3)\r\nreturning status TRUE\r\n\r\nAgain, the results for the last task (4) were not collected. Examining the state of the `redis` server shows that the results remained there in the queue and can be collected there:\r\n\r\n> redisKeys(\"*\")\r\n[1] \"jobs:counter\"   \"jobs:1.results\" \"jobs:workers\"  \r\n> redisGet(\"jobs:counter\")\r\n[1] \"3\"\r\n> redisGet(\"jobs:workers\")\r\n[1] \"2\"\r\n> r <- redisLRange(\"jobs:1.results\", 0, -1)\r\n> save(r, file = \"jobs_1.results.Rdata\")\r\n> str(r)\r\nList of 1\r\n $ :List of 1\r\n  ..$ 4:List of 7\r\n  .. ..$ ...\r\n"},"comment":{"url":"https://api.github.com/repos/bwlewis/doRedis/issues/comments/216507548","html_url":"https://github.com/bwlewis/doRedis/issues/14#issuecomment-216507548","issue_url":"https://api.github.com/repos/bwlewis/doRedis/issues/14","id":216507548,"user":{"login":"thomaskisler","id":4718006,"avatar_url":"https://avatars.githubusercontent.com/u/4718006?v=3","gravatar_id":"","url":"https://api.github.com/users/thomaskisler","html_url":"https://github.com/thomaskisler","followers_url":"https://api.github.com/users/thomaskisler/followers","following_url":"https://api.github.com/users/thomaskisler/following{/other_user}","gists_url":"https://api.github.com/users/thomaskisler/gists{/gist_id}","starred_url":"https://api.github.com/users/thomaskisler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thomaskisler/subscriptions","organizations_url":"https://api.github.com/users/thomaskisler/orgs","repos_url":"https://api.github.com/users/thomaskisler/repos","events_url":"https://api.github.com/users/thomaskisler/events{/privacy}","received_events_url":"https://api.github.com/users/thomaskisler/received_events","type":"User","site_admin":false},"created_at":"2016-05-03T12:02:20Z","updated_at":"2016-05-03T12:02:20Z","body":"Just for future reference: yes, it is possible that this are two unrelated problems. In any case, the error gets resolved with the new development version."}}