{"action":"created","issue":{"url":"https://api.github.com/repos/ropensci/drake/issues/280","repository_url":"https://api.github.com/repos/ropensci/drake","labels_url":"https://api.github.com/repos/ropensci/drake/issues/280/labels{/name}","comments_url":"https://api.github.com/repos/ropensci/drake/issues/280/comments","events_url":"https://api.github.com/repos/ropensci/drake/issues/280/events","html_url":"https://github.com/ropensci/drake/issues/280","id":299867125,"number":280,"title":"Master aborts with staged parallelism and failing dependencies even if keep_going is TRUE","user":{"login":"krlmlr","id":1741643,"avatar_url":"https://avatars0.githubusercontent.com/u/1741643?v=4","gravatar_id":"","url":"https://api.github.com/users/krlmlr","html_url":"https://github.com/krlmlr","followers_url":"https://api.github.com/users/krlmlr/followers","following_url":"https://api.github.com/users/krlmlr/following{/other_user}","gists_url":"https://api.github.com/users/krlmlr/gists{/gist_id}","starred_url":"https://api.github.com/users/krlmlr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/krlmlr/subscriptions","organizations_url":"https://api.github.com/users/krlmlr/orgs","repos_url":"https://api.github.com/users/krlmlr/repos","events_url":"https://api.github.com/users/krlmlr/events{/privacy}","received_events_url":"https://api.github.com/users/krlmlr/received_events","type":"User","site_admin":false},"labels":[{"id":701210128,"url":"https://api.github.com/repos/ropensci/drake/labels/high%20priority","name":"high priority","color":"d93f0b","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-02-23T22:13:27Z","updated_at":"2018-02-24T13:29:24Z","closed_at":"2018-02-24T03:47:42Z","author_association":"COLLABORATOR","body":"This isn't mentioned in the \"Caution\" vignette, looks like a bug to me. Workflow plan (from an earlier version with a different random seed) and reprex below:\r\n\r\n![rplot](https://user-images.githubusercontent.com/1741643/36619370-e3359218-18ee-11e8-97b0-d192436318a3.png)\r\n\r\n``` r\r\nlibrary(drake)\r\nlibrary(tidyverse)\r\n\r\nN <- 4\r\nR <- 2 / N\r\n\r\nset.seed(5)\r\n\r\nmake_job <- function(i, dep = NULL) {\r\n  if (is.null(dep)) {\r\n    deps_sym <- list()\r\n  } else {\r\n    deps_sym <- syms(paste0(dep, i))\r\n  }\r\n  \r\n  if (runif(1) < R) {\r\n    expr(fail(!!! deps_sym))\r\n  } else {\r\n    expr(succeed(!!! deps_sym))\r\n  }\r\n}\r\n\r\nfail <- function(...) {\r\n  stop(\"oops\")\r\n}\r\n\r\nsucceed <- function(...) {}\r\n\r\nplan_a <-\r\n  seq_len(N) %>%\r\n  as_tibble() %>%\r\n  transmute(\r\n    target = paste0(\"a\", value),\r\n    call = map(value, make_job)\r\n  ) %>%\r\n  deframe() %>% \r\n  drake_plan(list = .)\r\n\r\nplan_b <-\r\n  seq_len(N) %>%\r\n  as_tibble() %>%\r\n  transmute(\r\n    target = paste0(\"b\", value),\r\n    call = map(value, make_job, dep = \"a\")\r\n  ) %>%\r\n  deframe() %>% \r\n  drake_plan(list = .)\r\n\r\nplan <- \r\n  bind_rows(plan_a, plan_b)\r\n\r\nclean()\r\n#> cache /tmp/RtmpfQ4PI9/.drake\r\nmake(plan, jobs = 2)\r\n#> cache /tmp/RtmpfQ4PI9/.drake\r\n#> connect 8 imports: plan_a, plan_b, make_job, succeed, fail, N, plan, R\r\n#> connect 8 targets: a1, a2, a3, a4, b1, b2, b3, b4\r\n#> check 2 items: stop, succeed\r\n#> check 1 item: fail\r\n#> check 4 items: a1, a2, a3, a4\r\n#> Warning in mclapply(X = targets, FUN = drake_build_worker, meta_list =\r\n#> meta_list, : all scheduled cores encountered errors in user code\r\n#> check 4 items: b1, b2, b3, b4\r\n#> load 4 items: a1, a2, a3, a4\r\n#> Error: key 'a1' ('objects') not found\r\n\r\nclean()\r\n#> cache /tmp/RtmpfQ4PI9/.drake\r\nmake(plan, keep_going = TRUE, jobs = 2)\r\n#> cache /tmp/RtmpfQ4PI9/.drake\r\n#> connect 8 imports: plan_a, plan_b, make_job, succeed, fail, N, plan, R\r\n#> connect 8 targets: a1, a2, a3, a4, b1, b2, b3, b4\r\n#> check 2 items: stop, succeed\r\n#> check 1 item: fail\r\n#> check 4 items: a1, a2, a3, a4\r\n#> check 4 items: b1, b2, b3, b4\r\n#> load 4 items: a1, a2, a3, a4\r\n#> Error: key 'a1' ('objects') not found\r\nmake(plan, keep_going = TRUE, jobs = 2)\r\n#> cache /tmp/RtmpfQ4PI9/.drake\r\n#> connect 8 imports: plan_a, plan_b, make_job, succeed, fail, N, plan, R\r\n#> connect 8 targets: a1, a2, a3, a4, b1, b2, b3, b4\r\n#> check 2 items: stop, succeed\r\n#> check 1 item: fail\r\n#> check 4 items: a1, a2, a3, a4\r\n#> check 2 items: b2, b3\r\n#> load 2 items: a2, a3\r\n#> check 2 items: b1, b4\r\n#> unload 2 items: a2, a3\r\n#> load 2 items: a1, a4\r\n#> Error: key 'a1' ('objects') not found\r\nmake(plan, keep_going = TRUE, jobs = 2)\r\n#> cache /tmp/RtmpfQ4PI9/.drake\r\n#> connect 8 imports: plan_a, plan_b, make_job, succeed, fail, N, plan, R\r\n#> connect 8 targets: a1, a2, a3, a4, b1, b2, b3, b4\r\n#> check 2 items: stop, succeed\r\n#> check 1 item: fail\r\n#> check 4 items: a1, a2, a3, a4\r\n#> check 2 items: b2, b3\r\n#> check 2 items: b1, b4\r\n#> load 2 items: a1, a4\r\n#> Error: key 'a1' ('objects') not found\r\nmake(plan, keep_going = TRUE)\r\n#> cache /tmp/RtmpfQ4PI9/.drake\r\n#> connect 8 imports: plan_a, plan_b, make_job, succeed, fail, N, plan, R\r\n#> connect 8 targets: a1, a2, a3, a4, b1, b2, b3, b4\r\n#> check 2 items: stop, succeed\r\n#> check 1 item: fail\r\n#> check 4 items: a1, a2, a3, a4\r\n#> check 2 items: b2, b3\r\n#> target a1\r\n#> fail a1\r\n#> target a4\r\n#> fail a4\r\n#> check 2 items: b1, b4\r\n#> target b1\r\n#> fail b1\r\n#> target b4\r\n```\r\n\r\n> Created on 2018-02-23 by the [reprex package](http://reprex.tidyverse.org) (v0.2.0)."},"comment":{"url":"https://api.github.com/repos/ropensci/drake/issues/comments/368228564","html_url":"https://github.com/ropensci/drake/issues/280#issuecomment-368228564","issue_url":"https://api.github.com/repos/ropensci/drake/issues/280","id":368228564,"user":{"login":"krlmlr","id":1741643,"avatar_url":"https://avatars0.githubusercontent.com/u/1741643?v=4","gravatar_id":"","url":"https://api.github.com/users/krlmlr","html_url":"https://github.com/krlmlr","followers_url":"https://api.github.com/users/krlmlr/followers","following_url":"https://api.github.com/users/krlmlr/following{/other_user}","gists_url":"https://api.github.com/users/krlmlr/gists{/gist_id}","starred_url":"https://api.github.com/users/krlmlr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/krlmlr/subscriptions","organizations_url":"https://api.github.com/users/krlmlr/orgs","repos_url":"https://api.github.com/users/krlmlr/repos","events_url":"https://api.github.com/users/krlmlr/events{/privacy}","received_events_url":"https://api.github.com/users/krlmlr/received_events","type":"User","site_admin":false},"created_at":"2018-02-24T13:29:24Z","updated_at":"2018-02-24T13:29:24Z","author_association":"COLLABORATOR","body":"Not change `downstream_nodes()`, just the `targets` argument passed to it. What happens if `loadd()` succeeds because there is a cached but outdated value?"}}
{"actor":{"display_login":"krlmlr"}}