{"action":"created","issue":{"url":"https://api.github.com/repos/tidyverse/dplyr/issues/3788","repository_url":"https://api.github.com/repos/tidyverse/dplyr","labels_url":"https://api.github.com/repos/tidyverse/dplyr/issues/3788/labels{/name}","comments_url":"https://api.github.com/repos/tidyverse/dplyr/issues/3788/comments","events_url":"https://api.github.com/repos/tidyverse/dplyr/issues/3788/events","html_url":"https://github.com/tidyverse/dplyr/issues/3788","id":355557911,"node_id":"MDU6SXNzdWUzNTU1NTc5MTE=","number":3788,"title":"benchmark results filtering within and outside grouping ","user":{"login":"EdwinTh","id":7958036,"node_id":"MDQ6VXNlcjc5NTgwMzY=","avatar_url":"https://avatars2.githubusercontent.com/u/7958036?v=4","gravatar_id":"","url":"https://api.github.com/users/EdwinTh","html_url":"https://github.com/EdwinTh","followers_url":"https://api.github.com/users/EdwinTh/followers","following_url":"https://api.github.com/users/EdwinTh/following{/other_user}","gists_url":"https://api.github.com/users/EdwinTh/gists{/gist_id}","starred_url":"https://api.github.com/users/EdwinTh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/EdwinTh/subscriptions","organizations_url":"https://api.github.com/users/EdwinTh/orgs","repos_url":"https://api.github.com/users/EdwinTh/repos","events_url":"https://api.github.com/users/EdwinTh/events{/privacy}","received_events_url":"https://api.github.com/users/EdwinTh/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-08-30T12:13:53Z","updated_at":"2018-08-30T13:46:07Z","closed_at":null,"author_association":"NONE","body":"After discussion on [twitter](https://twitter.com/edwin_thoen/status/1034722004111446018) on how to keep the first `n` records for each group, I concluded that filtering outside grouping is way faster. This to the surprise of @hadley, who liked to see this issue filed. Results on my Macbook Pro are included. \r\n\r\n```r\r\nx <- data_frame(val = 1:10^7, group = rep(1:10^6, 10))\r\n\r\noriginal <- function(x, n, grouping_col) {\r\n   x %>%\r\n     group_by(!!enquo(grouping_col)) %>%\r\n     filter(row_number() %in% n) %>%\r\n     ungroup()\r\n}\r\n\r\n\r\n# suggested by Inaki Ucar @Enchufa2\r\nfilter_outside <- function(x, n, grouping_col) {\r\n  x %>%\r\n    group_by(!!enquo(grouping_col)) %>%\r\n    mutate(gr_nr = row_number()) %>%\r\n    ungroup() %>%\r\n    filter(gr_nr <= n) %>%\r\n    select(-gr_nr)\r\n}\r\n\r\n# suggested by Hadly\r\nsmaller_than_n <- function(x, n, grouping_col) {\r\n  x %>%\r\n    group_by(!!enquo(grouping_col)) %>%\r\n    filter(row_number() <= n) %>%\r\n    ungroup()\r\n}\r\n\r\nsystem.time(output_original <- original(x, 1, group))\r\n\r\n   user  system elapsed \r\n 21.068   0.915  22.646 \r\n\r\nsystem.time(output_outside  <- filter_outside(x, 1, group))\r\n\r\n   user  system elapsed \r\n  2.498   0.123   2.788 \r\n\r\nsystem.time(output_smaller  <- smaller_than_n(x, 1, group))\r\n\r\n   user  system elapsed \r\n 17.307   0.259  17.984 \r\n\r\nall.equal(output_original, output_outside)\r\n\r\n[1] TRUE\r\n\r\nall.equal(output_original, output_smaller)\r\n\r\n[1] TRUE\r\n``` \r\n"},"comment":{"url":"https://api.github.com/repos/tidyverse/dplyr/issues/comments/417325027","html_url":"https://github.com/tidyverse/dplyr/issues/3788#issuecomment-417325027","issue_url":"https://api.github.com/repos/tidyverse/dplyr/issues/3788","id":417325027,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNzMyNTAyNw==","user":{"login":"hadley","id":4196,"node_id":"MDQ6VXNlcjQxOTY=","avatar_url":"https://avatars3.githubusercontent.com/u/4196?v=4","gravatar_id":"","url":"https://api.github.com/users/hadley","html_url":"https://github.com/hadley","followers_url":"https://api.github.com/users/hadley/followers","following_url":"https://api.github.com/users/hadley/following{/other_user}","gists_url":"https://api.github.com/users/hadley/gists{/gist_id}","starred_url":"https://api.github.com/users/hadley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hadley/subscriptions","organizations_url":"https://api.github.com/users/hadley/orgs","repos_url":"https://api.github.com/users/hadley/repos","events_url":"https://api.github.com/users/hadley/events{/privacy}","received_events_url":"https://api.github.com/users/hadley/received_events","type":"User","site_admin":false},"created_at":"2018-08-30T13:46:07Z","updated_at":"2018-08-30T13:46:07Z","author_association":"MEMBER","body":"Minimal reprex with smaller data set and using `bench::mark()`:\r\n\r\n``` r\r\nlibrary(dplyr, warn.conflicts = FALSE)\r\n\r\nf1 <- function(x, n, grouping_col) {\r\n  x %>%\r\n    group_by(!!enquo(grouping_col)) %>%\r\n    filter(row_number() <= n) %>%\r\n    ungroup()\r\n}\r\n\r\n# suggested by Inaki Ucar\r\nf2 <- function(x, n, grouping_col) {\r\n  x %>%\r\n    group_by(!!enquo(grouping_col)) %>%\r\n    mutate(gr_nr = row_number()) %>%\r\n    ungroup() %>%\r\n    filter(gr_nr <= n) %>%\r\n    select(-gr_nr)\r\n}\r\n\r\nx <- tibble(val = 1:10^4, group = rep(1:10^3, 10))\r\nbench::mark(\r\n  f1(x, 1, group), \r\n  f2(x, 1, group)\r\n)\r\n#> # A tibble: 2 x 10\r\n#>   expression     min    mean  median     max `itr/sec` mem_alloc  n_gc\r\n#>   <chr>      <bch:t> <bch:t> <bch:t> <bch:t>     <dbl> <bch:byt> <dbl>\r\n#> 1 f1(x, 1, … 15.89ms 17.02ms 16.69ms 21.97ms      58.8    4.91MB    11\r\n#> 2 f2(x, 1, …  3.36ms  4.28ms  4.12ms  9.08ms     234.   746.41KB     7\r\n#> # ... with 2 more variables: n_itr <int>, total_time <bch:tm>\r\n```\r\n\r\n"}}
{"actor":{"display_login":"hadley"}}