{"action":"created","issue":{"url":"https://api.github.com/repos/Rdatatable/data.table/issues/3395","repository_url":"https://api.github.com/repos/Rdatatable/data.table","labels_url":"https://api.github.com/repos/Rdatatable/data.table/issues/3395/labels{/name}","comments_url":"https://api.github.com/repos/Rdatatable/data.table/issues/3395/comments","events_url":"https://api.github.com/repos/Rdatatable/data.table/issues/3395/events","html_url":"https://github.com/Rdatatable/data.table/issues/3395","id":409356678,"node_id":"MDU6SXNzdWU0MDkzNTY2Nzg=","number":3395,"title":"Performance drop with 1.12.0 (Selection + assignment)","user":{"login":"JeremyBesson","id":11754369,"node_id":"MDQ6VXNlcjExNzU0MzY5","avatar_url":"https://avatars2.githubusercontent.com/u/11754369?v=4","gravatar_id":"","url":"https://api.github.com/users/JeremyBesson","html_url":"https://github.com/JeremyBesson","followers_url":"https://api.github.com/users/JeremyBesson/followers","following_url":"https://api.github.com/users/JeremyBesson/following{/other_user}","gists_url":"https://api.github.com/users/JeremyBesson/gists{/gist_id}","starred_url":"https://api.github.com/users/JeremyBesson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JeremyBesson/subscriptions","organizations_url":"https://api.github.com/users/JeremyBesson/orgs","repos_url":"https://api.github.com/users/JeremyBesson/repos","events_url":"https://api.github.com/users/JeremyBesson/events{/privacy}","received_events_url":"https://api.github.com/users/JeremyBesson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2019-02-12T15:28:10Z","updated_at":"2019-02-13T07:24:30Z","closed_at":null,"author_association":"NONE","body":"I'm not sure if it's the same issue as #3330 because I haven't set the DTthreads parameters to 1.\r\nBut I could imagine that the default value is 1. \r\nThis list of operations take less than 1 second  to be executed with data.table  v1.11.8.\r\nWith data.table v1.12.0 it's take more than 4 seconds !\r\n\r\n\r\n```\r\n>   myDataTable[colJ != \"production\", colK:=na.omit(colK), by=.(colA)]\r\n>   myDataTable[colJ != \"production\", colH:=na.omit(colH), by=.(colA)]\r\n>   myDataTable[colJ != \"production\", colL:=na.omit(colL), by=.(colA)]\r\n>   myDataTable[colJ == \"code\",colM:=colE, by=.(colA)]\r\n>   myDataTable[,colM:=na.omit(colM), by=.(colA)]\r\n>   myDataTable[is.na(colM), colM:=\"\"]\r\n>   myDataTable[colJ == \"test\",colN:=colE, by=.(colA)]\r\n>   myDataTable[,colN:=na.omit(colN), by=.(colA)]\r\n```\r\n\r\nAnd there is only 4 rows in my data table : \r\n\r\n```\r\nprint(myDataTable)\r\n\r\n     colA colB colC colD colE colF colG colH colI colJ colK colL colM colN\r\n1 \t text text text text text text text text text text text text text text\r\n2 \t text text text text text text text text text text text text text text\r\n3 \t text text text text text text text text text text text text text text\r\n4  \t text text text text text text text text text text text text text text\r\n```                   \r\n(You can replace \"text\" by whatever) \r\n\r\nWe see this problem only in our production environment and not in other environment. This production server is really busy so it can explain why it's so long. But the test with  v1.11.8 was done in the same environment without performance issue (We done this test many times with both version).\r\n\r\nWe run R in a Docker container with the system Ubuntu 16.04.\r\n\r\n```\r\n> sessionInfo()\r\nR version 3.4.4 (2018-03-15)\r\nPlatform: x86_64-pc-linux-gnu (64-bit)\r\nRunning under: Ubuntu 16.04.5 LTS\r\n\r\nMatrix products: default\r\nBLAS: /usr/lib/libblas/libblas.so.3.6.0\r\nLAPACK: /usr/lib/lapack/liblapack.so.3.6.0\r\n\r\nlocale:\r\n [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C\r\n [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8\r\n [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8\r\n [7] LC_PAPER=en_US.UTF-8       LC_NAME=C\r\n [9] LC_ADDRESS=C               LC_TELEPHONE=C\r\n[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C\r\n\r\nattached base packages:\r\n[1] stats     graphics  grDevices utils     datasets  methods   base\r\n\r\nloaded via a namespace (and not attached):\r\n[1] compiler_3.4.4\r\n```\r\n\r\nI hope this will be fixed in a future data.table release because we will be blocked in v1.11.8 until this issue is not resolved.\r\nThank you for your support. \r\n\r\n\r\n\r\n  \r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/Rdatatable/data.table/issues/comments/463088360","html_url":"https://github.com/Rdatatable/data.table/issues/3395#issuecomment-463088360","issue_url":"https://api.github.com/repos/Rdatatable/data.table/issues/3395","id":463088360,"node_id":"MDEyOklzc3VlQ29tbWVudDQ2MzA4ODM2MA==","user":{"login":"HughParsonage","id":8111628,"node_id":"MDQ6VXNlcjgxMTE2Mjg=","avatar_url":"https://avatars1.githubusercontent.com/u/8111628?v=4","gravatar_id":"","url":"https://api.github.com/users/HughParsonage","html_url":"https://github.com/HughParsonage","followers_url":"https://api.github.com/users/HughParsonage/followers","following_url":"https://api.github.com/users/HughParsonage/following{/other_user}","gists_url":"https://api.github.com/users/HughParsonage/gists{/gist_id}","starred_url":"https://api.github.com/users/HughParsonage/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HughParsonage/subscriptions","organizations_url":"https://api.github.com/users/HughParsonage/orgs","repos_url":"https://api.github.com/users/HughParsonage/repos","events_url":"https://api.github.com/users/HughParsonage/events{/privacy}","received_events_url":"https://api.github.com/users/HughParsonage/received_events","type":"User","site_admin":false},"created_at":"2019-02-13T07:24:30Z","updated_at":"2019-02-13T07:24:30Z","author_association":"MEMBER","body":"Here you go: https://circleci.com/gh/HughParsonage/grattan/tree/population-forecast\r\n\r\nTurned out to be a 300x increase in runtime(!!):\r\n\r\nWith multithreading:\r\n\r\n![image](https://user-images.githubusercontent.com/8111628/52694012-8304d580-2fbc-11e9-8459-4150dbef8df7.png)\r\n\r\n\r\nand without:\r\n\r\n\r\n![image](https://user-images.githubusercontent.com/8111628/52693973-64064380-2fbc-11e9-8aaf-40e8365f8ba6.png)\r\n"}}
{"actor":{"display_login":"HughParsonage"}}