{"action":"created","issue":{"url":"https://api.github.com/repos/Rdatatable/data.table/issues/2707","repository_url":"https://api.github.com/repos/Rdatatable/data.table","labels_url":"https://api.github.com/repos/Rdatatable/data.table/issues/2707/labels{/name}","comments_url":"https://api.github.com/repos/Rdatatable/data.table/issues/2707/comments","events_url":"https://api.github.com/repos/Rdatatable/data.table/issues/2707/events","html_url":"https://github.com/Rdatatable/data.table/issues/2707","id":308601723,"number":2707,"title":"Segfault during sorting","user":{"login":"KyleDKavanagh","id":8503915,"avatar_url":"https://avatars3.githubusercontent.com/u/8503915?v=4","gravatar_id":"","url":"https://api.github.com/users/KyleDKavanagh","html_url":"https://github.com/KyleDKavanagh","followers_url":"https://api.github.com/users/KyleDKavanagh/followers","following_url":"https://api.github.com/users/KyleDKavanagh/following{/other_user}","gists_url":"https://api.github.com/users/KyleDKavanagh/gists{/gist_id}","starred_url":"https://api.github.com/users/KyleDKavanagh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KyleDKavanagh/subscriptions","organizations_url":"https://api.github.com/users/KyleDKavanagh/orgs","repos_url":"https://api.github.com/users/KyleDKavanagh/repos","events_url":"https://api.github.com/users/KyleDKavanagh/events{/privacy}","received_events_url":"https://api.github.com/users/KyleDKavanagh/received_events","type":"User","site_admin":false},"labels":[{"id":642468297,"url":"https://api.github.com/repos/Rdatatable/data.table/labels/dev","name":"dev","color":"5319e7","default":false},{"id":639641088,"url":"https://api.github.com/repos/Rdatatable/data.table/labels/seg.fault","name":"seg.fault","color":"000000","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/Rdatatable/data.table/milestones/8","html_url":"https://github.com/Rdatatable/data.table/milestone/8","labels_url":"https://api.github.com/repos/Rdatatable/data.table/milestones/8/labels","id":2291002,"number":8,"title":"v1.10.6","description":"","creator":{"login":"mattdowle","id":2779998,"avatar_url":"https://avatars3.githubusercontent.com/u/2779998?v=4","gravatar_id":"","url":"https://api.github.com/users/mattdowle","html_url":"https://github.com/mattdowle","followers_url":"https://api.github.com/users/mattdowle/followers","following_url":"https://api.github.com/users/mattdowle/following{/other_user}","gists_url":"https://api.github.com/users/mattdowle/gists{/gist_id}","starred_url":"https://api.github.com/users/mattdowle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattdowle/subscriptions","organizations_url":"https://api.github.com/users/mattdowle/orgs","repos_url":"https://api.github.com/users/mattdowle/repos","events_url":"https://api.github.com/users/mattdowle/events{/privacy}","received_events_url":"https://api.github.com/users/mattdowle/received_events","type":"User","site_admin":false},"open_issues":25,"closed_issues":265,"state":"open","created_at":"2017-02-01T00:17:34Z","updated_at":"2018-03-27T23:42:41Z","due_on":null,"closed_at":null},"comments":6,"created_at":"2018-03-26T14:41:30Z","updated_at":"2018-03-28T00:04:54Z","closed_at":null,"author_association":"NONE","body":"With v1.10.5 installed from source, newly seeing a segfault during standard grpby/assign operations on massive datatables\r\n\r\n``` *** caught segfault ***\r\naddress 0x7fc8862fa354, cause 'memory not mapped'\r\n\r\nTraceback:\r\n 1: fsort(as.numeric(irows))\r\n 2: `[.data.table`(dt, query %in% c(\"1\", \"2\"), `:=`(position = as.numeric(1:.N)),     by = grpvar)\r\n 3: dt[query %in% c(\"1\", \"2\"), `:=`(position = as.numeric(1:.N)),     by = grpvar]\r\nAn irrecoverable exception occurred. R is aborting now ...```\r\n"},"comment":{"url":"https://api.github.com/repos/Rdatatable/data.table/issues/comments/376714955","html_url":"https://github.com/Rdatatable/data.table/issues/2707#issuecomment-376714955","issue_url":"https://api.github.com/repos/Rdatatable/data.table/issues/2707","id":376714955,"user":{"login":"KyleDKavanagh","id":8503915,"avatar_url":"https://avatars3.githubusercontent.com/u/8503915?v=4","gravatar_id":"","url":"https://api.github.com/users/KyleDKavanagh","html_url":"https://github.com/KyleDKavanagh","followers_url":"https://api.github.com/users/KyleDKavanagh/followers","following_url":"https://api.github.com/users/KyleDKavanagh/following{/other_user}","gists_url":"https://api.github.com/users/KyleDKavanagh/gists{/gist_id}","starred_url":"https://api.github.com/users/KyleDKavanagh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KyleDKavanagh/subscriptions","organizations_url":"https://api.github.com/users/KyleDKavanagh/orgs","repos_url":"https://api.github.com/users/KyleDKavanagh/repos","events_url":"https://api.github.com/users/KyleDKavanagh/events{/privacy}","received_events_url":"https://api.github.com/users/KyleDKavanagh/received_events","type":"User","site_admin":false},"created_at":"2018-03-28T00:04:54Z","updated_at":"2018-03-28T00:04:54Z","author_association":"NONE","body":"Everything below run on v1.10.4-3 after rolling back \r\n\r\n```\r\n> dt[query %in% c(\"1\", \"2\"), `:=`(grpIdx=as.numeric(1:.N)),by=grpvar, verbose=T];\r\nUsing existing index 'query'\r\nStarting bmerge ...done in 0 secs\r\ni clause present and columns used in by detected, only these subset: grpvar\r\nDetected that j uses these columns: <none> \r\nFinding groups using forderv ... 0.094 sec\r\nFinding group sizes from the positions (can be avoided to save RAM) ... 0 sec\r\nGetting back original order ... 0.006 sec\r\nlapply optimization is on, j unchanged as 'list(as.numeric(1:.N))'\r\nOld mean optimization is on, left j unchanged.\r\nMaking each group and running j (GForce FALSE) ... \r\n  collecting discontiguous groups took 0.130s for 289239 groups\r\n  eval(j) took 4.950s for 289239 calls\r\n```\r\n\r\nResults from str()\r\n```\r\nClasses ‘data.table’ and 'data.frame':\t13539020 obs. of  4 variables:\r\n $ ts_ns             : chr  \"1522156437655270332\" \"1522156437655279062\" \"1522156437655286082\" \"1522156437655293372\" ...\r\n $ groupvar      : chr  \"\" \"\" \"\" \"\" ...\r\n $ query     : chr  \"1\" \"1\" \"1\" \"1\" ...\r\n $ SBE.SourceSequence: chr  \"0\" \"0\" \"0\" \"0\" ...\r\n - attr(*, \".internal.selfref\")=<externalptr> \r\n - attr(*, \"sorted\")= chr \"SBE.SourceSequence\"\r\n```\r\n\r\nResults from uniqueN()\r\n```\r\n             ts_ns          groupvar                query SBE.SourceSequence \r\n          13539020           12173006                  4           12735508 \r\n```\r\n\r\nValues for query:\r\n```\r\n   query     num\r\n1:     1  595317\r\n2:     0 6193535\r\n3:     2  758743\r\n4:     5 5991425\r\n```\r\n\r\nfsort debug - Not sure how helpful this is because of the number of rows\r\n```\r\nBrowse[2]> x\r\n   [1]     1     2     3     4     5     6     7     8     9    10    11    12    14    16    20    21    22    24    26    27    28    30    31    32\r\n  [25]    34    36    37    38    40    41    42    44    45    46    48    49    50    52    53    54    56    57    58    61    62    63    68    69\r\n  [49]    70    73    75    77    78    79    80    82    84    86    88    93    94    95    97    98    99   100   103   104   105   107   109   112\r\n  [73]   113   114   119   121   123   127   129   139   145   146   147   149   150   151   153   154   155   157   158   159   160   162   164   166\r\n  [97]   168   170   174   175   176   178   179   180   182   184   186   187   188   265   266   267   271   272   273   294   295   296   305   306\r\n [121]   307   308   309   310   326   327   328   333   334   335   348   349   350   382   383   384   385   386   387   419   420   421   488   489\r\n [145]   490   491   492   493   539   540   541   700   701   702   703   704   705   899   900   901   902   903   904   905   906   907   934   935\r\n [169]   936  1023  1024  1025  1026  1027  1028  1029  1030  1031  1032  1033  1034  1035  1036  1037  1038  1039  1040  1041  1042  1043  1130  1131\r\n [193]  1132  1133  1134  1135  1288  1289  1290  1291  1292  1293  1521  1522  1523  1591  1592  1593  1618  1619  1620  1621  1622  1623  1624  1625\r\n [217]  1626  2051  2052  2053  2083  2084  2085  2094  2095  2096  2117  2118  2119  2150  2151  2152  2173  2174  2175  2193  2194  2195  2226  2227\r\n [241]  2228  2298  2299  2300  2317  2318  2319  2337  2338  2339  2342  2343  2344  2367  2368  2369  2393  2394  2395  2462  2463  2464  2490  2491\r\n [265]  2492  2618  2619  2620  2641  2642  2643  2654  2655  2656  2692  2693  2694  2706  2707  2708  2716  2717  2718  2749  2750  2751  2763  2764\r\n [289]  2765  2766  2767  2768  2797  2798  2799  2837  2838  2839  2869  2870  2871  2913  2914  2915  2938  2939  2940  2969  2970  2971  3034  3035\r\n [313]  3036  3044  3045  3046  3053  3054  3055  3070  3071  3072  3076  3077  3078  3082  3083  3084  3126  3127  3128  3174  3175  3176  3184  3185\r\n [337]  3186  3212  3213  3214  3314  3315  3316  3392  3393  3394  3439  3440  3441  3446  3447  3448  3481  3482  3483  3493  3494  3495  3501  3502\r\n [361]  3503  3527  3528  3529  3542  3543  3544  3585  3586  3587  3712  3713  3714  3750  3751  3752  3784  3785  3786  3828  3829  3830  3863  3864\r\n [385]  3865  3904  3905  3906  3915  3916  3917  3978  3979  3980  3993  3994  3995  4008  4009  4010  4015  4016  4017  4029  4030  4031  4099  4100\r\n [409]  4101  4115  4116  4117  4136  4137  4138  4146  4147  4148  4159  4160  4161  4168  4169  4170  4216  4217  4218  4265  4266  4267  4270  4271\r\n [433]  4272  4283  4284  4285  4338  4339  4340  4374  4375  4376  4400  4401  4402  4515  4516  4517  4740  4741  4742  4743  4744  4745  4784  4785\r\n [457]  4786  4811  4812  4813  4873  4874  4875  4908  4909  4910  4932  4933  4934  4951  4952  4953  4957  4958  4959  5054  5055  5056  5107  5108\r\n [481]  5109  5127  5128  5129  5159  5160  5161  5209  5210  5211  5267  5268  5269  5318  5319  5320  5347  5348  5349  5378  5379  5380  5423  5424\r\n [505]  5425  5465  5466  5467  5642  5643  5644  5786  5787  5788  5797  5798  5799  5827  5828  5829  5840  5841  5842  6001  6002  6003  6019  6020\r\n [529]  6021  6122  6123  6124  6198  6199  6200  6219  6220  6221  6233  6234  6235  6254  6255  6256  6302  6303  6304  6314  6315  6316  6345  6346\r\n [553]  6347  6400  6401  6402  6428  6429  6430  6561  6562  6563  6596  6597  6598  6602  6603  6604  6666  6667  6668  6693  6694  6695  6703  6704\r\n [577]  6705  6739  6740  6741  6765  6766  6767  6923  6924  6925  6960  6961  6962  6978  6979  6980  7149  7150  7151  7193  7194  7195  7210  7211\r\n [601]  7212  7380  7381  7382  7529  7530  7531  7609  7610  7611  7664  7665  7666  7682  7683  7684  7686  7687  7688  7707  7708  7709  7797  7798\r\n [625]  7799  7852  7853  7854  7868  7869  7870  7887  7888  7889  7935  7936  7937  7941  7942  7943  7964  7965  7966  7979  7980  7981  8045  8046\r\n [649]  8047  8051  8052  8053  8085  8086  8087  8099  8100  8101  8208  8209  8210  8345  8346  8347  8360  8361  8362  8375  8376  8377  8463  8464\r\n [673]  8465  8498  8499  8500  8534  8535  8536  8543  8544  8545  8591  8592  8593  8657  8658  8659  8661  8662  8663  8699  8700  8701  8714  8715\r\n [697]  8716  8750  8751  8752  8773  8774  8775  8785  8786  8787  8946  8947  8948  9128  9129  9130  9155  9156  9157  9244  9245  9246  9252  9253\r\n [721]  9254  9274  9275  9276  9290  9291  9292  9293  9294  9295  9367  9368  9369  9447  9448  9449  9477  9478  9479  9484  9485  9486  9495  9496\r\n [745]  9497  9559  9560  9561  9695  9696  9697  9763  9764  9765  9786  9787  9788  9862  9863  9864 10013 10014 10015 10130 10131 10132 10197 10198\r\n [769] 10199 10235 10236 10237 10260 10261 10262 10266 10267 10268 10295 10296 10297 10313 10314 10315 10392 10393 10394 10413 10414 10415 10455 10456\r\n [793] 10457 10470 10471 10472 10502 10503 10504 10550 10551 10552 10560 10561 10562 10615 10616 10617 10677 10678 10679 10723 10724 10725 10751 10752\r\n [817] 10753 10754 10755 10756 10808 10809 10810 10867 10868 10869 10879 10880 10881 10893 10894 10895 10943 10944 10945 10980 10981 10982 11056 11057\r\n [841] 11058 11082 11083 11084 11106 11107 11108 11116 11117 11118 11127 11128 11129 11183 11184 11185 11203 11204 11205 11253 11254 11255 11259 11260\r\n [865] 11261 11297 11298 11299 11372 11373 11374 11404 11405 11406 11428 11429 11430 11444 11445 11446 11461 11462 11463 11504 11505 11506 11514 11515\r\n [889] 11516 11696 11697 11698 11816 11817 11818 11849 11850 11851 11897 11898 11899 11908 11909 11910 11962 11963 11964 11986 11987 11988 12036 12037\r\n [913] 12038 12058 12059 12060 12109 12110 12111 12141 12142 12143 12159 12160 12161 12166 12167 12168 12236 12237 12238 12307 12308 12309 12329 12330\r\n [937] 12331 12343 12344 12345 12365 12366 12367 12391 12392 12393 12421 12422 12423 12434 12435 12436 12442 12443 12444 12456 12457 12458 12472 12473\r\n [961] 12474 12488 12489 12490 12513 12514 12515 12562 12563 12564 12575 12576 12577 12610 12611 12612 12685 12686 12687 12696 12697 12698 12700 12701\r\n [985] 12702 12703 12704 12705 12709 12710 12711 12712 12713 12714 12989 13037 13041 13045 13056 13100\r\n [ reached getOption(\"max.print\") -- omitted 1353060 entries ]\r\n```"}}
{"actor":{"display_login":"KyleDKavanagh"}}