{"action":"opened","issue":{"url":"https://api.github.com/repos/JuliaLang/julia/issues/27809","repository_url":"https://api.github.com/repos/JuliaLang/julia","labels_url":"https://api.github.com/repos/JuliaLang/julia/issues/27809/labels{/name}","comments_url":"https://api.github.com/repos/JuliaLang/julia/issues/27809/comments","events_url":"https://api.github.com/repos/JuliaLang/julia/issues/27809/events","html_url":"https://github.com/JuliaLang/julia/issues/27809","id":335979861,"node_id":"MDU6SXNzdWUzMzU5Nzk4NjE=","number":27809,"title":"trouble with filter(!ismissing, x)","user":{"login":"GunnarFarneback","id":1903707,"node_id":"MDQ6VXNlcjE5MDM3MDc=","avatar_url":"https://avatars0.githubusercontent.com/u/1903707?v=4","gravatar_id":"","url":"https://api.github.com/users/GunnarFarneback","html_url":"https://github.com/GunnarFarneback","followers_url":"https://api.github.com/users/GunnarFarneback/followers","following_url":"https://api.github.com/users/GunnarFarneback/following{/other_user}","gists_url":"https://api.github.com/users/GunnarFarneback/gists{/gist_id}","starred_url":"https://api.github.com/users/GunnarFarneback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GunnarFarneback/subscriptions","organizations_url":"https://api.github.com/users/GunnarFarneback/orgs","repos_url":"https://api.github.com/users/GunnarFarneback/repos","events_url":"https://api.github.com/users/GunnarFarneback/events{/privacy}","received_events_url":"https://api.github.com/users/GunnarFarneback/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-06-26T20:33:09Z","updated_at":"2018-06-26T20:33:09Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Summary: `filter(!ismissing, x)` gives wrong results, is exceedingly slow, and occasionally leads to crashes. I would suspect code generation goes wrong.\r\n\r\nAs an accidental spinoff from #27781 I ran into this, which gives unexpected results and slowness.\r\n```\r\n   _       _ _(_)_     |  A fresh approach to technical computing\r\n  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org\r\n   _ _   _| |_  __ _   |  Type \"?\" for help, \"]?\" for Pkg help.\r\n  | | | | | | |/ _` |  |\r\n  | | |_| | | | (_| |  |  Version 0.7.0-beta.36 (2018-06-26 19:43 UTC)\r\n _/ |\\__'_|_|_|\\__'_|  |  Commit afbd699 (0 days old master)\r\n|__/                   |  x86_64-linux-gnu\r\n\r\njulia> y = rand(Float64, 1_000_000); y2 = ifelse.(rand(length(y)) .< 0.9, y, missing);\r\n\r\njulia> a = collect(skipmissing(y2));\r\n\r\njulia> b = filter(!ismissing, y2);\r\n\r\njulia> a == b\r\nmissing\r\n\r\njulia> sum(skipmissing(a))\r\n450111.2603673207\r\n\r\njulia> sum(skipmissing(b))\r\n450101.5954636309\r\n\r\njulia> @time collect(skipmissing(y2));\r\n  0.015578 seconds (25 allocations: 9.001 MiB)\r\n\r\njulia> @time filter(!ismissing, y2);\r\n  7.770327 seconds (24 allocations: 10.126 MiB, 0.02% gc time)\r\n\r\njulia> versioninfo()\r\nJulia Version 0.7.0-beta.36\r\nCommit afbd699 (2018-06-26 19:43 UTC)\r\nPlatform Info:\r\n  OS: Linux (x86_64-linux-gnu)\r\n  CPU: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz\r\n  WORD_SIZE: 64\r\n  LIBM: libopenlibm\r\n  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)\r\n```\r\nIncreasing the size from one million to ten million elements increases the running time from 8 seconds to 2700 seconds. Moreover I've had Julia crashing on me twice while experimenting with this, including this complete session:\r\n```\r\n               _\r\n   _       _ _(_)_     |  A fresh approach to technical computing\r\n  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org\r\n   _ _   _| |_  __ _   |  Type \"?\" for help, \"]?\" for Pkg help.\r\n  | | | | | | |/ _` |  |\r\n  | | |_| | | | (_| |  |  Version 0.7.0-beta.36 (2018-06-26 19:43 UTC)\r\n _/ |\\__'_|_|_|\\__'_|  |  Commit afbd699 (0 days old master)\r\n|__/                   |  x86_64-linux-gnu\r\n\r\njulia> y = rand(Float64, 1_000_000); y2 = ifelse.(rand(length(y)) .< 0.9, y, missing);\r\n\r\njulia> \r\n\r\njulia> a = collect(skipmissing(y2));\r\n\r\njulia> b = filter(!ismissing, y2);\r\n\r\njulia> a == b\r\nUnreachable reached at 0x7f7bc37cff38\r\n\r\nsignal (4): Illegal instruction\r\nin expression starting at no file:0\r\niterate at ./array.jl:682 [inlined]\r\nzip_iterate at ./iterators.jl:303 [inlined]\r\niterate at ./iterators.jl:324 [inlined]\r\n== at ./abstractarray.jl:1679\r\nunknown function (ip: 0x7f7bc37d000c)\r\njl_fptr_trampoline at /home/gunnar/julia0.7/src/gf.c:1813\r\njl_apply_generic at /home/gunnar/julia0.7/src/gf.c:2151\r\ndo_call at /home/gunnar/julia0.7/src/interpreter.c:324\r\neval_value at /home/gunnar/julia0.7/src/interpreter.c:428\r\neval_stmt_value at /home/gunnar/julia0.7/src/interpreter.c:363 [inlined]\r\neval_body at /home/gunnar/julia0.7/src/interpreter.c:671\r\njl_interpret_toplevel_thunk_callback at /home/gunnar/julia0.7/src/interpreter.c:788\r\nunknown function (ip: 0xfffffffffffffffe)\r\nunknown function (ip: 0x7f7bc510f78f)\r\nunknown function (ip: 0xffffffffffffffff)\r\njl_interpret_toplevel_thunk at /home/gunnar/julia0.7/src/interpreter.c:797\r\njl_toplevel_eval_flex at /home/gunnar/julia0.7/src/toplevel.c:814\r\njl_toplevel_eval_in at /home/gunnar/julia0.7/src/builtins.c:631\r\neval at ./boot.jl:319\r\njl_apply_generic at /home/gunnar/julia0.7/src/gf.c:2151\r\neval_user_input at /home/gunnar/julia0.7/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:85\r\nmacro expansion at /home/gunnar/julia0.7/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:116 [inlined]\r\n#28 at ./task.jl:257\r\njl_fptr_trampoline at /home/gunnar/julia0.7/src/gf.c:1813\r\njl_apply_generic at /home/gunnar/julia0.7/src/gf.c:2151\r\njl_apply at /home/gunnar/julia0.7/src/julia.h:1533 [inlined]\r\nstart_task at /home/gunnar/julia0.7/src/task.c:268\r\nunknown function (ip: 0xffffffffffffffff)\r\nAllocations: 4333365 (Pool: 4332618; Big: 747); GC: 9\r\nIllegal instruction (core dumped)\r\n```"}}
{"actor":{"display_login":"GunnarFarneback"}}