{"action":"created","issue":{"url":"https://api.github.com/repos/JuliaLang/julia/issues/13412","labels_url":"https://api.github.com/repos/JuliaLang/julia/issues/13412/labels{/name}","comments_url":"https://api.github.com/repos/JuliaLang/julia/issues/13412/comments","events_url":"https://api.github.com/repos/JuliaLang/julia/issues/13412/events","html_url":"https://github.com/JuliaLang/julia/pull/13412","id":109391246,"number":13412,"title":"WIP: redesign closures, then generic functions","user":{"login":"JeffBezanson","id":744556,"avatar_url":"https://avatars.githubusercontent.com/u/744556?v=3","gravatar_id":"","url":"https://api.github.com/users/JeffBezanson","html_url":"https://github.com/JeffBezanson","followers_url":"https://api.github.com/users/JeffBezanson/followers","following_url":"https://api.github.com/users/JeffBezanson/following{/other_user}","gists_url":"https://api.github.com/users/JeffBezanson/gists{/gist_id}","starred_url":"https://api.github.com/users/JeffBezanson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JeffBezanson/subscriptions","organizations_url":"https://api.github.com/users/JeffBezanson/orgs","repos_url":"https://api.github.com/users/JeffBezanson/repos","events_url":"https://api.github.com/users/JeffBezanson/events{/privacy}","received_events_url":"https://api.github.com/users/JeffBezanson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":105,"created_at":"2015-10-01T21:42:45Z","updated_at":"2016-01-26T20:53:15Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/JuliaLang/julia/pulls/13412","html_url":"https://github.com/JuliaLang/julia/pull/13412","diff_url":"https://github.com/JuliaLang/julia/pull/13412.diff","patch_url":"https://github.com/JuliaLang/julia/pull/13412.patch"},"body":"~~This is a very early preview of the likely future direction for how functions work. It's very much pre-alpha quality but does have a working REPL.~~ This is now essentially done! I have one cool demo:\r\n\r\n```\r\njulia> function foo()\r\n       a = rand(2000,2000);\r\n       @time 2*a\r\n       @time map(x->2x, a)\r\n       end\r\nfoo (generic function with 1 method)\r\n\r\njulia> foo();\r\n  0.011471 seconds (2 allocations: 30.518 MB, 44.34% gc time)\r\n  0.009771 seconds (2 allocations: 30.518 MB)\r\n\r\njulia> foo();\r\n  0.007800 seconds (2 allocations: 30.518 MB, 23.52% gc time)\r\n  0.006837 seconds (2 allocations: 30.518 MB)\r\n```\r\n\r\nOk, maybe two cool demos:\r\n\r\n```\r\njulia> function inner(a)\r\n        x->a*x\r\n       end\r\ninner (generic function with 1 method)\r\n\r\njulia> f = inner(6)\r\n#2#g0(6)\r\n\r\njulia> f(2)\r\n12\r\n\r\njulia> f.a\r\n6\r\n```\r\n\r\nFastAnonymous.jl, you are in my sights and your days are numbered.\r\n\r\nMy full plan has three stages, the first of which is now almost complete:\r\n\r\n1. Rewrite closures (inner functions) to new types that define `call`.\r\n2. Deprecate the `call` function, and instead put method tables inside TypeName. Remove the `Function` concrete type. All generic functions use the same basic design as closures in stage 1.\r\n3. Fix various problems and redesign parts of Base to exploit this change.\r\n\r\nSome next steps:\r\n\r\n- [x] Improve typing of closure fields\r\n- [x] Remove some unnecessary `Box`es\r\n- [x] Propagate static parameters to closure types\r\n- [x] `jl_instantiate_staged` is probably a bit buggy\r\n\r\nSome problems to solve:\r\n\r\n1. The situation with some kind of `Callable` type is now more urgent. I'd propose making `Function` an abstract type, and making anonymous functions subtypes of it by default. That will at least handle all the cases in Base where function arguments had `::Function`, and will also handle \"functors\" that had nothing to subtype from before.\r\n1. How to serialize closures? If all code is present on all nodes, then sending closures is actually way more efficient than ever, since you only need to send the (small) environment object. However we need to decide when to send the whole type definition as well.\r\n1. Decide how to deal with inner functions with multiple definitions. Currently they work fine assuming (1) all definitions are in the same basic block, (2) the type signatures can be evaluated at the top level. These are reasonable restrictions, but we need to solidify them and add good error messages etc.\r\n1. This uses way too many gensyms, and the generated names are *horrible*, as in the type of `f` above, `#2#g0`. There should be a nice standard name mangling, e.g. `__hidden__.inner_1`.\r\n1. Figure out how to pass static parameters for `--compile=all` mode. This currently depends on the old implementation of closures.\r\n1. Figure out how (if at all) to mitigate the additional excess code specialization. So far it's actually not too bad; the system image is maybe 10% bigger and the build takes a bit longer, but it's nothing horrible. However after stage 2 this will probably get worse.\r\n"},"comment":{"url":"https://api.github.com/repos/JuliaLang/julia/issues/comments/175224594","html_url":"https://github.com/JuliaLang/julia/pull/13412#issuecomment-175224594","issue_url":"https://api.github.com/repos/JuliaLang/julia/issues/13412","id":175224594,"user":{"login":"vtjnash","id":330950,"avatar_url":"https://avatars.githubusercontent.com/u/330950?v=3","gravatar_id":"","url":"https://api.github.com/users/vtjnash","html_url":"https://github.com/vtjnash","followers_url":"https://api.github.com/users/vtjnash/followers","following_url":"https://api.github.com/users/vtjnash/following{/other_user}","gists_url":"https://api.github.com/users/vtjnash/gists{/gist_id}","starred_url":"https://api.github.com/users/vtjnash/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vtjnash/subscriptions","organizations_url":"https://api.github.com/users/vtjnash/orgs","repos_url":"https://api.github.com/users/vtjnash/repos","events_url":"https://api.github.com/users/vtjnash/events{/privacy}","received_events_url":"https://api.github.com/users/vtjnash/received_events","type":"User","site_admin":false},"created_at":"2016-01-26T20:53:15Z","updated_at":"2016-01-26T20:53:15Z","body":"> Sundials: Passing Ref{Function} to cfunction doesn't work since Function is an abstract type. @vtjnash any ideas? One possibility is to wrap the callback in a concrete OpaqueFunction type, so you have a single concrete type to work with.\r\n\r\nas long as the method cache can returning the correct method for the signature, the cfunction code should handle this without an issue – the error is being overly cautious because it assumes that (a) `--compile=all` might be broken (so non-leaf-type method might compile incorrectly) and (b) the method cache signature might not be unique (the `linfo->specTypes` tuple might be more general than `argt`).\r\n\r\nthis is a problem for bits types, but not for `Ref`.\r\n\r\nfor example;\r\n```julia\r\nfoo(x::Integer) = typeof(x)\r\n\r\n# the following can be handled only if the linfo returned from the method cache is specialized for the right size int:\r\ncfunction(x, DataType, (Int64,)) # 1\r\ncfunction(x, DataType, (Int32,)) # 2\r\n# if the method cache were to return linfo->unspecialized (or equivalent), then cfunction would be unable to generate different code for the two type signatures (`jl_value_t* (*)(int64_t)` and `jl_value_t* (*)(int32_t)`) because cfunction does not maintain its own method type cache\r\n\r\n# these all get boxed types, so the compile-time type is relatively unimportant:\r\ncfunction(x, DataType, (Ref{Int64},)) # 3\r\ncfunction(x, DataType, (Ref{Int32},)) # 4\r\ncfunction(x, DataType, (Ref{Union{Int32, Int64}},)) # 5\r\ncfunction(x, DataType, (Ref{Integer},)) # 6\r\n\r\n# if we add another definition, however:\r\nfoo(x::Int16) = typeof(unsigned(x))\r\n# the method match for `Ref{Integer}` is now non-unique, so cfunction can't deal with case 6 anymore, but 3/4/5 are still fine\r\n```\r\n\r\ncase 6 can be fixed by having the cfunction wrapper call `jl_apply` after boxing all of the arguments. that would also work for the case where there are sparams missing that need to be filled in by the runtime."}}