{"action":"created","issue":{"url":"https://api.github.com/repos/JuliaLang/julia/issues/28878","repository_url":"https://api.github.com/repos/JuliaLang/julia","labels_url":"https://api.github.com/repos/JuliaLang/julia/issues/28878/labels{/name}","comments_url":"https://api.github.com/repos/JuliaLang/julia/issues/28878/comments","events_url":"https://api.github.com/repos/JuliaLang/julia/issues/28878/events","html_url":"https://github.com/JuliaLang/julia/pull/28878","id":353855811,"node_id":"MDExOlB1bGxSZXF1ZXN0MjEwNzkzNzM0","number":28878,"title":"Exception stacks","user":{"login":"c42f","id":601473,"node_id":"MDQ6VXNlcjYwMTQ3Mw==","avatar_url":"https://avatars1.githubusercontent.com/u/601473?v=4","gravatar_id":"","url":"https://api.github.com/users/c42f","html_url":"https://github.com/c42f","followers_url":"https://api.github.com/users/c42f/followers","following_url":"https://api.github.com/users/c42f/following{/other_user}","gists_url":"https://api.github.com/users/c42f/gists{/gist_id}","starred_url":"https://api.github.com/users/c42f/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/c42f/subscriptions","organizations_url":"https://api.github.com/users/c42f/orgs","repos_url":"https://api.github.com/users/c42f/repos","events_url":"https://api.github.com/users/c42f/events{/privacy}","received_events_url":"https://api.github.com/users/c42f/received_events","type":"User","site_admin":false},"labels":[{"id":119746705,"node_id":"MDU6TGFiZWwxMTk3NDY3MDU=","url":"https://api.github.com/repos/JuliaLang/julia/labels/error%20handling","name":"error handling","color":"02d7e1","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-08-24T16:49:45Z","updated_at":"2018-08-27T23:10:35Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/JuliaLang/julia/pulls/28878","html_url":"https://github.com/JuliaLang/julia/pull/28878","diff_url":"https://github.com/JuliaLang/julia/pull/28878.diff","patch_url":"https://github.com/JuliaLang/julia/pull/28878.patch"},"body":"# An exception stack system for julia\r\n\r\nHere's my attempt to fix #12485 / #19979 by introducing an exception stack\r\nsystem. The basic problem is that you may throw a new exception while handling\r\nan exception.  What do we do then? The existing runtime simply overwrites the\r\nfirst exception, loosing any information it might have had about the root\r\ncause.\r\n\r\nA distinct problem discussed in #12485 is that task switching looses all\r\nexception and backtrace information. I've tackled this at the same time\r\nthough in hindsight it might have been better to do the exception stack\r\nfirst, followed by fixing task switching.\r\n\r\n## Design\r\n\r\nTo solve the first problem, I've implemented an exception stack mechanism.\r\nConceptually:\r\n\r\n* Every `throw()` pushes an `(exception,backtrace)` pair onto a stack of current\r\n  exceptions.\r\n* When you exit a `catch` block normally, any exception generated within the\r\n  corresponding `try` is considered to have been handled. The stack is therefore\r\n  popped back to the state at the entry of the corresponding `try`.\r\n* The full stack is available for root cause analysis via `Base.catch_stack()`.\r\n\r\nFor example:\r\n\r\n```julia\r\ntry\r\n    error(\"A\")\r\ncatch\r\n    try\r\n        error(\"B\")\r\n    catch\r\n        for (e, bt) in Base.catch_stack()\r\n            showerror(stdout, e, bt)\r\n            println()\r\n        end\r\n    end\r\nend\r\n```\r\n\r\nProduces\r\n\r\n```\r\nA\r\nStacktrace:\r\n [1] error(::String) at ./error.jl:33\r\n [2] top-level scope at ./REPL[3]:2\r\n [3] eval(::Module, ::Any) at ./boot.jl:319\r\n [4] eval_user_input(::Any, ::REPL.REPLBackend) at /home/chris/dev/julia/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:85\r\n [5] macro expansion at /home/chris/dev/julia/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:117 [inlined]\r\n [6] (::getfield(REPL, Symbol(\"##28#29\")){REPL.REPLBackend})() at ./task.jl:259\r\nB\r\nStacktrace:\r\n [1] error(::String) at ./error.jl:33\r\n [2] top-level scope at ./REPL[3]:5\r\n [3] eval(::Module, ::Any) at ./boot.jl:319\r\n [4] eval_user_input(::Any, ::REPL.REPLBackend) at /home/chris/dev/julia/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:85\r\n [5] macro expansion at /home/chris/dev/julia/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:117 [inlined]\r\n [6] (::getfield(REPL, Symbol(\"##28#29\")){REPL.REPLBackend})() at ./task.jl:259\r\n```\r\n\r\n## Implementation\r\n\r\n#### Lowering\r\n\r\nA new lowered expression head `:pop_exc` is introduced and emitted in any\r\nlocation where a catch block exits normally (either by falling out the bottom,\r\nor using return, break, goto). This pops the exception stack back to the state\r\nof the associated `enter`. There needs to be some way to track the associated\r\n`enter` in the IR, so I chose to emit an SSAValue token from `enter` which is\r\nconsumed by `pop_exc`. I tried various alternatives, but this was by far the\r\nnicest in terms of non-disruptive integration into the SSAIR processing code,\r\nand supporting both the interpreter and codegen.\r\n\r\n#### Runtime\r\n\r\n* The exception in transit is no longer a single pointer in the thread local\r\n  storage. It's stored in `ptls->exc_stack` instead, along with associated\r\n  backtrace.\r\n\r\n* The exception stack is copied out onto the `Task` during task switching.\r\n  I've been back and forth on several variations here, as the exact detail\r\n  of how this is done depends on how the memory for the exception\r\n  stack is allocated (GC vs plain malloc).\r\n\r\n* `jl_current_exception()` is now the method to retreive the current exception\r\n  from within a `JL_CATCH` dynamic context.\r\n\r\n* `_resetstkoflw()` handling is consolidated into `jl_eh_restore_state`.\r\n  Needs proper testing on windows.\r\n\r\n* Several places where manual restoration of the exception and backtrace was\r\n  done are no longer necessary because the stack system handles these\r\n  automatically.\r\n\r\n* An exception stack type, `jl_exc_stack_t` and associated manipulation\r\n  functions has been added. Conceptually stores a stack of\r\n  (exception,backtrace) pairs. It's stored in a shared buffer so we can avoid\r\n  reallocating when throwing and catching the same exception or set of\r\n  exceptions repeatedly.\r\n\r\n* `jl_eh_restore_state` has become non-inline. It seemed good to get this out\r\n  of the public header.\r\n\r\n* `jl_sig_throw` is now used with `jl_throw_in_ctx` from signal handlers, to\r\n  make the special circumstances clear and to avoid conflation with rethrow\r\n  which now simply rethrows the existing exception stack.\r\n\r\n#### Interpreter / Codegen\r\n\r\nSome minor rearrangement in the interpreter to make the eh buffer management\r\nand `leave` handling a little easier to understand as I found this surprisingly\r\nconfusing.\r\n\r\nMostly small changes in codegen. Representing the enter token as an SSAValue\r\nreally integrates well here when you need to retreive it for the associated\r\n`pop_exc`.\r\n\r\n\r\n#### GC\r\n\r\nWhat? How did this rabbit hole lead down into the GC?\r\n\r\nUnfortunately the raw backtrace buffers contain references to GC allocated\r\nobjects. Since these buffers can now live on a `Task`, their contents need to\r\nbe GC rooted. Unfortunately the raw buffers are also in a non-julia native\r\nformat, so I had to write specialized GC code for them.\r\n\r\nI'm not sure this was a good idea, especially having extra code to maintain in\r\nthis rather sensitive part of the codebase. Perhaps it would be better to\r\nmodify our raw backtrace buffer format so that it can be understood by the GC\r\nwithout major changes.\r\n\r\n\r\n## TODO\r\n\r\n* [ ] Check some potential GC issues I found in testing.\r\n* [ ] Possibly export `Base.catch_stack()` / bikeshed API\r\n* [ ] Manually test consolidated `_resetstkoflw` handling?\r\n* [ ] Test exception preservation with task switching more carefully\r\n\r\n#### Optional\r\n\r\n* [ ] `jl_apply_with_saved_exception_state` may be unnecessary now\r\n* [ ] `jl_rethrow_other` is probably unnecessary now\r\n* [ ] Persist backtrace when task exits with an exception\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/JuliaLang/julia/issues/comments/416398086","html_url":"https://github.com/JuliaLang/julia/pull/28878#issuecomment-416398086","issue_url":"https://api.github.com/repos/JuliaLang/julia/issues/28878","id":416398086,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNjM5ODA4Ng==","user":{"login":"c42f","id":601473,"node_id":"MDQ6VXNlcjYwMTQ3Mw==","avatar_url":"https://avatars1.githubusercontent.com/u/601473?v=4","gravatar_id":"","url":"https://api.github.com/users/c42f","html_url":"https://github.com/c42f","followers_url":"https://api.github.com/users/c42f/followers","following_url":"https://api.github.com/users/c42f/following{/other_user}","gists_url":"https://api.github.com/users/c42f/gists{/gist_id}","starred_url":"https://api.github.com/users/c42f/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/c42f/subscriptions","organizations_url":"https://api.github.com/users/c42f/orgs","repos_url":"https://api.github.com/users/c42f/repos","events_url":"https://api.github.com/users/c42f/events{/privacy}","received_events_url":"https://api.github.com/users/c42f/received_events","type":"User","site_admin":false},"created_at":"2018-08-27T23:10:35Z","updated_at":"2018-08-27T23:10:35Z","author_association":"CONTRIBUTOR","body":"Looking into alternative storage for backtrace buffers, the `Union` isn't going to work because we really need to store pointers in there. So perhaps we're stuck with two parallel arrays instead like the current `bt`, `bt2` which appears in some places. Or just put up with having some special purpose code in the gc."}}
{"actor":{"display_login":"c42f"}}