{"action":"created","issue":{"url":"https://api.github.com/repos/nim-lang/Nim/issues/8363","repository_url":"https://api.github.com/repos/nim-lang/Nim","labels_url":"https://api.github.com/repos/nim-lang/Nim/issues/8363/labels{/name}","comments_url":"https://api.github.com/repos/nim-lang/Nim/issues/8363/comments","events_url":"https://api.github.com/repos/nim-lang/Nim/issues/8363/events","html_url":"https://github.com/nim-lang/Nim/issues/8363","id":342523667,"node_id":"MDU6SXNzdWUzNDI1MjM2Njc=","number":8363,"title":"Rework Nim's exception handling","user":{"login":"Araq","id":287816,"node_id":"MDQ6VXNlcjI4NzgxNg==","avatar_url":"https://avatars1.githubusercontent.com/u/287816?v=4","gravatar_id":"","url":"https://api.github.com/users/Araq","html_url":"https://github.com/Araq","followers_url":"https://api.github.com/users/Araq/followers","following_url":"https://api.github.com/users/Araq/following{/other_user}","gists_url":"https://api.github.com/users/Araq/gists{/gist_id}","starred_url":"https://api.github.com/users/Araq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Araq/subscriptions","organizations_url":"https://api.github.com/users/Araq/orgs","repos_url":"https://api.github.com/users/Araq/repos","events_url":"https://api.github.com/users/Araq/events{/privacy}","received_events_url":"https://api.github.com/users/Araq/received_events","type":"User","site_admin":false},"labels":[{"id":32256242,"node_id":"MDU6TGFiZWwzMjI1NjI0Mg==","url":"https://api.github.com/repos/nim-lang/Nim/labels/RFC","name":"RFC","color":"1d1d1d","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-07-18T23:14:17Z","updated_at":"2018-07-19T08:17:38Z","closed_at":null,"author_association":"MEMBER","body":"Nim's exception handling is currently tied to Nim's garbage collector and\r\nevery raised exception triggers an allocation. For embedded systems this is\r\nnot optimal, see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r1.pdf\r\nfor more details.\r\n\r\nSo, here is what I propose: Clearly distinguish between \"bugs\" and \"errors\". Bugs\r\nare not mapped to exceptions.\r\n\r\nThe builtin \"bugs\" are mapped to the new ``system.FatalError`` enum,\r\nruntime errors are mapped to ``system.Error``. FatalErrors are usually not\r\nrecovered from, unless a \"supervisor\" is used. (See the section about them.)\r\n\r\n```nim\r\n\r\ntype\r\n  FatalError* = enum         ## Programming bugs and other error\r\n                             ## conditions most programs cannot\r\n                             ## realistically recover from.\r\n                             ## However, a \"supervisor\" can be\r\n                             ## registered that does allow to\r\n                             ## recover from them.\r\n    IndexError,              ## index out of bounds\r\n    FieldError,              ## invalid object field accessed\r\n    RangeError,              ## value out of its valid range\r\n    ReraiseError,            ## nothing to re-raise\r\n\r\n    ObjectAssignmentError,   ## object assignment would lose data (object slicing problem)\r\n    ObjectConversionError,   ## object is not a subtype of the given type\r\n    MoveError,               ## object cannot be accessed as it was moved\r\n    DivByZeroError,          ## divison by zero\r\n\r\n    OverflowError,           ## integer arithmetic under- or overflow\r\n    AccessViolationError,    ## segfault\r\n    NilAccessError,          ## attempt to deref a 'nil' value\r\n    AssertionError,          ## assertion failed. Is used to indicate wrong API usage\r\n    DeadThreadError,         ## the thread cannot be accessed as its dead.\r\n    LibraryError,            ## a DLL could not be loaded\r\n\r\n    OutOfMemError,           ## the system ran out of heap space\r\n    StackOverflowError,\r\n    FloatInvalidOpError,\r\n    FloatDivByZeroError,\r\n    FloatOverflowError,\r\n    FloatUnderflowError,\r\n    FloatInexactError\r\n\r\n  Error* = enum\r\n    NoError,\r\n    SyntaxError,\r\n    IOError, EOFError, OSError,\r\n\r\n    ResourceExhaustedError,\r\n    KeyError,\r\n    ValueError,\r\n    ProtocolError,\r\n    TimeoutError\r\n\r\nproc isBug*(e: FatalError): bool = e <= AssertionError\r\n\r\n```\r\n\r\nEvery error that can be caught is of the type ``Error``. The effect system\r\ntracks if a proc can potentially ``raise`` such an error. This means that\r\nan \"empty\" ``except`` can be used to conveniently check for any meaningful exception,\r\ncomparable in convenience to an ``if`` statement/expression:\r\n\r\n```nim\r\n\r\nlet x = try: parseJson(input) except: quit(getCurrentExceptionMsg())\r\n\r\n```\r\n\r\nThe runtime calls ``system.panic`` on a detected bug, which roughly\r\nhas this implementation:\r\n\r\n```nim\r\n\r\nvar panicHook*: proc (e: FatalError; msg: cstring) {.nimcall.}\r\n\r\nproc panic(e: FatalError; msg: cstring) {.noreturn.} =\r\n  if panicHook != nil:\r\n    panicHook(e, msg)\r\n  else:\r\n    if msg != nil: stdout.write msg\r\n    quit(ord(e)+1)\r\n\r\n```\r\n\r\n# Out of memory\r\n\r\nOut of memory is a fatal error.\r\n\r\n**Rationale**: \"Stack unwinding with destructors invocations that free memory\" is too much gambling, if it works, that's mostly by chance as in more complex programs it becomes intractable.\r\n\r\nThe systems that are designed to continue gracefully in case of an OOM situation should make use of the proposed \"supervisor\" feature. The systems that are proven to work with a fixed amount of heap space should allocate this space upfront (the allocator already has some support for this) and then an OOM situation is a fatal, unrecoverable error too.\r\n\r\n\r\n# Implementation of 'try'\r\n\r\nThere are different implementation strategies possible for these light weight\r\nexceptions:\r\n\r\n1. C ``setjmp``. Since this is pretty expensive, this will not be used.\r\n2. C++'s ``try`` statement. Will be used for the C++ target unless the C++ dialect does not support exceptions. This is likely to offer the best performance.\r\n3. Map ``Error`` to an additional return value and propagate it explicitly in the generated code via ``if (retError) goto error;``\r\n4. Map ``Error`` to a hidden pointer parameter so and propagate it explicitly in the generated code via ``if (*errorParam) goto error;``. This is probably slower than using a return value, but this requires benchmarking.\r\n\r\n\r\n# The supervisor\r\n\r\nFatal errors cannot be caught by ``try``. For servers in particular or for embedded devices that lack an OS a \"supervisor\" can be used to make a subsystem shut down gracefully without shutting down the complete process. The supervisor feature requires extensive stdlib support; every ``open`` call needs to register the file descriptor to a threadlocal list so that they can be closed even in the case of a fatal error which does *not* run the destructors if any implementation strategy but (2) is used. Ideally also the memory allocator supports \"safe points\", so every allocation that was done by the failed subsystem can be rolled back. For the usual segregated free-list allocators this seems feasible to provide.\r\n\r\nThe supervisor sets the ``panicHook`` to a proc that does the ``raise`` or ``longjmp`` operation. The supervisor runs on the same thread as the supervised subsystem.\r\n\r\n**Rationale**: Easier to implement, stack corruptions are super rare even in low level Nim code, embedded systems might not support threads. Erlang's supervisors conflate \"recover from this subsystem\" with a concurrency mechanism. This seems to be unnecessary.\r\n\r\nA supervisor will be part of the stdlib, but one also can write his own. A possible implementation looks like:\r\n\r\n```nim\r\n\r\ntemplate shield(body: untyped) =\r\n  let oldPanicHook = panicHook\r\n  var target {.global.}: JmpBuf\r\n  panicHook = proc (e: FatalError; msg: cstring) =\r\n    log(e, msg)\r\n    longjmp(target)\r\n  let oldHeap = heapSnapshot()\r\n  var failed = false\r\n  if setjmp(target) == 0:\r\n    try:\r\n      body\r\n    except:\r\n      failed = true\r\n  else:\r\n    failed = true\r\n  if failed:\r\n    heapRollback(oldHeap)\r\n    closeFileDescs()\r\n  panicHook = oldPanicHook\r\n\r\n# in a typical web server, that must have some resistance against\r\n# programming bugs, this would be used like so:\r\nshield:\r\n  handleWebRequest()\r\n\r\n```\r\n\r\nHow well the heap can be rolled back and external resources can be freed is a quality of implementation issue, even in the best case, created files would not be removed on failure. Perfect isolation is not even provided by Erlang's supervisors, in fact, every write to a disk is not rolled back! The supervisor feature is not a substitute for virtualization.\r\n"},"comment":{"url":"https://api.github.com/repos/nim-lang/Nim/issues/comments/406194506","html_url":"https://github.com/nim-lang/Nim/issues/8363#issuecomment-406194506","issue_url":"https://api.github.com/repos/nim-lang/Nim/issues/8363","id":406194506,"node_id":"MDEyOklzc3VlQ29tbWVudDQwNjE5NDUwNg==","user":{"login":"andreaferretti","id":1962818,"node_id":"MDQ6VXNlcjE5NjI4MTg=","avatar_url":"https://avatars1.githubusercontent.com/u/1962818?v=4","gravatar_id":"","url":"https://api.github.com/users/andreaferretti","html_url":"https://github.com/andreaferretti","followers_url":"https://api.github.com/users/andreaferretti/followers","following_url":"https://api.github.com/users/andreaferretti/following{/other_user}","gists_url":"https://api.github.com/users/andreaferretti/gists{/gist_id}","starred_url":"https://api.github.com/users/andreaferretti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andreaferretti/subscriptions","organizations_url":"https://api.github.com/users/andreaferretti/orgs","repos_url":"https://api.github.com/users/andreaferretti/repos","events_url":"https://api.github.com/users/andreaferretti/events{/privacy}","received_events_url":"https://api.github.com/users/andreaferretti/received_events","type":"User","site_admin":false},"created_at":"2018-07-19T08:17:38Z","updated_at":"2018-07-19T08:17:38Z","author_association":"COLLABORATOR","body":"1. Using thread local variables is messy - like global variables you have the issue of choosing unique names, making sure you are not writing over something already used. This is a completely unstructured way of communicating errors. That the compiler does this under the hood is of little relevance - the compiler is there to help and translate a mantainable pattern into whatever is needed to make the thing work on actual hardware. The problem is when a human has to manually do this.\r\n\r\n2. All the things I listed may genuinely be not bugs. I made an example with `FloatOverflowError`, but an example can be made for all of these cases.\r\n\r\nMaybe I have a data structure like an array with holes. You can ask to get `myDataStructure[i]` for an integer `i`, but - unlike normal arrays - there is no simple rule to figure out whether index `i` is accessible, such as `0 <= i and i < len(myDataStrcuture)`. So if you are not sure you can do\r\n\r\n```nim\r\nlet x = try:\r\n  myDataStructure[i]\r\nexcept IndexError:\r\n  \"no value\"\r\n```\r\n\r\nNot to mention multithreading and the shared heap. For data stored on the thread-local heap, one can do\r\n\r\n```nim\r\nif 0 <= i and i < len(myArray):\r\n  x = myArray[i]\r\n```\r\n\r\nIn a multithreaded setting, this is a race condition, and the only safe way to do this may be (assume `myArray` is some kind of data structure stored on the shared heap but is not thread safe)\r\n\r\n```nim\r\nlet x = try:\r\n  myArray[i]\r\nexcept IndexError, MoveError:\r\n  \"no value\"\r\n```\r\n\r\nAll the cases I have listed can be used for actual runtime errors that **are not** programming bugs.\r\n\r\nMoreover, "}}
{"actor":{"display_login":"andreaferretti"}}