{"action":"created","issue":{"url":"https://api.github.com/repos/idris-lang/Idris-dev/issues/2628","repository_url":"https://api.github.com/repos/idris-lang/Idris-dev","labels_url":"https://api.github.com/repos/idris-lang/Idris-dev/issues/2628/labels{/name}","comments_url":"https://api.github.com/repos/idris-lang/Idris-dev/issues/2628/comments","events_url":"https://api.github.com/repos/idris-lang/Idris-dev/issues/2628/events","html_url":"https://github.com/idris-lang/Idris-dev/pull/2628","id":107030888,"number":2628,"title":"WIP: sticking string length in the RTS","user":{"login":"david-christiansen","id":115330,"avatar_url":"https://avatars.githubusercontent.com/u/115330?v=3","gravatar_id":"","url":"https://api.github.com/users/david-christiansen","html_url":"https://github.com/david-christiansen","followers_url":"https://api.github.com/users/david-christiansen/followers","following_url":"https://api.github.com/users/david-christiansen/following{/other_user}","gists_url":"https://api.github.com/users/david-christiansen/gists{/gist_id}","starred_url":"https://api.github.com/users/david-christiansen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/david-christiansen/subscriptions","organizations_url":"https://api.github.com/users/david-christiansen/orgs","repos_url":"https://api.github.com/users/david-christiansen/repos","events_url":"https://api.github.com/users/david-christiansen/events{/privacy}","received_events_url":"https://api.github.com/users/david-christiansen/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/idris-lang/Idris-dev/labels/A-In%20Progress","name":"A-In Progress","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":14,"created_at":"2015-09-17T17:13:14Z","updated_at":"2016-08-20T09:57:51Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/idris-lang/Idris-dev/pulls/2628","html_url":"https://github.com/idris-lang/Idris-dev/pull/2628","diff_url":"https://github.com/idris-lang/Idris-dev/pull/2628.diff","patch_url":"https://github.com/idris-lang/Idris-dev/pull/2628.patch"},"body":"This is a WIP pull request so that the skilled C hackers can get critique in early. It's not done yet!\r\n\r\nThe idea is that an explicit length field means we're not dependent on null-termination. This has, in my view, two benefits:\r\n 1. Less chance of accidental buffer overruns from malformed UTF-8 strings\r\n 2. It becomes possible to use \\0 inside of strings, giving the type `String` the semantics \"sequence of code points\" rather than \"sequence of nonzero code points\" which is at least elegant and possibly even useful.\r\n\r\nA null terminator is retained to avoid marshalling issues or excess copying when calling C functions.\r\n\r\nThis pull request does the following:\r\n - [X] Add a length field to the runtime representation of strings, tracking their size in bytes\r\n - [X] Refactor string allocations to do all object initialization in one place, to help avoid mistakes\r\n - [X] Modify the UTF-8 functions to not access memory outside the size size of the string\r\n - [X] Crash with informative messages on partial string operations (like `strTail \"\"`)\r\n - [X] Validate the output of primitive conversions, like `prim__floatToStr` to make sure that library code is returning what we expect\r\n - [ ] Stop using C string functions that assume null-termination (partially complete)\r\n - [ ] Check for `NULL` return from all `malloc` calls (partially complete)\r\n - [X] Pass the test suite\r\n - [ ] Validate that strings from calls to C code are actually real UTF-8 (I may punt on this or implement a poor version of it until we get a proper Unicode library integrated)\r\n - [X] Validate that purported characters (represented as `Int` under the hood) are actually in the Unicode range before consing onto a string\r\n - [X] Add `assert`s about the byte size invariants\r\n - [ ] Make the C codegen actually allow `'\\0'` in the middle of Idris strings, and display them appropriately using the print functions in the library\r\n - [ ] Factor out checks for `NULL` after `malloc` into a function\r\n\r\nPotential things to do:\r\n 1. Also cache the length in characters (would make some things more elegant, but adds overhead and complexity and may not be worth it)\r\n 2. More thorough Unicode testing"},"comment":{"url":"https://api.github.com/repos/idris-lang/Idris-dev/issues/comments/241190796","html_url":"https://github.com/idris-lang/Idris-dev/pull/2628#issuecomment-241190796","issue_url":"https://api.github.com/repos/idris-lang/Idris-dev/issues/2628","id":241190796,"user":{"login":"ahmadsalim","id":1647846,"avatar_url":"https://avatars.githubusercontent.com/u/1647846?v=3","gravatar_id":"","url":"https://api.github.com/users/ahmadsalim","html_url":"https://github.com/ahmadsalim","followers_url":"https://api.github.com/users/ahmadsalim/followers","following_url":"https://api.github.com/users/ahmadsalim/following{/other_user}","gists_url":"https://api.github.com/users/ahmadsalim/gists{/gist_id}","starred_url":"https://api.github.com/users/ahmadsalim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ahmadsalim/subscriptions","organizations_url":"https://api.github.com/users/ahmadsalim/orgs","repos_url":"https://api.github.com/users/ahmadsalim/repos","events_url":"https://api.github.com/users/ahmadsalim/events{/privacy}","received_events_url":"https://api.github.com/users/ahmadsalim/received_events","type":"User","site_admin":false},"created_at":"2016-08-20T09:57:51Z","updated_at":"2016-08-20T09:57:51Z","body":"@ericqweinstein Did you ever take a look at this issue?"}}
{"actor":{"display_login":"ahmadsalim"}}