{"action":"created","issue":{"url":"https://api.github.com/repos/randomness666/test/issues/28","repository_url":"https://api.github.com/repos/randomness666/test","labels_url":"https://api.github.com/repos/randomness666/test/issues/28/labels{/name}","comments_url":"https://api.github.com/repos/randomness666/test/issues/28/comments","events_url":"https://api.github.com/repos/randomness666/test/issues/28/events","html_url":"https://github.com/randomness666/test/issues/28","id":173018821,"number":28,"title":"new functions for multi-byte aware indexing and splicing","user":{"login":"randomness666","id":20402909,"avatar_url":"https://avatars.githubusercontent.com/u/20402909?v=3","gravatar_id":"","url":"https://api.github.com/users/randomness666","html_url":"https://github.com/randomness666","followers_url":"https://api.github.com/users/randomness666/followers","following_url":"https://api.github.com/users/randomness666/following{/other_user}","gists_url":"https://api.github.com/users/randomness666/gists{/gist_id}","starred_url":"https://api.github.com/users/randomness666/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/randomness666/subscriptions","organizations_url":"https://api.github.com/users/randomness666/orgs","repos_url":"https://api.github.com/users/randomness666/repos","events_url":"https://api.github.com/users/randomness666/events{/privacy}","received_events_url":"https://api.github.com/users/randomness666/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/randomness666/test/labels/encoding","name":"encoding","color":"ededed"},{"url":"https://api.github.com/repos/randomness666/test/labels/enhancement","name":"enhancement","color":"84b6eb"},{"url":"https://api.github.com/repos/randomness666/test/labels/viml","name":"viml","color":"ededed"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/randomness666/test/milestones/3","html_url":"https://github.com/randomness666/test/milestones/todo","labels_url":"https://api.github.com/repos/randomness666/test/milestones/3/labels","id":1961180,"number":3,"title":"todo","description":"\"todo\" items are *definitely not* planned for the *current* target, but *may* be given priority for a future target. Y'know, like todo.txt ;)","creator":{"login":"randomness666","id":20402909,"avatar_url":"https://avatars.githubusercontent.com/u/20402909?v=3","gravatar_id":"","url":"https://api.github.com/users/randomness666","html_url":"https://github.com/randomness666","followers_url":"https://api.github.com/users/randomness666/followers","following_url":"https://api.github.com/users/randomness666/following{/other_user}","gists_url":"https://api.github.com/users/randomness666/gists{/gist_id}","starred_url":"https://api.github.com/users/randomness666/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/randomness666/subscriptions","organizations_url":"https://api.github.com/users/randomness666/orgs","repos_url":"https://api.github.com/users/randomness666/repos","events_url":"https://api.github.com/users/randomness666/events{/privacy}","received_events_url":"https://api.github.com/users/randomness666/received_events","type":"User","site_admin":false},"open_issues":2,"closed_issues":0,"state":"open","created_at":"2016-08-24T17:53:29Z","updated_at":"2016-08-24T18:00:55Z","due_on":null,"closed_at":null},"comments":11,"created_at":"2016-08-24T18:00:55Z","updated_at":"2016-08-24T18:01:00Z","closed_at":null,"body":"<a href=\"https://github.com/OliverUv\"><img src=\"https://avatars.githubusercontent.com/u/374505?v=3\" align=\"left\" width=\"96\" height=\"96\" hspace=\"10\"></img></a> **Issue by [OliverUv](https://github.com/OliverUv)**\n_Sunday Feb 23, 2014 at 08:48 GMT_\n_Originally opened as https://github.com/neovim/neovim/issues/68_\n\n----\n\nfrogonwheels_ entered the irc with a question we should consider at some point. Apparently, multi-byte character handling is working unintuitively. Allegedly, it's an issue that Bram has been avoiding. Here is what he had to say (I have done some cleanup, summarized, clarified):\r\n\r\nIf you do `let idx=1 | while idx < strlen(txt) | let idx+=1 | let ch = txt[idx] | endwhile` then it's going through byte-wise rather than char wise. Essentially, Vim indexes lines per byte, not per character. This leads to complications when plugins try to get character n from the current line.\r\n\r\nAll the code that loops through characters in a string, or expects the column number to be the same as the index into the line (which happens all the time) will be broken as soon as you have multibyte characters.\r\n\r\n`col('.')` indexes the current column you are in (ie for display) and people do `getline('.')[col('.')]` to get the current character, but that doesn't work for multibyte. You'd have to do `matchstr(getline('.'), '^\\(\\zs.\\)\\{'.col('.').'}')` to get the correct character. Since regexp indexing is character-wise. This doesn't even take into account the fact that some multibyte chars go over 2 columns.\r\n\r\nCurrently the only safe way of indexing strings in a character-length independent way is to use `'.\\{'.n.'}`.\r\n\r\nThe Windows solution to this problem is to use ucs2-le character encoding - which will work until people start using multi-word characters (given that ucs2 is still multi-byte, just less likely to encounter it).\r\n\r\nSo either it's gotta be internally handled as wide characters (which is in itself limiting, but less so) which is the Windows approach.\r\n\r\nA problem arises if we don't have constant-size characters: If you define `txt[idx]` to be character-wise, then your `txt[idx]` ends up being an ord(n) operation and the loop becomes bad, so you have to have some kind of indexing or something.\r\n\r\ngrayb__ observes: There's nothing too expensive about keeping a codebase multibyte safe, the problem is all the code that assumes 7 or 8 bit ascii.\r\n\r\nI assume this is something we'll want to do far in the future, so it might be best to archive this issue under an appropriate label.\r\n\r\n<bountysource-plugin>\r\n\r\n---\r\nWant to back this issue? **[Place a bounty on it!](https://www.bountysource.com/issues/1428489-decide-on-how-to-handle-multi-byte-characters?utm_campaign=plugin&utm_content=tracker%2F461131&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F461131&utm_medium=issues&utm_source=github).\r\n</bountysource-plugin>\n"},"comment":{"url":"https://api.github.com/repos/randomness666/test/issues/comments/242155175","html_url":"https://github.com/randomness666/test/issues/28#issuecomment-242155175","issue_url":"https://api.github.com/repos/randomness666/test/issues/28","id":242155175,"user":{"login":"randomness666","id":20402909,"avatar_url":"https://avatars.githubusercontent.com/u/20402909?v=3","gravatar_id":"","url":"https://api.github.com/users/randomness666","html_url":"https://github.com/randomness666","followers_url":"https://api.github.com/users/randomness666/followers","following_url":"https://api.github.com/users/randomness666/following{/other_user}","gists_url":"https://api.github.com/users/randomness666/gists{/gist_id}","starred_url":"https://api.github.com/users/randomness666/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/randomness666/subscriptions","organizations_url":"https://api.github.com/users/randomness666/orgs","repos_url":"https://api.github.com/users/randomness666/repos","events_url":"https://api.github.com/users/randomness666/events{/privacy}","received_events_url":"https://api.github.com/users/randomness666/received_events","type":"User","site_admin":false},"created_at":"2016-08-24T18:01:00Z","updated_at":"2016-08-24T18:01:00Z","body":"<a href=\"https://github.com/ZyX-I\"><img src=\"https://avatars.githubusercontent.com/u/322097?v=3\" align=\"left\" width=\"48\" height=\"48\" hspace=\"10\"></img></a> **Comment by [ZyX-I](https://github.com/ZyX-I)**\n_Monday Feb 24, 2014 at 09:50 GMT_\n\n----\n\n\n\n24.02.14, 08:39, \"Michael\" <notifications@github.com>\":\n> \n> \n> @Shougo , I disagree about it breaking compatibilities.  Even if there's some really smart code out there that knows about all this stuff (ie reads Utf-8), it would be by far the exception.  Fixing this would fix incompatibilities far more than breaking compatibilities. \n> Code that knows about this issue is more likely to use the regexp solution (which shouldn't change since it seems quite well behaved), than to implement a utf-8 parser!     I believe (80% certainty) that vim stores everything internally as utf-8.\n\nBy the way, there is another thing that besides regex solution may be used for obtaining UTF-8 character: nr2char(char2nr(string[col - 1 :])). It should not break as well. Unlike regexes this variant takes exactly one *unicode code point* (there is a difference if you have combining characters).\n\n> \n> @ZyX-I certainly the idea of storing all strings (even if it were just in the scripting internals) depending on the widest character is appealing.  Simple and effective with not too much coding overhead (wrappers round all string operations.. hmm).\n> —\n> Reply to this email directly or view it on GitHub.\n"}}
{"actor":{"display_login":"randomness666"}}