{"action":"created","issue":{"url":"https://api.github.com/repos/stenskjaer/samewords/issues/25","repository_url":"https://api.github.com/repos/stenskjaer/samewords","labels_url":"https://api.github.com/repos/stenskjaer/samewords/issues/25/labels{/name}","comments_url":"https://api.github.com/repos/stenskjaer/samewords/issues/25/comments","events_url":"https://api.github.com/repos/stenskjaer/samewords/issues/25/events","html_url":"https://github.com/stenskjaer/samewords/issues/25","id":294394824,"number":25,"title":"Some Unicode blocks not supported","user":{"login":"floriandk","id":954598,"avatar_url":"https://avatars2.githubusercontent.com/u/954598?v=4","gravatar_id":"","url":"https://api.github.com/users/floriandk","html_url":"https://github.com/floriandk","followers_url":"https://api.github.com/users/floriandk/followers","following_url":"https://api.github.com/users/floriandk/following{/other_user}","gists_url":"https://api.github.com/users/floriandk/gists{/gist_id}","starred_url":"https://api.github.com/users/floriandk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/floriandk/subscriptions","organizations_url":"https://api.github.com/users/floriandk/orgs","repos_url":"https://api.github.com/users/floriandk/repos","events_url":"https://api.github.com/users/floriandk/events{/privacy}","received_events_url":"https://api.github.com/users/floriandk/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-02-05T13:36:33Z","updated_at":"2018-03-07T05:24:55Z","closed_at":"2018-03-06T04:10:58Z","author_association":"NONE","body":"While trying some real life examples with branch issue-24 I noticed that some Unicode characters break compilation, i.e. it says \"Starting conversion.\" and never comes to an end.\r\n\r\nIt is a bit curious which characters are affected. It seems to go by Unicode blocks and neither by frequency (e.g. typographical quotes and € don't work, Runes do) nor function.\r\n\r\n```latex\r\n\\documentclass[a5paper]{scrartcl}\r\n\r\n\\usepackage{fontspec}\r\n\\setmainfont{Arial}\r\n\r\n\\usepackage[series={A},noledgroup,draft]{reledmac}\r\n\r\n\\begin{document}\r\n\r\n\\beginnumbering\r\n\\pstart\r\nBasic Latin: o \r\n\r\nLat1: ô \r\n\r\nExtA: ő\r\n\r\nExtB: ǫ\r\n\r\nIPA: ɷ %all working\r\n\r\nSpacin Mod:% ˚ %breaks\r\n\r\nComb Diacritics:%  oͦ %breaks\r\n\r\nGreek: ώ %compiles\r\n\r\nCyrillic: ѻ %compiles\r\n\r\nCyr Supp: Ԛ %compiles\r\n\r\nRunic: ᚮ  %compiles\r\n\r\nIPA Ext: ᴏ ᴼ  %compiles\r\n\r\nIPA Ext Suppl: ᶱ  %compiles\r\n\r\nComb Dia Suppl: % ᷕ %breaks \r\n\r\nLat Ext A: ṓ %compiles\r\n\r\nGreep Add: Ὧ %compiles\r\n\r\nGeneral Punct: %“⁐ %breaks!\r\n\r\nSuperscripts/Sunscripts: ₒ %compiles\r\n\r\nCurrency: %€₰ %breaks\r\n\r\n[...]\r\n\r\nLat Ext C: ⱬ %compiles\r\n\r\n[...]\r\n\r\nSupplemental Punct: %⸀ %breaks\r\n\r\n[...]\r\n\r\nLat Ext D: ꝏ %compiles\r\n\r\n[...]\r\n\r\nPUA: % %breaks\r\n\\pend\r\n\\endnumbering\r\n\r\n\\end{document}\r\n```"},"comment":{"url":"https://api.github.com/repos/stenskjaer/samewords/issues/comments/371026949","html_url":"https://github.com/stenskjaer/samewords/issues/25#issuecomment-371026949","issue_url":"https://api.github.com/repos/stenskjaer/samewords/issues/25","id":371026949,"user":{"login":"stenskjaer","id":4047523,"avatar_url":"https://avatars0.githubusercontent.com/u/4047523?v=4","gravatar_id":"","url":"https://api.github.com/users/stenskjaer","html_url":"https://github.com/stenskjaer","followers_url":"https://api.github.com/users/stenskjaer/followers","following_url":"https://api.github.com/users/stenskjaer/following{/other_user}","gists_url":"https://api.github.com/users/stenskjaer/gists{/gist_id}","starred_url":"https://api.github.com/users/stenskjaer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stenskjaer/subscriptions","organizations_url":"https://api.github.com/users/stenskjaer/orgs","repos_url":"https://api.github.com/users/stenskjaer/repos","events_url":"https://api.github.com/users/stenskjaer/events{/privacy}","received_events_url":"https://api.github.com/users/stenskjaer/received_events","type":"User","site_admin":false},"created_at":"2018-03-07T05:24:55Z","updated_at":"2018-03-07T05:24:55Z","author_association":"OWNER","body":"Okay, so this is very useful. I am improving the punctuation tokenization now to include more characters. I will describe that in #24 when I'm done. \r\n\r\nAbout the composing characters (which I mistook the punctuation as): All composing characters such as the suggested 006F (o) and 0308 (̈) = 00F6 (ö) are normalized to the single character glyphs (00F6 here) before processing so that sholdn't be a problem.\r\n\r\nI am adding a note on this in the readme, as that actually is a change in the file that can be very hard to perceive. "}}
{"actor":{"display_login":"stenskjaer"}}