{"action":"created","issue":{"url":"https://api.github.com/repos/mapbox/mapbox-gl/issues/4","repository_url":"https://api.github.com/repos/mapbox/mapbox-gl","labels_url":"https://api.github.com/repos/mapbox/mapbox-gl/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/mapbox/mapbox-gl/issues/4/comments","events_url":"https://api.github.com/repos/mapbox/mapbox-gl/issues/4/events","html_url":"https://github.com/mapbox/mapbox-gl/issues/4","id":121336360,"number":4,"title":"Complex Text Rendering","user":{"login":"mikemorris","id":1149913,"avatar_url":"https://avatars.githubusercontent.com/u/1149913?v=3","gravatar_id":"","url":"https://api.github.com/users/mikemorris","html_url":"https://github.com/mikemorris","followers_url":"https://api.github.com/users/mikemorris/followers","following_url":"https://api.github.com/users/mikemorris/following{/other_user}","gists_url":"https://api.github.com/users/mikemorris/gists{/gist_id}","starred_url":"https://api.github.com/users/mikemorris/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikemorris/subscriptions","organizations_url":"https://api.github.com/users/mikemorris/orgs","repos_url":"https://api.github.com/users/mikemorris/repos","events_url":"https://api.github.com/users/mikemorris/events{/privacy}","received_events_url":"https://api.github.com/users/mikemorris/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"mikemorris","id":1149913,"avatar_url":"https://avatars.githubusercontent.com/u/1149913?v=3","gravatar_id":"","url":"https://api.github.com/users/mikemorris","html_url":"https://github.com/mikemorris","followers_url":"https://api.github.com/users/mikemorris/followers","following_url":"https://api.github.com/users/mikemorris/following{/other_user}","gists_url":"https://api.github.com/users/mikemorris/gists{/gist_id}","starred_url":"https://api.github.com/users/mikemorris/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikemorris/subscriptions","organizations_url":"https://api.github.com/users/mikemorris/orgs","repos_url":"https://api.github.com/users/mikemorris/repos","events_url":"https://api.github.com/users/mikemorris/events{/privacy}","received_events_url":"https://api.github.com/users/mikemorris/received_events","type":"User","site_admin":false},"milestone":null,"comments":5,"created_at":"2015-12-09T20:52:48Z","updated_at":"2016-04-19T09:17:59Z","closed_at":null,"body":"### What is the state of text rendering in Mapbox GL?\r\nWe currently do not render scripts that require bidirectional support or complex text shaping correctly in [mapbox-gl-native](https://github.com/mapbox/mapbox-gl-native) or [mapbox-gl-js](https://github.com/mapbox/mapbox-gl-js). This ticket will track adding proper support to both projects.\r\n\r\n![Broken Arabic labels example courtesy of @mushon](https://cloud.githubusercontent.com/assets/1149913/11675912/954b1232-9de2-11e5-9bad-58c3a1ce19f3.gif)\r\n\r\n### What's missing?\r\nWe need to add the following functionality for proper text rendering:\r\n- Unicode Bidi Algorithm to cut labels into logical segments and flip the display order of RTL (right-to-left) text.\r\n  - Necessary for proper rendering of RTL scripts and mixed script labels containing RTL scripts interspersed with LTR text runs like numerals.\r\n- Complex text shaping\r\n  - Necessary for proper display of scripts where adjacent glyphs should be transformed into new glyphs or rendered as combined glyphs or ligatures.\r\n\r\nIn terms of scripts affected by this, [Hebrew requires bidi support](https://github.com/mapbox/mapbox-gl-js/issues/471#issuecomment-119288570), Indic scripts like Hindi can require complex text shaping, and [Arabic requires both bidi and complex text shaping support](https://github.com/mapbox/mapbox-gl-native/issues/1259). Additionally, implementing the [Unicode line-breaking algorithm](http://www.unicode.org/reports/tr14/) should improve support for cases like [smarter line breaking in Chinese](https://github.com/mapbox/mapbox-gl-native/issues/1223).\r\n\r\n### How do we currently handle fontstack fallbacks?\r\nCurrently, the Protobuf-encoded \"glyph tiles\" we create with [node-fontnik](https://github.com/mapbox/node-fontnik) are a composited \"fontstack\" with missing glyphs in fonts higher in the stack being filled in by glyphs from fonts further down the stack and we therefore end up with a combined `Helvetica, Arial Unicode` fontstack with per-glyph fallbacks in rendered text.\r\n\r\nFontstack | Coverage\r\n--- | ---\r\n\"Helvetica\" | Latin\r\n\"Arial Unicode\" | Latin, Arabic\r\n\"Helvetica, Arial Unicode\" | Helvetica Latin, Arial Unicode Arabic\r\n\r\n### Why will this not work for complex text shaping?\r\nBecause shaping tables are specific to a font file, to apply shaping properly we will need to work exclusively with glyphs from a single font. Instead of using \"fontstack\" glyph tiles, we will need tiles which contain all the glyphs in a given range for a single font. This approach should also limit [glyph atlas duplication](https://github.com/mapbox/mapbox-gl-js/issues/141#issuecomment-98345854) for multiple fontstacks with a common fallback.\r\n\r\n### How will we do this?\r\nWe will first need to segment each label into text runs (splitting words into individual segments, and splitting Arabic text segments from numerical segments for example) with the Unicode bidi algorithm. Then, for each segment, we will attempt, with each font in the fontstack until a match is found, to shape the text segment with a single font's shaping table and check whether all characters in the shaped result can be rendered by that font (using a glyph coverage file). If coverage is incomplete, we will fall back to the next font in the stack.\r\n\r\n(It's possible we could check glyph coverage first, but the necessary glyphs may change after shaping, and the glyph coverage check would have to be repeated. We should test performance to determine whether a possibly inaccurate initial coverage check is faster than redundant shaping passes for fonts lacking glyph coverage.)\r\n\r\n#### Example\r\nFor the fontstack \"Open Sans, Arial Unicode\", no glyphs change when shaped with `Open Sans/gsub.sfnt` - do all characters in `résumé` exist in `Open Sans.coverage.json`? NO? Missing `é`? Reshape with `Arial Unicode/gsub.sfnt`, then check if all characters in `résumé` exist in `Arial Unicode.coverage.json`\r\n\r\nOnce a font with matching coverage has been determined, we can request glyph tiles from a single font containing the necessary glyphs, like `Arial Unicode Regular/0-255.pbf`.\r\n\r\n### How will we get/use these \"shaping tables\"?\r\n\r\nShaping tables are contained in font files as `GSUB` (glyph substitution), `GPOS` (glyph positioning) and `KERN` (kerning) tables, which can be read by the [FreeType](http://www.freetype.org/) function [`FT_Load_Sfnt_Table`](http://www.freetype.org/freetype2/docs/reference/ft2-truetype_tables.html#FT_Load_Sfnt_Table). We will need to [extract these tables from from uploaded font files](https://github.com/mapbox/node-fontnik/issues/85), then request them from the client through an API. We've [started work on extracted shaping tables](https://github.com/mapbox/node-fontnik/pull/95) but it isn't quite functional yet.\r\n\r\nTo use these shaping tables, we will need to pass them into [HarfBuzz](https://wiki.freedesktop.org/www/Software/HarfBuzz/) for mapbox-gl-native, or [an emscripten port](https://github.com/prezi/harfbuzz-js) for mapbox-gl-js. I'm not sure if HarfBuzz currently has an interface for reading raw shaping tables (it generally works with full font files). If this interface doesn't currently exist, we'll need to add it.\r\n\r\n### Resources\r\n**Universal**\r\n- http://www.unicode.org/Public/8.0.0/ucd/\r\n- Unicode Bidirectional Algorithm http://www.unicode.org/reports/tr9/\r\n- Unicode Line Breaking Algorithm http://www.unicode.org/reports/tr14/\r\n- Unicode Text Segmentation http://www.unicode.org/reports/tr29/\r\n- Unicode test data http://www.unicode.org/reports/tr41/tr41-17.html#Tests14\r\n\r\n**C++**\r\n- Unicode Bidi Reference Sample http://unicode.org/Public/PROGRAMS/BidiReferenceCpp/v26/\r\n- HarfBuzz API Design https://mail.gnome.org/archives/gtk-i18n-list/2009-August/msg00025.html\r\n- Mozilla HarfBuzz integration https://bugzilla.mozilla.org/show_bug.cgi?id=449292\r\n- http://site.icu-project.org/\r\n\r\n**JavaScript**\r\n- https://github.com/twitter/twitter-cldr-js#handling-bidirectional-text\r\n\r\n/cc @mapbox/gl"},"comment":{"url":"https://api.github.com/repos/mapbox/mapbox-gl/issues/comments/211820534","html_url":"https://github.com/mapbox/mapbox-gl/issues/4#issuecomment-211820534","issue_url":"https://api.github.com/repos/mapbox/mapbox-gl/issues/4","id":211820534,"user":{"login":"mushon","id":27123,"avatar_url":"https://avatars.githubusercontent.com/u/27123?v=3","gravatar_id":"","url":"https://api.github.com/users/mushon","html_url":"https://github.com/mushon","followers_url":"https://api.github.com/users/mushon/followers","following_url":"https://api.github.com/users/mushon/following{/other_user}","gists_url":"https://api.github.com/users/mushon/gists{/gist_id}","starred_url":"https://api.github.com/users/mushon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mushon/subscriptions","organizations_url":"https://api.github.com/users/mushon/orgs","repos_url":"https://api.github.com/users/mushon/repos","events_url":"https://api.github.com/users/mushon/events{/privacy}","received_events_url":"https://api.github.com/users/mushon/received_events","type":"User","site_admin":false},"created_at":"2016-04-19T09:17:58Z","updated_at":"2016-04-19T09:17:58Z","body":"@mikemorris do let us know if you need some help with testing as this is definitely a pressing issue for many of us. Thanks!"}}