{"comment":{"url":"https://api.github.com/repos/nalimilan/StringEncodings.jl/comments/16084939","html_url":"https://github.com/nalimilan/StringEncodings.jl/commit/167189715d763485ea369b932ea5d80389dce36e#commitcomment-16084939","id":16084939,"user":{"login":"ScottPJones","id":11798751,"avatar_url":"https://avatars.githubusercontent.com/u/11798751?v=3","gravatar_id":"","url":"https://api.github.com/users/ScottPJones","html_url":"https://github.com/ScottPJones","followers_url":"https://api.github.com/users/ScottPJones/followers","following_url":"https://api.github.com/users/ScottPJones/following{/other_user}","gists_url":"https://api.github.com/users/ScottPJones/gists{/gist_id}","starred_url":"https://api.github.com/users/ScottPJones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ScottPJones/subscriptions","organizations_url":"https://api.github.com/users/ScottPJones/orgs","repos_url":"https://api.github.com/users/ScottPJones/repos","events_url":"https://api.github.com/users/ScottPJones/events{/privacy}","received_events_url":"https://api.github.com/users/ScottPJones/received_events","type":"User","site_admin":false},"position":46,"line":125,"path":"src/encodings.jl","commit_id":"167189715d763485ea369b932ea5d80389dce36e","created_at":"2016-02-14T19:14:42Z","updated_at":"2016-02-14T19:14:42Z","body":"Actually, those properties have been the most important for making optimally performing conversion code in my experience.  Validated ASCII can simply be treated as UTF-8,\r\nor, like Latin1, widened directly from 8-bit to 16-bit UCS-2 or UTF-16 or 32-bit UTF-32,\r\nwith operations that can operate on whole chunks of data at once using SIMD instructions (I'd hand coded those in the past for different instruction sets).\r\n\r\nWhile it might seem by counting number of encodings, that these are just 3 out of many,\r\nif you look at people's data, I'd say most of the data can be represented by those 3.\r\nIf you want to leave it out of the table, I think we should still have methods that return that state, set up for those 3 (more really, because with UCS-2 you really have all the different encodings of that subset of Unicode, i.e. little/big endian, with or without BOM, etc.), and then just having a fallback to false for all other encodings.\r\n"}}