{"action":"created","issue":{"url":"https://api.github.com/repos/curl/curl/issues/654","repository_url":"https://api.github.com/repos/curl/curl","labels_url":"https://api.github.com/repos/curl/curl/issues/654/labels{/name}","comments_url":"https://api.github.com/repos/curl/curl/issues/654/comments","events_url":"https://api.github.com/repos/curl/curl/issues/654/events","html_url":"https://github.com/curl/curl/pull/654","id":133588896,"number":654,"title":"idn_win32: Fall back to ANSI encoding if UTF-8 fails","user":{"login":"jay","id":965580,"avatar_url":"https://avatars.githubusercontent.com/u/965580?v=3","gravatar_id":"","url":"https://api.github.com/users/jay","html_url":"https://github.com/jay","followers_url":"https://api.github.com/users/jay/followers","following_url":"https://api.github.com/users/jay/following{/other_user}","gists_url":"https://api.github.com/users/jay/gists{/gist_id}","starred_url":"https://api.github.com/users/jay/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jay/subscriptions","organizations_url":"https://api.github.com/users/jay/orgs","repos_url":"https://api.github.com/users/jay/repos","events_url":"https://api.github.com/users/jay/events{/privacy}","received_events_url":"https://api.github.com/users/jay/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-02-14T22:46:07Z","updated_at":"2016-02-22T09:43:03Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/curl/curl/pulls/654","html_url":"https://github.com/curl/curl/pull/654","diff_url":"https://github.com/curl/curl/pull/654.diff","patch_url":"https://github.com/curl/curl/pull/654.patch"},"body":"~~~\r\nWhen converting the hostname from IDN to punycode if the IDN is not\r\nvalid UTF-8, or if it is and fails to convert, fall back to converting\r\nthe IDN to punycode using the user's ANSI codepage.\r\n~~~\r\n\r\n@pierrejoye @mkauf @gvanem based on our discussion in #637.\r\n\r\nThis will bring the WinIDN code more in line with libidn. We try treating the IDN in UTF-8 encoding and if that doesn't work we use locale encoding.\r\n\r\nThis detection isn't 100% accurate since an ANSI encoded hostname could be determined to be valid UTF-8, though that seems unlikely. How unlikely? I don't know.\r\n\r\nAnother thing is a possible privacy issue where a server may be able to determine the user's codepage by sending a hostname for redirect with bad UTF-8 to force it to be converted using the user's codepage. This is already possible with libidn, afaics.\r\n\r\nIn terms of codepage conversion I went with CP_ACP which is the user's current ANSI codepage. I did not go with CP_OEMCP which may be more appropriate because it's the console codepage. I've never seen the latter used in any codebase I've worked on. Sometimes the OEM and ANSI codepages can differ, apparently. I would be curious how this affects those users but I don't know much about the differences. Microsoft's support [article](https://support.microsoft.com/en-us/kb/108450) on when to use ANSI and when to use OEM is lacking. I believe more people use ANSI simply because when you save a file for, say, running a script and you are not saving in Unicode then you're most likely saving in ANSI.\r\n"},"comment":{"url":"https://api.github.com/repos/curl/curl/issues/comments/187096731","html_url":"https://github.com/curl/curl/pull/654#issuecomment-187096731","issue_url":"https://api.github.com/repos/curl/curl/issues/654","id":187096731,"user":{"login":"mkauf","id":12849992,"avatar_url":"https://avatars.githubusercontent.com/u/12849992?v=3","gravatar_id":"","url":"https://api.github.com/users/mkauf","html_url":"https://github.com/mkauf","followers_url":"https://api.github.com/users/mkauf/followers","following_url":"https://api.github.com/users/mkauf/following{/other_user}","gists_url":"https://api.github.com/users/mkauf/gists{/gist_id}","starred_url":"https://api.github.com/users/mkauf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mkauf/subscriptions","organizations_url":"https://api.github.com/users/mkauf/orgs","repos_url":"https://api.github.com/users/mkauf/repos","events_url":"https://api.github.com/users/mkauf/events{/privacy}","received_events_url":"https://api.github.com/users/mkauf/received_events","type":"User","site_admin":false},"created_at":"2016-02-22T09:43:03Z","updated_at":"2016-02-22T09:43:03Z","body":"Table 3-7 in the Unicode 8 specification has the same content as in the Unicode 7 specification."}}