{"action":"created","issue":{"url":"https://api.github.com/repos/frerich/clcache/issues/64","repository_url":"https://api.github.com/repos/frerich/clcache","labels_url":"https://api.github.com/repos/frerich/clcache/issues/64/labels{/name}","comments_url":"https://api.github.com/repos/frerich/clcache/issues/64/comments","events_url":"https://api.github.com/repos/frerich/clcache/issues/64/events","html_url":"https://github.com/frerich/clcache/issues/64","id":131839105,"number":64,"title":"python3: encoding error","user":{"login":"timblechmann","id":792880,"avatar_url":"https://avatars.githubusercontent.com/u/792880?v=3","gravatar_id":"","url":"https://api.github.com/users/timblechmann","html_url":"https://github.com/timblechmann","followers_url":"https://api.github.com/users/timblechmann/followers","following_url":"https://api.github.com/users/timblechmann/following{/other_user}","gists_url":"https://api.github.com/users/timblechmann/gists{/gist_id}","starred_url":"https://api.github.com/users/timblechmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timblechmann/subscriptions","organizations_url":"https://api.github.com/users/timblechmann/orgs","repos_url":"https://api.github.com/users/timblechmann/repos","events_url":"https://api.github.com/users/timblechmann/events{/privacy}","received_events_url":"https://api.github.com/users/timblechmann/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-02-06T09:36:30Z","updated_at":"2016-02-26T13:42:48Z","closed_at":null,"body":"when using clcache/master via python 3, i encountered the following error:\r\n\r\n```\r\nException in thread Thread-1:\r\nTraceback (most recent call last):\r\n  File \"C:\\Python34\\lib\\threading.py\", line 920, in _bootstrap_inner\r\n  File \"C:\\Python34\\lib\\threading.py\", line 868, in run\r\n  File \"C:\\Python34\\lib\\subprocess.py\", line 1171, in _readerthread\r\n  File \"C:\\Python34\\lib\\encodings\\cp1252.py\", line 23, in decode\r\nUnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 161099: character maps to <undefined>\r\n\r\nTraceback (most recent call last):\r\n  File \"clcache.py\", line 1200, in <module>\r\n  File \"clcache.py\", line 1100, in main\r\n  File \"clcache.py\", line 1166, in processCompileRequest\r\n  File \"clcache.py\", line 1026, in processNoManifestMiss\r\n  File \"clcache.py\", line 769, in invokeRealCompiler\r\n  File \"C:\\Python34\\lib\\subprocess.py\", line 960, in communicate\r\n  File \"C:\\Python34\\lib\\subprocess.py\", line 1235, in _communicate\r\nIndexError: list index out of range\r\n```\r\n\r\nit seems to be related to the fact that the compiler is using a german locale :("},"comment":{"url":"https://api.github.com/repos/frerich/clcache/issues/comments/189282130","html_url":"https://github.com/frerich/clcache/issues/64#issuecomment-189282130","issue_url":"https://api.github.com/repos/frerich/clcache/issues/64","id":189282130,"user":{"login":"webmaster128","id":2603011,"avatar_url":"https://avatars.githubusercontent.com/u/2603011?v=3","gravatar_id":"","url":"https://api.github.com/users/webmaster128","html_url":"https://github.com/webmaster128","followers_url":"https://api.github.com/users/webmaster128/followers","following_url":"https://api.github.com/users/webmaster128/following{/other_user}","gists_url":"https://api.github.com/users/webmaster128/gists{/gist_id}","starred_url":"https://api.github.com/users/webmaster128/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/webmaster128/subscriptions","organizations_url":"https://api.github.com/users/webmaster128/orgs","repos_url":"https://api.github.com/users/webmaster128/repos","events_url":"https://api.github.com/users/webmaster128/events{/privacy}","received_events_url":"https://api.github.com/users/webmaster128/received_events","type":"User","site_admin":false},"created_at":"2016-02-26T13:42:48Z","updated_at":"2016-02-26T13:42:48Z","body":"Okay, here we go: http://stackoverflow.com/a/33291200\r\n\r\nProblem is that we do not know which encoding MSVC is throwing at us in STDOUT and STDERR. It is likely to be this semi-UTF-16 encoding called MBCS/DBCS ([Microsoft's definition of DBCS](https://www.microsoft.com/typography/unicode/cs.htm)).\r\n\r\nIt is true that 0x81 is not Windows-1251/CP-1251 because the [position is empty](https://en.wikipedia.org/wiki/Windows-1252#Code_page_layout).\r\n\r\nSo as described in the SO answer, in `invokeRealCompiler()` we should not use Popen with `universal_newlines=True` when calling `communicate()`. Instead we should take the plain bytes and do not convert them into strings. This is already done `ObjectCache.computeKey()`."}}