{"action":"created","issue":{"url":"https://api.github.com/repos/FreeRDP/FreeRDP/issues/4646","repository_url":"https://api.github.com/repos/FreeRDP/FreeRDP","labels_url":"https://api.github.com/repos/FreeRDP/FreeRDP/issues/4646/labels{/name}","comments_url":"https://api.github.com/repos/FreeRDP/FreeRDP/issues/4646/comments","events_url":"https://api.github.com/repos/FreeRDP/FreeRDP/issues/4646/events","html_url":"https://github.com/FreeRDP/FreeRDP/issues/4646","id":322746327,"number":4646,"title":"shared-folder: files with certain encoding leads to cut-off list","user":{"login":"sh-dvl","id":26329004,"avatar_url":"https://avatars2.githubusercontent.com/u/26329004?v=4","gravatar_id":"","url":"https://api.github.com/users/sh-dvl","html_url":"https://github.com/sh-dvl","followers_url":"https://api.github.com/users/sh-dvl/followers","following_url":"https://api.github.com/users/sh-dvl/following{/other_user}","gists_url":"https://api.github.com/users/sh-dvl/gists{/gist_id}","starred_url":"https://api.github.com/users/sh-dvl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sh-dvl/subscriptions","organizations_url":"https://api.github.com/users/sh-dvl/orgs","repos_url":"https://api.github.com/users/sh-dvl/repos","events_url":"https://api.github.com/users/sh-dvl/events{/privacy}","received_events_url":"https://api.github.com/users/sh-dvl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/FreeRDP/FreeRDP/milestones/10","html_url":"https://github.com/FreeRDP/FreeRDP/milestone/10","labels_url":"https://api.github.com/repos/FreeRDP/FreeRDP/milestones/10/labels","id":3262235,"number":10,"title":"2.0.0","description":"2.0.0 release","creator":{"login":"bmiklautz","id":195793,"avatar_url":"https://avatars0.githubusercontent.com/u/195793?v=4","gravatar_id":"","url":"https://api.github.com/users/bmiklautz","html_url":"https://github.com/bmiklautz","followers_url":"https://api.github.com/users/bmiklautz/followers","following_url":"https://api.github.com/users/bmiklautz/following{/other_user}","gists_url":"https://api.github.com/users/bmiklautz/gists{/gist_id}","starred_url":"https://api.github.com/users/bmiklautz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bmiklautz/subscriptions","organizations_url":"https://api.github.com/users/bmiklautz/orgs","repos_url":"https://api.github.com/users/bmiklautz/repos","events_url":"https://api.github.com/users/bmiklautz/events{/privacy}","received_events_url":"https://api.github.com/users/bmiklautz/received_events","type":"User","site_admin":false},"open_issues":59,"closed_issues":0,"state":"open","created_at":"2018-04-12T14:01:12Z","updated_at":"2018-05-14T11:21:27Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2018-05-14T09:52:31Z","updated_at":"2018-05-14T11:45:05Z","closed_at":null,"author_association":"NONE","body":"when a shared-folder (or one of its sub-folders) have a file with iso8859-encoding like \"Vorfälle.txt\" ('ä' == '\\xe4') this and all following(!!!) files won't be shown in Windows (which sometimes leads to directories, which seems to be empty but aren't)\r\n\r\nnormally there shouldn't be any files with such an encoding, but sometimes they are and the standard-users are not always able to have a look at the directory directly on linux to identify and rename such files.\r\n\r\nwe're using remmina on linux as interface, so we can't switch any config-items for testing, but we believe that we've traced the problem down to libwinpr:\r\n\r\nin file/generic.c in function FindNextFileW() the function ConvertToUnicode() is called for every filename and if this function returns a zero-lenght, the function returns false with OOM-error\r\n\r\nunfortunately other errors might have occured: ConvertToUnicode() is a wrapper to MultiByteToWideChar(), which itself either calls u_strFromUTF8() (if WITH_ICU=on) or ConvertUTF8toUTF16() (if WITH_ICU=off) for conversion.\r\n\r\nwe've compiled freerdp with ICU=off, so let's look what happens there:\r\nConvertUTF8toUTF16()  is called with \"strictConversion\" from MultiByteToWideChar()\r\nit looks at trailingBytesForUTF8 for the utf8-length and get 2 for '\\xe4' and then checks the possible utf8-char via isLegalUTF8(length=2+1). Because '\\xe4' is not between 0x80 and 0xbf, it returns false and so ConvertUTF8toUTF16() returns \"sourceIllegal\", which isn't checked at all.\r\n\r\nbut even if we've compiled it with ICU=on, nothing would change:\r\nif u_strFromUTF8() see a unconvertible character it stops and returns U_INVALID_CHAR_FOUND via a pointer. Even here the error is completly ignored by MultiByteToWideChar().\r\neither this should be checked, or u_strFromUTF8WithSub() should be called.\r\n\r\nbut also in ConvertUTF8toUTF16() there seems to be a solution coded, if strictConversion isn't set: in line 579 there'll be a UNI_REPLACEMENT_CHAR written to target.\r\nunfortunately it wouldn't help to siimply deactivate strictConversion, because the call to isLegalUTF8 is done either and I'm not as familar with the code to see, if this change would break other things.\r\n\r\nNevertheless, the current behaviour is wrong (not showing files at all)\r\nRight behaviour would be to either leave unrecognized chars as is, or replace them by the UNI_REPLACEMENT_CHAR\r\n(I think it should depend on the reverse way, what happens, when the file with bad encoding is accessed - which we've not traced, yet: the file should not only be seen in filelist, but accessible, despite of its bad name)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/FreeRDP/FreeRDP/issues/comments/388788706","html_url":"https://github.com/FreeRDP/FreeRDP/issues/4646#issuecomment-388788706","issue_url":"https://api.github.com/repos/FreeRDP/FreeRDP/issues/4646","id":388788706,"user":{"login":"akallabeth","id":3472254,"avatar_url":"https://avatars2.githubusercontent.com/u/3472254?v=4","gravatar_id":"","url":"https://api.github.com/users/akallabeth","html_url":"https://github.com/akallabeth","followers_url":"https://api.github.com/users/akallabeth/followers","following_url":"https://api.github.com/users/akallabeth/following{/other_user}","gists_url":"https://api.github.com/users/akallabeth/gists{/gist_id}","starred_url":"https://api.github.com/users/akallabeth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akallabeth/subscriptions","organizations_url":"https://api.github.com/users/akallabeth/orgs","repos_url":"https://api.github.com/users/akallabeth/repos","events_url":"https://api.github.com/users/akallabeth/events{/privacy}","received_events_url":"https://api.github.com/users/akallabeth/received_events","type":"User","site_admin":false},"created_at":"2018-05-14T11:45:05Z","updated_at":"2018-05-14T11:45:05Z","author_association":"MEMBER","body":"@sh-dvl your answer regarding `utf-8` is somewhat ambigous, do you use it as `locale` or just for new entries in the filesystem? `readdir` should return strings in current locale if I did not misinterpret."}}
{"actor":{"display_login":"akallabeth"}}