{"action":"created","issue":{"url":"https://api.github.com/repos/hydroshare/hydroshare/issues/3024","repository_url":"https://api.github.com/repos/hydroshare/hydroshare","labels_url":"https://api.github.com/repos/hydroshare/hydroshare/issues/3024/labels{/name}","comments_url":"https://api.github.com/repos/hydroshare/hydroshare/issues/3024/comments","events_url":"https://api.github.com/repos/hydroshare/hydroshare/issues/3024/events","html_url":"https://github.com/hydroshare/hydroshare/issues/3024","id":384837972,"node_id":"MDU6SXNzdWUzODQ4Mzc5NzI=","number":3024,"title":"unicode file names not handled properly by HydroShare","user":{"login":"alvacouch","id":3342292,"node_id":"MDQ6VXNlcjMzNDIyOTI=","avatar_url":"https://avatars1.githubusercontent.com/u/3342292?v=4","gravatar_id":"","url":"https://api.github.com/users/alvacouch","html_url":"https://github.com/alvacouch","followers_url":"https://api.github.com/users/alvacouch/followers","following_url":"https://api.github.com/users/alvacouch/following{/other_user}","gists_url":"https://api.github.com/users/alvacouch/gists{/gist_id}","starred_url":"https://api.github.com/users/alvacouch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alvacouch/subscriptions","organizations_url":"https://api.github.com/users/alvacouch/orgs","repos_url":"https://api.github.com/users/alvacouch/repos","events_url":"https://api.github.com/users/alvacouch/events{/privacy}","received_events_url":"https://api.github.com/users/alvacouch/received_events","type":"User","site_admin":false},"labels":[{"id":926946612,"node_id":"MDU6TGFiZWw5MjY5NDY2MTI=","url":"https://api.github.com/repos/hydroshare/hydroshare/labels/(2)%20Medium%20Effort","name":"(2) Medium Effort","color":"d3b326","default":false},{"id":729633515,"node_id":"MDU6TGFiZWw3Mjk2MzM1MTU=","url":"https://api.github.com/repos/hydroshare/hydroshare/labels/File%20System","name":"File System","color":"c2e0c6","default":false},{"id":433017943,"node_id":"MDU6TGFiZWw0MzMwMTc5NDM=","url":"https://api.github.com/repos/hydroshare/hydroshare/labels/High%20Priority","name":"High Priority","color":"b60205","default":false},{"id":233668032,"node_id":"MDU6TGFiZWwyMzM2NjgwMzI=","url":"https://api.github.com/repos/hydroshare/hydroshare/labels/REST%20API","name":"REST API","color":"fef2c0","default":false}],"state":"open","locked":false,"assignee":{"login":"alvacouch","id":3342292,"node_id":"MDQ6VXNlcjMzNDIyOTI=","avatar_url":"https://avatars1.githubusercontent.com/u/3342292?v=4","gravatar_id":"","url":"https://api.github.com/users/alvacouch","html_url":"https://github.com/alvacouch","followers_url":"https://api.github.com/users/alvacouch/followers","following_url":"https://api.github.com/users/alvacouch/following{/other_user}","gists_url":"https://api.github.com/users/alvacouch/gists{/gist_id}","starred_url":"https://api.github.com/users/alvacouch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alvacouch/subscriptions","organizations_url":"https://api.github.com/users/alvacouch/orgs","repos_url":"https://api.github.com/users/alvacouch/repos","events_url":"https://api.github.com/users/alvacouch/events{/privacy}","received_events_url":"https://api.github.com/users/alvacouch/received_events","type":"User","site_admin":false},"assignees":[{"login":"alvacouch","id":3342292,"node_id":"MDQ6VXNlcjMzNDIyOTI=","avatar_url":"https://avatars1.githubusercontent.com/u/3342292?v=4","gravatar_id":"","url":"https://api.github.com/users/alvacouch","html_url":"https://github.com/alvacouch","followers_url":"https://api.github.com/users/alvacouch/followers","following_url":"https://api.github.com/users/alvacouch/following{/other_user}","gists_url":"https://api.github.com/users/alvacouch/gists{/gist_id}","starred_url":"https://api.github.com/users/alvacouch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alvacouch/subscriptions","organizations_url":"https://api.github.com/users/alvacouch/orgs","repos_url":"https://api.github.com/users/alvacouch/repos","events_url":"https://api.github.com/users/alvacouch/events{/privacy}","received_events_url":"https://api.github.com/users/alvacouch/received_events","type":"User","site_admin":false}],"milestone":null,"comments":1,"created_at":"2018-11-27T15:04:02Z","updated_at":"2018-11-27T17:08:55Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"@martinseul @dtarb @sblack-usu\r\nThe plot thickens as I study the Unicode problem, as manifest in cuahsi-dev-2, resource `3135543760f24fa5a89c096d67b55e7f`, directory `data/contents`, filename `Appendix H â Monitoring and Assessment of Critical Thermal Dynamics in Upper Middle Fork of the John Day River, 2008-2016.pdf`. \r\n\r\nThe Unicode character in this filename is **â**.\r\n\r\nThe full story of this file's ingestion:\r\n \r\n* Windoze created a zipfile with Unicode in it. \r\n* Linux actually understood the zipfile Unicode. (verification: stack overflow post: https://unix.stackexchange.com/questions/38055/utf-8-filenames) \r\n* iRODS actually understood the zipfile Unicode. (verification: iRODS team) \r\n* The file was correctly ingested into Django. \r\n\r\nSo, what's the problem?\r\n\r\n* All of the URLs for this file are invalid, because they contain unescaped Unicode (and a bunch of other unescaped stuff like whitespace). This includes public URLS that a user would use to download the file, as well as private URLs that invoke REST AJAX features. Thus, almost any action directed at that specific file fails, due to Unicode encoding errors in the REST request!\r\n\r\nThus, the real solution is to fix the URLs, not the file. This must be a systematic solution in which **all URLs referring to files or file paths get fixed,** or there will be some case that doesn't work. Affected regions of code include large file download, resource landing page, etc. URLs listed to reference thsi file in resourcemetadata.xml, resourcemap.xml, and all composite resource metadata files are also invalid. \r\n"},"comment":{"url":"https://api.github.com/repos/hydroshare/hydroshare/issues/comments/442138987","html_url":"https://github.com/hydroshare/hydroshare/issues/3024#issuecomment-442138987","issue_url":"https://api.github.com/repos/hydroshare/hydroshare/issues/3024","id":442138987,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MjEzODk4Nw==","user":{"login":"dtarb","id":3456833,"node_id":"MDQ6VXNlcjM0NTY4MzM=","avatar_url":"https://avatars1.githubusercontent.com/u/3456833?v=4","gravatar_id":"","url":"https://api.github.com/users/dtarb","html_url":"https://github.com/dtarb","followers_url":"https://api.github.com/users/dtarb/followers","following_url":"https://api.github.com/users/dtarb/following{/other_user}","gists_url":"https://api.github.com/users/dtarb/gists{/gist_id}","starred_url":"https://api.github.com/users/dtarb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dtarb/subscriptions","organizations_url":"https://api.github.com/users/dtarb/orgs","repos_url":"https://api.github.com/users/dtarb/repos","events_url":"https://api.github.com/users/dtarb/events{/privacy}","received_events_url":"https://api.github.com/users/dtarb/received_events","type":"User","site_admin":false},"created_at":"2018-11-27T17:08:55Z","updated_at":"2018-11-27T17:08:55Z","author_association":"MEMBER","body":"As an additional test case, here is a zip file provided by a user a while back that was a problem.  \r\n[Filenames.zip](https://github.com/hydroshare/hydroshare/files/2621056/Filenames.zip)\r\nIt contains several files with unicode character names.  If I attempt to unzip in hydroShare today I get a folder with only one file, i.e. several of the files are not extracted.  \r\n\r\nWill the solution suggested resolve this.\r\n"}}
{"actor":{"display_login":"dtarb"}}