{"action":"created","issue":{"url":"https://api.github.com/repos/jjjake/internetarchive/issues/284","repository_url":"https://api.github.com/repos/jjjake/internetarchive","labels_url":"https://api.github.com/repos/jjjake/internetarchive/issues/284/labels{/name}","comments_url":"https://api.github.com/repos/jjjake/internetarchive/issues/284/comments","events_url":"https://api.github.com/repos/jjjake/internetarchive/issues/284/events","html_url":"https://github.com/jjjake/internetarchive/issues/284","id":386017952,"node_id":"MDU6SXNzdWUzODYwMTc5NTI=","number":284,"title":"UTF-8 error on upload","user":{"login":"vxbinaca","id":12656590,"node_id":"MDQ6VXNlcjEyNjU2NTkw","avatar_url":"https://avatars2.githubusercontent.com/u/12656590?v=4","gravatar_id":"","url":"https://api.github.com/users/vxbinaca","html_url":"https://github.com/vxbinaca","followers_url":"https://api.github.com/users/vxbinaca/followers","following_url":"https://api.github.com/users/vxbinaca/following{/other_user}","gists_url":"https://api.github.com/users/vxbinaca/gists{/gist_id}","starred_url":"https://api.github.com/users/vxbinaca/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vxbinaca/subscriptions","organizations_url":"https://api.github.com/users/vxbinaca/orgs","repos_url":"https://api.github.com/users/vxbinaca/repos","events_url":"https://api.github.com/users/vxbinaca/events{/privacy}","received_events_url":"https://api.github.com/users/vxbinaca/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2018-11-30T03:02:36Z","updated_at":"2018-12-03T17:51:24Z","closed_at":null,"author_association":"NONE","body":"`ia upload LFTP_art_ucla_edu_newgenres_student_videos art.ucla.edu/ --delete`\r\n\r\n```\r\nLFTP_art_ucla_edu_newgenres_student_videos:\r\nTraceback (most recent call last):\r\n  File \"/home/vxbinaca/.local/bin/ia\", line 11, in <module>\r\n    sys.exit(main())\r\n  File \"/home/vxbinaca/.local/lib/python3.6/site-packages/internetarchive/cli/ia.py\", line 159, in main\r\n    sys.exit(ia_module.main(argv, session))\r\n  File \"/home/vxbinaca/.local/lib/python3.6/site-packages/internetarchive/cli/ia_upload.py\", line 225, in main\r\n    for _r in _upload_files(item, files, upload_kwargs):\r\n  File \"/home/vxbinaca/.local/lib/python3.6/site-packages/internetarchive/cli/ia_upload.py\", line 86, in _upload_files\r\n    response = item.upload(files, **upload_kwargs)\r\n  File \"/home/vxbinaca/.local/lib/python3.6/site-packages/internetarchive/item.py\", line 828, in upload\r\n    request_kwargs=request_kwargs)\r\n  File \"/home/vxbinaca/.local/lib/python3.6/site-packages/internetarchive/item.py\", line 617, in upload_file\r\n    base_url, urllib.parse.quote(norm_filepath(key).lstrip('/').encode('utf-8')))\r\nUnicodeEncodeError: 'utf-8' codec can't encode character '\\udcf1' in position 7: surrogates not allowed\r\n```"},"comment":{"url":"https://api.github.com/repos/jjjake/internetarchive/issues/comments/443803306","html_url":"https://github.com/jjjake/internetarchive/issues/284#issuecomment-443803306","issue_url":"https://api.github.com/repos/jjjake/internetarchive/issues/284","id":443803306,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MzgwMzMwNg==","user":{"login":"ethus3h","id":10457522,"node_id":"MDQ6VXNlcjEwNDU3NTIy","avatar_url":"https://avatars3.githubusercontent.com/u/10457522?v=4","gravatar_id":"","url":"https://api.github.com/users/ethus3h","html_url":"https://github.com/ethus3h","followers_url":"https://api.github.com/users/ethus3h/followers","following_url":"https://api.github.com/users/ethus3h/following{/other_user}","gists_url":"https://api.github.com/users/ethus3h/gists{/gist_id}","starred_url":"https://api.github.com/users/ethus3h/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ethus3h/subscriptions","organizations_url":"https://api.github.com/users/ethus3h/orgs","repos_url":"https://api.github.com/users/ethus3h/repos","events_url":"https://api.github.com/users/ethus3h/events{/privacy}","received_events_url":"https://api.github.com/users/ethus3h/received_events","type":"User","site_admin":false},"created_at":"2018-12-03T17:51:24Z","updated_at":"2018-12-03T17:51:24Z","author_association":"NONE","body":"Cool, thanks! It looks like it is those two files 'art.ucla.edu/JaimeMu'$'\\361''oz_sm.jpg' and 'art.ucla.edu/JaimeMu'$'\\361''oz_Center15.mp4' causing the problem, and the difference in error messages is from a difference in our versions of ia, Python, or operating system.\r\n\r\nFor now, if you just want to get these files uploaded, you can manually rename those files (`mv '~/Documents/LFTP/art.ucla.edu/JaimeMu'$'\\361''oz_sm.jpg' '~/Documents/LFTP/art.ucla.edu/JaimeMu'$'\\303\\261''oz_sm.jpg'; mv '~/Documents/LFTP/art.ucla.edu/JaimeMu'$'\\361''oz_Center15.mp4' '~/Documents/LFTP/art.ucla.edu/JaimeMu'$'\\303\\261''oz_Center15.mp4'` to have the equivalent filenames in Unicode, after which the whole folder should upload normally. (Alternatively, you could put the folder in a tar file and upload that.)\r\n\r\nI can't really speak much on a proper bugfix in `ia`, but the essence of it would be to avoid converting the filename to a Python string, and leave it as plain old bytes. (I'm not much of a Python person, but evidently its strings aren't binary-safe.) That may expose other issues, though: I don't know if the way these names are sent to the server can handle non-Unicode names, or if the servers at IA are able to handle them correctly."}}
{"actor":{"display_login":"ethus3h"}}