{"action":"created","issue":{"url":"https://api.github.com/repos/ncw/rclone/issues/753","repository_url":"https://api.github.com/repos/ncw/rclone","labels_url":"https://api.github.com/repos/ncw/rclone/issues/753/labels{/name}","comments_url":"https://api.github.com/repos/ncw/rclone/issues/753/comments","events_url":"https://api.github.com/repos/ncw/rclone/issues/753/events","html_url":"https://github.com/ncw/rclone/issues/753","id":180397142,"number":753,"title":"s3: Support empty \"directory\" keys","user":{"login":"felixbuenemann","id":909587,"avatar_url":"https://avatars.githubusercontent.com/u/909587?v=3","gravatar_id":"","url":"https://api.github.com/users/felixbuenemann","html_url":"https://github.com/felixbuenemann","followers_url":"https://api.github.com/users/felixbuenemann/followers","following_url":"https://api.github.com/users/felixbuenemann/following{/other_user}","gists_url":"https://api.github.com/users/felixbuenemann/gists{/gist_id}","starred_url":"https://api.github.com/users/felixbuenemann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/felixbuenemann/subscriptions","organizations_url":"https://api.github.com/users/felixbuenemann/orgs","repos_url":"https://api.github.com/users/felixbuenemann/repos","events_url":"https://api.github.com/users/felixbuenemann/events{/privacy}","received_events_url":"https://api.github.com/users/felixbuenemann/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ncw/rclone/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-09-30T20:38:07Z","updated_at":"2016-10-01T16:02:48Z","closed_at":null,"body":"Many tools like the the AWS S3 Console, S3 network drives / fuse filesystems and various other tools create \"directories\" on S3 by creating zero byte keys that end with a slash to emulate empty directories.\r\n\r\nrclone currently can't handle these keys, because it tries to treat them like a file instead of a directory and fails with an error if a local directory with the same name exist.\r\n\r\nInstead rclone should detect these keys and treat them like a directory.\r\n\r\nThe keys are easy to detect, because they always end with a slash and have a content-length of 0 bytes.\r\n\r\nHere's an example key listed with the python aws cli:\r\n```\r\naws s3api head-object --bucket bito-pim-images --key 'images/WebShop/ita_IT/'\r\n{\r\n    \"AcceptRanges\": \"bytes\",\r\n    \"ContentType\": \"binary/octet-stream\",\r\n    \"LastModified\": \"Fri, 22 Jul 2016 12:32:11 GMT\",\r\n    \"ContentLength\": 0,\r\n    \"ETag\": \"\\\"d41d8cd98f00b204e9800998ecf8427e\\\"\",\r\n    \"Metadata\": {}\r\n}\r\n```\r\n\r\n> What is your rclone version (eg output from `rclone -V`)\r\n\r\nv1.33-DEV\r\n\r\n> Which OS you are using and how many bits (eg Windows 7, 64 bit)\r\n\r\nmacOS 10.12.1\r\n\r\n> Which cloud storage system are you using? (eg Google Drive)\r\n\r\ns3\r\n\r\n> The command you were trying to run (eg `rclone copy /tmp remote:tmp`)\r\n\r\n`rclone sync --size-only --delete-after remote:tmp /tmp`\r\n\r\n> A log from the command with the `-v` flag (eg output from `rclone -v copy /tmp remote:tmp`)\r\n\r\n```\r\n2016/09/30 22:09:55 Local file system at /Volumes/EXT4TBA/images: Waiting for transfers to finish\r\n2016/09/30 22:09:55 images/WebShop/ita_IT/: Failed to copy: open /Volumes/EXT4TBA/images/WebShop/ita_IT: is a directory\r\n2016/09/30 22:09:55 images/WebShop/nld_BE/: Failed to copy: open /Volumes/EXT4TBA/images/WebShop/nld_BE: is a directory\r\n2016/09/30 22:09:55 images/WebShop/nld_NL/: Failed to copy: open /Volumes/EXT4TBA/images/WebShop/nld_NL: is a directory\r\n2016/09/30 22:09:55 images/WebShop/pol_PL/: Failed to copy: open /Volumes/EXT4TBA/images/WebShop/pol_PL: is a directory\r\n2016/09/30 22:09:56 images/WebShop/slo_SK/: Failed to copy: open /Volumes/EXT4TBA/images/WebShop/slo_SK: is a directory\r\n2016/09/30 22:09:56 images/WebShop/spa_ES/: Failed to copy: open /Volumes/EXT4TBA/images/WebShop/spa_ES: is a directory\r\n2016/09/30 22:09:56 images/WebShop/ukrainian_ukraine/: Failed to copy: open /Volumes/EXT4TBA/images/WebShop/ukrainian_ukraine: is a directory\r\n2016/09/30 22:09:56 Local file system at /Volumes/EXT4TBA/images: not deleting files as there were IO errors\r\n2016/09/30 22:09:56 Attempt 3/3 failed with 30 errors and: not deleting files as there were IO errors\r\n2016/09/30 22:09:56 Failed to sync: not deleting files as there were IO errors\r\n```\r\n\r\nIt's very easy to create a test key by going into the AWS S3 Console and clicking on \"Create Folder\" to create an empty directory."},"comment":{"url":"https://api.github.com/repos/ncw/rclone/issues/comments/250920222","html_url":"https://github.com/ncw/rclone/issues/753#issuecomment-250920222","issue_url":"https://api.github.com/repos/ncw/rclone/issues/753","id":250920222,"user":{"login":"ncw","id":536803,"avatar_url":"https://avatars.githubusercontent.com/u/536803?v=3","gravatar_id":"","url":"https://api.github.com/users/ncw","html_url":"https://github.com/ncw","followers_url":"https://api.github.com/users/ncw/followers","following_url":"https://api.github.com/users/ncw/following{/other_user}","gists_url":"https://api.github.com/users/ncw/gists{/gist_id}","starred_url":"https://api.github.com/users/ncw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ncw/subscriptions","organizations_url":"https://api.github.com/users/ncw/orgs","repos_url":"https://api.github.com/users/ncw/repos","events_url":"https://api.github.com/users/ncw/events{/privacy}","received_events_url":"https://api.github.com/users/ncw/received_events","type":"User","site_admin":false},"created_at":"2016-10-01T16:02:48Z","updated_at":"2016-10-01T16:02:48Z","body":"Nice idea thanks.\r\n\r\nIf you had a list of applications which use a 0 sized `dir\\` object to mean a directory that would be very helpful.\r\n \r\nNote this is related to #14 which discussed s3fs\r\n"}}
{"actor":{"display_login":"ncw"}}