{"action":"closed","issue":{"url":"https://api.github.com/repos/ncw/rclone/issues/769","repository_url":"https://api.github.com/repos/ncw/rclone","labels_url":"https://api.github.com/repos/ncw/rclone/issues/769/labels{/name}","comments_url":"https://api.github.com/repos/ncw/rclone/issues/769/comments","events_url":"https://api.github.com/repos/ncw/rclone/issues/769/events","html_url":"https://github.com/ncw/rclone/issues/769","id":181290919,"number":769,"title":"rclone iterates through a large bucket to operate on a single object","user":{"login":"DanielDent","id":291138,"avatar_url":"https://avatars.githubusercontent.com/u/291138?v=3","gravatar_id":"","url":"https://api.github.com/users/DanielDent","html_url":"https://github.com/DanielDent","followers_url":"https://api.github.com/users/DanielDent/followers","following_url":"https://api.github.com/users/DanielDent/following{/other_user}","gists_url":"https://api.github.com/users/DanielDent/gists{/gist_id}","starred_url":"https://api.github.com/users/DanielDent/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DanielDent/subscriptions","organizations_url":"https://api.github.com/users/DanielDent/orgs","repos_url":"https://api.github.com/users/DanielDent/repos","events_url":"https://api.github.com/users/DanielDent/events{/privacy}","received_events_url":"https://api.github.com/users/DanielDent/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ncw/rclone/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/ncw/rclone/milestones/21","html_url":"https://github.com/ncw/rclone/milestones/v1.34","labels_url":"https://api.github.com/repos/ncw/rclone/milestones/21/labels","id":1890781,"number":21,"title":"v1.34","description":"","creator":{"login":"ncw","id":536803,"avatar_url":"https://avatars.githubusercontent.com/u/536803?v=3","gravatar_id":"","url":"https://api.github.com/users/ncw","html_url":"https://github.com/ncw","followers_url":"https://api.github.com/users/ncw/followers","following_url":"https://api.github.com/users/ncw/following{/other_user}","gists_url":"https://api.github.com/users/ncw/gists{/gist_id}","starred_url":"https://api.github.com/users/ncw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ncw/subscriptions","organizations_url":"https://api.github.com/users/ncw/orgs","repos_url":"https://api.github.com/users/ncw/repos","events_url":"https://api.github.com/users/ncw/events{/privacy}","received_events_url":"https://api.github.com/users/ncw/received_events","type":"User","site_admin":false},"open_issues":24,"closed_issues":23,"state":"open","created_at":"2016-07-18T22:13:42Z","updated_at":"2016-10-07T14:47:10Z","due_on":null,"closed_at":null},"comments":3,"created_at":"2016-10-05T23:29:13Z","updated_at":"2016-10-07T14:47:10Z","closed_at":"2016-10-07T14:47:10Z","body":"I have a swift bucket at Hubic with many thousands of objects in it, without any folder hierarchy. Copying a single object from this bucket is quite slow. It looks like rclone first iterates through the entirety of a 'folder' before performing an operation on a single object.\r\n\r\n> What is your rclone version (eg output from `rclone -V`)\r\nrclone v1.33\r\n> Which OS you are using and how many bits (eg Windows 7, 64 bit)\r\nDebian Linux 64 bit\r\n> Which cloud storage system are you using? (eg Google Drive)\r\nHubic\r\n> The command you were trying to run (eg `rclone copy /tmp remote:tmp`)\r\nrclone copy -v hubic:GA/afile .\r\n> A log from the command with the `-v` flag (eg output from `rclone -v copy /tmp remote:tmp`)\r\nrclone copy -v hubic:GA/afile .\r\n2016/10/05 16:24:19 rclone: Version \"v1.33\" starting with parameters [\"rclone\" \"copy\" \"-v\" \"hubic:GA/afile\" \".\"]\r\n2016/10/05 16:24:20 Hubic: Got swift credentials (expiry 2016-10-07 00:52:33 +0200 +0200 in 23h28m12.913624349s)\r\n2016/10/05 16:24:21 Local file system at /tmp/asd/9: Modify window is 1ns\r\n2016/10/05 16:24:21 existingfile1: Excluded from sync (and deletion)\r\n2016/10/05 16:24:21 existingfile2: Excluded from sync (and deletion)\r\n2016/10/05 16:24:21 existingfile3: Excluded from sync (and deletion)\r\n... and so-on ...\r\n\r\nDeleting a single object is also quite slow:\r\n\r\nrclone -v delete hubic:GA/thisisfile\r\n2016/10/05 16:19:21 rclone: Version \"v1.33\" starting with parameters [\"rclone\" \"-v\" \"delete\" \"hubic:GA/thisisfile\"]\r\n2016/10/05 16:19:22 Hubic: Got swift credentials (expiry 2016-10-07 00:52:33 +0200 +0200 in 23h33m10.930861748s)\r\n2016/10/05 16:19:22 Hubic Swift container GA: Modify window is 1ns\r\n2016/10/05 16:19:22 Waiting for deletions to finish\r\n2016/10/05 16:20:22 \r\nTransferred:      0 Bytes (0 Bytes/s)\r\nErrors:                 0\r\nChecks:                 0\r\nTransferred:            0\r\nElapsed time:        1m1s\r\n\r\n2016/10/05 16:21:22 \r\nTransferred:      0 Bytes (0 Bytes/s)\r\nErrors:                 0\r\nChecks:                 0\r\nTransferred:            0\r\nElapsed time:        2m1s\r\n\r\n2016/10/05 16:22:22 \r\nTransferred:      0 Bytes (0 Bytes/s)\r\nErrors:                 0\r\nChecks:                 0\r\nTransferred:            0\r\nElapsed time:        3m1s\r\n\r\n2016/10/05 16:23:22 \r\nTransferred:      0 Bytes (0 Bytes/s)\r\nErrors:                 0\r\nChecks:                 0\r\nTransferred:            0\r\nElapsed time:        4m1s\r\n\r\n2016/10/05 16:24:22 \r\nTransferred:      0 Bytes (0 Bytes/s)\r\nErrors:                 0\r\nChecks:                 0\r\nTransferred:            0\r\nElapsed time:        5m1s\r\n\r\n2016/10/05 16:25:22 \r\nTransferred:      0 Bytes (0 Bytes/s)\r\nErrors:                 0\r\nChecks:                 0\r\nTransferred:            0\r\nElapsed time:        6m1s\r\n\r\n2016/10/05 16:26:22 \r\nTransferred:      0 Bytes (0 Bytes/s)\r\nErrors:                 0\r\nChecks:                 0\r\nTransferred:            0\r\nElapsed time:        7m1s\r\n\r\n2016/10/05 16:27:14 thisisfile: Deleted\r\n2016/10/05 16:27:14 \r\nTransferred:      0 Bytes (0 Bytes/s)\r\nErrors:                 0\r\nChecks:                 1\r\nTransferred:            0\r\nElapsed time:     7m52.6s\r\n2016/10/05 16:27:14 Go routines at exit 2\r\n"}}
{"actor":{"display_login":"ncw"}}