{"action":"created","issue":{"url":"https://api.github.com/repos/ncw/rclone/issues/472","repository_url":"https://api.github.com/repos/ncw/rclone","labels_url":"https://api.github.com/repos/ncw/rclone/issues/472/labels{/name}","comments_url":"https://api.github.com/repos/ncw/rclone/issues/472/comments","events_url":"https://api.github.com/repos/ncw/rclone/issues/472/events","html_url":"https://github.com/ncw/rclone/issues/472","id":153374048,"number":472,"title":"Looks like amazon is sick of us. I downloaded a 15gigabyte rar file perfectly fine via rclone. Then a bit later I wanted to use rclone again and got this.","user":{"login":"left1000","id":16432525,"avatar_url":"https://avatars.githubusercontent.com/u/16432525?v=3","gravatar_id":"","url":"https://api.github.com/users/left1000","html_url":"https://github.com/left1000","followers_url":"https://api.github.com/users/left1000/followers","following_url":"https://api.github.com/users/left1000/following{/other_user}","gists_url":"https://api.github.com/users/left1000/gists{/gist_id}","starred_url":"https://api.github.com/users/left1000/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/left1000/subscriptions","organizations_url":"https://api.github.com/users/left1000/orgs","repos_url":"https://api.github.com/users/left1000/repos","events_url":"https://api.github.com/users/left1000/events{/privacy}","received_events_url":"https://api.github.com/users/left1000/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-05-06T04:04:04Z","updated_at":"2016-05-19T20:05:37Z","closed_at":null,"body":"Looks like amazon is sick of us. I downloaded a 15gigabyte rar file perfectly fine via rclone. Then a bit later I wanted to use it again and got this rate limited error.\r\nI was going to use the website to download the file, but it said I could only download files up to 5GB so I just used rclone to download a copy of the file I wanted. This never threw up any flags before.\r\n\r\nNow no matter what I do this is the only feedback rclone gives.\r\n\r\nI did upload around 3tb of data last week. This is a very odd time for amazon to get upset with me though. Would've made sense for them to have done it sooner (like last week). I didn't bother to log verbose the actual download though so this is all I have to show. \r\n\r\nI've no idea if amazon sent rclone any errors about my download or if there would've been anyway to avoid it. I should also mention my system clock was bugged during the download by about 4hours which resulted in the download taking negative 3.7hours or so.\r\n\r\nHas anyone ever gotten this feedback from amazon via rclone before? is it new? what in particular causes it? downloads of large files? My recent upload?\r\n\r\nUsing the website I can use amazon cloud drive perfectly normally to browse my files and download/upload files normally at full speed (only limited by the normal website limitations). It seems like amazon has specifically blocked ONLY rclone! I'm assuming(hopefully) in a day or two this will go away but I am curious what caused it and how amazon knows to block rclone but not it's own website. If I had logged the download -verbose would any message about this have popped up telling me what had happened or how long it would last? I cannot say because I didn't generate that log. Without -v on no messages about this appeared.\r\n\r\n\r\nBelow is the entire log I was able to generate from this command \r\n\r\nrclone --log-file=logtext.txt -v ls remote:C\r\n\r\n\r\n2016/05/05 23:55:51 pacer: Rate limited, sleeping for 947.77941ms (1 consecutive low level retries)\r\n2016/05/05 23:55:51 pacer: low level retry 1/10\r\n2016/05/05 23:55:51 pacer: Rate limited, sleeping for 1.082153551s (2 consecutive low level retries)\r\n2016/05/05 23:55:51 pacer: low level retry 2/10\r\n2016/05/05 23:55:52 pacer: Rate limited, sleeping for 3.666145821s (3 consecutive low level retries)\r\n2016/05/05 23:55:52 pacer: low level retry 3/10\r\n2016/05/05 23:55:53 pacer: Rate limited, sleeping for 2.235010051s (4 consecutive low level retries)\r\n2016/05/05 23:55:53 pacer: low level retry 4/10\r\n2016/05/05 23:55:56 pacer: Rate limited, sleeping for 287.113937ms (5 consecutive low level retries)\r\n2016/05/05 23:55:56 pacer: low level retry 5/10\r\n2016/05/05 23:55:59 pacer: Rate limited, sleeping for 20.54916732s (6 consecutive low level retries)\r\n2016/05/05 23:55:59 pacer: low level retry 6/10\r\n2016/05/05 23:55:59 pacer: Rate limited, sleeping for 23.632969758s (7 consecutive low level retries)\r\n2016/05/05 23:55:59 pacer: low level retry 7/10\r\n2016/05/05 23:56:19 pacer: Rate limited, sleeping for 1m57.331776148s (8 consecutive low level retries)\r\n2016/05/05 23:56:19 pacer: low level retry 8/10\r\n2016/05/05 23:56:43 pacer: Rate limited, sleeping for 29.183117216s (9 consecutive low level retries)\r\n2016/05/05 23:56:43 pacer: low level retry 9/10\r\n2016/05/05 23:58:40 pacer: Rate limited, sleeping for 1m28.480279449s (9 consecutive low level retries)\r\n2016/05/05 23:58:40 pacer: low level retry 10/10\r\n2016/05/05 23:58:40 Failed to create file system for \"remote:C\": Failed to get endpoints: HTTP code 401: \"401 Unauthorized\", reponse body: {\"message\":\"Token has expired\"}\r\n\r\n\r\nEDIT: the rate limited period is already over and rclone is working again (although my transfer speed is 90% lower than it usually is). This might render my issue pointless entirely. if you think so we can always just delete it. otherwise maybe leave this up as a pointless rant that someone can google up someday if the same thing happens to them and it can reassure them? It is still odd that while rclone was blocked the website still worked. As far as I could tell the website ran/runs at full speed while rclone is now in slowmo.\r\n\r\nEdit again: just a few minutes later and now my speed is up to 80% lower than normal, I guess ACD has implemented a series of throttling measures that are highly dynamic? (edit again: now it's up to full speed for me which is 10megabytes per second, looks like I'm entirely un-throttled now).\r\n"},"comment":{"url":"https://api.github.com/repos/ncw/rclone/issues/comments/220436903","html_url":"https://github.com/ncw/rclone/issues/472#issuecomment-220436903","issue_url":"https://api.github.com/repos/ncw/rclone/issues/472","id":220436903,"user":{"login":"left1000","id":16432525,"avatar_url":"https://avatars.githubusercontent.com/u/16432525?v=3","gravatar_id":"","url":"https://api.github.com/users/left1000","html_url":"https://github.com/left1000","followers_url":"https://api.github.com/users/left1000/followers","following_url":"https://api.github.com/users/left1000/following{/other_user}","gists_url":"https://api.github.com/users/left1000/gists{/gist_id}","starred_url":"https://api.github.com/users/left1000/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/left1000/subscriptions","organizations_url":"https://api.github.com/users/left1000/orgs","repos_url":"https://api.github.com/users/left1000/repos","events_url":"https://api.github.com/users/left1000/events{/privacy}","received_events_url":"https://api.github.com/users/left1000/received_events","type":"User","site_admin":false},"created_at":"2016-05-19T20:05:37Z","updated_at":"2016-05-19T20:05:37Z","body":"I'm back in now (at full speed). I guess logging the time of these events is meaningless, but I do stand by my suggestion to change the retry timer (or allow users to customize it with a flag)."}}