{"action":"created","issue":{"url":"https://api.github.com/repos/duplicati/duplicati/issues/1924","repository_url":"https://api.github.com/repos/duplicati/duplicati","labels_url":"https://api.github.com/repos/duplicati/duplicati/issues/1924/labels{/name}","comments_url":"https://api.github.com/repos/duplicati/duplicati/issues/1924/comments","events_url":"https://api.github.com/repos/duplicati/duplicati/issues/1924/events","html_url":"https://github.com/duplicati/duplicati/issues/1924","id":175351387,"number":1924,"title":"--throttle-upload option not working with Amazon Cloud Drive","user":{"login":"automorphism88","id":19507282,"avatar_url":"https://avatars.githubusercontent.com/u/19507282?v=3","gravatar_id":"","url":"https://api.github.com/users/automorphism88","html_url":"https://github.com/automorphism88","followers_url":"https://api.github.com/users/automorphism88/followers","following_url":"https://api.github.com/users/automorphism88/following{/other_user}","gists_url":"https://api.github.com/users/automorphism88/gists{/gist_id}","starred_url":"https://api.github.com/users/automorphism88/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/automorphism88/subscriptions","organizations_url":"https://api.github.com/users/automorphism88/orgs","repos_url":"https://api.github.com/users/automorphism88/repos","events_url":"https://api.github.com/users/automorphism88/events{/privacy}","received_events_url":"https://api.github.com/users/automorphism88/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-09-06T21:03:33Z","updated_at":"2016-10-02T06:33:35Z","closed_at":null,"body":"<!-- This is a bug report template. By following the instructions below and filling out the sections with your information, you will help the developers to get all the necessary data to fix your issue.\r\nYou can also preview your report before submitting it. You may remove sections that aren't relevant to your particular case.\r\n\r\nLet's begin with a checklist: replace the empty checkbox [ ] below with a checked one [x] if you already searched for duplicate bugs -->\r\n\r\nI have:\r\n- [x] searched open and closed issues for duplicates\r\n\r\nIt looks like this was a problem in the past, but was fixed in early 2015. I've only just started using duplicati and have only tried using it with Amazon Cloud Drive, so I'm not sure if the problem I'm experiencing is at all related to the past bug, or whether it's unique to the ACD backend or affects duplicati generally.\r\n\r\n----------------------------------------\r\n\r\n### Version info\r\n<!-- please replace the examples with your info -->\r\n**Duplicati Version:** 2.0.1.20_canary_2016-09-01\r\n**Operating System:** Linux Mint 18 sarah\r\n**Backend:** Amazon Cloud Drive\r\n\r\n### Bug description\r\nThe --throttle-upload command line option appears to have no effect, although I have confirmed that it is being set correctly by observing the \"input options\" output from duplicati-cli. When starting a backup with duplicati-cli, the output correctly displays the value that was entered for --throttle-upload, but it has no effect on the actual upload speed as observed with nethogs, or by observing from the timestamps in the log how long it's taking to upload each 50 MB dblock file.\r\n\r\nI've enabled --debug-output but haven't gotten any actual error messages in the log to report, at least not related to the bandwidth throttling. The failure of --throttle-upload to work can only be detected by using other software (in my case, nethogs) to monitor how much upload bandwidth the Duplicati.CommandLine process is actually using, or by calculating the upload speed from the log timestamps. I have tried several different values for --throttle-upload (100, 200, 300, and 400, tried both with and without \"kb\" after the number) and no matter what, duplicati maxes out my available upload bandwidth (4Mbps, with duplicati observed to be uploading at roughly 500KB/s via both nethogs and log timestamps, no matter what I do with --throttle-upload).\r\n\r\nI do see the put operations fail relatively often, generally either a GATEWAY_TIMEOUT error or sometimes \"Error writing request\" or \"The socket has been shut down\" but I think these errors are linked to the bandwidth throttling issue, since they seem to fit with the general flakiness of my connection when an application is trying to use as much upload bandwidth as it can. The same thing happens to my connection when I allow other applications to use as much upload bandwidth as they can, which is why I generally throttle them so that they can't use more than 80-90% of what's available.\r\n\r\nI've also noticed that after a file upload is complete, there is sometimes a substantial delay between when the command line output (and the log, observed with tail -f) say it is starting the next upload and when I actually see it start to use the bandwidth in nethogs. Usually the delay only lasts a second or two, but I've seen it sit there for over a minute using zero bandwidth, before the next file actually starts. \r\n\r\nThe one pattern I have noticed with these put operation errors is that they seem to occur more often when I've just started duplicati, or when I'm trying to do something else that uses my internet connection at the same time, whereas if I leave it running overnight with nothing else happening, the errors become much less frequent, and it might successfully upload a dozen or two files without getting any errors (or, judging by the timestamps, any delays of more than a couple of seconds in between files). This seems consistent with the hypothesis that the lack of upload throttling is responsible for the put errors, by causing the connection generally to become flaky, as does the fact that uploads to ACD go much more smoothly using rclone or acdcli with upload throttling enabled - it isn't perfect and I might have an occasional upload fail, but nowhere near the frequency I'm seeing with duplicati.\r\n\r\n### Steps to reproduce\r\n- use --throttle-upload to set a limit on upload bandwidth with duplicati-cli uploading to Amazon Cloud Drive\r\n- use nethogs or other network monitoring software to observe the actual bandwidth used by the \"Duplicati.CommandLine\" process while uploading a file\r\n- observe from timestamps in log that files are being uploaded faster than the throttle-upload value should allow\r\n- observe that further reducing the throttle-upload value makes no difference to either of these measurements of actual bandwidth being used\r\n\r\n**Actual result:** Duplicati uses all available bandwidth to upload files to Amazon Cloud Drive, regardless of the --throttle-upload parameter\r\n**Expected result:** Bandwidth usage should stay reasonably close to the maximum set with --throttle-upload, like it does for me when I use rclone to upload to ACD\r\n\r\n### debug log\r\n<!-- posting a debug log helps the developers to fix your issue -->\r\nSample output verifying that throttle-upload was set (but not obeyed):\r\n\r\n```\r\nInput command: backup\r\nInput arguments: \r\n        amzcd://[removed]\r\n        /storage/btrfs\r\n\r\nInput options: \r\nbackup-name: 1\r\ndbpath: /storage/shared/duplicati/1.sqlite\r\nencryption-module: gpg\r\ncompression-module: zip\r\ndblock-size: 50mb\r\npassphrase: [removed]\r\nzip-compression-level: 9\r\nlog-level: Information\r\nlog-file: /storage/shared/duplicati/1.log\r\nlog-retention: 1M\r\ndisable-module: console-password-input\r\nverbose: true\r\nasynchronous-upload-limit: 16\r\nasynchronous-upload-folder: /storage/tmp/duplicati\r\ntempdir: /storage/tmp/duplicati\r\nupload-verification-file: true\r\ndebug-output: true\r\ndebug-retry-errors: true\r\nthrottle-upload: 200kb\r\n\r\n```\r\n\r\nSample put error from log:\r\n\r\n```\r\n2016-09-06 13:16:08Z - Information: Backend event: Put - Started: duplicati-b2cdc0cb6e56549cbb22ad110569b58b4.dblock.zip.gpg (50.07 MB)\r\n2016-09-06 13:19:03Z - Warning: Operation Put with file duplicati-b2cdc0cb6e56549cbb22ad110569b58b4.dblock.zip.gpg attempt 1 of 5 failed with message: GATEWAY_TIMEOUT\r\nSystem.Net.WebException: GATEWAY_TIMEOUT\r\n  at Duplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper.GetResponseOrStream () <0x401d6840 + 0x0010b> in <filename unknown>:0 \r\n  at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse () <0x4028cd50 + 0x00163> in <filename unknown>:0 \r\n  at Duplicati.Library.JSONWebHelper.GetResponse (Duplicati.Library.Utility.AsyncHttpRequest req, System.Object requestdata) <0x4028c570 + 0x003fb> in <filename unknown>:0 \r\n\r\n2016-09-06 13:19:03Z - Information: Backend event: Put - Retrying: duplicati-b2cdc0cb6e56549cbb22ad110569b58b4.dblock.zip.gpg (50.07 MB)\r\n2016-09-06 13:19:03Z - Information: Backend event: Put - Rename: duplicati-b2cdc0cb6e56549cbb22ad110569b58b4.dblock.zip.gpg (50.07 MB)\r\n2016-09-06 13:19:03Z - Information: Backend event: Put - Rename: duplicati-b3973dbc5cf3f4e30be37f675ad963577.dblock.zip.gpg (50.07 MB)\r\n2016-09-06 13:19:03Z - Information: Renaming \"duplicati-b2cdc0cb6e56549cbb22ad110569b58b4.dblock.zip.gpg\" to \"duplicati-b3973dbc5cf3f4e30be37f675ad963577.dblock.zip.gpg\"\r\n2016-09-06 13:19:13Z - Information: Backend event: Put - Started: duplicati-b3973dbc5cf3f4e30be37f675ad963577.dblock.zip.gpg (50.07 MB)\r\n2016-09-06 13:21:13Z - Information: Backend event: Put - Completed: duplicati-b3973dbc5cf3f4e30be37f675ad963577.dblock.zip.gpg (50.07 MB)\r\n\r\n```\r\nYou can see from the log that the file uploaded successfully on the second try after a GATEWAY_TIMEOUT on the first try, and took exactly two minutes, which reflects a pause of about twenty seconds between when the log says the file started uploading and when it actually started. When things are running more smoothly, the delay with zero bandwidth being used between when duplicity reports that it's starting the upload and when it actually starts will only last a couple of seconds, resulting in a log that looks more like the following, where a dblock file of the same size took 1:42 to finish:\r\n\r\n```\r\n2016-09-06 13:00:35Z - Information: Backend event: Put - Started: duplicati-befb1323f91dd45849acc0bf375cc5d91.dblock.zip.gpg (50.07 MB)\r\n2016-09-06 13:02:17Z - Information: Backend event: Put - Completed: duplicati-befb1323f91dd45849acc0bf375cc5d91.dblock.zip.gpg (50.07 MB)\r\n```\r\n\r\nFrom watching nethogs, I know this difference is due to the upload taking longer to start, and not to any difference in speed once it does. Also, during both of these uploads, throttle-upload was set to \"200kb\" as shown above (though I see the exact same results if it's set to something smaller or larger, or not set at all), while the timestamps above are evidence that the files are actually being uploaded at 500 KB/s, as I would expect them to if duplicati is maxing out my connection's 4Mbps upload capacity. At 500 KB/s, it should take 100 seconds, or 1:40, to upload a 50MB file, plus the second or two delay observed with nethogs with bandwidth use at zero before the upload actually starts, which is exactly what these timestamps show.\r\n\r\n<bountysource-plugin>\r\n---\r\nWant to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/37632246-throttle-upload-option-not-working-with-amazon-cloud-drive?utm_campaign=plugin&utm_content=tracker%2F4870652&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F4870652&utm_medium=issues&utm_source=github).\r\n</bountysource-plugin>"},"comment":{"url":"https://api.github.com/repos/duplicati/duplicati/issues/comments/250956509","html_url":"https://github.com/duplicati/duplicati/issues/1924#issuecomment-250956509","issue_url":"https://api.github.com/repos/duplicati/duplicati/issues/1924","id":250956509,"user":{"login":"renestach","id":8270279,"avatar_url":"https://avatars.githubusercontent.com/u/8270279?v=3","gravatar_id":"","url":"https://api.github.com/users/renestach","html_url":"https://github.com/renestach","followers_url":"https://api.github.com/users/renestach/followers","following_url":"https://api.github.com/users/renestach/following{/other_user}","gists_url":"https://api.github.com/users/renestach/gists{/gist_id}","starred_url":"https://api.github.com/users/renestach/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/renestach/subscriptions","organizations_url":"https://api.github.com/users/renestach/orgs","repos_url":"https://api.github.com/users/renestach/repos","events_url":"https://api.github.com/users/renestach/events{/privacy}","received_events_url":"https://api.github.com/users/renestach/received_events","type":"User","site_admin":false},"created_at":"2016-10-02T06:33:35Z","updated_at":"2016-10-02T06:33:35Z","body":"We know that the throttle is broken at the moment. https://github.com/duplicati/duplicati/issues/1729\r\n\r\n"}}
{"actor":{"display_login":"renestach"}}