{"action":"created","issue":{"url":"https://api.github.com/repos/ncw/rclone/issues/2632","repository_url":"https://api.github.com/repos/ncw/rclone","labels_url":"https://api.github.com/repos/ncw/rclone/issues/2632/labels{/name}","comments_url":"https://api.github.com/repos/ncw/rclone/issues/2632/comments","events_url":"https://api.github.com/repos/ncw/rclone/issues/2632/events","html_url":"https://github.com/ncw/rclone/pull/2632","id":367866858,"node_id":"MDExOlB1bGxSZXF1ZXN0MjIxMTc2NjQy","number":2632,"title":"fstests: add TestFsPutChunked","user":{"login":"B4dM4n","id":448169,"node_id":"MDQ6VXNlcjQ0ODE2OQ==","avatar_url":"https://avatars0.githubusercontent.com/u/448169?v=4","gravatar_id":"","url":"https://api.github.com/users/B4dM4n","html_url":"https://github.com/B4dM4n","followers_url":"https://api.github.com/users/B4dM4n/followers","following_url":"https://api.github.com/users/B4dM4n/following{/other_user}","gists_url":"https://api.github.com/users/B4dM4n/gists{/gist_id}","starred_url":"https://api.github.com/users/B4dM4n/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/B4dM4n/subscriptions","organizations_url":"https://api.github.com/users/B4dM4n/orgs","repos_url":"https://api.github.com/users/B4dM4n/repos","events_url":"https://api.github.com/users/B4dM4n/events{/privacy}","received_events_url":"https://api.github.com/users/B4dM4n/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-10-08T16:31:31Z","updated_at":"2018-10-12T16:33:29Z","closed_at":"2018-10-11T13:47:59Z","author_association":"COLLABORATOR","pull_request":{"url":"https://api.github.com/repos/ncw/rclone/pulls/2632","html_url":"https://github.com/ncw/rclone/pull/2632","diff_url":"https://github.com/ncw/rclone/pull/2632.diff","patch_url":"https://github.com/ncw/rclone/pull/2632.patch"},"body":"#### What is the purpose of this change?\r\n\r\nThis adds a test for chunked uploads to the integration tests.\r\nEach remote now has a `ChunkedUploadConfig` field in the `fstests.Opt` struct, to specify the required settings for the test. In addition, each remote that supports chunked uploads has to implement the `fstest.SetUploadChunkSizer` interface, which allows to change the value at runtime. Remotes that do not implement the `fstest.SetUploadChunkSizer` interface will not be tested, since the test would be the same as `TestFsPut` with bigger files.\r\n\r\nThese are the currently minimum chunk sizes and the total number of bytes uploaded to the remote during the test:\r\n\r\nremote | min chunk size | test upload size\r\n------ | -------------- | ----------------\r\nb2 | 5M | 190M\r\ns3 | 5M | 190M\r\ndrive | 256K | 13M\r\nonedrive | 320K | 25M\r\n\\* | 1 | 28M\r\n\r\nThese numbers could probably be reduced a bit by removing some similar test cases. There should be no file download involved, as `file.Check` only compares the hash and modtime.\r\n\r\nThe output of the following tests can be viewed here: [chunked.log](https://github.com/ncw/rclone/files/2456792/chunked.log)\r\n\r\n`go test -v ./backend/drive ./backend/onedrive ./backend/s3 ./backend/local -test.run '^TestIntegration/((Setup|Init|Purge|Finalise)|FsPutChunked)$' | tee chunked.log`\r\n\r\n#### Was the change discussed in an issue or in the forum before?\r\n\r\n#2549\r\n\r\n#### Checklist\r\n\r\n- [x] I have read the [contribution guidelines](https://github.com/ncw/rclone/blob/master/CONTRIBUTING.md#submitting-a-pull-request).\r\n- [x] I have added tests for all changes in this PR if appropriate.\r\n- [x] I have added documentation for the changes if appropriate.\r\n- [x] All commit messages are in [house style](https://github.com/ncw/rclone/blob/master/CONTRIBUTING.md#commit-messages).\r\n- [x] I'm done, this Pull Request is ready for review :-)\r\n"},"comment":{"url":"https://api.github.com/repos/ncw/rclone/issues/comments/429385175","html_url":"https://github.com/ncw/rclone/pull/2632#issuecomment-429385175","issue_url":"https://api.github.com/repos/ncw/rclone/issues/2632","id":429385175,"node_id":"MDEyOklzc3VlQ29tbWVudDQyOTM4NTE3NQ==","user":{"login":"B4dM4n","id":448169,"node_id":"MDQ6VXNlcjQ0ODE2OQ==","avatar_url":"https://avatars0.githubusercontent.com/u/448169?v=4","gravatar_id":"","url":"https://api.github.com/users/B4dM4n","html_url":"https://github.com/B4dM4n","followers_url":"https://api.github.com/users/B4dM4n/followers","following_url":"https://api.github.com/users/B4dM4n/following{/other_user}","gists_url":"https://api.github.com/users/B4dM4n/gists{/gist_id}","starred_url":"https://api.github.com/users/B4dM4n/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/B4dM4n/subscriptions","organizations_url":"https://api.github.com/users/B4dM4n/orgs","repos_url":"https://api.github.com/users/B4dM4n/repos","events_url":"https://api.github.com/users/B4dM4n/events{/privacy}","received_events_url":"https://api.github.com/users/B4dM4n/received_events","type":"User","site_admin":false},"created_at":"2018-10-12T16:33:29Z","updated_at":"2018-10-12T16:33:29Z","author_association":"COLLABORATOR","body":"I can't tell what is wrong there. All the tests that fail are `chunkSize = fileSize`. A quick glance at the azureblob code has not revealed any problems."}}
{"actor":{"display_login":"B4dM4n"}}