{"action":"closed","issue":{"url":"https://api.github.com/repos/DanielDent/git-annex-remote-rclone/issues/1","repository_url":"https://api.github.com/repos/DanielDent/git-annex-remote-rclone","labels_url":"https://api.github.com/repos/DanielDent/git-annex-remote-rclone/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/DanielDent/git-annex-remote-rclone/issues/1/comments","events_url":"https://api.github.com/repos/DanielDent/git-annex-remote-rclone/issues/1/events","html_url":"https://github.com/DanielDent/git-annex-remote-rclone/issues/1","id":151430745,"number":1,"title":"documented chunk size is too large","user":{"login":"joeyh","id":16392,"avatar_url":"https://avatars.githubusercontent.com/u/16392?v=3","gravatar_id":"","url":"https://api.github.com/users/joeyh","html_url":"https://github.com/joeyh","followers_url":"https://api.github.com/users/joeyh/followers","following_url":"https://api.github.com/users/joeyh/following{/other_user}","gists_url":"https://api.github.com/users/joeyh/gists{/gist_id}","starred_url":"https://api.github.com/users/joeyh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joeyh/subscriptions","organizations_url":"https://api.github.com/users/joeyh/orgs","repos_url":"https://api.github.com/users/joeyh/repos","events_url":"https://api.github.com/users/joeyh/events{/privacy}","received_events_url":"https://api.github.com/users/joeyh/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-04-27T16:32:59Z","updated_at":"2016-04-27T22:56:24Z","closed_at":"2016-04-27T22:56:24Z","body":"It's not really a good idea to set chunk size to 1 gb, because git-annex currently has to buffer a whole chunk of a file in memory. So, that could make git-annex use 1 gb of memory or more.\r\n\r\nhttp://git-annex.branchable.com/chunking/ documents this, and suggests something in the 1 MB range for chunks. Partly due to memory size and partly because this minimizes the amount of redundant data transferred when resuming an upload/download of chunks.\r\n\r\nIf rclone supports resuming partial uploads and downloads of large files, it might make sense to pick a larger chunk size, since the latter concern wouldn't matter. The memory usage would still make 1 gb too large for chunks."}}