{"action":"created","issue":{"url":"https://api.github.com/repos/gilbertchen/duplicacy/issues/334","repository_url":"https://api.github.com/repos/gilbertchen/duplicacy","labels_url":"https://api.github.com/repos/gilbertchen/duplicacy/issues/334/labels{/name}","comments_url":"https://api.github.com/repos/gilbertchen/duplicacy/issues/334/comments","events_url":"https://api.github.com/repos/gilbertchen/duplicacy/issues/334/events","html_url":"https://github.com/gilbertchen/duplicacy/issues/334","id":291139915,"node_id":"MDU6SXNzdWUyOTExMzk5MTU=","number":334,"title":"Inefficient Storage after Moving or Renaming Files?","user":{"login":"jonreeves","id":32250664,"node_id":"MDQ6VXNlcjMyMjUwNjY0","avatar_url":"https://avatars0.githubusercontent.com/u/32250664?v=4","gravatar_id":"","url":"https://api.github.com/users/jonreeves","html_url":"https://github.com/jonreeves","followers_url":"https://api.github.com/users/jonreeves/followers","following_url":"https://api.github.com/users/jonreeves/following{/other_user}","gists_url":"https://api.github.com/users/jonreeves/gists{/gist_id}","starred_url":"https://api.github.com/users/jonreeves/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jonreeves/subscriptions","organizations_url":"https://api.github.com/users/jonreeves/orgs","repos_url":"https://api.github.com/users/jonreeves/repos","events_url":"https://api.github.com/users/jonreeves/events{/privacy}","received_events_url":"https://api.github.com/users/jonreeves/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":76,"created_at":"2018-01-24T09:34:47Z","updated_at":"2018-06-18T09:33:08Z","closed_at":null,"author_association":"NONE","body":"I think this may be related to issue #248 .\r\n\r\nI noticed that backups of my active folders were growing in size significantly where I expected them to change very little. The best example I can give is my \"Photos\" folder/snapshot...\r\n\r\nI regularly download my sd cards to a temporary named project folder under a \"Backlog\" subfolder. It can be days or months before I work on these images, but usually this will involve Renaming ALL the files and separating them into subfolders by Scene/Location or Purpose (print, portfolio, etc...). The project folder gets renamed too and the whole thing is then moved from out of \"Backlog\" to \"Catalogued\". None of the content of any of the files have physically changed during all this, file hashes should be the same.\r\n\r\nThe renaming and moving alone appears to be enough to make the backup double in size. Any image edits in theory shouldn't have an impact as typically the original file is untouched... The good thing about modern photo processing is edits are non destructive to the original image file. Instead and XML sidecar file is saved along side the image file with meta about the edits applied.\r\n\r\nI'm yet to test the impact of making image edits but I suspect it may make things worse because each file gets another file added after it, and it seemed like file order made a difference to how the rolling hash was calculated.\r\n```\r\nImage1.raw\r\nImage2.raw\r\n...\r\n```\r\nBecomes...\r\n```\r\nImage1.raw\r\nImage1.xmp\r\nImage2.raw\r\nImage2.xmp\r\n```\r\nThis should be perfect for an incremental backup system, but it seems like duplicacy struggles under these circumstances. I figured the `-hash` option might help, but it didn't seem to.\r\n\r\nAm I doing something wrong or missing an option?\r\n\r\nIs this a bug, or just a design decision?\r\n\r\nIs there any possible way to improve this?\r\n\r\nAlthough the above example may sound unique I find this happens in almost all my everyday folders. Design projects, website coding projects, Files and Folders are just often reorganized.\r\n\r\nI'm guessing the only way to reclaim this space would be to prune the older snapshots where the files were named differently?"},"comment":{"url":"https://api.github.com/repos/gilbertchen/duplicacy/issues/comments/397996978","html_url":"https://github.com/gilbertchen/duplicacy/issues/334#issuecomment-397996978","issue_url":"https://api.github.com/repos/gilbertchen/duplicacy/issues/334","id":397996978,"node_id":"MDEyOklzc3VlQ29tbWVudDM5Nzk5Njk3OA==","user":{"login":"kairisku","id":3451743,"node_id":"MDQ6VXNlcjM0NTE3NDM=","avatar_url":"https://avatars1.githubusercontent.com/u/3451743?v=4","gravatar_id":"","url":"https://api.github.com/users/kairisku","html_url":"https://github.com/kairisku","followers_url":"https://api.github.com/users/kairisku/followers","following_url":"https://api.github.com/users/kairisku/following{/other_user}","gists_url":"https://api.github.com/users/kairisku/gists{/gist_id}","starred_url":"https://api.github.com/users/kairisku/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kairisku/subscriptions","organizations_url":"https://api.github.com/users/kairisku/orgs","repos_url":"https://api.github.com/users/kairisku/repos","events_url":"https://api.github.com/users/kairisku/events{/privacy}","received_events_url":"https://api.github.com/users/kairisku/received_events","type":"User","site_admin":false},"created_at":"2018-06-18T09:33:08Z","updated_at":"2018-06-18T09:33:08Z","author_association":"NONE","body":"No I have not tried that, nor do I understand how that would make any difference. Chunks are immutable, once a chunk is created it will not be changed or touched in any way. During an incremental backup, only new and modified files are processed while any old unmodified files are skipped (unless you use the -hash option), so all new and modified files will end up creating new chunks. I see no benefit from changing the order in which files are bundled.\r\n\r\nI am not familiar with the worst case behaviour regarding git repositories you are referring to.\r\n\r\nMoving and copying of small files are already handled in my branch which causes them to never be included in new chunks if the identical data is already present in an existing chunk. I really should get around to making a PR of my improvements..  :)"}}
{"actor":{"display_login":"kairisku"}}