{"action":"created","issue":{"url":"https://api.github.com/repos/borgbackup/borg/issues/1766","repository_url":"https://api.github.com/repos/borgbackup/borg","labels_url":"https://api.github.com/repos/borgbackup/borg/issues/1766/labels{/name}","comments_url":"https://api.github.com/repos/borgbackup/borg/issues/1766/comments","events_url":"https://api.github.com/repos/borgbackup/borg/issues/1766/events","html_url":"https://github.com/borgbackup/borg/issues/1766","id":185347637,"number":1766,"title":"reduce cache size needs (txn.active + files)","user":{"login":"martin21","id":114577,"avatar_url":"https://avatars.githubusercontent.com/u/114577?v=3","gravatar_id":"","url":"https://api.github.com/users/martin21","html_url":"https://github.com/martin21","followers_url":"https://api.github.com/users/martin21/followers","following_url":"https://api.github.com/users/martin21/following{/other_user}","gists_url":"https://api.github.com/users/martin21/gists{/gist_id}","starred_url":"https://api.github.com/users/martin21/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martin21/subscriptions","organizations_url":"https://api.github.com/users/martin21/orgs","repos_url":"https://api.github.com/users/martin21/repos","events_url":"https://api.github.com/users/martin21/events{/privacy}","received_events_url":"https://api.github.com/users/martin21/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-10-26T09:56:59Z","updated_at":"2016-10-26T12:55:48Z","closed_at":null,"body":"Related to \"significantly reduce cache space needs\" #235. But here it isn´t large chunks.archive.d, but rather txn.active + files. This is a cache for /home which has an excessive number of files, but its not that large:\r\n\r\nmerkaba:~> du -sh /home\r\n164G    /home\r\n\r\nmerkaba:~> find /home | wc -l\r\n2865746\r\nmerkaba:~> find /home -type f | wc -l\r\n2797574\r\nmerkaba:~> find /home -type d | wc -l\r\n64319\r\nmerkaba:~> find /home/martin/.local/share/local-mail | wc -l\r\n2128451\r\nmerkaba:~> find /home/martin/.local/share/local-mail -type f | wc -l\r\n2127093\r\nmerkaba:~> find /home/martin/.local/share/local-mail -type d | wc -l\r\n1358\r\n\r\nToday a borgbackup run into exception due to insufficient free space (see issue \"limit cache space needs to avoid out of space exceptions\" #1765)\r\n\r\nAnd really:\r\n\r\nmerkaba:~/.cache/borg> du -sch * | sort -rh | head -4\r\n5,4G    insgesamt\r\n4,9G    673e17ea929[…]\r\n420M    809d8850c0bd[…]\r\n44M     723a580c358b[…]\r\n\r\nmerkaba:~/.cache/borg/673e17ea92925012e32db8d4d92c6ec18a2a08f490d431b068fe6bdaae073737> LANG=C du -sch * | sort -rh | head -5\r\n4.9G    total\r\n2.3G    txn.active\r\n2.1G    files\r\n296M    chunks.archive.d\r\n253M    chunks"},"comment":{"url":"https://api.github.com/repos/borgbackup/borg/issues/comments/256338434","html_url":"https://github.com/borgbackup/borg/issues/1766#issuecomment-256338434","issue_url":"https://api.github.com/repos/borgbackup/borg/issues/1766","id":256338434,"user":{"login":"martin21","id":114577,"avatar_url":"https://avatars.githubusercontent.com/u/114577?v=3","gravatar_id":"","url":"https://api.github.com/users/martin21","html_url":"https://github.com/martin21","followers_url":"https://api.github.com/users/martin21/followers","following_url":"https://api.github.com/users/martin21/following{/other_user}","gists_url":"https://api.github.com/users/martin21/gists{/gist_id}","starred_url":"https://api.github.com/users/martin21/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martin21/subscriptions","organizations_url":"https://api.github.com/users/martin21/orgs","repos_url":"https://api.github.com/users/martin21/repos","events_url":"https://api.github.com/users/martin21/events{/privacy}","received_events_url":"https://api.github.com/users/martin21/received_events","type":"User","site_admin":false},"created_at":"2016-10-26T12:55:48Z","updated_at":"2016-10-26T12:55:48Z","body":"Thomas, I switched off files cache and got:\r\n\r\n2016-10-26T12:20:33+02:00 Sichere /home.\r\n2016-10-26T13:19:11+02:00 /home gesichert.\r\n\r\nThats not too bad.\r\n\r\nLast backups with enabled cache where:\r\n\r\n2016-10-26T10:33:32+02:00 Sichere /home.\r\n2016-10-26T11:06:14+02:00 /home gesichert.\r\n\r\n2016-10-17T12:36:56+02:00 Sichere /home.\r\n2016-10-17T13:38:13+02:00 /home gesichert.\r\n\r\n2016-10-14T10:02:19+02:00 Sichere /home.\r\n2016-10-14T11:28:09+02:00 /home gesichert.\r\n\r\nWhen I look at the variations I find it difficult to say what performance impact will be. I am using borgbackup only from office / work related networks with 1 GBit link to backup VM and today ping times of about 1 to 1,5 ms. Backup is to slow with my current DSL uplink tomorrow.\r\n\r\n> The txn.active directory is used for transaction processing (while the transaction is active,  removed after transaction is completed).\r\n\r\nSo theoretically I could make a symlink from txn.active to /tmp/txn.active to store this directory in tmpfs. This would add another about 2 GiB to RAM usage, in addition to the about 5,4 GiB RSIZE borgbackup used, but on laptop with 16 GiB that would be bearable.\r\n"}}
{"actor":{"display_login":"martin21"}}