{"action":"created","issue":{"url":"https://api.github.com/repos/QubesOS/qubes-issues/issues/4217","repository_url":"https://api.github.com/repos/QubesOS/qubes-issues","labels_url":"https://api.github.com/repos/QubesOS/qubes-issues/issues/4217/labels{/name}","comments_url":"https://api.github.com/repos/QubesOS/qubes-issues/issues/4217/comments","events_url":"https://api.github.com/repos/QubesOS/qubes-issues/issues/4217/events","html_url":"https://github.com/QubesOS/qubes-issues/issues/4217","id":350908923,"node_id":"MDU6SXNzdWUzNTA5MDg5MjM=","number":4217,"title":"Move .cache to a separate volume for efficient backups","user":{"login":"tasket","id":14337937,"node_id":"MDQ6VXNlcjE0MzM3OTM3","avatar_url":"https://avatars3.githubusercontent.com/u/14337937?v=4","gravatar_id":"","url":"https://api.github.com/users/tasket","html_url":"https://github.com/tasket","followers_url":"https://api.github.com/users/tasket/followers","following_url":"https://api.github.com/users/tasket/following{/other_user}","gists_url":"https://api.github.com/users/tasket/gists{/gist_id}","starred_url":"https://api.github.com/users/tasket/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tasket/subscriptions","organizations_url":"https://api.github.com/users/tasket/orgs","repos_url":"https://api.github.com/users/tasket/repos","events_url":"https://api.github.com/users/tasket/events{/privacy}","received_events_url":"https://api.github.com/users/tasket/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-08-15T18:09:25Z","updated_at":"2018-08-16T05:31:21Z","closed_at":null,"author_association":"NONE","body":"### Qubes OS version:\r\n<!-- (e.g., `R3.2`)\r\n     You can get it from the dom0 terminal with the command\r\n         `cat /etc/qubes-release`\r\n     Type below this line. -->\r\nR4.0\r\n\r\n\r\n\r\n\r\n<!-- Type above this line. -->\r\n### Affected component(s): \r\n<!-- Type below this line. -->\r\ndomU storage\r\nany dom0-based backup tools like `qvm-backup`\r\n\r\n\r\n<!-- Type above this line. -->\r\n### General notes:\r\n<!-- Type below this line. -->\r\nA large but seldom-discussed problem we've had with backups is that application cache contents are backed-up along with the normally saved/wanted data in guest VMs. By definition cache is a type of data is only retained between sessions for efficiency, yet it generates a lot of churn, bandwidth and space usage in backup data sets. This is a significant burden on user time and resources.\r\n\r\nAnd with some use cases generating cache churn reaching into the gigabyte(s) range for individual VMs, this problem largely negates the development of _efficient_ incremental backups.\r\n\r\n### Proposal\r\n\r\nAlthough traditional backup tools commonly avoid cache contents, unfortunately the current practice of storing application cache in *private means this data cannot be avoided by a dom0-based backup tool. A bit of added structure in the virtual storage layer is required in the form of a cache volume.\r\n\r\n### Cache volume characteristics\r\n\r\n* Administered in identical fashion to *private volumes for VM create, start, stop, clone, etc.\r\n* Mounted at VM startup, with .cache in /rw/home linking or binding to it\r\n* Cache volumes are only \"volatile\" to certain tools, e.g. backups\r\n* Cache volumes can also be deleted in system emergencies, such as pool out-of-space situations\r\n* May also serve as an info security perk\r\n\r\nTreating /home/*/.cache as a separate class of data in this way with its own separate virtual volume would dispel a shadow hanging over Qubes backups: It would immediately improve `qvm-backup` and enable developments in efficient incremental backups, for example ones based on dom0 [COW metadata](https://github.com/tasket/sparsebak/blob/master/sparsebak.py).\r\n\r\nIn terms of implementation difficulty, I think this rates as low-moderate. The main difficulty will be in making sure *private and *cache are snapshotted atomically together.\r\n\r\n<!-- Type above this line. -->\r\n---\r\n<!--------------------------->\r\n### Related issues:\r\n<!-- Type below this line. -->\r\nIssue #858 \r\n\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/QubesOS/qubes-issues/issues/comments/413430595","html_url":"https://github.com/QubesOS/qubes-issues/issues/4217#issuecomment-413430595","issue_url":"https://api.github.com/repos/QubesOS/qubes-issues/issues/4217","id":413430595,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMzQzMDU5NQ==","user":{"login":"t4777sd","id":2925965,"node_id":"MDQ6VXNlcjI5MjU5NjU=","avatar_url":"https://avatars0.githubusercontent.com/u/2925965?v=4","gravatar_id":"","url":"https://api.github.com/users/t4777sd","html_url":"https://github.com/t4777sd","followers_url":"https://api.github.com/users/t4777sd/followers","following_url":"https://api.github.com/users/t4777sd/following{/other_user}","gists_url":"https://api.github.com/users/t4777sd/gists{/gist_id}","starred_url":"https://api.github.com/users/t4777sd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/t4777sd/subscriptions","organizations_url":"https://api.github.com/users/t4777sd/orgs","repos_url":"https://api.github.com/users/t4777sd/repos","events_url":"https://api.github.com/users/t4777sd/events{/privacy}","received_events_url":"https://api.github.com/users/t4777sd/received_events","type":"User","site_admin":false},"created_at":"2018-08-16T05:31:21Z","updated_at":"2018-08-16T05:31:21Z","author_association":"NONE","body":"Borg scans files, but it can also backup from a block device. So, the .cache solution you propose would be pretty great because then borg users can gain the benefits of dom0-like security in their backups by backing up the entire block device in their BackupVM without having to mount it which avoids a whole class of security vulns.\r\n\r\nI hope it can make it into 4.1\r\n\r\nBtw, I looked at your script and you might want to consider borg as the backup engine. For no-cost you will gain incremental, encryption, deduplication, purging, and remote backup capabilities. To backup a block device you could use something like this:\r\n\r\ndd if=/dev/xxx | borg create repo::disk -"}}
{"actor":{"display_login":"t4777sd"}}