{"action":"closed","issue":{"url":"https://api.github.com/repos/borgbackup/borg/issues/4218","repository_url":"https://api.github.com/repos/borgbackup/borg","labels_url":"https://api.github.com/repos/borgbackup/borg/issues/4218/labels{/name}","comments_url":"https://api.github.com/repos/borgbackup/borg/issues/4218/comments","events_url":"https://api.github.com/repos/borgbackup/borg/issues/4218/events","html_url":"https://github.com/borgbackup/borg/issues/4218","id":391703001,"node_id":"MDU6SXNzdWUzOTE3MDMwMDE=","number":4218,"title":"\"Repository has no manifest.\" - very likely after NFS mount lost...","user":{"login":"pmorch","id":159292,"node_id":"MDQ6VXNlcjE1OTI5Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/159292?v=4","gravatar_id":"","url":"https://api.github.com/users/pmorch","html_url":"https://github.com/pmorch","followers_url":"https://api.github.com/users/pmorch/followers","following_url":"https://api.github.com/users/pmorch/following{/other_user}","gists_url":"https://api.github.com/users/pmorch/gists{/gist_id}","starred_url":"https://api.github.com/users/pmorch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pmorch/subscriptions","organizations_url":"https://api.github.com/users/pmorch/orgs","repos_url":"https://api.github.com/users/pmorch/repos","events_url":"https://api.github.com/users/pmorch/events{/privacy}","received_events_url":"https://api.github.com/users/pmorch/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-12-17T13:04:31Z","updated_at":"2018-12-17T19:13:11Z","closed_at":"2018-12-17T19:13:10Z","author_association":"NONE","body":"## Have you checked borgbackup docs, FAQ, and open Github issues?\r\n\r\nYes\r\n\r\n## Is this a BUG / ISSUE report or a QUESTION?\r\n\r\nBUG\r\n\r\n## System information. For client/server mode post info for both machines.\r\n\r\n#### Your borg version (borg -V).\r\nborg 1.1.5\r\n\r\n#### Operating system (distribution) and version.\r\nUbuntu 18.04\r\n\r\n#### Hardware / network configuration, and filesystems used.\r\nPretty standard amd64 desktop machine with an NFS mounted ~/store (that, yes, has problems)\r\n\r\n#### How much data is handled by borg?\r\n\r\n```\r\n> du -sh mozilla-borg\r\n770M\tmozilla-borg\r\n```\r\n\r\n#### Full borg commandline that lead to the problem (leave away excludes and passwords)\r\n\r\n```\r\n> borg list mozilla-borg\r\nRepository has no manifest.\r\n```\r\n\r\n## Describe the problem you're observing.\r\n\r\nI have an NFS mounted ~/store folder. Sometimes I loose access to the NFS server for reasons unrelated to borg. When it occurs, it occurs suddenly after which the NFS server needs a hard restart to come back online.\r\n\r\nI'm now seeing the `Repository has no manifest.` error. The NFS server is currently working fine, but it is possible the problem was introduced in the past when there were NFS problems.\r\n\r\n#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.\r\n\r\nNo\r\n\r\n#### Include any warning/errors/backtraces from the system logs\r\nI have a cron.d file:\r\n```\r\n> cat /etc/cron.d/backup-mozilla \r\nMAILTO=XXXXX@gmail.com\r\n\r\nSHELL=/bin/sh\r\nPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin\r\n\r\n0 *    * * *   peter /etc/backupstore/cronic bash -c 'mountpoint /home/peter/store && borg create --stats /home/peter/store/backup/mozilla-borg::mozilla-{now} /home/peter/.mozilla >> /var/log/borg-mozilla.log 2>&1'\r\n30 3    * * *   peter /etc/backupstore/cronic bash -c 'mountpoint /home/peter/store && borg prune -P mozilla- -p --keep-hourly 8 --keep-daily 3 --keep-weekly 3 --keep-monthly 3 --keep-yearly 3 /home/peter/store/backup/mozilla-borg 2>1'\r\n```\r\nThat should create new backup every hour. (And has done so fine in the past).\r\n\r\nAnd from this, the last lines of /var/log/borg-mozilla.log are:\r\n```\r\nArchive name: mozilla-2018-12-16T20:00:01\r\nArchive fingerprint: d662d5a301d0d96cefaf76450507720cbd3b2e2dae0c4d94cb4b40279f3654af\r\nTime (start): Sun, 2018-12-16 20:00:01\r\nTime (end):   Sun, 2018-12-16 20:00:03\r\nDuration: 2.09 seconds\r\nNumber of files: 4074\r\nUtilization of max. archive size: 0%\r\n------------------------------------------------------------------------------\r\n                       Original size      Compressed size    Deduplicated size\r\nThis archive:              439.09 MB            255.00 MB              9.41 MB\r\nAll archives:               14.25 GB              8.23 GB              1.09 GB\r\n\r\n                       Unique chunks         Total chunks\r\nChunk index:                    6047               132918\r\n------------------------------------------------------------------------------\r\n------------------------------------------------------------------------------\r\nArchive name: mozilla-2018-12-16T21:00:01\r\nArchive fingerprint: b1315830d52f66c444b8dafce175a27c35086f183787e95984729a87cc07ae75\r\nTime (start): Sun, 2018-12-16 21:00:01\r\nTime (end):   Sun, 2018-12-16 21:00:03\r\nDuration: 2.07 seconds\r\nNumber of files: 4074\r\nUtilization of max. archive size: 0%\r\n------------------------------------------------------------------------------\r\n                       Original size      Compressed size    Deduplicated size\r\nThis archive:              439.09 MB            255.00 MB              9.49 MB\r\nAll archives:               14.69 GB              8.49 GB              1.10 GB\r\n\r\n                       Unique chunks         Total chunks\r\nChunk index:                    6075               137085\r\n------------------------------------------------------------------------------\r\nRepository has no manifest.\r\nRepository has no manifest.\r\nRepository has no manifest.\r\n<serveral more of such lines - one every hour I presume>\r\n```\r\n\r\nSo the last one at 21:00 ended without problems emitted from borg and no more entries after that.  At no point did borg emit any error or warning messages, because then they'd be in the log output above.\r\n\r\n[an old comment](https://github.com/borgbackup/borg/issues/2132#issuecomment-278759197) in #2132 suggests to run `borg check --repair`. When I do that (after keeping a copy of the original folder), I get:\r\n```\r\n> borg check --repair mozilla-borg\r\n'check --repair' is an experimental feature that might result in data loss.\r\nType 'YES' if you understand this and want to continue: YES\r\nRepository manifest not found!\r\n(many, many lines like:)\r\nhome/peter/.mozilla/firefox/vxcggtho.dev-edition-default/xulstore.json: New missing file chunk detected (Byte 0-437). Replacing with all-zero chunk.\r\n42 orphaned objects found!\r\nArchive consistency check complete, problems found.\r\n```\r\n\r\nAfter which I see some of the backups but many are also missing. At least all the backups between 2018-12-11 23:00:03 and 2018-12-16 21:00:01 :-(\r\n\r\n```\r\n> borg list ~/store/backup/mozilla-borg/\r\nmozilla-2018-11-25T23:00:01          Sun, 2018-11-25 23:00:02 [d8d6db0f2d79aa906fe15f435d65c3289cd92e2ccac353847164751607b5c2dd]\r\nmozilla-2018-11-30T23:00:02          Fri, 2018-11-30 23:00:02 [c4b36d6b09fdecf3d51e7ea705fe0c79a76638f4c44619598ff2013cf288221e]\r\nmozilla-2018-12-02T23:00:02          Sun, 2018-12-02 23:00:02 [c65756514ef98006edd820af713e7c77fe18ccedafbfd27e967cdfa1bc45f855]\r\nmozilla-2018-12-08T17:00:01          Sat, 2018-12-08 17:00:01 [bb77c48d42778212f664373dd4d38ea04f2b153cf4cf7ccee5be3182afbacb9b]\r\nmozilla-2018-12-11T23:00:02          Tue, 2018-12-11 23:00:03 [7d62cb404301a7c6fb7f614850a5a25e56e7f098c24261e7e56f81850c1f8255]\r\n```\r\n\r\nBut uhm, I used to store things once an hour. They're all lost. And what about the  `New missing file chunk detected (Byte 0-437). Replacing with all-zero chunk.` errors.\r\n\r\nHmm. I don't like it. Something is pretty bad here.\r\n\r\n\r\n\r\n"}}
{"actor":{"display_login":"pmorch"}}