Page 1 of 1

Synology RAID Recovery

Posted: Tue Dec 29, 2015 1:35 pm
by jwaldeck
An ongoing project that's become an unfortunate hobby...

I had a Synology NAS crash many months ago. It was during the expansion of a 12 disk RAID6 (hybrid, using LVM2) to 14 disks. Not sure what happened other than some sort of software fail that caused a kernel panic and hang. The drives are still fine.

I made a mistake of letting a Synology tech into the system to recover it, and they did some destruction in trying to be helpful by reverting the RAID(s) to their last known good state, wiping out things like where it was in the reshape when it failed. So I'm working off of var/log kind of messages to recover the structure, etc.

One huge problem is that I've had no luck having R-Studio validate the RAID integrity. Despite that, it walks the ext4 file system directory structure (inode tables) until I suspect the reshape point and the resulting "directory entries" go crazy. My theory was that the drives configured as the new RAID structure would look fine up to a particular stripe, at which the parity would fail because the stripes beyond that were in the original RAID configuration (12 drives). So then it's a matter of tying the old structure with the new structure and the system would be recovered.

But R-Studio reports RAID integrity bad throughout--or at least crazy "Good/Bad/Zero" sectors with no patterns to speak of.

So I went and took small images of the existing, working Synology raid (15 partitions, RAID6), with absolutely known parameters. Fed those into R-Studio, and the results are very similar.

The md device is typical: 64kB chunks, RAID6, Left Synchronous, 15 devices. Each device data section starts at sector 2048 (1MB). Order is consistent with the RAID superblocks. I've tried reassembling using device data offsets of 2048 sectors, and I've created regions that start at 1MB and used those instead of the whole partition, and the results are the same.

When assembled, R-Studio does not recognize that an ext4 superblock sits in the first block, either. I have no idea what it thinks is wrong with it, unless it parses it deep enough to realize the GDT includes pointers beyond the size of the image. I can walk things manually (I need to find an ext template for the hex editor!) and the system looks fine.

Any thoughts?