Data Recovery from ReiserFS RAID Array?

Ruatha asks: "We've recently had a problem with a ReiserFS RAID-5 array - two of the disks failed and, of course, some of the people using the array didn't have backups of their data...Ontrack have returned the disks because they can do nothing with them due to the FS we used on the array. Does anyone know of a company that can deal with data recovery from a ReiserFS RAID-5 array?"

Comments

Linux FS recovery techniques

[0x0d0a]

Could they have recovered from ext3?

Very likely, yes. Ext2 recovery techniqes are well known, documented, and tools exist (if rudimentary) for recovering files from it. I believe that this translates well to ext3.

Also, if you want someone to recover the data and you're willing to drop some money on it...I strongly suspect that there are people on the reiserfs team that would take on a recovery job quite happily. No one knows reiser better!

That being said, there is currently no good, easy-to-use, powerful recovery tool for Linux filesystems, rather depressingly. Now, you *could* argue that this is because the filesystems are so great, but even so...

You probably know this, but..

[arcade]

First of all, since _two_ disks got screwed at the same time, you've lost the "normal" chance of getting the data back. RAID-5 ensures that if ONE disk fails, you can get the data back due to the parity-stuff - but not if two disks fail. You probably know this.

So, what needs to be done, is to get one of the disks back online again. That *should* be possible, and if nothing really really bad happened, should, i think, in theory, be as easy as getting a lab to pull the disk-platters out and put a new motor / new electronics on them. I'm not sure about this though ;)

Preferrably it could be done by the disk-manufacturer.

You could also check out an excellent company in Norway called IBAS. Check out http://www.ibas.com/america/index.html for their american office. They are really excellent at data reconstruction.

quick fix

[austad]

This may or may not work, however, I've successfully recovered data about 10 or 12 times using this method.

Find a working drive of the same model, take the electronics board off of it and swap it onto the bad drive. Typically when I have a drive fail, it's the electronics, not the mechanical portion of it. So far, this has worked every time for me, one was a Quantum Fireball, and the rest were all Seagate SCSI drives (some FCAL and some ultrawide).

If you had two disks fail at the same time, chances are it's the electronics. Once you recover the data, I would take a serious look at your RAID controller and possibly replace it. I had a bad RAID controller that kept frying drives, and once I replaced it I didn't have anymore problems.

Re:quick fix

[ChadN]

I've had drives which were fixed by the same method; apparently heat is a major cause of this. I would also suggest trying to operate the drives in a cold room (or with some suitable extra cooling), to see if they'll work long enough to recover the data.

Re:Related question::

[Dashslot]

Yes. I once managed to trash my /home partition. (Ironically, I was upgrading from ext2 to ext3 and I ran mke2fs -j instead of tune2fs -j!)

To recover the files, all I did was..

1) Use dd to make a copy of the partition

2) Consult /etc/magic for the headers of the file types I wanted.

3) Used a binary grep utitlity to find the starting point of those files

3) Used dd to grab an arbitary sized chunk of the partition

4) Found out that after exactly 48k of each file there was a 4k block of junk

5) used dd to assemble a new file with the 4k of junk missing

6) Found what end tag each file type used and remove the junk after it (with dd of course). Alternatively, I could have opened and resaved the file in whatever app that could handle it, and that would have trimmed the junk as well.

It took about 2 hours total to figure out how to do it, despite having no previous experience with filesystem debugging/etc

Of course, I wasn't using RAID, but I would bet that your chances of recovering ext2/3 stuff are much much higher than reiser, whatever the underlying hardware.

Data Recovery efforts...

[ComputerSlicer23]

Assuming, it's really, really necessary, there are ways of gleaming data off of a drive that has died. You can send your drives off to said company and for a thousand dollars a disk they take it into a clean room, disassmeble the entire platter assembly, physically prepare the platters so they are as good as can be, then put them under their own super duper head assembly.

Assuming you can find one of them, use google for gods sake. Get them to read the drive linearly. At this point, they should be able to put each drive onto new drives for you. Build a raidtab that matches the new drives they give you, do raidstart /dev/mdXX, do a fsck.reiserfs on it. Mount the thing. Poof like magic you have data again.

I've heard of several people recovering Ph.d and masters disertations this way. It's why the DoD has such strict standards about destruction and disposal of a hard drive. It's very difficult to delete data from a drive so it can't be recovered. It's can just get out of control expensive to recover the data.

If it's worth $5-10K, and you can deal without having it for several days, this might actually work... If the company bitches, just tell that it's about what a good tape setup would have cost if they had bought one.

The two guys who said try stripping the electronics are pretty brave SOB's, but that might be worth trying if you can't spring for some experts to deal with that for you.

Kirby

Re:Data Recovery efforts...

[duffbeer703]

Remember that IRIX and AIX use XFS and JFS, respectively.

If you sent corrupted volumes to a shop that can recover commerical Unix disks, they should be able to recover data on a Linux box. JFS=JFS.

Re:Freeze it

[Komarosu]

I may get shouted at for slander (we've been "warned" by our suppliers and fujitsu) but... We've had a few (more like 100+) fujitsu drives fail over the last few months with "known manafacturing problems", and its becoming more of a headache for us and our supplyer. We've worked out that if you leave the hard-drive to cool for a good few days then you will get a final 15-20mins out of it, enough for using Norton Ghost or DD to image the drive. Note: just look for Fujitsu MPG or MPAT 10gb drives...

data recover specialists

[martin]

For me the guys are www.vogon-data-recovery.com

There are others, but I always seem to see these guys at the forefront...

just my 2 pence..

Re:What happened?

[duffbeer703]

That is not a smart strategy.

Alot of storage is very finicky -- and odd low-level problems emerge when you mix disk manufacturers or even disk models in some cases.

When Sun 5200 fibre arrays first came out, if you got a batch of Sun-branded drives that were manufactured by different vendors, you would have all sorts of odd locking issues and other goodies.

A good strategy for reliable storage:

- Don't be cheap. You get what you pay for.
- Plan for Disk-to-Tape (or DVD) backup, actually test restores regularly-
- Use well-supported, stable versions of filesystems
- Don't buy the latest and greatest unless there is a business reason to do so