Slashdot Mirror


Fault Tolerant Archive Solutions?

Bob Washburne asks: "Does anyone know of a file system or storage protocol which allows you to recover a file even when sections of the media have become corrupt? This would be for archival storage - tapes, CR-R's, etc. - rather than on-line. I have been doing a lot of digital retoration/preservation work. Digitising home recordings from the 50's, photos from the 1890's, etc. and cleaning them up. I now have several gig of files - and it will continue to grow - representing hundreds of hours of work and I'm starting to get nervous about losing it to wear or bad media."

"There are several solutions for on-line storage; RAID, UPS, and frequent backups. As I fill a CD-R I make several copies of it and send them to reletives who live out of state. So I am fairly well protected against local disaster.

But what happens when the CD-R itself becomes degraded - possibly from scratches or bad lamination - and cannot be read by the normal file system? Murphy's Law would guarentee that all the backup CD-s were from the same bad batch or were lost, etc. So I am left with a CD that is 90% good, but that ugly 10% prevents me from getting my file.

I remember studying about N-dimentional parity and Hamming codes in Comp Sci class, so I know that it is possible to store a file with signifficant error correction capabilities. But has any such scheme been actually implemented?

I would expect any such scheme to include the ability to adjust the degree of recoverability (size vs. robustness) and to be able to span volumes. Since most physical damage is contiguous, you would hope that the storage would be non-contiguous. And you would think that this would either represent a unique file system or a custom raw storage methodology useable only by the storage aplication.

Thanks for your insights."

0 of 18 comments (clear)

No comments match the current filter.