TrueDisc Error Correction for Disc Burning?

An anonymous reader asks: "Macintouch has a link to a new piece of software — TrueDisc — which claims to make data burned to record-able discs more reliable. More specifically it uses interleaved redundant cells to rebuild data should part of the disc be scratched. On the developer's blog they say they plan to create an open-source implementation of the TrueDisc system, now that it is not going to be included in the Blu-ray/HD-DVD standards. Have any of you used this software before, and what alternatives are already available?"

Comments

Re:Is this enough?

[loraksus]

Your average blank disk out there is pretty poor quality, if anything, this lets you burn on crap disks with at least the chance of reading the data a year or more down the road.
Par does take a while to generate the recovery files though...

Re:Or use par2

[dgatwood]

I tried to use it to send my parents a copy of their problematic hard drive that I scraped for them, spread across a handful of DVDs. Turns out that at least on Mac OS X, I couldn't find a PAR decoder implementation that worked correctly if the total data size was over 2GB (or maybe 4GB). It was mistakenly using a 32-bit value for some of its internal math instead of a 64-bit, thinking (incorrectly) that Mac OS X's seek only supported a 32-bit offset. I sent the UnRarX folks a hackish workaround patch to the open source (or was it GPL) tool that they used under the hood. Not sure if they've actually fixed it or not. It also took an eternity to process such a large volume.

Anyway, IMHO, anything that intends to solve the fragility problem of optical media must do ALL of the following:

• Be no worse than half the performance of direct writes. That means that multi-gigabyte Reed-Solomon codes are right out. That also means a very intelligent checksumming mechanism that avoids unnecessary write/seek/settle cycles of the hard disk that will almost inevitably be providing the backing storage for data prior to writing it onto the disk. I'm not sure what the access pattern should look like, but I can think of a good number of examples of what it shouldn't look like.... :-D
• Be BELOW the filesystem level, not above it (on a per-file basis). If a file gets corrupted, per-file protection is okay, but such a protection scheme still leaves lots of very critical metadata without any backup. What happens when a block in the root directory structure fails? You're thoroughly screwed if you just have per-file redundancy.
• Provide backups of data that are sufficiently distributed both in rotational angle and in distance from the edge. Scratches by clumsiness usually occur radially, outwards from the center or inwards from the edge. They may also be arc-shaped. Either way, those are unlikely to destroy a huge amount of data. Scratches due to a piece of sand in your DVD player, spinning the DVD in a roughly manufactured case, etc. happen in a circular pattern, and are, IMHO, one of the major reasons why the mechanisms built into most of the optical disc formats aren't sufficient.... Putting in some n ECC bytes every n*k bytes of data only solves that problem for very large values of n and k so that several entire rotations of the media can be obliterated without losing any data. Unfortunately, that sort of strategy then can quickly turn into a "multi-gigabyte Reed-Solomon code" problem again. Again, on this one, I don't know the solution (though I have some vague notions). I mainly just know a lot of examples of what the solution isn't. :-)
• Take into account that the last few percent of storage on optical media are notoriously unreliable on cheap media and limit the total capacity appropriately to avoid relying on that space for anything important.

After the failure rate I saw trying to do that data transfer (90% immediate verify failure on discs that were over about 98% full, no burn failures ever on discs that were under 90% full), coupled with the failure rate I saw with Retrospect Remote (almost 100% of DVDs stopping with only 15% of the disc used because the craptastic software didn't support burn-safe and wasn't smart enough to pause the burn while it waited for data over the network), at this point, I trust optical media about as far as I can throw it... though maybe not AOL CDs---I can throw them pretty far.... It would be really cool if this sort of tech works and were implemented broadly, as it might make optical media actually useful instead of it just being a nice pretty round disc to slide label-side down against a plasterboard wall in office pachinko.

Just my $0.0195, adjusted for consumer products deflation.