Slashdot Mirror
TrueDisc Error Correction for Disc Burning?
An anonymous reader asks: "Macintouch has a link to a new piece of software — TrueDisc — which claims to make data burned to record-able discs more reliable. More specifically it uses interleaved redundant cells to rebuild data should part of the disc be scratched. On the developer's blog they say they plan to create an open-source implementation of the TrueDisc system, now that it is not going to be included in the Blu-ray/HD-DVD standards. Have any of you used this software before, and what alternatives are already available?"
Okay, so you store redundant (maybe error-correcting) stuff on top of the existing file system (or in otherwise unused sectors), so you can recover your files if the original sectors succumb to bit rot. For fifty bucks.
Why not just store the files twice? It would be a whole lot cheaper...
Any sufficiently advanced technology is insufficiently documented.
Isn't there already an open source program which does this called DVDisaster?
There's already good parity software available. Parchive will create redundant data that can be burned on the same disc or a separate one. You can create up to 100% redundant data so even if the original disc is lost you can completely restore the files. The software is free and open source. The windows version is called quickpar.
Or use PAR2? It's free.
Tony
http://www.bruguier.com
what alternatives are already available?
RAR compression has an option for redundancy. You set what % you want to be able to recover if it becomes corrupted.
Libertarian Leaning Political Discussion Forum.
These optical disk formats already use error correction codes in an attempt to recover from small scratches. If you have a big enough error on one part of the disk, chances are lots of the disk will be unreadable. Unless you create multiple disks and spread the redundant data across all of them, this isn't going to add a lot of protection.
09F91102 no, 455FE104 nope, F190A1E8 uh-uh, 7A5F8A09 that's not it, C87294CE no. Ah! 452F6E403CDF10714E41DFAA257D313F.
Looks like they're using some kind of error correction code that's better than just parity, and distributing the blocks in each ECC group around the disk. It's better than writing the file twice if this (as it sounds like) happens below the file system level.
It's kind of analogous to a super-RAID, except with the "disks" that are being redundantly striped and mirrored are all on the same physicl DVD or CDR.
This is just plain stupid. Why don't you divide a CD into three virtual sectors, and treat each as an independent storage area to be joined in RAID 5?
That way, you lose 1/3 of your capacity, but would have to lose the exact same bit position in two different thirds of the disc to actually lose information. It's so easy it's stupid. I was surprised HD/BR-DVD didn't spec it in when they were released.
=P
You mean, if Slashdot stored every comment twice, there would be no more redundant posts? What about incorrect or off-topic posts? Would these, also, be a thing of the past?
Does this apply to articles as well? No more dupes? No more FUD? No more slashvertizements?
You, my friend, are a genius!
And I was trying not to post in this thread, so I could moderate something! Damn you!
In Mode 1 CD-ROM, for every 2048 bytes of data, there's 276 bytes of Reed-Solomon error correcting code and 4 bytes of error detection. Considering we're talking bytes, that's pretty reliable, and as you know, a single scratch often doesn't render a CD totally useless. Usually, a CD has to look like an ice skating rink after an hour of skating for it to fail miserably, and light scratches, even in high numbers, are generally not a factor.
So what the hell? Why is this even necessary, unless you're using a Mode 2 CD (and then, Mode 2 is usually used for videos/streaming data, which requires a more sequential read, where adding ECC would defeat the purpose).
Waste of money.
Screw the rules, I have green hair!
Get some Brasso (brass polish) and a soft, lint free cloth and you are in business. Really. You just polish the surface so it's all even and thus reflects light equally. If you are nervous about using Brasso, there's a number of products designed just for this purpose, though they are way more expensive and Brasso does just as good a job.
Either way the point is that with error correction as it is now, it's not hard to fix a CD if needed.
It really is open source if you just store the data twice. The TrueDisk recovery part is proprietary so it's not really open, you will only be able to read the 'master data', still have to buy their software to recover the undamaged data from their 'recovery sections'. Is the author of this article part of the OOXML team?
My guess is, speed would be the problem. If it's anything like par2, the recovery process takes too long to do in realtime; therefore, only useful if you were allowed to burn a backup copy (and if it was economically feasible to do so).
Don't thank God, thank a doctor!
From article: "Since the TrueDisc format is open and the master copies stored by TrueDisc are located in the standard ISO 9660 filesystem"
That pretty much fucks up anyone's day when they wanna burn a UDF DVD doesn't it? ISO9660 doesn't support files greater than 4GB, you can only have 8 directories deep (until the 1999 spec but I always had a hell of a time reading this stuff on anything but XP), stupid filename restrictions (and then do you use Joliet or RockRidge or whatever to fix it or not?)..
This $89 (or $52 intro price) TrueDisc sounds rather similar to the open source dvdisaster. It builds Reed-Solomon error correction data from CD or DVD iso images, which can be either augmented to the image and burned on the same disc, or stored separately. It's somewhat similar to par2/quickpar, but dvdisaster is more specialized for CDs and DVDs.
So far all the comments I've read are way off the mark about what is interesting about TrueDisk. Yes it's true that TrueDisk is just yet another error correction scheme. What is slick about it is it's high usability. This comes from two things
1) It writes the correction bits to a separate partition from the "regular" bits. As a result, the primary partition looks exactly like a regular CD. put it in any computer, even one not equipped with the TrueDisk Software and it can be read normally.
2) The amount of the redundancy is automatically chosen. It just uses any left over space when it finalizes the CD.
As a result the operation of TrueDisk is pretty much transparent. You only need to invoke the truedisk software to read a disk that has been corrupted. Uncorrepted disks can be read normally. So You won't lose your data if you don't have the software or the company goes out of bussiness and it stops working on newer OS's. (All you would lose without the software is the ability to recover from the redundant bits. ).
In comparison to PAR or RAR, you are not compressing the data so it's faster. Now I note that if you compress and then add redundancy you could potentially have higher redundancy for a given amount of data on a fixed CD size. So there could be some theoretical advantages to RAR and PAR. However, those PAR/RAR disks cannot be read in-place (they have to be expanded) nor in "real time" (say if you are playing video). They are very slow to write. They can't be read on any computer without the same verison of par/rar. And if you do lose bits beyond the point of recovery the compressed bits will span a much greater extent in the data space--you might even lose the entire CD with PAR/RAR. So you can see that TrueDisk has usability advantages even if it's redundancy is less and it's uncompressed.
Some drink at the fountain of knowledge. Others just gargle.
Burnable optical media is going to go bad, sooner rather than later. I have disks in my collection which are already unreadable after five years. So this is how I went paranoid with reliability for my important stuff:
1) Set up Solaris 10 x86 file server runing ZFS so that I can guarentee the data on the hard drives.
2) Use raidz2 as 6+2 drives. When one drive fails, I can power down the system until the replacement drive comes. I still have one disk protection when I power back up and wait for the re-sync.
3) Back up data to multiple DVDs and share them with out-of-state friends and relatives.
4) Rate each DVD with Nero's CD/DVD Speed.
5) Each year re-test the DVDs to note any degredation. Reburn before they tank.
6) Every other year re-silver the hard drives.
Not that I am obsessive, you understand.
This kind of thing already exists in the form of PAR files. Basically RAID5 but on a set of files, with arbitrary amounts of parity data (from 1% to 100%). The advantage of using PAR files, created with a program like QuickPAR, is that you can burn the parity files to a different disc.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Is PAR2 open-source? It seems like it is, but I certainly wouldn't want to do long-term archives of my data in any format where there were only binary decoders available.
A while back I had an absolute devil of a time trying to unpack some Compact Pro archives (.sea), and that's not really even that old a format -- it was last released in 1995 -- and there are still a lot of Classic Macs around that will run the software. However, in another 10 years, I'd imagine that it would be a lot tougher, since Macs being manufactured now won't run the Classic OS. (Unless someone reverse-engineers it, which I think has been done or at least discussed; according to WP macutils will do it. But hoping that somebody will reverse-engineer your proprietary archive format before you need it isn't the sort of risk you'd want to consciously take.)
At least with formats like TAR and GZIP, not only are they very well understood, but you could easily put a copy of the source code onto each piece of backup media; that way if somewhere down the road, you needed to get your data off on a machine that didn't have the proper decoder or didn't have the capability of running the binary decoder (Microsoft is going to have to break backwards-compatibility sometime...), you wouldn't be completely screwed. And most of all, you'd still be OK even if you turned out to make a bad decision on your choice of format, and it maybe didn't end up being as overwhelmingly popular as you thought it'd be.
I'd really hate to plop all my data into some proprietary archive or compression format in order to save a few percent, and then end up cursing myself (or having someone else curse me) a few decades down the line, when they're left with a binary blob and a decoder that will only run on obsolete hardware, using an obsolete architecture, running an obsolete OS.
People seem to chronically underestimate the lifespan of data. The backup of your vacation photos that you make now, may very well be something that you're going to want to get at in 20 or 50 years. Heck, I've scanned slides that are older than that. A whole lot of folks seem to not really consider the long term when they're backing up and archiving data, and in an average day, it's probably one of the more longer-lived decisions you're likely to make.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."