Slashdot Mirror

← Back to Stories (view on slashdot.org)

Error-Proofing Data With Reed-Solomon Codes

Posted by kdawson on from the trust-but-verify dept.

ttsiod recommends a blog entry in which he details steps to apply Reed-Solomon codes to harden data against errors in storage media. Quoting: "The way storage quality has been nose-diving in the last years, you'll inevitably end up losing data because of bad sectors. Backing up, using RAID and version control repositories are some of the methods used to cope; here's another that can help prevent data loss in the face of bad sectors: Hardening your files with Reed-Solomon codes. It is a software-only method, and it has saved me from a lot of grief..."

5 of 196 comments (clear)

  1. Re:It can make files a bit hard to read, though by xquark · · Score: 4, Insightful

    It really depends where you store the FEC, some techniques store the fec separately others concatenate and others interleave the FEC. Each method has its own advantages and disadvantages.

    --
    Arash Partow's Philosophy: Be a person who knows what they don't know, and not a person who doesn't know.
  2. Yes, RS should be a file system service. by Futurepower(R) · · Score: 3, Insightful

    "... data integrity MUST be an operating/file system service."

    I agree. I'm willing to have a small loss in speed and a small increase in price to have better data integrity.

    There is already data integrity technology embedded in hard drives, and I support making it more robust.

  3. Re:Version control != backups by ceswiedler · · Score: 3, Insightful

    The best solution is for developers to use their own private branches. Then they can commit as much as they want, and integrate into the main branch when they're ready. Unfortunately subversion has crappy support for integration (even with version 1.5 AFAICT) compared to something like perforce.

  4. Re:As I understand it... by Pseudonym · · Score: 3, Insightful

    Ok, lets assume its a 128-bit hash. For a 1GB file how many combinations of 1GB will produce the same hash?

    You're asking the wrong question.

    The right question is: Given a 1Gb file, how much "mutation" do you have to do to it to produce a file with the same hash? And the answer to that is: Enough to make the data unrecoverable no matter what you do.

    --
    sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
  5. Datarecovery "data". by rew · · Score: 4, Insightful

    Working for a datarecovery company, I know that about half the cases where data is lost the whole drive "disappears". So, bad sectors? You can solve that problem with reed solomon! Fine! But that doesn't replace the need for backups to help you recover from: accidental removal, fire, theft and total disk failure (and probably a few other things I can't come up with right now)... .