Slashdot Mirror


Ask Slashdot: Data Remanence Solutions?

MightyMartian writes "The company I work for has just had their government contract renewed, which is good news, giving me several more years of near-guaranteed employment! However, in going through all the schedules and supplementary documents related to the old contract, which we will begin winding down next spring, we've discovered some pretty stiff data remanence requirements that, for hard drives at least, boil down to 'they must be sent to an appropriately recognized facility for destruction.' Now keep in mind that we are the same organization that has been delivering this contract all along, so the equipment isn't going anywhere. What's more, destruction of hard drives means we have to buy new ones, which is going to cost us a lot of money, particular with prices being so high. I've looked at using encryption as a means of destroying data, in that if you encrypt a drive or a set of files with an appropriately long and complex key, and then destroy all copies of that key, that data effectively is destroyed. I'd like to write up a report to submit to our government contract managers, and would be interested if any Slashdotters have experience with this, or have any references or citations to academic or industry papers on dealing with data remanence without destroying physical media?"

27 of 209 comments (clear)

  1. Why not digital destruction? by quanticle · · Score: 4, Insightful

    There is software out there (like D-BAN) which will repeatedly overwrite the data on a hard drive, rendering it unrecoverable. Why not use that, rather than relying on encryption?

    --
    We all know what to do, but we don't know how to get re-elected once we have done it
    1. Re:Why not digital destruction? by 1729 · · Score: 3, Interesting

      There is software out there (like D-BAN) which will repeatedly overwrite the data on a hard drive, rendering it unrecoverable. Why not use that, rather than relying on encryption?

      How do you verify that the software does this correctly, and that it hasn't been tampered with? What if a drive is mishandled and doesn't get wiped? And if there's a process to do this correctly and with no chance of failure, is it worth that effort to recycle some old hard drives?

      Where I work, hard drives with less-sensitive data can be reused; other ones are ground up into little bits. Data cannot be recovered(*) from a thoroughly destroyed hard drive. What assurance is there for a software solution?

      (*) To the best of my knowledge. Maybe NSA can piece together the dust of a hard drive, but I highly doubt it.

    2. Re:Why not digital destruction? by Joce640k · · Score: 3, Insightful

      The old "You can recover data even after it's overwritten" thing is a myth.

      Today's bit densities are so high that it simply isn't going to happen.

      Format them. Run a small program to write a file (can be the output of a RNG if you want) until the disk is full. Job done.

      Or, as mentioned, use one of the many programs available for this.

      Take the "repeatedly overwrite" thing with a pinch of salt unless you really enjoy sitting there watching hard drive lights blinking.

      --
      No sig today...
    3. Re:Why not digital destruction? by Anonymous Coward · · Score: 5, Funny

      How much checking could a checker check if a checker checkering checked checks to check the checks that checked the checkering checker?

    4. Re:Why not digital destruction? by mlts · · Score: 4, Informative

      I like combining DBAN with HDDErase.

      HDDErase will do an ATA low-level secure erase that tells the controller to zero out all sectors. Even though that are on the relocated table which would be inaccessible via normal software solutions.

      After HDDErase does its job (which it does in a pretty quick amount of time since there is no I/O involved, but just the write head laying down zeros), running DBAN on the drive adds further insurance. Realistically, this will remove all data.

      Of course, prevention is a good idea as well. This is why I have some type of FDE software on my drives. This way, a simple zeroing out of the drive will be enough. In fact, the format command in Windows will check to see if a disk is BitLocker protected and zero out the places where the volume key resides, so even if someone knew the password to the drive, it will do them no good.

    5. Re:Why not digital destruction? by Anonymous Coward · · Score: 4, Informative

      There is software out there (like D-BAN) which will repeatedly overwrite the data on a hard drive, rendering it unrecoverable. Why not use that, rather than relying on encryption?

      Some classifications of data require destruction of media. See NIST SP 800-88:

      http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf

      In NIST/DoD parlance, what DBAN is cleaning/purging; i.e., either overwrite, or invoke the SATA Secure Erase command. Degaussing is also classified as purging (though the disk becomes unusable AFAIK); degaussing is better suited towards tapes IMHO.

      You also need to Validate that it has been done, and document that fact for each drive that has been sanitised.

      The OP will have to ask the contract manager at what level the information is considered at (low, medium, high) and then make plans accordingly. If it's high security, one can simply purge the media if you want to re-use the media with-in an organization, but if you ever want to toss the disk (or even if it's in a RAID array and you need to replace because it died), you need to destroy it and record that fact.

      So if your EMC/NetApp/Dell array has sensitive information, you can't send it back to the OEM if sensitive data ever touched it: you have to make arrangements with the OEM so that you can destroy it. Ditto for your laptop/desktop drives: if Lenovo/HP want/s the drive back, they can't have it as otherwise you'll be breaking your contract with the government.

    6. Re:Why not digital destruction? by EdZ · · Score: 3, Informative

      No need even for DBAN. Unless you're using truly ancient decade-old HDDs, use the ATA SECURE ERASE command built into the HDD controller. Much faster than DBAN, and wipes not only the accessible sectors but sectors in the G-list. Plus it's NIST and NSA approved, so it should be complaint with any government requirements for data destruction.

      It also effectively returns non-TRIM SSDs to a factory state. Remember: when used on SATA drives, set your bios to IDE compatibility mode, not AHCI.

    7. Re:Why not digital destruction? by Local+ID10T · · Score: 5, Insightful

      D-BAN is great... but if the contract says "Thou shalt turn over thy hard drives for destruction..." then its already been agreed on, and the cost was factored into the bid. Deal with it.

      --
      "You want to know how to help your kids? Leave them the fuck alone." -George Carlin
    8. Re:Why not digital destruction? by Sancho · · Score: 4, Insightful

      Yes, but this is a government contract with specific destruction requirements. Go complain to the feds if you don't like the myth. Or maybe the government knows something we don't. Who knows?

    9. Re:Why not digital destruction? by SnarfQuest · · Score: 5, Interesting

      A lot of disks have "bad sector" replacement. When a sector starts to be unreadable, it replaces that sector with a spare one set aside for that purpose. Does the software wipe out these revectored sectors, or can someone read those old sectors after software overwrite?

      It depends on the security threat on how serious you need to be about wiping data off drives. Sometimes just 'rm'ing files is enough. Sometimes dropping them in a volcano isn't enough.

      --
      Who would win this election: Andrew Weiner vs Andrew Weiner's weiner.
    10. Re:Why not digital destruction? by Anonymous Coward · · Score: 5, Informative

      Yea, you're remembering that contest how you want to remember it. The prize was a pittance, and the "company" offering it was a handful of people. There were also ridiculous restrictions, such as not damaging the single physical drive the whole challenge was based around. And several data companies said they likely could recover some data, just not necessarily the specific file that that the challenge was based around (as a general rule, you can't target a file, you get whatever it is you get). But the process involves ripping the drives to pieces and costs significantly more than the challenge was worth. And since the challenge was issued by a handful of guys rather than an actual, large company, very little publicity would have been generated, so it wasn't worth it to anyone.

      Now, even if that story happened exactly as you remember it, it's still irrelevant. The point isn't that that it's currently possible, it's that it's theoretically possible and thus may be trivial in the near or distant future. For certain kinds of data, that is a world of difference.

    11. Re:Why not digital destruction? by nabsltd · · Score: 3

      Sure, the process can still be subverted, but it's a lot easier to verify that a hard drive has been destroyed

      Imagine, if you will, someone who wanted your data and could intercept the drive for long enough to swap the platters on a drive (thus taking the important data with them). How do you verify that your data was destroyed?

      One way would be to send a backup (or SHA1 hash) of the data on the drive to the data destruction facility and have them verify that the data on drive serial number 123456789 is what it is expected to be before destruction. If you aren't doing something like this, then you have no way of knowing whether your data is really gone or not. If you think this sort of thing can't happen, read some of the stories about how people get back the wrong ashes from cremations.

    12. Re:Why not digital destruction? by flonker · · Score: 4, Interesting

      Yea, you're remembering that contest how you want to remember it. The prize was a pittance, and the "company" offering it was a handful of people. There were also ridiculous restrictions, such as not damaging the single physical drive the whole challenge was based around. And several data companies said they likely could recover some data, just not necessarily the specific file that that the challenge was based around (as a general rule, you can't target a file, you get whatever it is you get). But the process involves ripping the drives to pieces and costs significantly more than the challenge was worth. And since the challenge was issued by a handful of guys rather than an actual, large company, very little publicity would have been generated, so it wasn't worth it to anyone.

      Now, even if that story happened exactly as you remember it, it's still irrelevant. The point isn't that that it's currently possible, it's that it's theoretically possible and thus may be trivial in the near or distant future. For certain kinds of data, that is a world of difference.

      +1 for AC

      In addition, they required that you release your methods for recovering the data, which I'm sure is worth a lot more than the 3-4 digits they were offering.

    13. Re:Why not digital destruction? by LordLimecat · · Score: 3, Insightful

      Its not a myth, its a theoretical possibility that either noone has the current capability to do, or they do and its simply too cost prohibitive, or else we simply dont know about it. Thats not terribly reassuring if youre dealing with data whose leak might cause jail time.

      As for formatting, depending on how you format the drive, it may or may not overwrite the data at all and may leave it ripe for the picking.

      Honestly, if youre dealing with government and they say "we want the drives shredded", DBAN set to a DoD approved setting MIGHT be a reasonable suggestion, as would encryption (as we can actually quantify the risk there, and it is vanishingly small), but saying "ah, just zero it once or format it, it doesnt make a difference" sounds incredibly foolhardy.

  2. All you have to do is... by WhitePanther5000 · · Score: 3, Funny

    ...burn it to an optical disc, then shred the disc! :)

    1. Re:All you have to do is... by PhilHibbs · · Score: 4, Insightful

      You've said it better than I could - and I'd go further to say that the fact that he considered encrypting the data and then destroying the key indicates that the OP is incompetent to be doing this kind of work. You don't destroy data by making an unreadable copy of it. You destroy it by destroying it, which could mean physical destruction, or could mean multiple overwrites (but the face that the government requirements state physical destruction implies that they have already considered and rejected this option).

  3. DBAN by jd142 · · Score: 5, Informative

    DBAN, Darik's Boot and Nuke, will wipe a hard drive to any of several government standards. If they are fine with mere software disposal of data, then DBAN is the way to go. http://www.dban.org/.

    If they insist on physical destruction, I'm sure there are companies in your area that will handle that for you.

  4. Easy Peasy by danwesnor · · Score: 5, Insightful

    If you believe the data shouldn't be destroyed, have your contracting office send the government contracting officer letter requesting the requirement be deffered until the end of the new contract.

    1. Re:Easy Peasy by rjstott · · Score: 4, Informative

      Totally agree, if the contract is renewed the destruction can't be necessary until termination of the extension UNLESS this is not a renewal but a NEW contract. THEN you need to ask for a WAIVER

  5. The contract... by Taelron · · Score: 4, Insightful

    The contract states that it must be physically destroyed. Depending on what kind of business you are in, the government will only accept physical destruction of a drive if classified data was ever on it.
    You will need to adhere to the contract and destroy and replace drives or the Government will rake your company over the coals during an audit. They will also then demand monies paid back, tack on a huge fine, and possibly criminal charges on anyone that failed to properly dispose of and destroy the data per the contract.

  6. Why would they agree? by sirwired · · Score: 4, Insightful

    Your old contract requires the destruction of the equipment. Your new contract failed to price in its replacement. Why is this the agency's problem? If I were the client, I'm not going to go out of my way to evaluate your data destruction ideas and instead would simply request you perform the contract as agreed.

    Make sure your negotiators don't foul this up for future contracts.

    1. Re:Why would they agree? by tlhIngan · · Score: 4, Insightful

      Exactly. They'll want certificates proving the drives were destroyed per the contract.

      Part of your contract bottom line includes the cost of replacing those drives. If your company bid too low and won't make a profit, that's really a shame, but that's something you'll have to take up with the salesperson who wrote the proposal.

      Also, realize that hard drives are only expensive *NOW*. Remember what happened in Japan that was supposed to kill the electronics market until the end of the year? In 6 month's time, the prices of hard drives will come back down. Unless your contract is only a month long, the destruction probably won't happen until then, which is probably a year or more down the road (unless it gets renewed again). In the mean time, you only destroy hard drives of PCs that are being decomissioned, so they've already been replaced and no issue at all.

      Also - why are you trying to find ways around it? It's in the contract and you wouldn't have gotten it if you didn't agree to the requirement. Is it really to save the company a few bucks? Or is it the inner geek who can't see the sight of tossing a 500GB drive away?

  7. Re:Zero-fill? by ajlitt · · Score: 4, Informative

    You mean like this? Maybe you should read the articles you cite before you use them to correct someone else.

  8. Re:Zero-fill? by Baloroth · · Score: 3, Informative
    Oh yes, that challenge totally answered the question once and for all! What drive recovery or intelligence agency could resist the reward of... wait for it,

    $40.00 USD and the title "King (or Queen) of Data Recovery".

    $40.00 US DOLLARS!!! And they can keep a 60$ HDD!!! For performing a time-intensive, expensive procedure! Yeah, that totally shows everyone...

    Oh and most challengers also wouldn't be able to disassemble the drive. And would have to publicly disclose the method used (heh, yeah, I can totally see the NSA jumping at the opportunity to prove some random Internet blogger wrong while disclosing all their methods). I'm sorry, but that challenge is so obviously a joke, it's actually sad, because people think it answers... well, anything. (source, BTW. Original source has absolutely zero info AFAICT.)

    --
    "None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
  9. Poor negotiation is not best fixed technically by malx · · Score: 3, Insightful

    I agree. You're trying to solve a commercial issue (and possible mistake) with a (poor) technical solution.

    As you describe it, the original contract wanted the data destroyed at the end of the contract term. You've just had the contract *renewed*, which is another word for "extended". Why exactly would anyone want the data destroyed in mid-contract?

    Your contact negotiators ought to have realised that the government didn't need you to destroy the data until the end of the new contract, and written that into the new contract, thereby over-riding the old one. More than saving you the money, it was one of your advantages as the incumbent contractor: compared with a competitor, you could perform the second contract term at lower cost simply because you could off-set the data destruction cost for which you were already contracted simply by writing into the new contract permission to defer that destruction! This would allow you to underbid any potential competitor - or if there is no likely competitor, writing deferral in would be a straight profit to you at no cost to the customer. That kind of win-win is *exactly* what your contract negotiators are paid to spot and capitalise on.

    As poster above says, your contract office can still possibly rescue this by simply writing and asking for permission to not destroy the data until the end of the renewed contract term. All the same, missing this at contract negotiation time is something that should come up in somebody's annual performance assessment.

  10. It's not up to us, or the submitter by DragonHawk · · Score: 4, Informative

    It really depends on the terms of the contract. That's what controls. You can theorize and speculate and pontificate all you want, that contract is what they agreed to, and what the government agreed to pay for.

    Now, the phrases "sent to an appropriately recognized facility" and "data remanence" make me suspect this is classified information, which would mean the contract is under NISP (National Industrial Security Program) jurisdiction. There are four possible CSAs (Cognizant Security Authorities) -- DoD, DoE, CIA, and NRC. I'm really only familiar with DoD, but I believe the rest follow suit on this. To wit:

    Since Oct 2007, when ISL 2007-01 (Industrial Security Letter) was issued, overwrite methods are not acceptable for fixed disks. Degaussing or physical destruction are the only acceptable methods.

    Degaussing has to be done using a deguasser which is on the NSA EPL (Evaluated Products List). This generally renders the hard disk inoperable. (Modern hard disks have their servo tracks encoded at the factory, and generally don't have field low-level format capability.)

    Physical destruction has to cover the entire recording media. (e.g., "target practice" isn't acceptable.) They generally want the entire recording surface ground off, melted down, shredded to dust, and/or raised above the curie point. You can't just toss it in any old shredder.

    You have to provide a certificate of destruction, saying you've done this. Failure to do so results in loss of Security Clearance, loss of contract, loss of future contract opportunities, fines, and/or jail. I wouldn't recommend it.

    Now, submitter mentions they're going on to a new contract. If this is DoD, they should check the DD254 to see if it's the same classification derivation. If it is, they should be able to get approval to continue using the old systems. They should have a formal ATO (Approval To Operate) that identifies who to contact.

    Now, if I'm way off base, and this isn't classified, then it's still up to what the contract says. There are various government standards that might be called out. NIST 800-88 is one; it allows for sanitization by overwrite, IIRC.

    --

    dragonhawk@iname.microsoft.com
    I do not like Microsoft. Remove them from my email address.
  11. Re:Zero-fill? by Electricity+Likes+Me · · Score: 3, Informative

    The original reason HDD data was recoverable was because the head did not perfectly create or remove magnetic regions on the media. Imperfections, head wobble, electrical noise - all contributed to creating variable sized domains. Now, magnetic polarization of materials has some odd effects - one is that inducing a region of magnetic polarity doesn't swamp out a neighboring region, it will first "push" it away. So if you write "1", then "0", then "1", the thin band of magnetism from the first "1" will be at the outer most edge of the track, with another thin band of "0" and finally the actual "1" that the head sees.

    The "killer app" of magnetic force microscopy was then that you could stick the platters under MFM and beat the resolution of the head for reading the data - the oldest copy of the data would be squished up at the edge of the track, the second oldest further in and so on and so forth - you could actually read back several generations of hard disk data.

    Of course, since that age, technology has changed - hard disks now use RF modulation to store multiple bits per space, bit densities have shot up, and heads track much more accurately - basically, the physics has been beaten out since we are now writing much more complex data, and almost every single bit of magnetically encodeable space on a hard disk is now used to encode data - there's (very little) space between platters, and what signal you get there is likely irrecoverably fuzzed RF if you can even see it at all.