Slashdot Mirror
Data Recovery & Solid State
theoverlay writes "With all of the recent hype about solid-state drives in both consumer applications and enterprise environments I have a real concern about data recovery on these devices. I know there are services for flash memory restoration but has anyone been involved in data restoration projects on ssd drives? What are the limits and circumstances that have surfaced so far? What tools will law enforcement and government use to retrieve data for investigations and the like?"
What tools will law enforcement and government use to retrieve data for investigations and the like?"
Waterboarding, tasers, sleep deprivation, bright lights and loud obnoxious music.
I'd figure the same as with regular harddisks apply. One pass and gone the data is.
Is it "How can I recover data from a failing/failed solid-state drive?"? Or is it "How easily can someone else find my 'deleted' data on my solid-state drive?"?
I'm not sure of the answer to either question, directly, but I'd suggest multiple backups for the first one, and encryption for the second one (full/near-full disk encryption is quite fast on a multi-core system).
It appears that solid state drives are going to have several times the MTBF of conventional media, and thus a failure rate several times lower. Sure, data recovery is much less likely to work when SSDs fail-- as it's more likely to be the actual memory failing than controller chips or ancillary electronics. However, normal disk recovery places can only recover your data from a failing/failed drive perhaps 60-75% of the time. Thus, the actual incidence of unrecoverable data on a SSD is likely to be much lower than with rotating media, and the overall failure rate lower still. This is nothing but a win, as the normal data recovery rackets are made irrelevant in the case of media failure and overall reliability is improved.
I don't know about NAND chips , but apparently ram isn't all that "volatile" as it should be( http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html , part 7). If nand flash is anything like ram the ware leveling algorithms would still ruin any forensics in a system were data changes frequently.
I realize there are "professional" companies that specialize in data recovery, but in my ( admittedly limited) experience I've only heard of sob stories of people paying $$$ and not getting any data back. On the plus side, Its always taught them to back up their data.
Well.. maybe. Or Maybe not. But Definitely not sort of.
-1, didn't read the question. He is NOT asking about how reliable the drives are, since he acknowledges that ANY media can fail. Instead, he asks about recovery options when there are no other alternatives, such as extreme disasters or criminal cases where data was intentionally lost. This is a good question, I look forward to constructive answers and the discussion that follows. Yours, however, is a dead end.
Perhaps as these types of media become cheap enough we will all be able to run our own media with the GMail-esk mantra "Never delete data again!". But seriously, Data Recovery exists through a flaw(?) in old media types. If I delete something, I want it gone. If I want to get it again, or insure it from loss, I should make backup. This is all well and good until FBI/NSA/DHS decides to install rootkits on every media type we buy... that'll be the day.
Where genius and insanity become confused true wisdom is found
A dolphin with a SQUID would seem to be the obvious choice.
From the first sentence's "there is a significantly less number of qualified technicians" to "However, none of this any consolance to the customer who has just lost critical business material", there is no content in this blog. Worst blog-slashvertisement ever.
If you want security, encrypt before you store. If you want recoverability, get a real backup. Seriously, this has been this way ever since computers got fast enough to do AES on the fly against disk. Ubuntu supports it in the alternate installer, Debian and probably the rest too. On Windows various closed source software like DriveCrypt++, Bitlocker and whatnot is available. This isn't really all that difficult...
Live today, because you never know what tomorrow brings
Not recovering the data you want is always a risk. In my experience I have recovered everything I've needed using a pay-for service. Expensive? Yes, but you (or your client) must weigh benefit.
Backup, backup, backup. Those that don't will pay the price. Literally.
When someone asks a question like: "What tools will law enforcement and government use to retrieve data for investigations and the like?"
The issue isn't just 'how do I recover data' it's also 'how do I erase it permanently'
In my experience, you can recover anything that hasn't been overwritten on a flash drive with most recovery programs.
Keep in mind, that even if you've "erased" your files, not all wipe/erase programs will delete the file & folder names from your drive. Programs like DirSnoop can recover the names, if not the files.
[Fuck Beta]
o0t!
Actually my concern would be more the exact opposite, what are the implications for secure erasure of these drives? Before we could just open the drives and smash the platters if you wanted to be really paranoid. Now, do we have to make sure we find all the flash chips and ensure each one of them is destroyed? Are there other implications because of this flash memory for secure erase utilities?
;-)
If your hard drive dies and you don't have a backup, I have very little sympathy for you. You should know better. Especially anyone reading slashdot. Let's get back to our NSA fearing roots and talk about how to protect ourselves with the latest in encryption technology.
Ask Slashdot: For when you've got time to write up a whole paragraph, but not a 5-word google search...
Google results, which seem rather informative
"Quoting famous computer scientists out of context is the root of all evil (or at least most of it) in programming." - K
Just put your drug deals, k1dd13 pr0n, and terrorist plans in a file called attorneyconfidential.doc. That way when you erase them you can claim attorney-client privilege with a straight face.
The second question here is if it is possible to recover data that has been overwritten on a solid state device. It is possible on magnetic disks, but a solid state device is encapsulated in a much more rigorous manner which means that it will be a lot harder. However, it may still be possible using the right equipment.
And don't forget: Never store your important data under the directory /tmp or /var/tmp on any *NIX machine. It will be erased! I know that this has happened, since I was working for a company where a consultant did EXACTLY this. That consultant stored all his sources there! And the system erased all files older than 14 days, and since it was /tmp there was no backup. That person had to do it the HARD way because there was no way that there was any possibility to recover that data. I have no idea what became of that consultant after that was cleaned up, but I sure hope that he at least didn't make that mistake again!
One of the classical Murphy's law moments...
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
And why is it considered a desirable effect that someone can forensically recover data that the owner indended to destroy? If SSD really does not allow data to be recovered like this, then in general thats good, IMO. Not just for legal reasons, but for any reason of privacy.
If you are concerned about protecting against data loss there are other more effective ways like implementing RAID and maintaining off-site backups.
I Heart Sorting Networks
There are ways to destroy solid-state disks that don't require a hammer.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
One confounding aspect of trying to permanently erase things from solid state drives is the fact that most flash drives incorporate wear-leveling. You may not be able to over write specific physical sectors without just overwriting the whole drive several times.
"Prefiero morir de pie que vivir siempre arrodillado!"
Which is the same infallible data erasure option for any media. Incineration.
Trusting data loss to just one delete command is being broken in the head.
Makes you wonder if you could quadruple the capacity of the harddrives that way too.
I think you just proved to us why your statement is false.
If old data is recoverable, the disk would hold more data.
No sig today...
The chinese used some very impressive tech to read the hard drives from a US surveillance plane, where the data was overwritten, and then melted with thermite. Magnetic domains aren't that easy to erase, it like erasing a whiteboard with a slotted eraser, there will still be traces of the magnetic domains even after two rewrites. And the extra data that drives store for CRC info helps a bunch in getting the data right.
Awwe, how cute! Would you like an apple shaped cookie?
Copyright 2010. All rights reserved. This comment may not be copied in any way including, but not limited to caching.
I work for www.harddisk-recovery.com .
We will gladly reverse engineer the data-distribution algorithms that the SSD device uses on a case-by-case basis. We have done so in the past for several different USB sticks. We will desolder and read the individual data-holding chips and then reverse engineer their scrambling algorithms. We will then recover your data from whatever chips still work sufficiently to provide us with some data.
The first time this will take us a few days extra. Expect about a week turnaround time the first time anyone sends us a failed SSD disk.....
If you have any data that you may need to destroy quickly and permanently, I would suggest using DVDs. Sure, it's slow and a hassle but, when you need to get rid of a large volume of information in a hurry, you just take your DVDs and put them in a microwave for a few seconds.
The damage microwave radiation causes to the data on the DVD extends beyond visible damage to the metal layer. That is to say that, even though it may seem like there are undamaged areas left on the DVD's surface, they are still unreadable. And it only takes 2-3 seconds to completely destroy a whole stack of DVDs, if they are arranged in a microwave with some space between them. Rewriting a hard drive with multiple passes may take hours and still leaves a possibility that some data may be recovered.
It seems to me that with SSD data recovery should work better than with conventional hard drives. You may need to overwrite the entire disk multiple times, as opposed to overwriting just the selected data, as you would with a conventional hard drive.
If you're wanting to know about recovery for security purposes, as in, "how do I destroy this thing so that no one can recover data from it?", that's an interesting and useful question. If you're just wanting to know out of general curiosity, it's also an interesting question.
But if you're thinking about what might be possible as part of disaster recovery, you've completely lost the plot. This thought seems to spring from the same well as the idea that "mirroring" can be used for backups. No, no, a thousand times no.
"Not an actor, but he plays one on TV."
A relative of mine paid some $2500 for what probably were a few broken sectors. Years later, the recovered data (and all the stuff accumulated in between) was, without any backups, stored on the disk he got it from the recovery service. Which started failing, too.
Some people never learn.
I've seen a lot of comments about using whole-drive encryption on these flash drives. However, flash drives balance the load across multiple blocks in order to extend the life of the device. Anybody want to take a guess at how less secure your encryption becomes if there are mutliple historical copies of a block around to use for comparison?
...criminal cases where data was intentionally lost
You can completely and unretrievable wipe data from both paper and disk drives. With paper, shredding is no good but a single match or Bic will do the trick. Cheaper than a shredder, too. With a disk drive, just disassemble it and sand off all the oxide. Or alternatively, if you have a smelter or other really really hot mass of molten metal, you can just drop the thing in there. The smelter option works for CDs and tape as well.
Or you can bury it in the bridge abutment your construction company is building with tax dollars, right next to Jimmy Hoffa.
Oh oh, am I on my way to Gitmo now?
-mcgrew
(still no journal although the last one was updated Friday. Mod me down for this?)
mcgrew's razor: Never attribute to stupidity that which can be explained by greedy self-interest
My experience with Flash medium has been extremely impressive (especially versus harddrives):
I've encountered a nearly a dozen hard drive and micro-drive failures in recent years. Meanwhile, I have experienced only one partial failure of a flash device - it had a bad sector. I could extract all the rest of the data except for the file written in that sector of a 512mb Compact Flash card. So it was merely a partial loss and very small percentage. While this was enough to lead me to cease using this card, it was a very very minimal loss of data.
Now, I haven't even addressed the accolades of flash based devices. I have one thumb drive, it's a few years old now and still running. That may not be all that surprising. But I think it is unlikely that a 3 yr old hard drive would still be running after having gone thru the washing machine and the dryer....twice!
***
So back to the point of my reply....
The recovery options seem very similar to me. Clean room, magnetic readers, etc. I expect the same basic processes as are used to recover data on hard drives and floppies. However, I expect there to be a lot lot less need to do so.
The problem is see is that the small number of recovery centers may become even fewer. And the issue might be finding a company to extract the data. Especially after disaster situations (ie: regional flood, etc) where a large number of individuals & companies desire data recovery. We could see a large backlog occur as there might not be enough business out there to keep a large number of companies operating in this very unique field.
- The Saj
Yup, it can be extra hard to wipe a flash drive without knowledge of its particular wear-leveling algorithm. In these cases, Fe2O3+2Al is your friend.
The recovery options seem very similar to me. Clean room, ...
Clean room? Why?
Having operated a makeshift incinerator a few times, I have to point out that fire can be insufficient in and of itself.
I've actually held bits of ash with legible writing still on it. I was burning old checks for my parents.
I wouldn't count it destroyed until the ashes are stirred well.
I don't read AC A human right
Okay, so the new wear-levelling ability of SSDs, (where if it cannot write to a block/bit/whatever, it marks that as bad and writes somewhere else), brings a question to mind:
Let's say you have had your SSD for awhile, and some data is in areas that subsequently get marked as 'bad'. You 'format' your SSD clean, but does the format change those marked-bad bits? If not, just because they cannot be written to, doesn't necessarily mean they couldn't be READ from by some utility that ignores the marked-bad flags, in theory. So, is it possible for an SSD to have data recoverable from 'marked bad' areas, that might even pass a format/multi-write randomizing utility? Something to think about. Hopefully someone knows the answer...
Seriously--does anyone know how to wipe a SSD? It's my understanding that these things have wear-leveling built into the firmware--I tell this to write 0's to some sector, and it might just reorder the device and write there instead...
I admit I don't understand exactly how this works, but it strikes me as trying to wipe a journaled file system...
Anyone care to contribute thoughts?
DoD5220.22-M is what most use and is becomming old-school. That means three passes. Ones, Zeros, then Random. However, the national standard in America is NIST 800-88. Newer drives have a function built into the firmware that do a secure erase in one pass, even covering spare sectors. It's called Secure Erase or SE. The NSA likes it, rating it higher than using an external program. It meets security requirements of HIPAA, PIPEDA, GLBA, and Sarbanes-Oxley. If you want it, check into this man's utility and its educational document.
I guess these drives are going to put Steve Gibson out of business; unless he's currently writing ReadRite!
yes, yes it does =) (haven't tried though. Should not be considered technical advice =))
Overwriting with a single pass of /dev/urandom will only make recovery very labour intensive and hugely expensive, but not impossible.
Two wipes makes it harder still. It is a statistics game, each write makes the odds go down (and steeply at that) that the data can be recovered.
Anyway, wiping once is not enough to keep our lab from looking at your pr0n.
They can't be moles. Secret Squirrel would never stand for that.
Being one who is an owner of a data recovery company, I have been contemplating the idea of writing an article about the implications of SSHD and data recovery. I guess this discussion has beaten me to it.
I have a few thoughts on this matter and will post them in point form:
1. The elimination of the clean room?
- For obvious reasons, the necessity of a clean room for solid state devices will be drastically reduced. However, due to the price and size constraints, I don't foresee the elimination of the traditional hard drive for some time to come. Of course, that could be 5 years or 15 years, depending on industry trends.
2. The stability of solid state hard drives?
- I'd say that SSHD are more stable from the perspective of being bumped around. However, a simple power surge could render the data lost forever. This is where the traditional drive has a hope. The electronics can be toast, but the data is still on the platters.
- To the most part, traditional hard drives show signs of dying before they completely crash where a SSHD is going to work or not work, with the exception of failing bits.
3. Will SSHDs be the data recovery lab killer?
- I doubt it. It is true that hardware failure is the number one reason for data loss. But, a close second is human failure and I believe that will never change. So, the SSHD may become a more stable drive, but it won't be the end of data loss. If anything at all, the SSHD technology will create more false security, making for more critical data loss.
4. Will SSHDs affect the cost of data recovery?
- I suspect that we will see three different quotes for these devices: 1. around $500, 2. around $2000 and 3. unrecoverable.
All in all, I am excited about the technology and look forward to putting my first 250GB SSHD into my MacBook Pro. But, until we see the prices drop and the capacities increase, we won't be seeing these drives in anything other than a few overpaid executive's laptops.
I don't see the troll rating as being accurate. Overrated ... perhaps. I didn't think someone was going to be posting the answer to my question a few seconds before I asked it. In any case, that was not an attempt to troll. Meta mods... do your magic.
Well.. maybe. Or Maybe not. But Definitely not sort of.
I call shennanigans. Recovery after thermite? Not a chance.
Any ferrous material brought above the Curie Point is no longer magnetic, and looses any magnetism it had prior to heating. You can test this yourself with a magnet, a butter knife and a blowtorch. No matter what combination of iron and impurities your drive surface has, its Curie Point is easily below the temperature of molten iron - the product of your thermite reaction.
So even if the discs were heated by thermite, rather than just plain destroyed, it's unlikely that the heating would allow any data to survive unless the iron was already pretty cold.
That said, this was a surveillance plane flying over a foreign country in a (presumably) covert fashion. If it had such a self-destruct, it would be a mil-spec component. In case of a crash, I doubt there would be much of a plane left, let alone drive platter pieces to be recovered.
Most of a hard drive is made of aluminum. Disassemble, mix, and sell to local recycling plant.
There is no "-1 offended" or "-1 you don't agree with me" mod options for a reason.
I just feed all my shredded documents to my worms(no joke)
In today's political climate, this probably should have been modded insightful or informative instead.
Our disk drives were RM-05s, which had stacks of a dozen or so 14" platters. Most computer administrators had one on their wall showing the effects of a head crash, with various tracks scraped into the oxide finish. I was no longer running the lab when we decommissioned the VAX, but my successor got to take the disks down to the machine shop in the basement to have them sandblasted. The platter on her wall didn't have any oxide left - it was smooth and shiny metal.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Law enforcement organizations aren't going to waterboard you, which would be against the law, though they might have fun tasing you. And courts have simpler methods - they issue you a subpoena that says to turn over any information you've got, and can make you sit in jail or pay heavy fines for not handing it over, or if it's a civil lawsuit they can decide that you're acting in bad faith and decide in favor of your opponent and make you pay their attorney's costs.
Law enforcement organizations are also highly unlikely to get out the electron microscopes and look for fuzzy bits around the edges of your disk tracks; that's more of an NSA/CIA spy-vs-spy kind of threat model. On the other hand, they are often willing to have some sleep-deprived technician who likes bright lights and loud obnoxious music do the kind of disk recovery that looks at your file systems for the data sitting around in unerased blocks or marked deleted in directory listings.
Fundamentally, if you're storing data on a computer that you don't want anybody else to recover, you need to store it in encrypted form so the only thing that can be recovered is the cyphertext.
For most people, though, the real threat model is that Murphy and BillG gang up on you. For that you need backups, and you need to periodically make sure you can recover your backups, and every couple of years you need to copy the data from old media to new media because otherwise your only copy will be on a 9-track tape or MFM disk. And BillG's still going to make sure that you can't read that proprietary file format that was used by some word processor in 1994. And your corporate IT staff are going to write a backup script that only copies files in Microsoft Office formats, which don't include the
Fortunately, storage costs have been dropping much faster than Moore's Law predicts, so in theory it's getting easier or at least cheaper to do backups. In practice, Murphy's taken out one of my new 500GB drives, and Maxtor's turned the other one from 500GB into 128/137 GB because the old Maxtor USB-drive case didn't know if the new Maxtor drive supported 48-bit addressing....
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Likely what you'll need is a program that fills the drive several times.
Chas - The one, the only.
THANK GOD!!!
You can use sdelete on Windows
http://www.microsoft.com/technet/sysinternals/Security/SDelete.mspx
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
http://en.wikipedia.org/wiki/RAS_syndrome
Just a thought:
Powdered magnesium & aluminum 60/40 in a plastic capsule with the same dimensions as the SSD unit in question, perhaps 2mm thick with a small plastic tube connected to a solenoid valve and a tiny O2 bottle. When activated by a panic switch the 6000 degree F temp should vaporise the chip (vary the thickness to increase the burn time).
I killed da wabbit -Elmer Fudd
I have a shitty portable hdd that I bought about 6 years ago. It still works perfectly. A cheap bus powered 20gb drive (1.8 inches I believe). I have dropped it and knocked it off the table (while it was running) and generally carried it about for ~3 years before moving on to a smaller flash drive and use of the internet for storage.
I had a flash drive that I paid just as much as the HDD cost me. $70 for 256MB (which actually was a steal at the time). I used it for less than a year (and not all that much because I had two of them.) It just died one day. Do not know why, but it did. My data? No more. Unreadable. At least if the HDD died I could still have a chance at data recovery.
In this case the shitty HDD with its moving parts ended up being more durable. The stupid less used flash drive sucked.
This is not the only reason flash scares me. I have really never lost anything important to drive failure as I have always been at least able to access enough to get my files that I wanted (I do have backups, but that is not the point.)
The day someone can recover data from media I've killed with thermite, then we'll talk.
Right now, if someone sketchy wants to cover their tracks, it's cheap and relatively easy. I've personally witnessed the awesome destruction that thermite does to a hard drive, it leaves a big drippy hole where the platters once sat. It's basically super-welding the drive into one big block. I don't think there's any way to get data back, at least not with current nor near-future technology.
-Billco, Fnarg.com
And the Chinese did manage to recover the data... I cant find the article right off.
Storm
EnCase® Forensic
http://www.guidancesoftware.com/products/ef_index.asp
I'm surprised no one has mentioned it yet.
It clones your HD and provides an image that law enforcement can work from & admit as evidence.
[Fuck Beta]
o0t!
No, you can pick up the old signals even in new drives. It's more complicated now, since you need to know the encoding schemes and ECC strategy (there are some wild ones out there now with fancy LDPC structures and the like), the fact that media noise is actually the dominant factor in modern encoding schemes, and tracks are pretty tight. But if you're willing to go the distance you can pull stuff off. And if you're the NSA or a drive manufacturer you can go great guns and use a interferometer controlled spin stand to read the off track footprint from the slight servo misalignment of the head and track when you did the erase. Not cheap, not quick, but it'll usually work. We do stuff like that to make sure that overwriting performance is "good enough" to dominate the signal, but you can still see the old signal down in the noise if you need it badly enough.
Yeah, I work on the things. So what's your point? Nothing's changed so badly that a single write can wipe out the data completely unless you're very (un)lucky if you want it badly enough. You still have to overwrite a fair number of times to really wipe your disk.
And before you CS types go all whacko on best theoretical patterns for erasure, we encode your bits ourselves into our own codespaces and usually use sequencers to scramble the bits to whiten out the frequency bands for more typical input patterns, so without knowing what we're doing your efforts to optimize erasure are dubious at best.
As anyone who has used Norton WipeInfo can attest, there is a U.S. Government "Data Encryption Standard" for wiping disks. It involves multiple writes of different bit patterns to the drive. Now, the standard is probably old, and made for those old disk technologies... but since when have you known the government to keep up with technology?
But no banana.
Recovery from formatting has very little to do with forensic data recovery. The other posters in this thread are correct: with modern drives, there is very little magnetic "slop" left once a bit has been written.
True, it is not enough to just format the drive or erase files; one MUST overwrite the bits to actually destroy the information. But as for recovering data that has been overwritten on a modern drive, forget it. If anyone took even minimal care to make sure it was fully overwritten once, or even better twice, an "analyst" and his tools can get as "anal" as he/she wants pursuing the data, and will get nowhere. It just doesn't exist anymore.
What you claim may have been true in the bad old days (between about 5 and 15 years ago, give or take) but is simply not true true today. Try to keep up.
http://www.microsoft.com/technet/sysinternals/Security/SDelete.mspx I'm a little suspicious of a microsoft command which would permanently erase my data. Are you sure they are not sending it to themselves first?
There is nothing random about it. The DES calls for specific patterns of bits to be written to the disk repeatedly.
(*ducks and runs*)
May contain traces of nut.
Made from the freshest electrons.
Sdelete is a sysinternals utility. You can get the source code, or at least you could. Even if you can't you can still use File monitor to see what it does. It would seem like a very bad move for Microsoft to change it in a way that makes it send data off to them.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
SSD is 100x easier to strip the data from than traditional technology. You can pull the data directly using any chip reader / programmer. It should actually be a CONCERN for you that it's so easy. Security on old drives is much tougher. On an SSD drive you can strip the data so much easier, it's a joke.
Hey, just fill up the drive with puns like that and your data will be lost forever.....It'll never be able to recover. ;-)
"Damn you worms, eat FASTER!!!! I can't stall the SWAT Team much longer!"
Thanks for the info. I was being a little facetious with a subtle reference to past policies which sent off user data at installation even though the user declined sending the information.
With a reputable company, you wouldn't have to pay if you didn't get your data back. That's fairly standard now.
If you really need it, then you pay for it, don't you. Yes, everyone should backup.
simple, fast homepage with your links: http://www.ngumbi.com/
Great response BTW