Slashdot Mirror
SSDs Cause Crisis For Digital Forensics
rifles only writes "Firmware built into many solid state drives (SSDs) to improve their storage efficiency could be making forensic analysis at a later date by police forces and intelligence agencies almost impossible to carry out to legally safe standards, Australian researchers have discovered. They found that SSDs start wiping themselves within minutes after a quick format (or a file delete or full format) and can even do so when disconnected from a PC and rigged up to a hardware blocker." So either SSDs are really hard to erase, or really hard to recover. I'm so confused.
Deleted, should mean deleted.
No really! What's the down side?
Lately all you have heard is the complete opposite. That they are impossible to completely erase so it's unsafe to store company/secure data on them. Because even if you erase the file its still left on the disk and just marked as empty. Now they say they erase them self.
At a guess this is caused by mounting with the discard option, or trim as its called in Windows. It tells the drive you don't need the data stored where a deleted file used to be.
Maybe it's still there if you look with a microscope but who really does that?
You need to disassemble the drive and read the memory chips independently of the controller. I believe I read this is how one of the major drive recovery companies is handling SSDs.
Problem solved. People need control over their own privacy. Tough luck Digital Forensic folks.
Next, expect law enforcement to clamor for a new law that mandates persistent data retention for all types of storage devices.
..destroyed overnight, go with the SSDs. The melting point of a surface mount IC is a lot less than that of a spinning platter.
So either SSDs are really hard to erase, or really hard to recover. I'm so confused.
All I know is that if SSDs were really hard to erase, and I was in the business of recovering data that other people didn't want recovered, this is exactly the kind of story that I would tell them so that they would continue using SSDs.
Not that I'm paranoid or anything.
On magnetic storage I can change controller boards, even swap out the
platters in a clean environment into another drive with working heads.
For a few hundred to some thousands, your poor choice of having no backup media
can be resolved.
On SSD I can desolder the chips, dump them and then tell you there's nothing recoverable.
For a few hundred to some thousands, your poor choice of having no backup media
can be resolved.
Ultimately since the Flash Translation Layer goes and does things under-the-hood that are not externally visible, it is hard to be sure your data were erased, and it's also hard to be sure they were not erased... Essentially since there is an opaque interface at the logical-block level and the device is internally free to behave as it chooses so long as that interface is maintained, it makes it tricky to guess how the internal implementation will behave.
Plain old magnetic disks used a fairly predictable implementation of that interface so forensics goons got used to having an easy task on their plates.
---
Play Six Pack Man. I
Why the confusion, dear editor? This should be well understood.
If you want to recover, you can't. If you want to erase, you can't. It's Murphy's Law of Data Storage.
Forgive my ignorance, but how is this possible? Does this mean that the drives understand NTFS and are actually zeroing out data on the drive when the OS simply deletes the entry from the FAT table? How can the SSD second guess what the OS is doing? I thought that SSD's use the same interface as regular HD's and should behave the same.
...we better ban them, then.
-- Even if a god did exist, why the fsck should I worship it?
The whole point of the referenced article is that it is somehow a "problem" that data deleted (and intended to be deleted) by the owner of the SSD cannot be later recovered. Why should deleted data be recoverable? Will "police state" now require SSDs to stop this seemingly desirable behavior to ensure evidence be recoverable from an impounded device? I for one applaud the behavior of these new storage devices.
So either SSDs are really hard to erase, or really hard to recover. I'm so confused.
Just RTF studies and I'll think you'll find the first (the one that concludes current SSD's are not purging their stored data as they should according to standards) is more thought out and thoroughly tested. IOW, I trust the first paper over this latest one.
would mod you up one for that.
So the bad news is that an exploit of an accidental side effect of an existing technology is not always possible to duplicate in newer technologies. I guess that means the digital forensics folks will have a harder job doing things with disk drives that they were never intended to do. I don't see the "problem."
The real problem, as I see it, will come when the digital forensics groups push back on disk manufacturers to change their purging routines in order to improve data retrieval (possibly at the cost of performance). You know, to keep the accidental exploit backwards-compatible.
Quote: "So either SSDs are really hard to erase, or really hard to recover. I'm so confused."
;)
I work in a professional environment where we attempted to recover data from a crashed SSD. Nothing can be recovered. Consider the way an SSD Works. They are extremely expensive because each one contains a memory bank like RAM and a processor to handle reading and writing. If an operating system has "TRIM" enabled (or implemented to work like in Windows 7) then it will delete when a user deletes a file. It writes over the blocks with blank space. This ensures that writing speed does not slow down during the use of the device. So anynill delete when a user deletes a file. It writes over the blocks with blank space. This ensures that writing speed does not slow down during the use of the device. If thing deleted on a drive like that is really DELETED and cannot be recovered. -- Little google goes a long way
The drives have internal overprovisioning and perform internal garbage collection. This means that marked for deletion data has an unknown lifetime and may disappear at any point without interaction from a controller.
The hard to erase bit means that you really can't be sure something is totally erased without a full specific erase command to all flash blocks. Without that a page marked unused but not erased may be nestled in with a bunch of valid pages. As all pages in a block are erased together that marked unused page can hang around for a wile.
On the other side the firmware does garbage collection it actively looks for blocks with many erased pages and then tries to consolidate things so it can create more free blocks. This means if the drive is powered but not connected to a host machine it can still be doing data moves for consilidation and erasing marked for deletion pages.
There are thresholds for the garbage collection so it won't overwork and try for 100% consolidation. Thus you get both the presence of some really sticky stale marked unused pages and some active erasing of others.
I'm on the fence about this, and it's possible neither pasture is green. On the one hand, I might be the victim of a genuine crime, evidence of which happens to be hiding in an SSD drive. On the other hand, these techniques are just as routinely abused now to go after people for political noncriminal reasons that don't serve the Common Good at all, people and organizations like Julian Assange, Wikileaks, Bradley Manning, the U.S. Chamber of Commerce opponents... you name it.
These techniques are like nuclear physics: just as easily applied for Bad Things as Good. If we can't selectively prevent the abuses, maybe we should err on the side of caution and ban the techniques altogether. They aren't being universally applied to serve justice.
I thought that this was particularly telling. In the article it said:
... the state of the drive cannot be taken to indicate that its owner did or did not interact with it in ways that allow prosecutors to infer guilt or innocence. The fact that data has been purged does not mean a human knowingly did it (e.g. accidental guilt)...
So in other words, until SSDs came along, evidence of purged data was evidence of guilt... at least in Austrailia.
"So either SSDs are really hard to erase, or really hard to recover. I'm so confused"
It's easy - if you need it back, it will be hard to recover. If you desperately depend on nobody ever seeing it, it will be hard to erase. I'm pretty sure this is a consequence of the Uncertainty Principle, but I have not yet completed my paper proving it.
"Firmware built into many solid state drives (SSDs) to improve their storage efficiency could be making forensic analysis at a later date by police forces and intelligence agencies almost impossible to carry out to legally safe standards, Australian researchers have discovered..."
So expect some government intervention on matters concerning which firmware should be built into the devices we use.
I cannot see any government worth its credibility endorse a product which if employed in crime and confiscated (by police), it is almost impossible to use it to prosecute the perpetrators by government agencies and the FBI in the case of these United States.
You might wonder how a government might endorse a product:
By allowing its importation or production and subsequent collection of taxes from transactions related to the product.
could be making forensic analysis at a later date by police forces and intelligence agencies almost impossible to carry out to legally safe standards
So then they're sending SSDs out of the country for hard-core, waterboarding-style data extraction?
sysadmins and parents of newborns get the same amount of sleep.
...a foregone conclusion ever since ATA Secure Erase and TRIM were introduced?
Secure Erase basically tells the SSD that all of its cells are now blank (AFAIK implementations actually zero the drive as well but I'm happy to be corrected on that); therefore as soon as anything is written to the disc, it will be written here, there and everywhere. It took about 30s to run on my first vertex and I couldn't find any trace of
TRIM support in the ATA spec, along with kernel/filesystem support, tells the disc that when file A is deleted, cells X, Y, and ABQ are now officially "empty" and that if the controller feels like it, it can zero them out, shunt other data in there, or have a mardi gras for all it cares. The same happens when a drive is formatted; OS tells drive controller "I've just formatted you" and for the sake of preserving performance the controller goes "Brilliant! I can chuck out all this shit I've been saddled with."
As soon as hard drives start intelligently erasing/shuffling bits of themselves about so that cells are utilised to their utmost efficiency this was bound to happen. Unlike spinning platters where bad blocks were reallocated only if a) the hard disc knew about it and b) the data could actually be read/recovered, it becomes terribly obvious that data on SSD's is going to be read and written and deleted completely and utterly all over the place, without sequential series of sector found in slackspace like you would on a magnetic drive.
Magnetic drives have no performance penalties for not actually erasing the data, so if you work your way around that double negative you'll see that one of the staples of digital forensics (e.g. recovering files from slack) is a by-product of people trying to make magnetic platters as fast as possible by not actually erasing stuff, because as long as the controller knows that sector is blank then it'll just be overwritten as needed. Technology has now changed sufficiently that the performance gains from new solid state tech are helped by a drive controller that erases stuff as soon as possible, since writing over an occupied cell is slower than writing over a blank one.
I'm sure there'll be new methods to mitigate the change in tech, we're just somewhat on the cusp of a completely new tech. They'll probably come to an agreement that TRIM doesn't actually delete stuff until the amount of free space in the cells reaches a certain threshold or something like that.
Disclaimer: I'm not a digital forensic scientist, but am friends with one and we discussed this problem over some exquisite cocktails a few months back. And I don't think TRIM instructions follow the exact specifications I laid out above (e.g. using Brilliant! as an ACK).
Moderation Total: -1 Troll, +3 Goat
It was the other way around last week, no? If you really care about privacy you encrypt your personal data, so irrevelant for most folks here.
(1.) It may be hard to securely erase an SSD. Due to things such as wear leveling, the relationship between sector addresses and physical flash cells isn't transparent to the OS. And ATA Secure Erase isn't implemented or isn't implemented correctly on all SSDs. (2.) SSDs are hard to recover. That's because they may start erasing some blocks containing data (and not just the entry in the file allocation table) shortly after you delete a file in the file system. Again, this happens due to things such as wear leveling and isn't transparent to the OS. Contrast this to a hard drive where, following a file delete, only the entry in the allocation table is deleted but no actual data. I don't see anything contradictory or confusing here
Why does the government have this expectation that technology should be built in order to make it easy to spy on citizens?
I don't now remember what the case was, but: a few months ago a read about a guy who was charged with some crime or other. They were unable to convict him of whatever it was, but they did convict him of obstruction of justice. Why? Because the computer forensics expert stated that he had deliberately deleted some files and then run a defrag.
Enjoy life! This is not a dress rehearsal.
I think I've read somewhere that evidence has also be reproducible by the defense. If you destroy the device in the process of recovering data, that might be hard to do; or not ... I'm just guessing really.
I think some tests inherently destroy evidence. For such cases it may be that the defense has the right to observe the testing to ensure that it was done properly.
In case the harddrive is full disc encrypted it all should not matter...
dd if=/dev/sdx of=/dev/random :P
(Please don't copy and try to run this....you might regret it.)
... which might be considered tampering and leaves room for the other side's lawyers to ask "and then you took a soldering iron to a delicate IC?" ...
Because the odds of the randomly generated bits creating an email to Bernie Madoff discussing the ponzi scheme falls within a range considered to be reasonable doubt? You would need a fairly ignorant and gullible jury to buy that ... oh wait ... OK that may work for a celebrity defendant but I wouldn't count on that saving the average guy.
I liked Reiserfs. Now you can bellyache about Hans Reiser and his murderin' ways, but the filesystem was the first really nice self-cleaning file system I had seen. The packed tails is nice (20% more disk space utilization), and the journaling (which makes the filesystem look as secure as a database). But when you delete a file, its self-cleaning, self-pruning. If you delete a file, blocks which had some of that data may be overwritten with new data in the name of file system efficiency, and all of the old data may be scrambled in an incoherent mess. Its kinda like the digital version of a paper shredder. There is no guarantee that the data is wiped out, but it takes a lot more than a hidden bit to get flipped to bring it all back. You have to dump the contents of the entire drive, and then try to reassemble the tree, some of which might be overwritten by other parts of the tree. Enjoy!
The TRIM command lets the OS tell an SSD that specific blocks no longer in use should be zeroed. Why do this?
On traditional magnetic hard drives leaving a block's contents untouched while marking it empty in the filesystem was standard practice. It's fast and easy and if the block we reused the data would simply be overwritten with no performance penalty.
*****For various reasons related to flash technology itself this is not the case with an an SSD. Overwriting previously used non-zeroed blocks involves a significant performance penalty. (This is a gross oversimplification but is more or less correct.)*****
TRIM lets the OS clean up old previously used blocks on an SSD so you have pristine, clean, fast 0'd flash for new data. - But the OS has to initiate it.
What the article is referencing here is a different mechanism other than TRIM causing the same thing to happen. That is, many new SSDs have automatic internal routines that achieve the same goal that the TRIM command does. During idle time they search for unused space and clean it up automatically to avoid performance degradation. - It seems that if you quick format an SSD it will go about automatically zeroing out all the previously used flash cells simply whenever it's powered on. That makes forensic recover all but impossible for conventional tools.
I don't see TRIM-like schemes being outlawed.. But maybe a gentleman's agreement from drive manufactures that will prevent a drive from trashing data if, say, a specific type of 'police security/forensics' dongle was attached to the drive.
My data shouldn't be easy to get off, I know a while ago there was a post about SSD's being high risk for data but now it appears there more safe? If the police want my data they can figure out how to get it off, of course if I'm doing something illegal I'd have it highly encrypted but then again why would I care. They want the data they can work to get it!
Plaintiff's Attorney: "Sir, what are the chances of the drive automatically generating the exact sequence of bits required to form this email?"
Expert Witness: "Billions to one, certainly."
Defendant's Attorney: "And how many times will that this 2KB email fit on the drive?"
Expert Witness: "Well, it's a 2TB drive, so... about a billion, give or take."
Defendant's Attorney: "So, assuming the data on the drive is random, then it's safe to say there are at least two billion opportunities on this drive to produce this email?"
Expert Witness: "That's not what I meant..."
Defendant's Attorney: "Yes or no?"
Expert Witness: "Well, yes, but..."
Defendant's Attorney: "No further questions"
Learn about Photography Basics.
Everything government does can be explained by cash flow. The simple reality is that this is an opportunity to expand the business of government in terms of power and revenue. They can sit back and do nothing (what's in that for them?) or they can exploit the situation, demanding more power and revenue which will be "required" to "solve" the "problem".
Knowing that governments only expand in power and revenue throughout their lifetimes, never willingly or permanently relinquishing power or revenue (history proves it), what do you think a person at the top of the power pyramid would do? It's a no branier: seize the opportunity to further enrich and empower their business.
It's all in how you perceive it. So delete might really work now and how is this bad ?
It used to be the drive remapping meant that if you deleted something (or even overwrote it), there was no guarantee it would be gone - SSD controllers do wear leveling to avoid having part of the drive get used excessively. Forensic analysts could go pull the raw data from the NAND flash.
The problem was that the wear leveling algorithms need a "free block" pool to work well. Drives that have been used heavily deplete the free block pool, and the drive slows down. For a long time, SSDs would have no knowledge of whether a file had been deleted or not.
The ATA TRIM command was added for just this purpose - with TRIM, when the OS deletes a file that references a block, it can tell the SSD controller that those blocks are free. The SSD controller will then begin erasing those blocks in its free time. (SSDs can be written one block at a time, but must be erased one page at a time. A page consists of multiple blocks. Oh, and I may have page/block swapped here.) So you get a lot of performance improvement by having a bunch of pre-erased pages - these can just have individual blocks written without a read/erase/modify/write on a whole page.
Pre-TRIM, SSDs were probably great for forensic analysts. Post-TRIM, SSDs are not. Oh, and I think the latest ATA standard added a "sanitize" command to make life easier for information assurance types, for whom SSDs have always been a pain.
retrorocket.o not found, launch anyway?
Some good news for a change.
This is great news for r@ygold, hussyfan, kingpass, vicky series collectors course knowing hard drive manufacturers they will cripple the ssd's somehow so they keep ghosts of the files like older drives did
Only criminals are worried about protecting their personal data. Really? Doctors, lawyers, researchers, need I go on?
Only criminals have guns. You what? Police are criminals in the US then? Soldiers are criminals?
Grow up and get a new perspective.
Encrypt everything - even if you have nothing to hide, it's a good habit to get into. SSD's are simply different from standard HDD's - it'll take a while for the various agencies to catch up. Of course, if your goal is to hide data - multiply encrypted drives with hidden partitions should be part of your standard-operating-procedures. You should also have empty drives filled with nothing but random data - like is usually done to test for bad sectors - connected to your network just to keep life interesting.
So either SSDs are really hard to erase, or really hard to recover. I'm so confused.
It's both. The internal optimization of SSDs includes, essentially, a degree of abstraction between what the computer says to do and what is actually done. With that kind of direct low-level control taken away, any task related to direct, granular control over what happens to what the computer sees as 'sectors' on a disk (but which don't really exist, since there is no such physical form) becomes unreliable. The reasons why it is unreliable differ between deletion and recovery, but the effect is the same and has the same basic root cause.
For your security, this post has been encrypted with ROT-13, twice.
The expert witness is patently incorrect: Billions is 2^32, but with 2KB that's 2^2048 which is somewhere around a 1 with 616 zeroes after it.
Support my political activism on Patreon.
So, if I am using another filesystem, and have something on it which happens to look like NTFS metadata and happens to sit in a place where NTFS metadata is usually found, will these SSDs start erasing random sectors of the disk?
Everyone involved in that exchange failed basic math.
Assuming seven bit ANSI, that's not even five characters to get a billion combinations.
And a 2k e-mail would be 2^2048 combinations.
Plaintiff's Attorney: "Sir, what are the chances of the drive automatically generating the exact sequence of bits required to form this email?"
Expert Witness: "Billions to one, certainly."
errmm, a quick run with numbers: 128 possible values for a single byte and lets say 1 KB messages, that would be 128^1024 possible combinations, wouldn't it? Which is WAY more than 'billions'. Not much of an expert that witness.
Here we go again!
I've spent the last hour writing replies for questions/comments on the thread using this throwaway account: could someone with uber-powers please mod them up a bit?
Thanks very much in advance, and thanks also to everyone making suggestions/comments about the article. :-)
Graeme.
The melting point of a surface mount IC is a lot less than that of a spinning platter.
Considering that all you need to melt a hard disk platter is a flower pot, a haird rier, and some charcoal that shouldn't be any problem.
Hello, I'm one of the authors of the paper. To explain the apparent paradox in rough terms:
Drive data was traditionally purged manually, by having the computer tell the drive to write something else over the top of the old data. In the absence of such an overwrite, magnetically stored data persists. However, if you try that trick on an SSD, it may not work. The logical address you try to overwrite may be remapped on the fly, so that your 'overwrite' goes to some other physical cell rather than the one which stored the data. From a logical viewpoint, it looks like the overwrite worked - you can't access the data any more through your computer's OS. But from the drives point of view, the data is still there, lurking in some physical cell that is presently out of use as far as the logical sector list is concerned. A cunning firmware or a hacker with a soldering iron might still get at it.
However, separately to this, modern SSD drives use tricks to try and automatically improve their performance, and one of these tricks is to pre-empetively wipe data cells that contain data no longer referenced by the filesystem. Here, the drive is actively attempting to permanently purge everything it can from the drive, all of it's own accord, in the interests of accelerating future writes by having a pool of completely unused cells available.
Summary:
- If you're a computer telling a drive to zero over some data, the drive may lie to you a bit, and not bother to zero it.
- If you're a drive, you do whatever the heck you like, and you see the physical layer directly (unlike the computer). That means the drive can open up the NTFS metadata, looking for data cells which could be preemptively reset, and nuking that data out of existence (when it might traditionally have been recoverable to an expert).
In summary. If your drive wants to nuke something, (and we've shown, they really DO want to nuke everything they can at a few minutes notice), it gets nuked. If your PC wants to nuke something, it may or may not get nuked by attempting an overwrite.
Finally, separate to this is TRIM, which is a hybrid of the two situations - an ATA command by which the OS can signal to the drive that it would like the corresponding physical cell for a particular logical sector address to be nuked, thank you very much.
Hope that clears things up.
Graeme.
Grow up and get a new perspective.
*whoosh*
Dead iPad? $1,000 can bring your data back. It explains the process for recovering data from the iPad's flash storage, but SSDs would be fairly similar, although I imagine each one is just different enough to cause some pain.
Come on, even I knew that was supposed to be funny.
So, certain SSDs have a firmware "garbage collection" that analyzes the file system and marks blocks that are unused, even when the OS does not issue any trim commands.
While perhaps a nightmare for forensics, this seems like a particularly useful thing for normal use.
Presumably it only works for NTFS, but it would be very useful for windows xp, which I assume does not fully support TRIM.
How can I tell which ssd supports this option?
is there a marketing name for it?
Whoosh.
OK, so police and intelligence agencies now have a very hard time reading the private data of other people. And yes, I can see how government worshippers who get agitated at the thought of any limit to the absolute power of governments might be upset at this.
Myself, I don't think governments have any more right to peruse my private data on my private hardware than, say, Microsoft or AT&T do. So I don't find any problem here.
Actually the number of possible combinations are 2^(2048*8) bits. I'm just going to guess here that 2^16384 is more than the number of atoms in the galaxy, or even the universe.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
There's still "reasonable doubt" that the prosecution wasn't corrupt and faked the data, if you can't reproduce the recovery a second time. One time is good enough if you've lost important data, but not good enough for criminal evidence.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
What about, not answering yes or no with a yes or no. Also the attorney can't cut the witness off, he can just say, "I wan'st finished answering". They don't have the right to edit / truncate your answers.
Defendant's Attorney: "Yes or no?"
Expert Witness: I was using "billions" metaphorically not as an exact figure. There is about 3240 characters in that email so it would occur randomly about 2^3240 ~ 10^1080
you're right, although the relevance of the point still stands. Billions vs 1 billion is different from billions vs 10^616. But yeah.
Support my political activism on Patreon.
This source estimates the number of atoms as 4*10^79, which is between 2^264 and 2^265, which is negligible compared to 2^16384. Even if the estimate should be a few dozen orders of magnitudes wrong, it still wouldn't come anywhere near.
The Tao of math: The numbers you can count are not the real numbers.
Sure, it makes forensic data recovery harder, but I see no reason other than $ that the feds can't disassemble the things, desolder the RAM from the controller, and attach it to their own custom-built data-extraction device.
Sure, there may be some devices out there that are pretty much impossible to disassemble in this way, but I'm sure Congress will quietly pass a bill making any future device that isn't "forensically recoverable" in this way illegal without special government permission.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
The same properties that make it hard to prove you deleted the contents make it hard to *prove* what the contents were. No surprise here really. SSDs are a b*tch no matter which side you're on.
For hiding stuff, the best policy is to never store unencrypted data on the device. Install full-disk encryption, then data.
For recovering stuff *to legal standards*.. Who knows. The courts are fickle at first, but standards of evidence will emerge over time.
Blessed are the pessimists, for they have made backups.
You're missing the point, though. The questions isn't "are there are enough monkeys locked in the room to write the complete works of Shakespeare?" - it's about the fact that the numbers are extremely large, and difficult to comprehend for the average juror. If they don't understand the math, then there is a reasonable (in the mind of the juror) doubt.
And yes, a lawyer can ask a yes/no question, and the judge will typically compel the witness to answer it in that fashion. There may be further discussion before or after, but a yes or a no will be the result.
Learn about Photography Basics.
It's too bad - or perhaps good if the cops seized your PCs and you have a good lawyer - that the devices don't have both a "high-level" interface and a "low-level" interface, where the "low-level" interface gives you complete control over the device - no writes happen without your explicit say-so. Couple this with a "disable automatic background behavior" pin that's checked on power-up, and you should have a device that's not only forensically readable, but much easier to do scientific research or any other task that requires predictable repeatability.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Is this only true for NTFS, or does the firmware understand ext3/4? I know that in-kernel TRIM for ext3/4 is out in Linux >2.6.34.
I hate my 2001 SL-2.
Why is it so hard to only have politicians for a few years, then have them go away?
This story comes just 2 weeks after another story on slashdot saying that your data doesn't actually get deleted:
http://hardware.slashdot.org/story/11/02/17/1911217/Confidential-Data-Not-Safe-On-Solid-State-Disks?from=rss
An earlier responder said this:
"......since writing over an occupied cell is slower than writing over a blank one."
I'll believe that this is true (because I read it on the Internet).
But I don't know anything about SSD technology, or why it is slower to write onto a non-blank thing than it is to write on a blank thing. Why is that true?
The solution to this would be to add a new command to the ATA spec, call it FREEZE or something. If a drive receives this command, it won't do any writing/erasing, neither on its own or in response to ATA commands. Then you just create write blockers that also send this command.
Of course, this both requires new drives and new write blocker hardware, so it probably won't be implemented.
Want to wipe deleted data off a drive? It's really simple, there is a method I always use before making an image of a drive. dd if=/dev/zero of=/location_on_drive Wait for the file to use all the space on the drive then delete it. I use it to make drive ghosts more compressable. Though if you wanted it for privacy you could always source from urandom instead of zero.
You're off by so many orders of magnitude, it hurts.
The chance of generating that particular e-mail, if it's 2 KiB, is 1 in 2^16384. (That's not really the number you want -- you want the chance of generating a similarly-incriminating e-mail. They're roughly equally improbable, though.)
So that's 1 in 2^16384 compared to 2^30 copies of the e-mail that would fit on the drive.
If the prosecution scoured every drive ever made for a random sequence of bits that looked like that e-mail, they'd never find it.
Plus, the expert witness generally has a better and less technical answer. Defense lawyers don't ask technical questions like that: the expert witness will undoubtedly have an answer, and you'll bore the jury to tears, which they hate.
I don't really care what argument they make, police should not be asking for this kind of thing. Either a law enforcement agency can make a case against you with evidence they already have, OR THEY CANNOT MAKE A CASE AGAINST YOU. What's in your possession or on your hard drive or whatever is irrelevant. If they don't have a case against you long before they need to look there, they shouldn't need to look there in the first place.
Billions to one isn't the right answer. Answer is 2kb = 16,000 bits = 2^16000 = the probability.
the odds are one in a number beginning with 3 and having 4,816 digits.
That's a big hard drive!
You are missing the point. "How many 2KB sets of data could be on the disk? A billion" is being played off against "What is the chance that this email appeared in random data? 128^1024" which is also true.
So the weasel question is "So, assuming the data on the drive is random, then it's safe to say there are at least two billion opportunities on this drive to produce this email?" - (Actually it is even more - who says that the email needs to be byte aligned?).
The answer is "Because the size of the email is relatively larfge there is about the same amount of chance as all the atoms in your underwear jumping a meter to the left in the next 5 seconds - it is very, very, very improbable".
In my opinion, all filesystems should operate this way. Digital forensics is a joke. With all the rootkits available today, who is to say that digital "evidence" isn't planted? How can you tell that a set of ones and zeroes was "without any doubt" placed there by the defendant? If the government wanted to finger anyone, at any time, all they need to do is "find" a reasonable cause to search someone's computer where they happen to know evidence will be found--because they put it there.
Ever been to court? Whether you can answer more than "yes or no" to a "yes or no" is rather up to the whims of the bewigged asshat at the front of the room.
Jesus was all right but his disciples were thick and ordinary. -John Lennon
Yes I feel your confusion too. The wear-levelling electronics makes it in theory hard to erase data from SSDs. But this is suggesting 'efficiency' electronics (details) may scramble things anyway.
Do remember, however, that a lot of forensics is "CSI'd up" compared to what is actually achievable. It is very difficult and usually impossible to recover files from a harddrive that has had even a basic multipass erase performed on it. The trick of retrieving data from chilled RAM chips just post shutting off the motherboard is actually very difficult to do usefully under non controlled cirumstances. Tracing network activity.... well we all know about that one and the MAFIAA is really finding out about it too recently.
So - I imagine things will err more in favour of NOT being able to easily recover secrets off SSD's. <usualcopout>Time will tell.</usualcopout>
Yes I have, I've testified in multiple trials. And the "bewigged asshats" have never pulled anything remotely like that.
Which is WAY more than 'billions'. Not much of an expert that witness.
lol...well it is billions, just an absolute freakin' shitload of billions ;)
Well what are the laws right now for servers? Mainframes/Minis/Servers for decades have had automated processes running on them. They have all sorts of custom data formats so you can't use standardized tools....
Why can't we just tell the drive to write 0's to fill up ALL remaining space. That should guarantee that all "erased" space is filled with 0 regardless of how SSD remaps blocks.
Lucky you.
Jesus was all right but his disciples were thick and ordinary. -John Lennon
Well it was never my money at stake.
What judge would want:
During appeal: "Mr Bolden you were cut off by the judge is that correct"
JB: Yes
DA: Does this answer accurately reflect your opinion at the time?
JB: No
DA: Could this answer have misled the jury
JB: Yes.
etc...
Ah, I see the trick: better attorneys than I could afford.
Jesus was all right but his disciples were thick and ordinary. -John Lennon
Doing that is insane. It breaks the contract of a block device. If I stripe a logical volume across multiple SSDs, I don't want one of the SSDs to think it's holding a regular filesystem and start erasing the wrong blocks.
This was known way back in 2009.
"There is much pleasure to be gained from useless knowledge." - Bertrand Russell.
Isn't it more like that window7 support TRIM, meaning that it will issue a trim command to the SSD after a delete? The SSD firmware already has enough worries then to dive into a native file system.
Are you sure you are not mistaking this for simply queued trims? A drive can be told to trim extreme amounts of data, and it can even store these requests, and continue them the next time it is powered up.
If what you said is true, that drives can read the file system format and decide to clear sectors on its own accord, all of the following things would be true:
-Encrypted drives would be slower than non encrypted drives, because the drive wouldn't know the file system.
-The NTFS journaling undo system would not work, because the drive would have deleted sectors that the journal tries to recover.
-NTFS extensions would not work, because they can specify the use of sectors in a format that the drive cannot possible know about. For example complicated virus protection systems hook into the ntfs windows system and prevent it from overwriting sectors that contain the anti virus programs. These sectors are not allocated in the bitmap or in the MFT, even disk ghosting programs can miss them, yet with what you are saying SSD's would completely wipe them.
-Encryption programs with plusible deniability have partitions in the unallocated blank space of a main partition. The programs work by stopping windows from assigning the sectors. If what you are saying is true, the drive would go around deleting everything.
Infact I think you are deeply mistaken. SSD's will dynamically reallocate sectors so to prevent fragmenting in portions that are not equal to the wiping block size. But this is just moving data about. The very idea that a drive would delete data of its own accord is unbelievable and completely breaks the abstraction between the drive and the computer.
Hope that clears things up.
No pun intended, eh?
SSDs can do this? Call me when they can do STDs and I'll be more impressed.
"Be polite, be professional, but have a plan to kill everybody you meet." General James Mattis
So does running the Linux command shred completely obliterate data stored on an SSD, or not?
Ceci n'est pas une
A typical SSD is hampered by it's HDD FTL that *cannot* know what you want to do, so all operations except reading are very slow.
The only reason we use the HDD FTL is because of some operating systems don't support any good filesystems.
Since flash memory is different than a HDD, pretending it's not, is always going to be a headache, and cause unintended problems for the user.
It seems retarded that FTL coders are now trying to second-guess the operating systems by trying to understand what filesystem *may* be on it.
This will obviously cause any amount of headache for users, if data written on the block device just happens to look like fat32 or ntfs...
There's a much cooler and older technology out there, which is connecting flash directly to any of your fast buses. And using a modern filesystem that works well on flash, such as ubifs.
Nokia n900 was a good device to test the pros and cons of using flash directly (via ubifs) or indirectly on an MMC.
The performance on the direct flash memory totally overwhelms that of the indirect memory. in addition, you know things like bad blocks, can execute in place, append and proper compression support.
And most likely your flash memory will last a decade longer in normal use.
Most importantly, using flash directly makes this article pointless, since you *know* what has and hasn't been deleted.
Surely you can still securely erase a drive by zeroing the whole drive (either by making a file the size of the filesystem, or by dd'ing the entire drive under /dev)?
Also, how does the drive firmware know what blocks the filesystem has allocated? Reading metadata has got to be risky and difficult (NTFS is proprietary too). Furthermore people are free to reformat their drives to some other filesystem.
I, for one, welcome our new ssd overlords.
PURGE PURGE PURGE
Bonus. Captcha: monotone
On a magnitude scale, you would have reached atoms in the known universe not too long after 2^256. 2^16384 is thousands of magnitudes more than that.