Slashdot Mirror
Why 'rm -R star' Isn't Enough
zdburke writes: "Short but interesting article in the New York Times (free reg req'd) about how difficult it is to cover your digital tracks because electronic documents are so well distributed -- on your lap top, on your workstation, on the server... Yes there are tools to thoroughly delete files on your computer, rather than just unlinking them when they're put in the trash, but it's the distributed nature of content these days that poses a special problem to the Ollie North's of the world."
On my harddrive space challenged machine, usually the reason I delete something is to make room for something else. So, chances are if they want "super-secret-MS-secrets.txt", the sectors have already been overwritten by "bspears-nude.jpg"
:)
It's quite possible to recover files, because, much like PCs nothing actually gets 'deleted'. The inode is marked as 'available for reuse' and removed from the directory entry, but doesn't actually remove anything. /dev/zero over a file just prior to erasing it work?)
Looking for an undelete? Take a look at the coroners toolkit. There's even instructions on how to recover files from a unix partition (any unix). It's one of those ones which you'd _really_ need to recover the data because it's hard work and a pain, but it is possible.
I don't recall seeing and 'write with zeros' program for Unix. I guess there must be some out there, since at a guess it's fairly trivial. (would dding
Of course, there's always disk analysis with an electron microscope, which I've always heard was possible but it's not one I've ever had substantiated.
Say you have important information on your hard drive. You only want one other person to see that information so you put that information on a floppy disk then give it to that person. No one else can see this information. You then take a pencil and stab the magnetic film of the floppy about 30 - 40 times. You then take lighter fluid and douse the entire floppy and light it. Stomp on the ashes for extra measure. Since the data has been on your computer. You must first take your hard drive out. Expose it to a giant magnet, then shoot it with a 12 guage (twice). Take all the IC's out of your computer and smash them with a sledgehammer, then run over them with your truck. Burn those with lighter fluid too. Since your monitor most likely displayed that sensitive information, you must take it to a helicopter and drop it. Have the helicopter land on the debris for safe measure.
Ensure that the other person gives your data to no one. Do a thorough background check on him and his closest 50 living relatives. After he is done processing the information; shoot him.
No need to worry about any information getting anywhere.
If you are concerned enough about your data to want to permanently delete it, or at least keep your tracks covered, you'll use PGP and either wipe your freespace multiple times to completely obscure data, and/or keep your important files encrypted.
Although encryption is, in theory, breakable, the resources to do so don't exist (unless the NSA has some quantum computers squirreled away somewhere), your files will be safe.
In short, if you want to keep files private, use PGP, and use it wisely. If you don't make more of an attempt, other than "well, if I tell Windows to delete it, it's gone", to keep files hidden/gone for good, you deserve to have your data recovered.
Gawyn
Freedom of Speech?
Yes there are tools to thoroughly delete files on your computer, rather than just unlinking them when they're put in the trash, but it's the distributed nature of content these days that poses a special problem to the Ollie North's of the world.
:D
Well, I don't think any OS has ever been short of undeletion tools - in unix, one can grep the inodes on a disk for a particular known string of a file and recover it fron a known template. Tools like gpart (a partition guesser) also easily recover those vital 512 bytes of your hard disk.
Where Unix has been lacking, behind most other systems, is the opposite - a good, reliable, trashcan. It might be interesting to note that there's now a reliable trashcan for Linux, BSD and other glibc systems th simply preloads and wraps unlink, `move and a couple of other system calls.
Since glibc is a part of the Linux Standard base, it works along with every LSB standard app. Even better, it doesn't matter whether you delete the file from KDE, GNOME, shittyunixtoolkitforhellcirca1980something or a terminal.
Anyway, check out Libtrash. And if you're a GNOME or KDE hacker, I'll give you a big hug if you use this as the default trashcan or your next release.
PGP is a brillient tool for encryption (esp. e-mail) and PGP disk or Scramdisk are great for secure archiving on windoze machines. However the PGP wipe isn't very good. This link explains why and gives good alterantives for windoze users.
Linux users already have encrypted filesystems and secure file wipeing as standard in all(?) common distro's. (I know that SuSE even lets you overwite the wiped files with zeros to hide its very existance)
Anyone quoted by a reporter knows how little they understand
Don't believe what you read is the truth.
I was hired to recover files from a hard drive by a woman who was getting a divorce. Her husband had been cheeting on her. The moron had norton systemworks installed on his system and never defraged his drive. I was able to recover over a years worth of incriminating emails with nortons undelete. Boy was that easy money
http://Lenny.com
4 great justice!
If you have problems destroying documents, you could always ask Arthur Anderson or Enron.
On some systems, rm has an option to nuke the contents of the file before unlinking it:
man rm
<snip>
-P Overwrite regular files before deleting them. Files are overwritten
three times, first with the byte pattern 0xff, then 0x00, and then 0xff
again, before they are deleted.
</snip>
You can just put "alias rm rm -P" in your login script to make this the default.
The man page for shred says
CAUTION: Note that shred relies on a very important assumption: that the filesystem
overwrites data in place. This is the traditional way to do things, but many mod
ern filesystem designs do not satisfy this assumption. The following are examples
of filesystems on which shred is not effective:
* log-structured or journaled filesystems, such as those supplied with
AIX and Solaris (and JFS, ReiserFS, XFS, etc.)
Using shred on ext3 does not seem to be a good idea. I use srm instead. srm overwrites the data 30+ different times using bit patterns and random patterns. The high number of overwrites is supposed not only to allow for slight deviations in alignment betweeen the drive heads and track on the platter, but also meets some very high (you might say "federal") standards, short of (or in some cases, followed by) incinerating the disk.
To-do List: Receive telemarketing call during a tornado warning. Check.
From the GNU shred info node:
shred overwrites devices or files, to help prevent even very expensive hardware from recovering the data.
Ordinarily when you remove a file (*note rm invocation::), the data is not actually destroyed. Only the index listing where the file is stored is destroyed, and the storage is made available for reuse. There are undelete utilities that will attempt to reconstruct the index and can bring the file back if the parts were not reused.
GNU shred is very featerful, as costumary in GNU utils, and has many flags to modify the behaviour.
BSD ppl are always praising the 'Unix Way' of small utilities that do a very defined job and nothing more, and hate the extended features that GNU utils provide; in this case it's BSD rm that is doing something that could be done by another tool by adding a flag! Horror!
Seriously, GNU shred is a good tool, and it can receive some interesting flags that a simple rm -P doesn't support.
cheers,
fsmunoz
For most of us here, the gov'ts electron-microscope method of determining old data is irrelevant. How many of you here think that it'll be employed against you? That said, I suppose for those of us who engage in a big-time trading of files via P2P networks, & DeCSS, etc, there's always the possibility of criminal prosecutions. So, let me go over the 3 types of "data deletion", and say where each should be used:
1. Typical deletion. Files are unlinked with their directories, so your OS does not "see" them and has more space available to write with. If the information is not sensitive, or you don't fear intrusion, this is the fastest, and also best, method of deletion. It simply changes the first character of a file name do something that your OS doesn't recognize -- a very fast process. The Advantage: data is recoverable via a data-recovery utility. The Disadvantage: the data has not been securely eliminated.
2. Simple once-sweep wipe-over deletion. Either random 1s and 0s, or wholly 1s, or wholly 0s, are written over an entire file. Use this for data that is sensitive, or where you fear cyber-intrusion by hackers. The Advantage: data is securely eliminated, beyond the reach of anyone who hacks into your computer. The Disadvantage: data is irrecoverable to you, should you realize you made a mistake, and this process is slower.
3. A multi-sweep wipe. Same as above, but many sweeps are performed, enough to make typical electron-microscopy methods of data-recovery inviable. This method effectively makes data irrecoverable by any means. Electron microscopes can detect "old zeros" by ghost-patterns, a slight trace. But if data has been written over many times, the older data is impossible to recover even by those methods. The Advantage: this method securely removes the data, beyond the reach of any technological means. The Disadvantage: this method is very slow, and again, data is irrecoverable should you learn you made a mistake.
It should be noted that whenever you want to securely delete data, not only do you need to wipe the file, but you also need to wipe your swap files and your temporary files.
So, let me summarize when each of the methods of "data-removal" should be used, starting with the strongest method (a multi-sweep wipe), and ending with the weakest method (the renaming of the first filename character to something unrecognizable):
1. A multi-sweep wipe. Use this when you have data on your computer that could be used against you in a lawsuit or prosecution. For example, certain kinds of pornography, copyrighted files, warez, and other various information that's been deemed "illegal" by the Information Police in the MPAA, RIAA, MS, and the US Gov't.
2. A single-sweep wipe. Use this for information that is sensitive, but that you need not fear should the government get ahold of. For exmaple, financial files, files containing credit-card information, etc -- anything you'd want to protect from online-hackers using data-recovery programs. The government, though draconian, has not been known to steal people's credit cards using electron-microscopy. Similarly, hackers have not the resources to use electron-microscopy to acquire your credit cards -- nor would it be worth it. However, if your a high-tech company selling your computer equipment to another company, a multi-sweep delete of your files may be necessary to protect your information from competing companies, who may have bought your machinery through another company as a front.
3. A deletion that dissociates the file from the directory (renames the 1st character). Use this for non-sensitive data. For example, stories you've written, calendars, lists, ideas, old programs, pictures, etc etc.
Hope this has been helpful -- and please, remember, if you want to securely remove sensitive data either by a single-sweep wipe (to protect it from hackers) or a multi-sweep wipe (to protect it from the government), please remember to also securely remove swap files and temporary files as well!
social sciences can never use experience to verify their statemen
Was it the orange stains on his hands and the faint odor of cheese that gave him away?
I'm surprised I've seen no discussion here of the very basic problem of file slack space - that unallocated space at the end of the last sector of every data file, except those that exactly fill a disk sector. Most of the methods described here for easy ways to wipe empty hard drive space do not overwrite all the file slack space. You need a program that does that explicitly. Otherwise every sector with the tail end of a file contains easily recoverable data, although disassociated from any filename. Given that the slack space on a hard drive averages out to $sectorsize*$numfiles/2 (on average, 1/2 of a sector, times the number of files), the average 40Gb hard drive with 10,000 files might have 50Mb or more of recoverable data, even if the "empty" space were completely and unrecoverably wiped.
I learned about this while preparing to publish a program commercially, and discovered that (at least at the time) files I copied to the distribution media master sometimes contained sensitive data, such as the source code, from my own hard drive. Basically, DOS wasn't very picky about copying a few extra bytes along with the actual file length, as long as the extra bytes didn't go past the end of the destination sector. The answer? I used a slack wiping program on the master disk before sending it for duplication.
--Brandon / Split Infinity Music