Slashdot Mirror
Why 'rm -R star' Isn't Enough
zdburke writes: "Short but interesting article in the New York Times (free reg req'd) about how difficult it is to cover your digital tracks because electronic documents are so well distributed -- on your lap top, on your workstation, on the server... Yes there are tools to thoroughly delete files on your computer, rather than just unlinking them when they're put in the trash, but it's the distributed nature of content these days that poses a special problem to the Ollie North's of the world."
I always like to add the "f" right after that "-R"...
:>
That good enough for ya?
It's quite possible to recover files, because, much like PCs nothing actually gets 'deleted'. The inode is marked as 'available for reuse' and removed from the directory entry, but doesn't actually remove anything. /dev/zero over a file just prior to erasing it work?)
Looking for an undelete? Take a look at the coroners toolkit. There's even instructions on how to recover files from a unix partition (any unix). It's one of those ones which you'd _really_ need to recover the data because it's hard work and a pain, but it is possible.
I don't recall seeing and 'write with zeros' program for Unix. I guess there must be some out there, since at a guess it's fairly trivial. (would dding
Of course, there's always disk analysis with an electron microscope, which I've always heard was possible but it's not one I've ever had substantiated.
If you are concerned enough about your data to want to permanently delete it, or at least keep your tracks covered, you'll use PGP and either wipe your freespace multiple times to completely obscure data, and/or keep your important files encrypted.
Although encryption is, in theory, breakable, the resources to do so don't exist (unless the NSA has some quantum computers squirreled away somewhere), your files will be safe.
In short, if you want to keep files private, use PGP, and use it wisely. If you don't make more of an attempt, other than "well, if I tell Windows to delete it, it's gone", to keep files hidden/gone for good, you deserve to have your data recovered.
Gawyn
Freedom of Speech?
http://archive.nytimes.com/2002/01/14/technology/e business/14DELE.html
Yes there are tools to thoroughly delete files on your computer, rather than just unlinking them when they're put in the trash, but it's the distributed nature of content these days that poses a special problem to the Ollie North's of the world.
:D
Well, I don't think any OS has ever been short of undeletion tools - in unix, one can grep the inodes on a disk for a particular known string of a file and recover it fron a known template. Tools like gpart (a partition guesser) also easily recover those vital 512 bytes of your hard disk.
Where Unix has been lacking, behind most other systems, is the opposite - a good, reliable, trashcan. It might be interesting to note that there's now a reliable trashcan for Linux, BSD and other glibc systems th simply preloads and wraps unlink, `move and a couple of other system calls.
Since glibc is a part of the Linux Standard base, it works along with every LSB standard app. Even better, it doesn't matter whether you delete the file from KDE, GNOME, shittyunixtoolkitforhellcirca1980something or a terminal.
Anyway, check out Libtrash. And if you're a GNOME or KDE hacker, I'll give you a big hug if you use this as the default trashcan or your next release.
for the most part, you don't even need to take the hard drive apart to do this. I have seen and even once had the oppurtunity to use a forensic computer that had the ability to scan through the hard drive, and determine just about anything that had been deleted, wiped, or over written. It has the hard drive read the magnetic echo left on each sector. The machine is mostly used as evidence against kiddie porn fuckers to prosecute them. I didn't really believe it until I see it... so anything I don't want seen... I shoot it with my 12 guage (repeatedly). Seriously.
Depending on the level of "security" you are looking for overwriting a file is not good enough. With proper analysis files can be recovered when they have been over written several times. This is expensive and time consuming, but it can be done.
-- Tim
TKrabec Pahh
For the lazy: substitute 'archives' where www appears
--El Linuxero
Actually, the idea that just because you have nothing to hide means that you shouldnt have the ability to hide something is an interesting fallacy.
Foucalt was a 60's "post-modern" French philospher who studied how systems of control are used to keep a Power in place. One of his most interesting insights was the more you can observe something the more the you can label it, quantify it, and more important the easier it becomes to define a Norm. Once you have a defined a norm, you know have the means to control the subject you were initally just merely observing.
I think this is a case of being able to keep something from observation, ie keep it away from ouside powers
anyway, thanks
Sigs are dangerous coy things
Of course `rm -R *` isn't enough -- it just unlinks files, but doesn't delete datablocks. To delete datablocks, try the -P option which overwrites the file data before unlinking. Unfortunately, this option is not available on GNU `rm` which is used on most Linux systems.
Norton has nothing to do with Midnight Commander.
Ahhh...the great dumpster continuum. Many a free computer will be found there. -- sowth (748135)
shred -f *.*
It kill DATA DEAD
That is a terrible example. Snopes has a good article explaining the problems with using pencils (the inhalation and electrical problems, as well as fire hazards in a pure oxygen environment made wood pencils problematic). It also explains that NASA never paid for the R&D, both Americans and Russians used graphite pencils in the beginning and both Americans and Russians switched to Fisher Space Pens. Also it gives the cost of 400 pens NASA initially bought: $2.95 each (granted that was in 1967 dollars).
-no broken link
There is a program called shred that comes with most distributions nowadays that overwrites the files with different patterns before unlinking them. There was something about this on Slashdot a while ago. This program seems to use a simular algorithm.
Presumably PGP runs on unix?
PGP 6.5.8, the last freeware version
GnuPG 1.0.6, the GNU Privacy Guard, is a free implementation of the OpenPGP spec.
Will I retire or break 10K?
On some systems, rm has an option to nuke the contents of the file before unlinking it:
man rm
<snip>
-P Overwrite regular files before deleting them. Files are overwritten
three times, first with the byte pattern 0xff, then 0x00, and then 0xff
again, before they are deleted.
</snip>
You can just put "alias rm rm -P" in your login script to make this the default.
Considering most systems come with 15-60gig drives now, it would take a long time to actually write over all the sectors used for that file in its entire lifetime.
No longer than a couple defrags. Simply open thousands of multimegabyte files, and then in each file, write a layer of 0's, a layer of 1's, and a couple layers of random data, and you're pretty safe. Five passes on a 20 GB partition shouldn't take more than a few hours depending on the transfer rate from computer to drive.
Will I retire or break 10K?
From the GNU shred info node:
shred overwrites devices or files, to help prevent even very expensive hardware from recovering the data.
Ordinarily when you remove a file (*note rm invocation::), the data is not actually destroyed. Only the index listing where the file is stored is destroyed, and the storage is made available for reuse. There are undelete utilities that will attempt to reconstruct the index and can bring the file back if the parts were not reused.
GNU shred is very featerful, as costumary in GNU utils, and has many flags to modify the behaviour.
BSD ppl are always praising the 'Unix Way' of small utilities that do a very defined job and nothing more, and hate the extended features that GNU utils provide; in this case it's BSD rm that is doing something that could be done by another tool by adding a flag! Horror!
Seriously, GNU shred is a good tool, and it can receive some interesting flags that a simple rm -P doesn't support.
cheers,
fsmunoz
Apparently, this isn't 100% effective:
Contrary to conventional wisdom, "volatile" semiconductor memory does not entirely lose its contents when power is removed. Both static (SRAM) and dynamic (DRAM) memory retains some information on the data stored in it while power was still applied. SRAM is particularly susceptible to this problem, as storing the same data in it over a long period of time has the effect of altering the preferred power-up state to the state which was stored when power was removed. Older SRAM chips could often "remember" the previously held state for several days. In fact, it is possible to manufacture SRAM's which always have a certain state on power-up, but which can be overwritten later on - a kind of "writeable ROM".
This is from Peter Gutmann's paper Secure Deletion of Data from Magnetic and Solid-State Memory
why don't you just overwrite the file with the same name, just some bogus data
This method does not ensure that any of your data is actually overwritten because the operating system is free to decide where on the disk it locates a file (or portions of it). Even though the filesystems references to that file name are destroyed by this method, the user has no guarantees that the data in the original file is overwritten. This is especially true in the case of remotely mounted filesystems which may not even implement the type of filesystem they appear to (e.g. Samba on Linux looks like it implements a MS filesystem).
This is the major shortcoming in most of the "secure delete" tools I have looked at.
For most of us here, the gov'ts electron-microscope method of determining old data is irrelevant. How many of you here think that it'll be employed against you? That said, I suppose for those of us who engage in a big-time trading of files via P2P networks, & DeCSS, etc, there's always the possibility of criminal prosecutions. So, let me go over the 3 types of "data deletion", and say where each should be used:
1. Typical deletion. Files are unlinked with their directories, so your OS does not "see" them and has more space available to write with. If the information is not sensitive, or you don't fear intrusion, this is the fastest, and also best, method of deletion. It simply changes the first character of a file name do something that your OS doesn't recognize -- a very fast process. The Advantage: data is recoverable via a data-recovery utility. The Disadvantage: the data has not been securely eliminated.
2. Simple once-sweep wipe-over deletion. Either random 1s and 0s, or wholly 1s, or wholly 0s, are written over an entire file. Use this for data that is sensitive, or where you fear cyber-intrusion by hackers. The Advantage: data is securely eliminated, beyond the reach of anyone who hacks into your computer. The Disadvantage: data is irrecoverable to you, should you realize you made a mistake, and this process is slower.
3. A multi-sweep wipe. Same as above, but many sweeps are performed, enough to make typical electron-microscopy methods of data-recovery inviable. This method effectively makes data irrecoverable by any means. Electron microscopes can detect "old zeros" by ghost-patterns, a slight trace. But if data has been written over many times, the older data is impossible to recover even by those methods. The Advantage: this method securely removes the data, beyond the reach of any technological means. The Disadvantage: this method is very slow, and again, data is irrecoverable should you learn you made a mistake.
It should be noted that whenever you want to securely delete data, not only do you need to wipe the file, but you also need to wipe your swap files and your temporary files.
So, let me summarize when each of the methods of "data-removal" should be used, starting with the strongest method (a multi-sweep wipe), and ending with the weakest method (the renaming of the first filename character to something unrecognizable):
1. A multi-sweep wipe. Use this when you have data on your computer that could be used against you in a lawsuit or prosecution. For example, certain kinds of pornography, copyrighted files, warez, and other various information that's been deemed "illegal" by the Information Police in the MPAA, RIAA, MS, and the US Gov't.
2. A single-sweep wipe. Use this for information that is sensitive, but that you need not fear should the government get ahold of. For exmaple, financial files, files containing credit-card information, etc -- anything you'd want to protect from online-hackers using data-recovery programs. The government, though draconian, has not been known to steal people's credit cards using electron-microscopy. Similarly, hackers have not the resources to use electron-microscopy to acquire your credit cards -- nor would it be worth it. However, if your a high-tech company selling your computer equipment to another company, a multi-sweep delete of your files may be necessary to protect your information from competing companies, who may have bought your machinery through another company as a front.
3. A deletion that dissociates the file from the directory (renames the 1st character). Use this for non-sensitive data. For example, stories you've written, calendars, lists, ideas, old programs, pictures, etc etc.
Hope this has been helpful -- and please, remember, if you want to securely remove sensitive data either by a single-sweep wipe (to protect it from hackers) or a multi-sweep wipe (to protect it from the government), please remember to also securely remove swap files and temporary files as well!
social sciences can never use experience to verify their statemen
FreeBSD users have the program obliterate in the sysutils part of the ports collection. It takes pains to overwrite the data in order to make sure the file, even if re-linked, is unusable.
/dev/rand.
If I understand correctly, it open the file for writing multiple times first. First it writes 0s, then 1s, then alternate beginning 0s and 1s, then 1s and 0s, then patterns of 1s and 0s of all descriptions, then several passes from
The upshot is that even if you find the inode and relink to the data, it's been overwritten so many times than you really can't possibly recover it even using forensic methods.
A big 'old electromagnet.
Degauss the disk and it's gone for good
Could you describe this big 'old electromagnet?
I've tried this with speaker magnets and bulk tape erasers like Radio Shack sells and they didn't erase floppies, zip disks or hard drives. In fact, it didn't seem to do squat to them. If you have a electromagnet that will, I'd like to know how it's made.
Wansu, th' chinese sailor
I always use the shred program that comes with my linux disto (Mandrake, but I think Red Hat also comes with shred). It's a great program, has many command line arguments and options, and is designed specifically to thwart those disk analysis techniques that you talk about.
Here's what I do: shred -fuzv *
The only problem is that there is no 'recurse' flag, so if you have an entire subtree that you want to erase, you have to manually 'cd' to each directory, and then 'shred -fuzv *'.
dd if=/dev/zero of=/dev/hda works for me ;)
Repeate 4 or 5 times, and good luch recovering anything...
LedgerSMB: Open source Accounting/ERP
Worrying about wiping isn't as big of a deal if you make the data indecipherable in the first place. :-) Check out rubberhose for more information: Basically an encrypted file system that's free and gives deniability. Who cares about file echoes when they're all chaos?
We can face anything... except for bunnies.
a large number of passes from /dev/random would in fact solve the problem.
/dev/random writes will thoroughly scramble a "narrow" stripe. You would have to wait a long time between each pass - each wait would have to be about as long as the original data was on the disk. Even then you have to worry about the write head drifting off center and leaving traces of your data off to one side. Even a BFM (big magnet) leaves traces of the data.
Nope. The longer data is on the disk the "wider" it gets. A large number of
This is why the Govt requires that any disk that ever contained classified data must be INCINERATED.
-
- - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.