Slashdot Mirror

There is no easy answer to this question. It certainly depends on the alogorithms used. It depends on who implemented it, tamperfree devices, and much more. Here are a couple of links that might give the interested reader some points to start:

Peter Gutmann's excellent crypto tutorial
Some information on Blind Signatures
A very nice link page for privacy and encryption
Ron Rivest's (the R in RSA) homepage with an excellent link section
And a link to buy Applied Cryptography, even if the stories lack accuracy it is a good read

Happy reading!

I actually figured out a usable way to do this. It wasn't pretty, but...

I got this ASN.1 dumper, but found out that it can't tell where in the data to start (though you can give it an offset). This is even worse because ASN.1 structures tend to encapsulate entire other ASN.1 structures as opaque (to the dumper) "octet strings."

So I rigged up a script like this:

#!/usr/bin/perl
for $i (0..200) {system "dumpasn1 -$i $ARGV[0]";}

...to try every possible offset in a reasonable range. I dump the output to a file, then browse through it looking for structure. Feh.

And those OIDs are the ultimate in separating unique definition from actual meaning.

If you think about it, the digits of pi are not really a "random sequence" at all, at least according to Gregory J. Chiatin's theory of algorithmic information theory. The digits of Pi are of course compressible. You can write a computer program which is of finite size that will generate the digits of Pi, and that's definitely smaller than all the digits! The "randomness" only arises from our choice of base, actually. If you would use a factorial base representation (for instance) to write Pi, it wouldn't look very random...

Also check out the source for GNU fileutils-4.1. (ftp://ftp.gnu.org/gnu/fileutils/fileutils-4.1.tar .gz). The src dir contains "shred.c" which more or less implements the contnent of the paper.

See e.g. "Secure Deletion of Data from Magnetic and Solid-State Memory"
--

for i in `seq 1 16`; do cat /dev/urandom | cat > /dev/hda; done what the fuck can you recover after that? you can't do wizard things... What the hell can the NSA know more about filesystems that us, who understand and write filesystems?

There is a world beyond software, fool. Read this article. In case you're too lazy to click the link, you can recover data from a drive by disassembling it and using magnetic force microscopy with a scanning tunneling microscope. Even after you overwrite a bit, there are still traces of its previous value.

... even the computers that had classified information on their harddrives should be allowed to be re-used.

No, no, no! Information can be recovered long after the second or third overwrite. Here's my
source. And that's just what the public knows how to do. Who knows what the NSA and their foreign counterparts can manage.

Remember that, when dealing with data security, you don't just need to worry about what your enemy can do now, you need to worry about their capabilities 20 years from now. If the data has to stay secure for 50 years, then the only choice is to destroy the drive (we may have nanotech by then, and then you'd probably be able to uncover everything that was ever written to the drive).

Please, people, the NSA and the Pentagon have people who know way the fuck more about security than you or me. Leave matters of national security up to them, and go back to worrying about how to make your home linux box secure... Write to your politicians, and tell them to stop meddling. Getting 486s for schools may sound like a noble cause, but if it costs the lives of American soldiers 5 years from now, was it worth it?

Others supported it after an audit found sensitive information such as lists of names and addresses had been left on hard drives of donated computers. Though unclassified, they said such cases still present risks.

He should have used Gregory Chaitin's Omega number to generate the challenge file.

Actually I really don't understand Chaitin's work well enough to know if that would have saved him the $5000, but at least he (and the challenger) would have learned something about algorithmic complexity theory.

Zooko

A quick search on google turns up the masters homepage. There.

The guys seems to be something of a pop star among mathematicians.

And I'm now looking forward for the obligatory halfdozen proofs that 2=1 in the next fifty comments. Yay for Slashdot...


/* And you'll never guess what the dog had */
/* in its mouth... */

You correctly point out that physical destruction of media is the only way to be sure.

One thing to be aware of when overwriting data with patterns is that what you think you write to disk isn't what you write to disk.

A string of "00000000" isn't "all magnetic north poles up", and a string of "11111111" isn't "all magnetic north poles down".

Drive firmware maps these bit streams into encodings that are broken up into patterns of ones and zeroes that the heads can always read - much the same way that your serial port would get very confused if you tried to download a 100K file of "all zeroes" by just holding the ReceiveData line low for 30 seconds with no parity or stop bits.

The actual encoding method by which the bitstream is encoded into alternating magnetic patterns is probably drive-dependent. As a result, the "ideal" pattern of bytes the controller should write to the drive to create patterns of alternating, or mostly-North, or mostly-South, magnetism, will also be drive-dependent.

Practical application: The Apple ]['s "disk ][" floppy controller used to have a feature where you could tell the floppy drive to give you the data as seen by the read/write head. By changing the encoding scheme to a less-redundant, but equally-reliable one, you went from 13 sectors per track to 16 sectors per track. Many copy-protection-breaking programs of the day would give you the bytes as seen by the drive head and use this to determine what encoding (or if a custom encoding) was in use.

In hard drives - MFM and RLL are two encoding schemes. RLL drives were exactly the same hardware inside, but used a different encoding scheme. RLL stood for Run-Length-Limited, where "Run-Length" can be loosely translated into "number of consecutive all-north-poles-in-a-row the drive firmware will tell the head to read/write for any given input bit sequence. As such, the RLL version of a drive typically had 30M of user space, whereas the MFM-encoded drive - same hardware - had 20M of user space.

Today's drives work on the same mechanism at the head/platter level, it's just buried under many more levels (BIOS, C/H/S remapping, LBA, etc. etc. etc.) of abstraction.

Others have posted links to this paper. I've merely summarized section 3. It's a damn good paper.

If it's important - whether military or corporate secrets - physically destroy the media and buy a new drive.

From Secure Deletion of Data from magnetic and solid state memory.

In conventional terms, when a one is written to disk the media records a one, and when a zero is written the media records a zero. However the actual effect is closer to obtaining a 0.95 when a zero is overwritten with a one, and a 1.05 when a one is overwritten with a one. Normal disk circuitry is set up so that both these values are read as ones, but using specialised circuitry it is possible to work out what previous "layers" contained. The recovery of at least one or two layers of overwritten data isn't too hard to perform by reading the signal from the analog head electronics with a high-quality digital sampling oscilloscope, downloading the sampled waveform to a PC, and analysing it in software to recover the previously recorded signal. What the software does is generate an "ideal" read signal and subtract it from what was actually read, leaving as the difference the remnant of the previous signal. Since the analog circuitry in a commercial hard drive is nowhere near the quality of the circuitry in the oscilloscope used to sample the signal, the ability exists to recover a lot of extra information which isn't exploited by the hard drive electronics (although with newer channel coding techniques such as PRML (explained further on) which require extensive amounts of signal processing, the use of simple tools such as an oscilloscope to directly recover the data is no longer possible).

So, in conclusion - Sorry, can't be done without modifying the electronics in the drive. That might be a challenge to the drive manufacturers though... how to get your density doubled by purposefully use the overwrite and read both the previous and current data! Kind of the same idea as the two bits per cell technology used by flash memory manufacturers...

See Peter Gutmann's Usenix paper on secure deletion of data from magnetic and solid state memory for some truly impressive data recovery methods.

--

See "Secure Deletion of Data from Magnetic and Solid-State Memory" by Peter Gutmann, Department of Computer Science, University of Auckland for the Sixth USENIX Security Symposium Proceedings.

Several wiping programs are available that will overwrite data multiple times with binary patterns - checkerboards, solid 0's, solid 1's, random patterns, etc. Even after all of that, it is still possible for an organization with lots of resources such as a data recovery service or a three-letter agency to recover the residual remains of the data, though it would be very difficult.

About a year ago I worked for a company (which shall remain nameless) that tasked me with writing a secure disk wiping algorithm. I did a little homework, and found that the US Department of Defense had a recommended 7-pass algorithm. A little more homework, and I discovered a crypto guru named Peter Gutmann who had a 35-pass algorithm. I implemented both of those, and then took it a step further and allowed the user to create a custom wiping algorithm, up to 99 passes (I figured any more than that would probably be overkill). We had a professional cryptographer on staff who assured me that even after 99 passes of overwriting the data on disk with different patterns, the NSA or some other such agency could still recover the data if they wanted to badly enough. I had my doubts, but then I've never ventured into the field of electron microscopy.

Anyway, the project was killed due to management/marketing cluelessness, so we never actually shipped it, but it sure was educational to implement. I had been under the impression that simply overwriting a file even once with 0's and 1's would be enough to render it unrecoverable, how naive I was.

--

For the theory behind it, see "Secure Deletion of Data from Magnetic and Solid-State Memory".

On-line documentation (if you have the package installed already):

$ info 'file util' basic shred

I wouldn't exactly say "fool", though, although I agree with you on the asinine nature of his writings on machine intelligence. He did crack Enigma, afterall.

Terrorist in NZ were targeting Aussies. You know its true 'cause its on the news.

At least there hasn't been a major power outage in a long time.

If you want to migrate to NZ you need to buy this first.

Im sure i found this link on /. a few weeks ago but for the life of me i cant find the story it was in. Anyhow the link is here and basically is a paper discussing the secure deletion of data, it goes into a little detail on erasure of information using magnetic fields and according to the paper it is currently not possible to completely erase most magnetic media using current degaussing equipment. Naturally if someone was wanting to read your data after it had been through this process as outlined in the paper they would have to go to considerable trouble but still i guess it depends on what stands to be gained from getting access to the data.
The paper also has an interesting bit on recovering data from RAM after power has been removed.
Anyways food for thought :)

The university's robotics group that's affliated to this venture/spam went to the robocup 2000. Their team is called All Boltz. I can't remember their exact place (and the robocup site is down) but I 100% sure that they're one of the teams who score the least.

If you guys want more information on exactly HOW they recover this information, check out this page: this page.

Uhhhh....this is what overwriting does. It overwrites over the sectors/inodes over and over again with bullshit. Read this for information: bleh.

Believe it or not, that may not be enough. Guttmann's paper goes into some detail as to how much magnetic field is required to fully reset the media. As an example of an "adequate" field, he mentions a DOD device which produced a field so strong it actually bent the drive platter.

I think many of these are serial port attachments, obviously a lot slower than what /dev/urandom is producing so dumping straight to disk isn't an option, but I think what such hardware gets you is a reliable, high-quality source of random bits to seed a pseudo-random process. Looking at the man page for urandom on OpenBSD (I'm assuming the one in Linux is no different), it doesn't check the entropy pool for quality so without a high-quality source of randomness, in 20GB you're entropy pool's quality is pretty likely to run low, relying just on system activity.

/dev/urandom isn't *that* slow, producing 364KB/sec (unless you meant 20Gb, then it's only about 45KB/sec). I don't know how cryptographically sound it would be but you could consider using a CD-ROM of random data (see the link above) as a starting point. A CD-ROM drive should be able to deliver a lot more KB/sec for /dev/urandom or something else to process to get your 20GB.

After reading this very nice paper, I would go to even greater lengths to destroy evidence. I would make a quick trip down to the local computer store and buy more RAM a new HD, and maybe a few floppies, then I would completely destroy my current RAM, HD, and any floppies with incriminating evidence on it. And I mean destroy it to the extent that even God himself would have trouble putting it back together. Then I would go to some lengths to make sure my system doesn't look like it has been completely replaced from the inside out in the last couple of days (which it has). If you have 2 or 3 days forewarning, there is no reason why you should have any incriminating evidence left on your system, unless you are dumb enough to write a virus with malicious intent in the first place.

http://www.cryptome.org
http://jya.com/crypto-free.htm
Learning About Cryptography
Ritter's Crypto Glossary and
Dictionary of Technical Cryptography
Encryption & Security Tutorial
N.A. Crypto Archives
International PGP site
NSA National Cryptologic Museum
EFF
attrition.org crypto archive
Bruce Schneier's Crypto-Gram

and last, but not least (the archive i developed) ....

PacketStorm Crypto Archives

there are lots and lots of excellent tutorials, docs, glossaries, and links to many of the great crypto sites in the world at all of the URLs above.

for the best info on NSA, ECHELON, misc paranoia, you should first check out Cryptome/JYA. i archived quite a bit of stuff related to your questions at the packetstorm site too - packetstorm.securify.com/crypt/nsa/.

feel free to email me directly if you like too. over the years, i have had some interesting experiences with the NSA, BXA, etc - primarily regarding my hosting of crypto archives, and personal investigations of NSA, ECHELON. if you want to discuss these things, get the pgp key for ken.williams@ey.com from www.keyserver.net, and send your key(s) and crypted msgs to tattooman@genocide2600.com

Surprisingly, there are collaborating groups around the world (e.g. Australia) that are in the process of designing building working prototypes of some of these weird and wonderful machines. The problem is that we still don't really have a good grasp of what commercially useful domain will drive the need for mass demand. I suppose it was the same electronically in that early boards were more toys until people mastered the assembly of megagates into useful building blocks. Peole like Pen rose have speculated on physiological process underlying a given thought that may initially involve a number of superposed quantum states. I hope there are some really smart guys out there who can take some of the ideas through to the next stage. Now if someone could come up with decent quantum algorithms for massive parallel search and comparisons of multiple genetic strands databases, they'd make a killing. LL

Of course this leaves the question: how did the frickin' logic get there?

It was a human-made invention of Aristotle. There's plenty of logic defying randomness in nuclear decay, the Uncertainty principle, and heck, one fellow used the Gödel theorm to show that there's randomness in arithmetic!! The atomic API is still not completely defined.

Like a decent grounding in physics.

There is an 11 dimension theory, called M-theory, sounds fascinating, I'm gonna get fired today. Not getting any work done at all.

With all the universe present (latent?) at the initial singularity, why didn't it just remain the Mother of all Black Holes ?
I've seen two plausible answers to this; one is what I like to call the 'just because' theory; a quantum flip made the big bang, well, bang. I'm not fond of this theory as I don't understand how something can change its state without time being present.
The other, cooler theory involves time loops. Relativity tells us that time can go backwards as long as the loop is closed, so its effects are never noticed. But if one of these time loops tried to loop back to before the big bang, weirdness would happen. This time loop would not only start the big bang, it would be the big bang.
So the universe cloned itself. However wrong that theory may eventually turn out to be, I think it's cool and I really hope it's right.

John Gribbin wrote about this in New Scientist last year; I can't find it in the online edition but the piece is recreated her e (scroll down to In the Beginning 24 Jan 98 New Scientist).

Monsanto is the last company I'd want producing plastic, oil, or any other product crucial to the US economy. Greenpeace crazies and eco terrorists are certainly right about one thing - dealing with Monsanto is dangerous for your long-term independence. Their clever mechanism for ensuring repeat buyers is to build infertility into the plants they sell. Farmers buy them because they are indeed very good crops for certain purposes, namely for surviving the popular but toxic herbicide RoundUp, which Monsanto also sells. Monsanto works vigorously to bankrupt competing seed sellers, so that only their perishable brand is available, thus locking farmers into their system for life. Prior to the development of these terminator genes, Monsanto would actually maraud around the countryside burning "illicitly stocked" seed.

http://www.mat.auckland.ac.nz/~king/Preprints/book /upd/umar99/monsan/ecol1.htm#anc hor52768

A recent company tactic as been to push this "system" as a solution for hunger in third-world countries. Of course, what it would really entail would be a complete regional ownership by Monsanto of the food supply.

http://www.greenpeac e.org/~geneng/highlights/food/98_10_15.htm

Monsanto is also renowned for suing magazines and television stations when they are about to produce an article critical of the company. Most news providers can't fight them, so they buckle and the issues are never aired.


http://www.inmotionmagazine.com/fox.html

And much like certain proprietary software companies, Monsanto patents its creations. We all are familiar with the stupidity of patenting ideas, and genetic engineering, especially of plants, is quite simply that. One plant can turn into two plants with only a negligable investment of soil, water, and sun. This means they are not a zero-sum game, and hence the arguments against patenting software apply to them.

Monsanto is one of the least palatable companies out there. They are easily the Microsoft of genetic science. I think I'd rather stick to the Sheiks for my gallon of gas and pound of shrink-wrap, thank you very much.

-konstant

An interesting tangential point (well, not open source related) is how all the other neat stuff in the lab works, and how you would go about making things *really* unrecoverable... There is an interesting paper on the subject available form Auckland university. Worth a read.

Possibly. It may be possible to refresh your brain using stem cells. And more stem cells. And yet more.

The link below will take you to what I believe to be the most extensive webpage on Encryption and Security. From free win based ssh clients to information about the Australian NSA.

Here it is!

Read it first time - really tackles a big subject, easily misunderstood. Would like to take a course in Gödel, like one GJ teaches (complete w/ source code).

I am. NOT!

Chuck

Synchronisation can be achieved by self-clocking codes, of course, rather than relying on a separate clock channel. This can be made error-resistant in various ways; a good example is Mark Titchener's T-codes.