Crypto Snake Oil

An anonymous reader writes "Luther Martin of Voltage Security has published an article about the perception of cryptography today with regards to quality and honesty in vendors. From the article: 'Products that implement cryptography are probably credence goods. It requires expensive and uncommon skills to verify that data is really being protected by the use of cryptography, and most people cannot easily distinguish between very weak and very strong cryptography. Even after you use cryptography, you are never quite sure that it is protecting you like it is supposed to do.'"

Comments

Still not too bad

[legoburner]

Even though in many cases this might be true, and product prices are increased because of it, weak encryption is a lot better than no encryption at all. There are many people out there who might go as far as casual data theft (eg; taking someone at their school's USB memory stick), but even a weak layer of encryption will stop all but those who know what encryption is and where to start breaking it.

Re:Still not too bad

[TCM]

I'm not so sure. Once a flawed implementation has been broken, there will be tools to crack it.

Take WEP for example. I personally wouldn't know how to crack it. But others do. They develop tools. Et voila, today it's trivial to download some tool and break WEP, even for novices.

Weak encryption is never good and should be strongly discouraged.

Re:Still not too bad

[Snarfangel]

I'm not so sure. Once a flawed implementation has been broken, there will be tools to crack it.

Plus, if there is *no* encryption, people are less likely to put sensitive information in the application.

To use an analogy, consider two locker rooms. Room A does not have locks on any of the lockers. Room B has locks, but all of them have the same combination. In which one is a person more likely to leave their wallet?

Re:Still not too bad

[Lord Ender]

I would say that there is an inverse relation (at least somewhat) between price of crypto software and real security.

The cheaper the software is, the greater the number of people who could have peer-reviewed it for correctness. The more open the software, likewise.

Really expensive software could only have been peer-reviewed by a small number of people, while free, open source software could have been reviewed by a huge number of people.

I recently was asked to recommend a way for my CEO and several other executives to securie thier IMs. I recommended gaim + gaim-encryption because it was all open source and free, so if there were a flaw in the crypto implementation, it would likely have been discovered already.

I also made sure the CEO knew that he was using open source software, and I told him why. He was totally down with it :-)

Re:Still not too bad

[RoboSpork]

gaim-encryption is flawed in that it is a weak encryption scheme. Off The Record is a far superior gaim plugin providing a much stronger encrytion, authentication, deniability, and secrecy into the future. Read how it compartes to gaim-encryption on their website. Their whitepaper is really good introduction to what can make encryption strong vs what can make it weak, definitely worth a read for anyone new to crypto. And besides all that, open source != secure. That is a really bad assumption to make.

Crypto is scary stuff

[smilindog2000]

So, for example, with a post like this, will somebody in a dark suit and glasses show up at my door tomorrow?

Blasphemy #1: I've heard from a claimed friend of one of the inventors of RSA that it was cracked it years ago. Yet, it continues to get worldwide use. Sure my friend was probably full of it... but who am I suppose to trust here? The government?

Blasphemy #2: One of my close friend's mother had to switch fields from Numerics after she published some papers considered too sensitive. It had something to do with factoring.

Blasphemy #3: Anybody else notice that quantum computers have been proven to be capable of factoring really well, but no one has shown that they can solve any NP-hard algorithms? Come on... factoring isn't NP hard.

Then, there's just some silly stuff I've noticed about crypto. Why do we always seem to use encryption just a generation or so ahead of what is needed to crack it? SHA-1 for example... And, why do we encrypt one small block at a time. Each encrypted file usually gives many independent chances to crack the key, and in many cases, some of those blocks have known data. Also, public key is great, but secret key can be easily shown NP-hard to crack (in terms of secret key length) with semi-reasonable assumptions, while public key has no such simple proof. I personally have been trying to prove that no public key system can be NP-hard, but what the heck... I'm not that good. However, I do believe it's probably true.

It seems any time you start talking about crypto, you get assailed by experts telling you just how full of it you are. Consider something simple, like generation of random numbers. Just claiming you can do a good job brings nay-sayers out of the woodwork. See:

        http://linux.slashdot.org/comments.pl?sid=193904&c id=15899118
        http://www.billrocks.org/rng

for how to do it well. Any child could do it (well at least my geeky 6-year-old).

Everything about crypto is scary... Are we being manipulated into using weak encryption? Is there some invisible line, which if crossed, bad things can happen? The scary part is the unknown.

--

Just because your paranoid doesn't mean the world isn't out to get you.

Re:Crypto is scary stuff

[Anonymous Coward]

Is there some invisible line, which if crossed, bad things can happen? The scary part is the unknown.

That's exactly what it is, I think. Crypto is so complex that, unless you are absolutely sure wtf you're doing, you're better off NOT trying to implement your own crypto algorithm, random number generator and whatnot. Without the mathematical knowledge, you can never completely assess side effects, for example.

A nice page about how novice understandig of crypto can turn into horribly insecure software: http://www.cs.auckland.ac.nz/~pgut001/pubs/linux_v pn.txt

Re:Crypto is scary stuff

[Realistic_Dragon]

sci.crypt is a good read if you are interested in Crypto. However it does tend to get a bit antagonistic towards newbies - and it's not hard to see why.

Approximatly every 12.5 minutes someone turns up claiming to have invented a new:

Random number generator
Unbreakable encryption method
Implimentation of old methods that makes them unbreakable
Proof that shows that all crypto is worthless

The percentage of loons is *so* high that anyone who does have an interesting idea (and who doesn't publish in reputable journals) is dismissed out of hand.

For example, here is a typical conversation from the one sane new poster (posted somewhere between the 999,999 people trying to sell "200000 bit quantum crypto based on the randomness of STARS!!!!!"):

<i>** Hi, I'd like to find out if there's a RNG sandbox somewhere so I can play about with some ideas.</i>

<i>* ARGH! Dont impliment your own RNG! It'll be crap! Here, use product X.</i>

Well, yes, that's true. When it comes to crypto there is a 99% chance that what you impliment will not work properly and as a result will be insecure... but stoping on someone who wants to try some ideas out is just plain wrong. All research doesnt have to take place in academic institutions.

or

[xmodem_and_rommon]

or you could just take the common sense approach and use products that rely on algorithms that are open, widely tested and reviewed, and known secure. Algorithms like Blowfish, AES, etc. I use Apple's built-in Filevault protection to encrypt my Powerbook's hard disk, in the event that it is ever stolen. It uses AES-128, which means I know that no-one is getting in without my password.

Any vendor that relies on a custom algorithm for their encryption technology shouldn't be trusted.

Re:or

[TCM]

Any vendor that relies on a custom algorithm for their encryption technology shouldn't be trusted.

Of course.

But even then there are vendors who claim to be using AES and end up introducing implementational flaws that are not obvious to the user. It's not just algorithms that need to be reviewed but complete implementations.

Nice read: http://www.schneier.com/crypto-gram-9902.html#snak eoil

Re:or

[swelke]

...use products that rely on algorithms that are open, widely tested and reviewed, and known secure.

Just because the algorithm is widely tested and known to be secure doesn't make the software based on it secure. It's very easy to take a secure algorithm like AES and make a totally insecure program by, for example, not encrypting all of the data it should, or by selecting the encryption key poorly so that it's easy to "guess",meaning you might only have to check 2^20 keys to decrypt that email of yours instead of 2^128, like you're supposed to have to. So instead of being secure against years of hard cracking, your data is compromised in seconds. Besides that, there are other ways to build a crappy program that I'm not a good enough cryptographer to know.

Some insights about the article

[owlstead]

It's pretty well known that there are many snake oil products that deploy cryptography. Bruce Sneier frequently displays snake-oil cryptography products in his newsletter, for instance. And these are just the really obvious ones.

Some time ago, I tried to evaluate if a Enterprise Service Bus (intercomponent communication) was fit enough to be put into a production environment. It said that it had AES encryption build in. When I looked at the manual, it displayed a pop up window where you could choose the key-size. It listed exactly all key sizes that were *not* possible for AES. This was a very short evaluation, I can tell you. This also shows a very important thing about cryptography: the algorithms used say very little about the security of an application.

Generally, the manual for cryptographic services is easy to find. This is simply because cryptography is added at the end of the development lifecycle. This is logical because cryptography is not part of the main functionality of most applications (e.g. mime encryption in email products). It's something that was added after the products main functionality was finished. So just look at the last paragraph, or Appendix Z and you are looking at it.

Sometimes it is easy to see why so many products contain bad cryptography. Take XML signatures for instance. XML signatures themselves contain *references* to the data that is signed and the cryptographic techniques used. If you are to verify an XML digital signature, you *must* check if these are not altered. Furthermore, you must keep the XML schema-definitions on your own disk, and not retrieve them from the internet. Nevertheless, I've not seen any API-documentation even mentioning this rather obvious cryptographic insight. You can rest assured that there will be many implementations that will get this wrong.

Cryptography is hard.

The real insight of this story is the listing of the products into "credence goods". If you can call this new insight. Otherwise, it's just stating the well known/obvious.

Truecrypt

[urikkiru]

This is something I've often considered about commercial encryption software. There's just no way to be sure of their validity, as they are closed source implementations. Open source solutions like Truecrypthttp://truecrypt.sourceforge.net/ are at least somewhat more trustworthy, in that they can be openly reviewed by anyone. Despite the fact that I know jack all about the specific math behind AES and such, at least I can read some simple explanations of the concepts, read the source, and decide if I want to trust my data to it. Honestly, unless we get down to the fraction of the population that actually does understand these bits at a deep level, that's the best any of us can do really.

Sure, large clusters of powerful servers working in tandem(or quantum computing) may render the factoral math behind crypto obsolete. A nice thing though, is that those kind of solutions are limited to those that can afford them. Still, even if it's all true, and I'm wasting my time encrypting things, what better solutions do we have?

Re:Truecrypt

[kasperd]

I agree TrueCrypt is well documented, and in addition to that the source is available. I have the necesarry knowledge to actually review such a design, and in the case of TrueCrypt I must say it is not the worst I have seen, but it is certainly not perfect either. There are some subtle watermarking attacks if you can get access to different encryptions of the same sector. Still in spite of that I'd much rather rely on TrueCrypt than some closed source products. So far all storage encryption products I have seen have had some weakness, I'd much rather use one where I know what it is and to what extend it could be a problem to me.

Re:Truecrypt

[Anonymous Coward]

There are some subtle watermarking attacks if you can get access to different encryptions of the same sector.

Any efficient 1-to-1-mapped disk encryption software (not just TrueCrypt) is subject to "water mark" attacks based on comparison of re-encrypted blocks. Increased granularity (i.e. wide-block modes) will not prevent the attack either. There is no mode of operation that can feasibly prevent it (you can prevent it only if you accept dramatic and generally unacceptable degradation of performance).

Re:Truecrypt

[kasperd]

Any efficient 1-to-1-mapped disk encryption software (not just TrueCrypt) is subject to "water mark" attacks based on comparison of re-encrypted blocks.

That is absolutely correct. In fact you didn't even have to use the word efficient here. Any 1-to-1-mapped encryption is subject to such attacks. The point is that if you are willing to sacrifice a few percent of disk space, you can improve security.

One encryption sacrificing about 3% of disk space is GBDE. Unfortunately GBDE suffers from a few other problems. The author designed his own weak pseudo random number generator. And GBDE does mean you have a risk of loosing data as the consequence of an incomplete write.

Those problems can be solved, and the overhead could be reduced from 3% to 1%. And I believe this can be done at only a few percent of cost in performance, though I don't yet have the complete solution, I'm pretty close.

Classic snake oil: Blitzkrieg!

[WWWWolf]

Anyone remember the Blitzkrieg server, which seems like the solution to all of the world's security needs? The expression Bruce Schneier used was "just too bizarre for words". I don't know if this was an elaborate trolling attempt or an actual real honest scam to deceive the terminally dumb, but it's fun to read, still, just for the amazing technobabble and ludicruous claims.

Article taken from Wikipedia???

[transporter_ii]

So did he write the article and then post it on wikipedia, or did he swipe it from wikipedia and post on his site?

http://en.wikipedia.org/wiki/Snake_oil

Not trying to troll, I just couldn't figure out which it was and I don't have a lot of time to investigate.

Transporter_ii

Re:It requires expensive... blah blah blah

[zolaris]

Yeah sure they can get a great understanding of crypto... with inexpensive books. Just curious do you know how many crypto courses at top level universities rely on textbooks for teaching crypto? I'd suggest discounting any books where the professor is the author. But even with that, it will probably be very small. There are recommended books but in my crypto classes (granted Johns Hopkins isn't exactly the number one crypto school in the country or world but I'd like to think we are half way decent) we never cracked a textbook. Sure we read a bit of papers but is average Joe developer really going to read through any crypto papers? I know I wouldn't unless I had to.

[Sarcasm captioning*]On a side note, let me know what project you are working on where developers employ crypto after about two weeks of reading some books.[/sarcasm captioning]

*Sarcasm captioning provided for cya purposes only and not for any public benefit.

an old problem

[v1]

I worked for sevearal years on a programming language called REALbasic. In the latter releases that I saw, it featured "encryption". A compiler is basically a tool that takes human readable commands and turns them into a program that a computer can run. This process is not easily reversable, and once compiled, it's difficult at best to make changes to the progam.

Encryption was added to RB so that it was possible for you to give away portions of your program's "source code" (the human readable part) without anyone actually being able to READ it. They could incorporate your souce into their new project and use it normally, they could just not read it or make changes to it.

This sounds like a nice idea, until you realize that when you get someone's "encrypted" source code and add it to your program, the compiler has to be able to read the source code, because it needs to translate it for your new program. This means one thing: the encryption is not secure because the compiler itself must somehow posess a "master key" of sorts so that it can read the source code to do its thing. So... when you select the module and try to open it to look at it, it's not that it can't read it.. it's that it won't read it. A sufficiently skilled programmer could go into the compiler and flip a switch inside it and basically say "ignore that", and you would have unrestricted access to the so called "encrypted" informataion.

I assisted with a project where we found out how this information was encrypted. In short, a fixed key was used to encrypt the project data. Then a different fixed key was used to encrypt the passcode you would use to "protect" the project. Thus, the compiler could ask you for the password if you wanted to read your own project, and it could verify you typed in the correct passcode. If you did, it would decrypt the project for you to view. So you see, the compiler does not NEED the passcode, it simply WANTS it.

It took us about a week to write a program that would read in the projects, decrypt them using the fixed key and completely ignoring the passcode thing, and saved an unprotected naked project file that anyone could edit or view.

This is probably not too far from the mark on how a LOT of programs "protect your privacy". In reality they are only protecting you from the casual inspection. Anyone that really wants your data can get it, all too easily. Be sure that with any program you are certain that the program NEEDS the passcode to unlock your data. If it only WANTS it, (is there a password reset option available?) then you know it's "security through obscurity", and we know how totally worthless that is.

You thought your windows or OS X keychain was secure? You have auto login turned on? Does the computer need your password? Think about it.