On Choosing Encryption ...

A crafty owl hooted this query into my ear: "How do developers decide which encryption they are going to use? If you do a search for 'blowfish' on freshmeat.net it turns up 14 hits. Blowfish isn't even an AES candidate. Counterpane's submission to the AES contest -- Twofish -- turns up a meager five hits, only two of which are really applications. Serpent, another strong AES candidate isn't used by any applications listed on freshmeat. OpenBSD can use Blowfish passwords. Not Serpent or Twofish but Blowfish. Why? How do the developers decide which to use? Do they actually evaluate the different candidates? On what criteria are the decisions made? Security? Speed? Ease of implementation? Is it based simply on what everyone else is doing or which algorithm's author is more well known?"

Comments

theres more to security than choosing an algorithm

[orabidoo]

First, I have to say that choosing Blowfish, Twofish, IDEA, 3DES, Serpent, Rijndael or something else, is not the first, nor the most important question, in a cryptograhpical product. There's *much* more to cryptography than choosing a symmetric cypher, and it's very easy to use a great algorithm in a broken way. As the saying goes, security is a process, not a product. And within a product, it's a protocol, not an algorithm. You need to design the entire way that the various clients and servers will talk, to be secure: how to choose and agree on the keys, how and when to rotate them or generate new ones, where to get your crypto-strong random seeds, using the right cyphering modes for the situation (ECB is not usually a good idea), whether to use a block or stream cypher, whether and how to interface to public key crypto, how to fend off replay attacks, and so on and so forth, the whole protocol. Only when you've figured out all this stuff, does it start to make sense to evaluate individual cyphers, for speed and perceived security. There, again, the general idea for safety is to go for something that has withstood serious analysis for a long time. I wouldn't count out Blowfish just because Twofish is its successor; Twofish looks very impressive, but has been around for a much shorter time. Right now I'd still trust Blowfish or 3DES better.

Just like telephones

[SamIIs]

On communication issues, being the "Best" isn't all that matters. If you're trying to talk to other people, it's important that they have the same protocols and languages, etc.

The first telephone ever created was totally useless, and the second was only barely better.

The usefullness of any communication device, whether encryption or telephones, is proportional to the square of the use.

The algorithm is irrelevant

[chazR]

Well, not quite irrelevant. It should be one that has been bounced on for a few years by serious cryptanalysts with no serious (i.e. non-academic) weaknesses found.

Triple DES is *probably* strong and has stood years of abuse.
RC4 is also thought to be strong, and is in the public domain whether RSA like it or not. It has the advantage that it can be coded in about 15 lines.
Blowfish is also probably strong. So are *all* the AES candidates. I like Rijndael with 18 rounds and Twofish (I'm betting on Twofish becoming the standard). Serpent is also nice.

But all this is irrelevant. *All* the algorithms I have mentioned are strong against any meaningful attack, and unless you are encrypting gigabytes they all perform well enough on modern hardware.

None of this matters if your implementation sucks. What really matters are things like key generation, key management, ensuring keys are never stored anywhere inappropriate (like on a disk - how are you going to control swapping?), enforcing strong passphrases (this is very hard). Then there's system security. What if someone patches a binary to email them the key during encryption?

Basically, spend your time hardening the implementation. That's where you will be attacked.

Also, don't code them yourself (except for fun) - there are a number of free (usually public domain) implementations of these algorithms. These have been peer-reviewed and tested. The chance of blowing the strength of the algorithm with a stupid coding error is too high to risk.

Dilbert

[Robert S Gormley]

Vendor: "Our device conforms to all known standards for communications protocols."

Dilbert: "You mean it does nothing useful and isn't your fault?"

Vendor: "Is there someone less knowledgeable I can talk to?"

:-)

crypto mini-howto

[Zooko]

There are several issues here: peer review, architecture, algorithm and implementation.

Peer Review: At each step in the process (architecture, algorithm, and implementation), you should publish your ideas for criticism by experts. slashdot, Advogato, the Cypherpunks mailing list, sci.crypt, and the Crypto++ mailing list might (or might not) be good places to find such people.

Architecture: You should do a public key architecture where every participant has a public/private key pair and the public keys are used to sign and encrypt symmetric keys that are then used for encryption and authentication of messages. There are three feasible architectures for public key distribution. You have to choose one based upon your threat model. Almost all realistic threat models should be handled using the first option: "opportunistic public key distribution". If you don't have a threat model in mind at all then you might as well use the first option. If you do have a threat model which precludes using the first option then I'd like to hear about it -- you must be doing something very interesting indeed.

• Option one: "opportunistic public key distribution". The first time any pair of people talk to one another, they exchange public keys in an un-authenticated exchange. (Also: you could just do Diffie-Hellman key generation here.) After that, they remember each other's public keys for future use. This is susceptible to an active attack (a "Man In The Middle Attack"), during the first step (though not afterwards). However the cost to the attacker of executing a MITM attack is probably far more than the payoff. This depends on your threat model.
• Option two: make it the user's problem; Each user decides whether to use a given key to talk to another user or not. This is the user interface nightmare that single-handedly prevented strong crypto from becoming standard in e-mail, but for a few applications it might be the right thing.
• Option three: hardcoded; Generate key pairs yourself and include them in the application. For example if you are going to have a single central server in your system which you operate then you can generate a key pair for it, put the secret key on the server, and put a copy of the public key into each copy of the client (e.g. include the public key hardcoded into the client source code). This doesn't work as well if you want to distribute copies of your server for other people to operate, but refer to "Option one"...

Algorithm: You probably just want Triple-DES and RSA (after September of this year, when RSA becomes free of patents) or else Triple-DES and Diffie-Hellman. It should be easy to switch to a different symmetric cipher later after the new ones have been peer-reviewed and tried by fire, but for starters you want the old standbys that have already withstood the test of time. They will be fast enough for you at first and if you need more speed later you can switch.

Implementation: Your first choice should be to use an extant implementation. Don't try to implement it yourself no matter how simple it looks. Satan's Computer is deceptively subtle to people who are used to hacking Murphy's Computer.

I prefer Wei Dai's Crypto++ library, but that is because I'm doing complex non-standard crypto tricks. If you just want simple "encrypt/authenticate a stream" functionality then use a TLS implementation like OpenSSL. By the way, if anyone wants to make Python wrappers for Crypto++ (possibly with the aid of Swig) then I would love to hear about it!

Okay that's my advice. Specific pitfalls to avoid are: skipping peer-review, trying to design a generalized perfect public key architecture to handle all possible threat models, using a newfangled or non-standard algorithm ("In open source hackery, newfangled is good. In crypto, not."), and implementing it yourself instead of using a library.

Please direct all flames and accolades to: zooko@schowto.mad-scientist.com

Blowfish, 3DES, Twofish

[Eric Green]

Well, Blowfish is also significantly faster than 3DES. In reality, I would say use 3DES unless speed matters, because 3DES has stood the test of time -- if it has not been cracked in almost 20 years, it probably won't be cracked within the near future. If you're going to do something that needs speed, on the other hand, Twofish and Blowfish are both much faster than 3DES. While neither has the history of 3DES, both have received significant cryptanalysis, and Twofish is faster and has a larger block size than Blowfish (should be more secure for many types of cipher feedback modes).

As for why Twofish is not yet widely used -- it's a new cipher. Only two years old. Most of the programs you cite as using Blowfish pre-date Twofish, and it is only recently that I would trust Twofish (after every reputable cipher designer in the world has had two years of trying to crack Twofish). I can't say the same about Serpent, BTW, various AES commentators have nasty things to say about it (mostly that it needs more rounds to ensure proper bit diffusion, which would erase its speed advantage). About all that's been said bad about Twofish is that it is rather klunky and ugly compared to the more elegant ciphers in the AES contest (but a secure, fast klunky!).

Disclaimer -- I did the Twofish module for Python... probably am biased :-).

-E

Cipher selection (professional)

[rjh]

I'm an InfoSec consultant IRL, but this is not professional advice. (And soon I'll be a full-time paid PGP hacker. Yay. )

There are literally dozens of perfectly good ciphers out there. Blowfish, 3DES, Twofish, IDEA, RC2, RC4, SEAL... the list goes on and on and on.

No two ciphers are exactly identical. Cipher selection is based on these minute tradeoffs between ciphers. For instance, 3DES is solid as a rock but is quite slow. I wouldn't use 3DES to encrypt a high-bandwidth realtime communications link, obviously, although I would readily use it to encrypt a 100k file on a disk.

Blowfish is very fast, but key setup is slow--so I wouldn't use it in an environment where keys would be changed frequently.

IDEA is strong and fast, and is well-suited in a whole range of endeavors. It's also encumbered by patent, though, and some of my clients don't want to shell out money to Ascom-Tech A.G..

... In the final analysis, cipher selection is dictated more by secondary factors than by cipher security. There are so many strong ciphers out there that we can engage in personal preference and pay attention to "other details", like performance. It's pretty simple stuff, really. I have a harder time selecting a mode for a cipher than I do selecting a cipher. :)

My own personal fave cipher is GOST, mainly because you get retro-cool points for using it. :) I wouldn't use it in a production system, but when it comes to noodling around with novel crypto (to quote Enoch Root), GOST is my favorite.

Suitability to Application

[Dungeon Dweller]

Well, different encryption technologies are better suited to different applications. Some are designed for streaming media vs files on your HD for instance, or for file compression. However, I can see the point that you are making is quite different in nature from that. In my point of view, people should pic encryptions that fit the following criteria.

1) Usful. You want an encryption that is actually worth using. Not worth encrypting something if it can be cracked without any effort.
2) Suited to the application, as I stated above.
3) Available Internationally (nobody wants to be stuck with a product that gets you arrested on export).
4) Open spec (you want it to work with other products.
** This list is in order of importance.

On top of that, your implementation should take the encryption seriously. Putting in a security measure, and then allowing someone to circumvent that is about as dumb as it gets. One example of this is window's passwords. It's case sensitive in Windows NT, but not in 95, on the same network. This means that, if your computer at least claims to be running Windows 95, you can cut the number of keys you need to check in order to crack into the network exponentially.

Just my $.02

Ease

[Chairboy]

On a product I worked on until recently, encryption selection was done using the following formula:

(sensitivity of data)+(ease of implementation) = (arbitrary decision about how we felt about securing stuff)

An example, we would use a form of blowfish encryption for a configuration file that's super easy to decrypt. Because of the nature of the product, the content would be sniffable when being sent upstream anyhow, so we made the encryption just strong enough to resist a notepad surfer.

On another product, however, we were sending up private data. Now, it was usual registration data during install, but we had real privacy concerns, so we looked at our options and chose SSL because it was easy on our part to implement, but unlikely to be easilly sniffed/decrypted.

Conclusion of ramble: Whatever's easiest to implement with some allowance for the dsecurity of the data.

Security through obfuscation is coming to a close, so this method won't work much longer.

Picking encryption

[Garin]

I find a few things are important in choosing an algorithm. From a math point of view, the optimal solution is NOT to go with the latest-and-greatest algorithm. Serpent, Twofish, and other new algorithms might be all fancy and wonderful, but they simply do not have the history behind them. They don't have years of continual battering that others do. 3DES, for example, has been around for ages. It has been beaten to death by cryptanalysts the world over, and it's still holding up very well. THIS is what makes someone confident in their selection of crypto algorithm. Sure, 3DES may not have a bazillion bits, but there are most likely no bone-headed mistakes in it. Serpent and Twofish probably don't have any either, but there just isn't the history there to say for sure. Encryption algorithms are like wine... Over time, some sour, and some develop into mature, robust, beautiful works of art. In this light, Blowfish becomes attractive. Its advantage over Twofish et al. is age. Its advantage over 3DES is its freedom. This is part of the reason why Blowfish is ubiquitous in the real world.

Why fucking bother?

[Semuta]

The entire world eats, sleeps, breathes, and shits in plaintext, yellow-stickies, credit card numbers on disposed reciepts, telnet, rsh, chatty cordless phones, unencrypted mail, chatty websites, open scripts, broken permissions, pages left on the xerox, unlocked files, and phone numbers given to nasty skanks at a dirty bar that has a scan of your driver's license.

The only real way to keep your information private is to be a boring ugly uninteresting fucking nobody with no life and piece-of-shit computers on crappy dialup lines with no money, bad credit, and massive personality flaws. I got those bases covered!

And if you're reading this, you probably do too.

Actually, Twofish is basically improved Blowfish..

[Sir_Winston]

Twofish isn't all that more difficult to understand than Blowfish, because it's basically just Blowfish with a few improvements and a change in block size as per AES standards.

The key to the popularity of Blowfish over Twofish is merely that Blowfish has been around for many more years. That would normally mean that the cipher is more "proven" than Twofish and others, except that the AES candidates which are still in the running have defied more cryptanalytic attacks than most other ciphers because of the barrage of attacks engendered by the AES process. This leaves the fact that Blowfish has been around longer and therefore gotten into more stuff as its only advantage.

Blowfish and Twofish are both very, very fast on 32-bit microprocessors, which is why they're generally the best contenders for inclusion in crypto programs. Triple-DES, on the other hand, is ungodly slow anywhere but in specialized hardware--because it was meant only as a stop-gap measure for when the 56-bit key length of DES became too short. As for AES candidates with potential, MARS is slower than Twofish but has an extraordinary security margin. It's the only cipher which uses all forms of cryptological tranformations, and therefore should be resistant to most forms of attack if not all. MARS is the one AES contender I'd trust as much as Twofish.

How to chosse an encryption algorithm

[Kiwi]

When choosing a crypto system to use, I make the following considerations:

• The algorithm has to be free and unencumbered by patents. The ElGamal public key algorithm is an example of an unencumbered algorithm, but it did not become unencumbered until the DH patent expired in 1997. The symmetric key algorithm Blowfish is the first unencumbered post-DES algorithm that people believe is secure, which is why it is so popular.
• The algorithm has to feel secure. This is way people stull use 3DES, even though it is far slower than Blowfish and most of the other new SK algorithms at the block cipher lounge and the AES candidates.
• The algorithm has to, once it meets the above two criteria, be efficient. Blowfish has an inefficient key generation cycle, but is otherwise efficient. Rijndael is the most efficient of the AES candidates.

One thing people can do is use a cryptosystem instead of a single algorithm. This makes implememtation much easier, since people don't need to become familiar with Applied Cryptography and the literature on crypto. This is why people like SSL--it is free outside of the US, and will become free in the US on September 20th, and is a complete system belived to be secure.

One of the nice things about crypto research is that most of the research papers out there are freely available on the internet.

- Sam