Slashdot Mirror

> This release is focusing on deprecating weak and unsafe cryptographic methods,

awwwwwwwwwwwwwwww!!!!!

> though some of the work won't be complete until 7.1.

well I guess that is okay then...

> This release removes support for the following: In the next release, they plan to retire more legacy cryptography. This includes refusing RSA keys smaller than 1024 bits,

LOL! Well, If it makes u feel safer! hahahahaha! http://gizmodo.com/nsa-paid-se...

> disabling these ciphers: blowfish-cbc,

Damn you Bruce Schneier! ''Please take a moment to read Authenticated Encryption and understand why you should prefer to use CCM or EAX over other modes, such as CBC or CTR'' http://www.cryptopp.com/wiki/A...

Encryption is not very expensive. A single core without any hardware help encrypts at a rate of 1Gbit/sec. With hardware carry-less multiplication, the rate at least 6x higher and with aes specific instruction at least 8x.

The cost of encryption is insignificant in comparison of the available power. If your heavy loaded web site have to send at gigabit, you need to dedicate a core or 0.10 to 0.2 core with specific instruction. If your system need to handle gigabit, it's probably a >8 cores system. You have consequently from 1% to 12.5% (without specific instructions) of your system used by the encryption. Most new systems embed specific instructions to handle encryption, we are thus converging to the 1%.

The Core i7 CPU the OP has should be able to do MD5 or SHA256/512 sums at a rate of at least 100 MBytes/second. (See here and here.) Any reasonably modern storage system should be able to feed data that quickly.

At 100 MBytes/sec, 1GB takes 10 seconds, 1TB takes 10,000 seconds. 10,000 seconds is somewhat under 3 hours (800 seconds less), so let's assume 3 hours per TB. With 5TB to hash, it should take around 15 hours, or one overnight plus a little bit.

Surprisingly, it's not that big a problem for a modern PC.

I would make a series of tables as you suggested, split by size. Maybe have separate tables by order of magnitude, FS<1k, 1K<FS<10K, 10K<FS<100k ... In each table, store the file size, file path, create/modify date (may not be accurate, but could be useful), and the hash for that file.

After the first run, this will also provide a mechanism for determining if files have changed.

The Core i7 CPU the OP has should be able to do MD5 or SHA256/512 sums at a rate of at least 100 MBytes/second. (See here and here.) Any reasonably modern storage system should be able to feed data that quickly.

At 100 MBytes/sec, 1GB takes 10 seconds, 1TB takes 10,000 seconds. 10,000 seconds is somewhat under 3 hours (800 seconds less), so let's assume 3 hours per TB. With 5TB to hash, it should take around 15 hours, or one overnight plus a little bit.

Surprisingly, it's not that big a problem for a modern PC.

I would make a series of tables as you suggested, split by size. Maybe have separate tables by order of magnitude, FS<1k, 1K<FS<10K, 10K<FS<100k ... In each table, store the file size, file path, create/modify date (may not be accurate, but could be useful), and the hash for that file.

After the first run, this will also provide a mechanism for determining if files have changed.

Recall however that most implementations have a fixed short encryption exponent (e = 65537, for example) requiring a constant number of multiplications. So, one could argue that RSA encryption has superlinear but subquadratic complexity.

I didn't go into that much detail but performance is asymmetric as well. Encryption and verification is cheap, decryption and signing is expensive. I'll just quote from this table: http://www.cryptopp.com/benchmarks.html (time in ms):

RSA 1024 Encryption 0.08
RSA 1024 Decryption 1.46
RSA 2048 Encryption 0.16
RSA 2048 Decryption 6.08

RSA 1024 Signature 1.48
RSA 1024 Verification 0.07
RSA 2048 Signature 6.05
RSA 2048 Verification 0.16

For the most common case of a secure web server, the client encrypts using the server's public key and the server decrypts, so they get the expensive part of the deal. I also realize my last post was a bit imprecise, yes it's a^b mod n but both b and n will be much bigger on decryption/signing, not just n. I just guesstimated the complexity based on it going from 1.5ms to 6ms.

The solution to this is simple: just iterate the hash function many times so that the time to hash the password is (say) 300ms - unnoticeable to an interactive user, but significant for a brute force attacker. This is called password stretching, and is as important as salt.

See http://www.openwall.com/articles/PHP-Users-Passwords for a review of this and other password hashing issues - not just for PHP, this article gives the thinking behind phpass which is now used in Drupal, and has been reimplemented in other languages. phpass includes bcrypt() as an option but can work even with really old PHP versions that only have MD5. Just because MD5 and SHA1 have been cracked to some degree doesn't invalidate them for password hashing with salt and stretching.

Key derivation functions perform essentially the same operation as password stretching, see http://en.wikipedia.org/wiki/Key_derivation_function - there is an IETF RFC for this.

Digression: Windows 7 still doesn't use salted passwords, which is why it's so easy to crack Win7 passwords given the hashed password, using Rainbow Tables - see http://en.wikipedia.org/wiki/Ophcrack - try the vendor's scarily good online password hash cracker for yourself...)

Most importantly: don't even think of implementing your own crypto code unless the above is very old news to you, because you WILL get it wrong - the examples of unsalted and unstretched passwords are only the beginning. Instead, search for a credible crypto library in your chosen language, and if necessary write a C wrapper so that your preferred scripting language can access a good C/C++ library such as Crypto++ - http://www.cryptopp.com/

It has been shown that E&A (encrypt & authenticate) is inherently insecure: http://www.cryptopp.com/wiki/Authenticated_Encryption. And this is is exactly the scheme used by ASP.NET

But technically the interesting point of the certification id that they managed to get the source code certified. There is at least one other open souce product Crypto++ that is also FIPS 140.2 validated (Certificate #343). But they only managed to get a compiled package validated, which does help me to trust the code but not really to "sell" the library to PHBs. The article doesn't really go into how they did get NIST to validate the source code. Anybody know more details?

Another open source crypto package (actually, it's public domain code) that has received FIPS 140-2 certification is crypto++, a set of C++ crypto classes and such.

It should be noted that if (or rather, when) OpenSSL is FIPS 140-2 certified, it doesn't mean that you can use OpenSSL and claim that your code is FIPS 140-2 certified. Technically, you can't even recompile OpenSSL yourself and claim certification on the resulting binaries, you need to go through the certification process again.

Even still, this is definitely nice to see. Congrats to the OpenSSL team.

J

Crypto++ seems like a better alternative then.

Really? Is Java 2 even available for FreeBSD yet?

Zooko goes to check.

Looks to me like it is still in beta. I strongly doubt that we would have more platforms if we had used Java.

This is not even counting the fact that we developed faster in Python than we would have in Java (I say this based on spending one year working full-time on a Java app and getting basically nothing shippable, and then spending one year working in Python and getting, well, Mojo Nation.) (Not, obviously, that I wrote Mojo Nation all by myself, but the point remains that I've seen Python apps come together way faster than Java apps.)

And this isn't counting the fact that the standard libraries that we used are in C/C++ and are very mature and widely supported than their Java equivalents. Don't get me wrong, I love Cryptix, not only because it was founded by some friends of mine, but because it enables cool tricks like hushmail , but it would've been dog slow to do our crypto in pure Java. The crypto library we use, Wei Dai's Crypto++ is about as fast as can possibly be (including hand-optimized assembly inner loops, if you are on x86). I know you can marry Java app code to native code (and I have done so, way back in the dark ages of Java 1), but the culture of Java frowns on this, whereas the culture of Python has fully embraced native-code integration from day one. Perhaps as a consequence of this, it seems easier to do in Python.

As to your comments about UI, I have to defer to your authority as a user. Personally, I like the HTML UI, and I would much rather have an HTML UI than an actual widget UI, but apparently most users in this world disagree, so you are right -- we should provide the latter.

To bring this conversation back to the topic at hand, has this argument been productive or destructive? I think that it has been useful to talk about the actual features offered by different languages / tools. It is good for programmers to learn from the experience of others. I do not think that Python is Good and Java is Bad. Java has several good features, such as integration with current browsers (allowing cool tricks like hushmail) and possibly good cross-platform UI (although I don't personally know how Swing compares to wxPython), and it has a security model that allows untrusted code, but cross-platform availability and rapid development are not among Java's strengths.

Regards,

Zooko