NIST Opens Competition for a New Hash Algorithm

Invisible Pink Unicorn writes "The National Institute of Standards and Technology has opened a public competition for the development of a new cryptographic hash algorithm, which will be called Secure Hash Algorithm-3 (SHA-3), and will augment the current algorithms specified in the Federal Information Processing Standard (FIPS) 180-2. This is in response to serious attacks reported in recent years against cryptographic hash algorithms, including SHA-1, and because SHA-1 and the SHA-2 family share a similar design. Submissions are being accepted through October 2008, and the competition timeline indicates that a winner will be announced in 2012."

Comments

hash algorithm hash recipe

[Briden]

i prefer the bubble bag method for making hash

Encryption == Something to Hide

[explosivejared]

Why does the government promote creating new encryption methods when encrypting data so clearly means you have something to hide and are therefore guilty? I mean COME ON!

Re:Encryption == Something to Hide

[flosofl]

One very simple attack was changing the grades in a school system. The school encrypted the grades, so they thought they were safe from change. The failed students hacked into the system, and just changed their data to the same data held against students they knew had done well. Anyway, that's just one way that encryption doesn't protect you against malicious modification. It gets a lot sneakier the more you look into it.

What was the school using, ROT13? It sounds like they were using a substitution cipher not a modern algorithm. If they had been using any kind of real encryption, there would be no way that technique would be possible. Some of the tests that modern encryption algorithms have to face are frequency analysis (which substitution ciphers fail) and known plain text attacks (I assume the students had access to the encrypted txt and their real information). Other than the school using a centuries old, easily defeated technique I call bullshit.

Modern encryption *does* protect you from malicious altering of information. I encourage you to read up on Message Authentication Code (and all it's sundry relatives, UMAC, HMAC, CMAC). By changing just one character in an encrypted block, you have just caused the MAC to show a mismatch and invalidate the integrity and authenticity of the data. Unless they have the key used for encryption (which would raise the question of why they simply substituted characters in an encrypted field), they are shit out of luck trying to fool anyone. Yes, the cipher block is useless, but no one will be "tricked" by the changed grade, either.

Encryption != Hashing

[rock217]

Encryption implies that you can reconstruct the original string from the encoded. Methods like md5, sha1, etc are one way algorithms that cannot be reversed* in a realistic amount of time.



* - Rainbow tables

Re:Encryption != Hashing

[TechyImmigrant]

When hashing a data set larger than the resulting digest, it cannot be reversed at all. However you can find collisions which is handy if you want to subvert the PKI hierarchy that protects web transactions.

Re:Encryption != Hashing

[Comatose51]

Rainbow tables won't help you get the old message back since pretty much by definition or pigeonhole theorem there is more than one plaintext that can generate the same hash. Breaking a hash algorithm usually involves finding a plaintext that generates the specific hash, thus fooling the victim into thinking that plaintext was the original one.

Or imagine this: you have a simple hash function that takes all the letters in a message, turns them into number based on their place in the alphabet, and adds them up to generate the sum. If that sum goes over 10,000 then it would do a mod 10,000 to wrap it around. There's an infinite number of plaintexts that can generate the exact same hash based on this hash algorithm. However, what you can never do is figure out which specific one generated it.

Re:What would happen if...

[SigILL]

What would happen if you wrote a program to randomly create algorithms? Most of them would be rubbish, but occasionally you'd hit gold.

Yes, and you'd spend most of your time trying to prove those algorithms are any good. That's the hard part anyhow, coming up with new algorithms isn't.

Oblig. xkcd link

[hellergood]

http://xkcd.com/257/

Re:I know I'm paranoid, but...

[kebes]

I know I'm being paranoid, but did anybody else think that this is a way for the gummint to get a look at the various methods people are using to secure their data? I think you are being a bit paranoid! NIST is proposing an open competition to develop a new open standard for hashing. Anyone who wants to participate can do so. Anyone who wants to retain their "secret hashing method" can continue to keep it secret. It's not like the government is demanding anything. This is just a research agency promoting open research.

Not to mention that I sincerely doubt that anyone is currently using some super-secret ultra-elite hashing algorithm that no one else knows about. This field of mathematics and security is quite mature and very much open to scrutiny currently. The current solutions are fully documented. I think the point here is that further progress isn't going to be made by lone researchers hiding their results: the only way forward is via more open collaboration.

What better way to get the methods than to have a 'competition', something that will stroke the egos of crackers? If a cracker wants to sell his secrets at the cost of an ego-stroke, that's his choice. Nothing nefarious here. Again, NIST is not going to take these results and use them for evil ends (or even for commercial gain): they are hoping to create an open, public standard that everyone will benefit from (and which international experts in mathematics, cryptography, and computer security will analyze in detail). That's what NIST does.

Sorry, but I think your paranoia is unfounded in this case!

(Disclosure: I work with NIST, but have nothing to do with this project. Note that my opinions are my own and should not be construed as official statements from NIST.)

Re:I don't get it

[bmac83]

Pay attention. You will be given a short string of characters that describes how to get from the prize to where you currently are, but from the directions it will be impossible to find your way back to the prize.

Re:I don't get it

[Lord Ender]

If you can claim to be the author of the US government standard cryptographic hash, you get to charge pretty much whatever you want in consulting fees.

Re:New Hash Algorithm Submission #1

[spun]

As you've admitted to being a libertarian, I suppose I should make one for you, too:

1. Declare war on Big Government with bogus slogan "Let the free market fix __________"
2. Announce plans to decrease funding to social programs
3. Figure out that you have no one in any elected office in any country anywhere who can carry out 2.
4. Announce that someone who has never professed to be a libertarian but holds a few libertarian ideals, is in fact a libertarian. Do the same for historical figures, especially anarchists.
5. Make up bogus arguments about the magical free market that will never be put to any sort of test, due to 3., above.
6. Parrot back tired arguments that were disproved hundreds of years ago, back in the days of lassez-faire. Conveniently forget about child labor, horrid working conditions, rampant pollution, institutionalized racism, debt slavery, and any other facts that show unregulated free market capitalism destroys lives.
7. Cherry pick examples of deregulation and privatization, ignoring any cases that prove libertarian methods wrong.
8. Try to convince other libertarians to all move to the same state so you can remedy point 2.
9. Realize that convincing self-centered libertarians to do anything is like trying to herd cats.
10. The rest of us grow bored with your childish, self involved, "Nyah nyah, you're not the boss of me!" political stance and ignore you, as libertarians have never managed to do anything more than talk.

Wait, that's not funny, it's just sad.

Re:I don't get it

[click2005]

The cake is a lie
The cake is a lie
The cake is a lie
The cake is a lie

Re:Just use Identity...

[marcello_dl]

No.

Very similar to the AES competition

[Sycraft-fu]

Also done by NIST. I suppose you could be all paranoid and claim that AES was chosen so the that US government could snoop on you since, after all, the NSA signed off on it as being secure and they'd never tell the truth, right? Well, except for the fact that it was designed by a couple of Belgians and has also been signed off on by essentially every other respected crypto expert and organization there is.

So that leaves you with two possible situations:

1) That the NSA is so amazingly far ahead of everyone else in crypto that they were able to find something in AES that no one else has in over a decade. Also they are so confident in their knowledge that they believe nobody else will find it since if they did the results would be a big problem (AES is approved for classified data, and is used by US financial institutions).

or

2) AES is really secure, and the NSA is telling the truth.

Now which is more likely? Also, supposing you believe option #1 then why trust any crypto? If the NSA really is so good that they can outdo the entire rest of the crypto community, well then they can probably break pretty much any of the cryptosystems out there. You can't trust any of them since the only people who would really know if they were insecure won't say.

Seems extremely unlikely.

Well, same deal with this hash competition. If you believe that the government will be able to pick one that is in fact something they can break, but that nobody else in the world will know about this then it doesn't matter, because their understanding is so far advanced that all hashes would have to be suspect.

Given the extremely public, international, nature of things like this there really isn't any room for mistrust. I again point to the results of the AES competition. You want to talk about a cypher that has stood up to some extreme scrutiny, there you go.

Re:Very similar to the AES competition

[lgw]

1) That the NSA is so amazingly far ahead of everyone else in crypto that they were able to find something in AES that no one else has in over a decade. When the DES standard was created, the NSA was so amazing far ahead of everyone else that they were able to find somehting in DES that no one else found for over a decade. The NSA provided very specific technical advice (without explanation) that was followed in the creation of DES. Many years later, the rest of the world caught up and discovered that the NSA had corrected a very subtle weakness in DES.

The NSA has an actual track record here, and their motives have proven good so far. However, they claim that (due to lack of funding and too much competition from financial firms for math PhDs) they aren't so far ahead any more.

Re:Very similar to the AES competition

[Llywelyn]

It is worth emphasizing that the NSA has said that AES 128/192/256 can be used to protect information up to the secret level, and that top secret information can be secured with AES 192 or 256. That's a pretty strong statement coming from the NSA, which if acting rationally they would not want to leave weaknesses in something that is used to secure information that would be, by definition, "very damaging to the US and its interests if released."

Now, it is possible that such statements are just for show, but it takes a belief that they are playing an incredulously deep game that they would make those statements as a denial and deception practice.

Re:Very similar to the AES competition

[evilviper]

However, they claim that (due to lack of funding and too much competition from financial firms for math PhDs) they aren't so far ahead any more.

You've got it wrong. They were decades ahead because nobody outside of the NSA was doing cryptography AT ALL. There was no real effort at all from the private sector.

DES was really the ONE cryptographic algorithm that existed, anywhere, and even that could only be found internal to IBM, which was by far the biggest digital equipment company anywhere at the time.

It isn't "too much competition" now, it's simply that, for the first time, they've got any competition at all.

Re:Very similar to the AES competition

[James Youngman]

If the NSA really is so good that they can outdo the entire rest of the crypto community, well then they can probably break pretty much any of the cryptosystems out there.

Actually I think you're right, but to play Devil's Advocate for a moment, I will note that the UK government agency GCHQ developed a public-key cryptosystem between 1969 and 1973, significantly before Diffie and Hellman's (apparently) ground-breaking paper. So, government agencies are quite capable of beating the public state of the art and not telling anyone about it.

Re:New Hash Algorithm Submission #1

[Archangel Michael]

First off, Touche. I love a good ribbing ... :-D

1) Never been tried.
2) What's wrong with this?
3) Sad, isn't it?
4) Huh?
5) Again haven't been tried in a while
6) I actually believe GVMT Roll in some of these things
7) No Cherry Picking here
8) Whatever
9) Whatever
10) Too many people being (D) or (R) because of Fear and Fear.

Lets just deal with #1

Free Markets are easy to control. Corporate Charters are given by the GVMT, why aren't they revoked more often? Why aren't assets seized? Why aren't boards of directors arrested and charged for lack of proper stewardship?

Much of the problems seen in the free market isn't the fault of free markets. It is the fault of interference when it isn't needed, and non-interference when it is needed. Indeed, there hasn't really been a "free market" in 150 years or so. Closest we have right now is the Internet, and with Congress getting involved it's only going to ruin it.

We don't need more laws, we need more responsibility.

Re:I don't get it

[ajlitt]

If you cannot comprehend the string, assume the party escort submission position. A party representative will arrive shortly to escort you to your prize and a party celebrating your reception of said prize. There will be cake.

Re:No, you're right.

[smallfries]

Maybe you should chase the etymology one level deeper. If the original data cannot be recovered then it is not "hidden" but "destroyed". You may not believe that the term encryption means a two-way process with an available decryption function - but that is the definition that the crypto community uses, and so it's good enough for me.

Oh no doubt

[Sycraft-fu]

And there's evidence that the NSA understood quite a bit more about cryptography back in the DES days based on a change they made ot it that hardened it against an as of yet unknown kind of attack.

However being a bit ahead in terms of creating a system is real different form being far enough ahead to break systems. To mistrust the NSA on AES means you figure that they know enough to know how to break it, and that they figure the knowledge is so far advanced that no one else will figure it out. One of the NSA's jobs is actually "To achieve information assurance for information infrastructures critical to U.S. national security interests." They are tasked with things like making sure that US financial systems aren't broken in to, hence things like DES/AES. As such if they knowingly allowed a breakable cryptosystem to become the standard and it was in fact broken, they'd have failed in that and have shit to answer for.

So while I certainly believe they are the best in the business, and while I'd not be surprised to discover they know things that public does not, it would imply a staggering advance in cryptography for them to be able to break AES and figure that the public can't. In fact, it would probably imply something along the Tom Clancy lines of a computer that could break ANY machine based cypher and as such no matter what crypto you used short of a one time pad, you'd be screwed.

I just don't find it reasonable to believe that. I find it more reasonable to believe that since good crypto is out there anyhow, and since their job is to protect US interests, that they did an honest analysis of AES and found it to be highly secure, just as everyone else did.