Hashing Email Addresses For Web Considered Harmful

cce writes "The MicroID standard, despite getting thrashed soundly by Ben Laurie two years ago, has since been recommended by the DataPortability Project and published on the user profiles of millions of users at Digg and Last.fm. MicroID is basically a hash calculated using a user's profile page URL and registered email address, producing a token that makes the email address vulnerable to dictionary attacks. To see how easy it was to crack these tokens, I conducted a small study, choosing 56,775 random Digg users, and cracking the email addresses of 14,294 of them (25%) using just their MicroID, username, and a list of popular email domains. Digg has more than 2 million users, and that means half a million of them — mostly people who had never heard of MicroID, and had probably not logged in for a long time — had their email addresses exposed to this trivial attack. I also applied this attack to Last.fm (19%) and ClaimID (34%). Digg and Last.fm have since removed support for MicroID, but the lesson is clear: don't publish a hash of my email address online, guys!"

Comments

Re:What does MicroID actually do for the user?

[Fred Ferrigno]

I read up on it and I'm still confused, but I think this is the idea:

1. You set up an account at website Alpha.
2. You have a publicly-viewable profile page at Alpha. On the page is your MicroID.
3. You set up an account at website Beta.
4. You tell Beta about your Alpha profile page.
5. Beta verifies that your Alpha profile page is really yours by checking the MicroID.

Beta can't really do anything with your Alpha page except link to it. I guess the point would be to prevent people who aren't you from linking to your Alpha page on their Beta pages. That way, other people can be sure that the same person owns both accounts.

The attack mentioned in the article doesn't compromise the proper use of the MicroID, since Beta is assumed to have verified that you own your email address and you wouldn't link to a profile page claiming to be yours that wasn't. All it does is make it possible for spammers to harvest your email.

even the spec admits it is retarded

[hdon]

I wrote about this earlier this year. My conclusion, more or less, was to carefully read the specification, which Iâ(TM)ll excerpt here:

By itself, a MicroID has no inherent meaning, since it is simply a string created from two URIs. Any entity can generate a MicroID even if it has not verified the identity of the resources associated with one or both URIs. Furthermore, a MicroID is easily copied by an entity that did not generate it. Finally, a MicroID is not digitally signed by the entity that generated it and therefore cannot be cryptographically associated with the generating entity.

Re:Solution: salt your emails

[geekgirlandrea]

Except that lots and lots of web sites fail at RFC 822 and think + isn't a valid character in an e-mail address. Usually the same sort of maldesigned horrors that make you type your e-mail address twice even though, unlike your password, you can read it as you type to make sure it's correct, or have a single free-form blank for credit card numbers and enforce some idiosyncratic rule on separators (really, is $cc =~ s/-//g; that hard?), or enforce strong passwords and then cripple them with mandatory 'security' questions that allow anyone who knows you halfway well to reset your password.

Yeah, I use them too, and if web designers were a whole lot smarter they would be a better solution to things like this, but in practice lots of web sites just refuse to accept addresses like that. I should get around to making sendmail let me use an underscore instead of a + for that purpose.

Re:Okay...

[WuphonsReach]

Do they still do that? I know from a distant past they tried it with smaller providers too, but haven't seen them for a long time. As far as I can tell, spammers do still use malware which harvests/sniffs email-address directly from peoples computers.

This is a definite tactic. I see it all the time on a mail server that I administer. From the results, there are definitely spammers that monitor user's e-mail, address book, or other sources of e-mail addresses on their computer. (Basically, on a brand new e-mail address, the user started getting spam within a few hours of contacting someone else.)

But we still see dictionary attacks on our mail server, so that's a popular tactic too.

Re:Solution: salt your emails

[statemachine]

Giving out e-mails with "+something" is worthless for spam. The malicious spammers will just strip the "+something" from address, as both can be delivered, but the short form will be less likely filtered, and you won't know which service it was sold/stolen from.

I actually make a separate alias for each site eg. name-something@example.com. If you shorten my alias to the part before the hyphen, it won't deliver. Yes, spammers have tried.

If you're using "+something" just know that you might as well not append that onto your e-mail address, for all the good that it does, as you're giving out your primary address anyway. Cat, bag, already open.

Re:Solution: salt your emails

[aj50]

Except that some web forms (and some mail servers) won't accept an email address with a '+' in it.

We use these types of addresses at work to organise replies to tickets and some people's mail set-ups really screw things up.

Re:Flawed study?

[QuantumG]

Offline attacks are better because they:

1. can't be monitored
2. can't be blocked
3. are not limited by bandwidth
4. can be sped up by throwing more hardware at them

This is basically why salting was added to the unix password file. And that failed.. so /etc/shadow was introduced. Revealing hashes is just unnecessary, so don't do it.

Re:Solution: salt your emails

[mi]

+ is a bad delimiter.

It is the delimiter, originally created as such by the authors of the very first MTA... There is no other character, that:

1. Can be part of an e-mail address.
2. Can not be part of a username.

Many web-forms don't accept email addresses with '+' in the username portion. Attempts to educate webmasters to the information in the relevant RFC's is usually met with silence or worse...

This is, unfortunately, the truth... Far too many programmer wannabees around... It is a good fight, however, and kudos to GMail for keeping support for it (unlike Yahoo! Mail).

I use this whenever I can, when giving my address to web-sites (including Slashdot)...

Re:Don't use Gravatar

[Anonymous Coward]

Uh... Do you realise what the article you're commenting on is about?

Just use dots, then

[Cow Jones]

Apart from the fact "+" is a perfectly valid character in an email address, if you're using Gmail, you can insert random dots in your address, and your mail will still get delivered.

my.name@gmail.com

is equivalent to

my.na.me@gmail.com
my....name@gmail.com
m.y.n.a.m.e@gmail.com
etc

Re:Just use dots, then

[funfail]

This is completely different. What the grandparent said that "username.ml@gmail.com" would automatically go to "usernameml@gmail.com". Gmail just ignores dots in e-mail addresses.

Re:Solution: salt your emails

[mcrbids]

Let's see... Large email provider, throwaway addresses, access until you don't want it anymore...

You mean, kinda like Mailinator??

There are others, Mailinator is the easiest.

Re:Solution: salt your emails

[skeeto]

This is, unfortunately, the truth... Far too many programmer wannabees around...

It is also unfortunate that perfect e-mail parsing is extremely complex. The Perl regexp for e-mail address validation according to RFC 822 is about 6.3 kilobytes. If you try to do it yourself you are pretty much guaranteed to get it wrong.

Those crappy programmers could still make things much better with liberal validation, allowing some invalid addresses to make validation simpler. Something simple like /[^@]+@[^@]+\.[^@]+/, will match all valid e-mail addresses (I think, and the /. filter won't let me write anything more complex than that anyway) plus a bunch of invalid ones.