Novel Method for Universal Email Authentication

MKaplan writes "Most spam is sent using spoofed domains. Email authentication schemes such as SPF attempt to foil spoofing by having domain administrators publish a list of their approved outgoing mail servers. SPF is sharply limited by incomplete domain participation and failure to authenticate forwarded email. A paper describes a novel method to rapidly generate a near-perfect global SPF database independent of the participation of domain administrators. A single email from an unauthenticated domain is bounced and then resent — this previously unauthenticated domain and the server listed in the return path of the resent bounce are entered into a globally accessible database. All future emails sent from this domain via this server will be authenticated after checking this new database. Mechanisms to authenticate forwarded email and to nullify subversion of this anti-spam system are also described."

Comments

Fails to account for SMTP farms...

[pathological liar]

So what happens when you receive an email from a big site like Sympatico, Hotmail, or any number of other places that have farms of SMTP servers, where your message isn't guaranteed to be resent from the same IP?

This also requires users to install software to use effectively, and features CAPTCHAs which are a usability nightmare and not nearly as impregnable as the author thinks.

All that effort instead of just adding a TXT record to their domains.

FUSSP

[Just some bastard]

Basically this guy is proposing an automated whitelist (for domains without SPF records) via a local database. At least I think what the paper is about, I gave up reading it earlier. It lacks a concise summary, doesn't read like a well researched paper and the diagrams don't even display without javascript.

The author may be an anti-spam kook but the paper is so badly written I can't be bothered identifying which.

Not exactly. I think.

[khasim]

He's talking about "bouncing" messages ... but I cannot tell if he means resending an accepted message or denying it at SMTP time.

Then he talks about having people install software:

Auto-Resend software will ensure that almost no one will see or be required to manually respond to the email seen in Figure 2. Auto-Resend software is a simple onetime update for webmail systems, email clients, and local mail servers.

Yeah, installing new software is a great solution.

Major flaw in methodology

[Todd Knarr]

The proposed scheme ignores one thing: the majority of bounce messages today are false bounces caused by spammer joe-jobs, therefore they themselves get flagged as spam and deleted/ignored. In addition, it also increases the annoyance of greylist authentication schemes, since a spammer forging my address in the From field will cause every host participating in this scheme to send me a verification e-mail for a message I didn't send which I'll have to deal with. The proposed scheme makes a very fundamental mistake: assuming that you can trust the sender's address in a message to be the true sender's address. You can do that only after you've determined the message is authentic and not spam, at which point you don't need this scheme anymore.

Re:No, I didn't RTFA..

[ScrewMaster]

Which just continues to show that all sophisticated security systems can and will be defeated by morons. There is no force on the planet more powerful than human stupidity.

Re:Greylisting?

[MightyMartian]

How many times have we heard the "this will fix Spam real good" claim? First it was "close those open relays, ye bastards", and lo, that worked for about a week. Then it was "Well, we'll just keep these black lists, and that'll fix things", until of course the complexity of maintaining such lists and the harsh consequences for any poor bastard who somehow found himself the victim of a false positive tried to get himself off said lists. Then there was "We'll just tarpit consumer IPs based upon some nifty string-matching" and the matching "we'll check reverse IPs, and if they don't match, fuck ya!" which of course buggered up all those poor guys using their cable and DSL connections to run small personal mail servers, or anyone with a retarded or miserable provider who refused to alter reverse DNS entries. Then there was "Hey, you don't have an MX record for that IP, so down the shitter ye go!", which nailed anyone who might be sending from sort of a proxy, and didn't want their actual mail servers advertised as such so that they didn't become victims of joe jobs and distributed dictionary attacks. Then there came greylisting, which actually worked for a while, but seriously screwed with "immediate delivery" that all those in the post UUCP world had become accustomed to with email, not to mention the smart spammers learning from the trick and just retrying. SPF was then heralded as the end-all and be-all, but of course has its own problems (particularly with message forwarding, which requires rewriting the header), not to mention that everyone came into compliance with neutral records, so at least the big guys wouldn't jettison mail from their server due to lack of an SPF record.

At the end of the day, you're right. Statistical filtering, with the careful use of all of the above solutions (though I think whitelists/blacklists are as bad as the problem they attempt to solve) is the only way to reliably filter spam. You're never going to catch it all, but the ISP I worked at was catching, by my estimate, about 90% to 95%, which meant that a guy getting about fifty spam a day was down to three or four, and in many cases less than that. It does mean work, there's no solution that doesn't require monitoring, management and tweaking, because the spammers are smart bastards who learn the tricks as fast we can come up with them.