Slashdot Mirror
Novel Method for Universal Email Authentication
MKaplan writes "Most spam is sent using spoofed domains. Email authentication schemes such as SPF attempt to foil spoofing by having domain administrators publish a list of their approved outgoing mail servers. SPF is sharply limited by incomplete domain participation and failure to authenticate forwarded email. A paper describes a novel method to rapidly generate a near-perfect global SPF database independent of the participation of domain administrators. A single email from an unauthenticated domain is bounced and then resent — this previously unauthenticated domain and the server listed in the return path of the resent bounce are entered into a globally accessible database. All future emails sent from this domain via this server will be authenticated after checking this new database. Mechanisms to authenticate forwarded email and to nullify subversion of this anti-spam system are also described."
So what happens when you receive an email from a big site like Sympatico, Hotmail, or any number of other places that have farms of SMTP servers, where your message isn't guaranteed to be resent from the same IP?
This also requires users to install software to use effectively, and features CAPTCHAs which are a usability nightmare and not nearly as impregnable as the author thinks.
All that effort instead of just adding a TXT record to their domains.
The author may be an anti-spam kook but the paper is so badly written I can't be bothered identifying which.
Then he talks about having people install software:
Yeah, installing new software is a great solution.
The proposed scheme ignores one thing: the majority of bounce messages today are false bounces caused by spammer joe-jobs, therefore they themselves get flagged as spam and deleted/ignored. In addition, it also increases the annoyance of greylist authentication schemes, since a spammer forging my address in the From field will cause every host participating in this scheme to send me a verification e-mail for a message I didn't send which I'll have to deal with. The proposed scheme makes a very fundamental mistake: assuming that you can trust the sender's address in a message to be the true sender's address. You can do that only after you've determined the message is authentic and not spam, at which point you don't need this scheme anymore.
Which just continues to show that all sophisticated security systems can and will be defeated by morons. There is no force on the planet more powerful than human stupidity.
The higher the technology, the sharper that two-edged sword.
This is exactly why greylisting is effective. It pushes the cost of spamming back on the spammers. Now they have to have a semi-legitimate mail relay, vs. fire and forget. If everyone greylisted, then the spammer's mail queues would be huge.
Of course, all bets are off with zombies that start using legitimate SMTP servers, but there are solutions to that already in place:
The only place this fails is if the spammers as part of their owning of zombie hosts begin to check for the proper SMTP server to relay through and configure accordingly. Admittedly, this is not too difficult to do, but they aren't doing it yet.
I receive approximately one spam email every 45 seconds. Constantly. Without spam filtering, I would go to bed with an empty inbox and wake up to 500 spam emails. Spam filtering, far from being futile, is the only thing that makes email usable for me. Without spam filtering, I would simply have to give up on email.
Can it stop all spam? No. Do filters have to adapt? Yes. But that hardly means that filtering is futile, it just means that it's not as easy as we'd all like it to be.
Bogtha Bogtha Bogtha
Nope. We need a solution involving cruise missiles though bedroom windows late at night.
We need Spam Assasin Ninjas clad in impregable black carbon-fibre capes with the knives of cutting edge technology and the deadly intent of artificial intelligence enhanced mania.
We need mountains of spammer bodies piled high on the forefront of technological .
We need chain gangs of spammers publicly televised chanting "The Only Good Spammer is a dead Spammer" to the sound of hammers hitting rocks.
IN Summary: Cruel and inhuman tortue is not enough for these guys
Sent from my ASR33 using ASCII
Huh? When I take a look at how many mails are bounced on all my domains, thanks to greylisting, each day, and hold it against how much spam actually enters my mailbox, i'd say they haven't adapted at all.
When you are sending millions of mails, retrying is far, far more expensive than just ignoring it.
I don't know, I didn't get that far. The article and the concept is bullshit.
The 'From' field is the keystone of their identification process. Well, I got news for you if you bothered to read the RFC. 'From' does not have to represent the real sender. I can forge it up all I want into anything I want and you can't tell. I didn't get past section 3 where this is before I determined the rest isn't worth reading.
Once again we have another company trying to come up the next Big Thing and they don't know what the hell they are talking about. SPF is cute -- but relies too much on people setting it up and correctly. I suppose you could pay a service to act as a third party validator, but that's turning into a boondoggle too.
I don't think bouncing email at valid senders is going to win any friends.
Perhaps there is a way to do it successfully and with great accuracy. I would love to say I'm working on it. But quite frankly, if I do figure it out I probably won't mention to anyone since I really don't want the legal hassle of trying to defend my idea against someone else's billions. I can block spam. I can block spam to the tune of 99+%. The rest is trivial. I was even surprised to hear them say 94% was the average. Perhaps people would be better off if they stopped using SpamAssassin.
Sorry, my opinion is that statistical filtering is more than sufficient if it's managed well. I think few people are willing to do the work required of them to make them spam free. Kind of like locking the door to keep out the crooks.
How many times have we heard the "this will fix Spam real good" claim? First it was "close those open relays, ye bastards", and lo, that worked for about a week. Then it was "Well, we'll just keep these black lists, and that'll fix things", until of course the complexity of maintaining such lists and the harsh consequences for any poor bastard who somehow found himself the victim of a false positive tried to get himself off said lists. Then there was "We'll just tarpit consumer IPs based upon some nifty string-matching" and the matching "we'll check reverse IPs, and if they don't match, fuck ya!" which of course buggered up all those poor guys using their cable and DSL connections to run small personal mail servers, or anyone with a retarded or miserable provider who refused to alter reverse DNS entries. Then there was "Hey, you don't have an MX record for that IP, so down the shitter ye go!", which nailed anyone who might be sending from sort of a proxy, and didn't want their actual mail servers advertised as such so that they didn't become victims of joe jobs and distributed dictionary attacks. Then there came greylisting, which actually worked for a while, but seriously screwed with "immediate delivery" that all those in the post UUCP world had become accustomed to with email, not to mention the smart spammers learning from the trick and just retrying. SPF was then heralded as the end-all and be-all, but of course has its own problems (particularly with message forwarding, which requires rewriting the header), not to mention that everyone came into compliance with neutral records, so at least the big guys wouldn't jettison mail from their server due to lack of an SPF record.
At the end of the day, you're right. Statistical filtering, with the careful use of all of the above solutions (though I think whitelists/blacklists are as bad as the problem they attempt to solve) is the only way to reliably filter spam. You're never going to catch it all, but the ISP I worked at was catching, by my estimate, about 90% to 95%, which meant that a guy getting about fifty spam a day was down to three or four, and in many cases less than that. It does mean work, there's no solution that doesn't require monitoring, management and tweaking, because the spammers are smart bastards who learn the tricks as fast we can come up with them.
The world's burning. Moped Jesus spotted on I50. Details at 11.
This scheme seems every bit as awful as those "Hi! Before anyone e-mails me the first time, I make them go through these steps" filters
- It causes backscatter
- It doesn't work with mail from mailing lists
- It's not accessible
Additionally:
- It doesn't work well with sites that have many MTAs (requires one bounce/CAPTCHA per MTA)
- It doesn't work well with an SMTP server that sends for many domains (requires one bounce per MTA per outgoing domain)
- It merely confirms that "this server can send mail for domain X". If you've got a spambot and can determine your user's domain name (e.g. comcast.com), this won't stop anything at all.
The author brushes off concerns with bold (well, italic now) statements like:
Resend software is a simple onetime update for webmail systems, email clients, and local mail servers...Universal Distribution of Auto-Resend Software is a Surprisingly Simple Thing to Achieve
Hah! A simple one-time update for all servers and clients everywhere! Granted, RIA doesn't depend on that update happening, but it's clear even the author thinks it'd be a pain without auto-resend.
There is little disincentive to implement Auto-Resend software as it is a one-time upgrade that remains dormant until needed.
There is a huge disincentive; looking up a user's mailbox to see if he did, indeed, send the message you claim he sent is a ridiculously expensive operation, if it's even possible at the server level. It could also lead to a privacy leak if done wrong; people could forge RIA bounces to probe outgoing mail flows.
At best, it potentially doubles the volume of outgoing mail, which deepens queues, requires more disk space, etc. etc.
I'm guessing the author is unfamiliar with high-volume mail sites - the very ones he wants to implement this scheme first.
Suspicious Domains Will Be Neutralized By CAPTCHA Encoded Sub-addresses
Great. So now e-mail that's "suspicious" requires intervention from a sighted human, and all his "auto-resend" silver bullets are used up. He does imagine yet another client change that will "nicely reformat" a CAPTCHA. Yeah, right. Oh, and now he's e-mailing me graphics on my Blackberry.
In general, he seems to imagine that he personally runs the One True RIA list, and we all trust his determinations of what is and isn't "suspicious", with reputation scores, rate limiting, etc. That is, of course, ridiculous; the original MAPS RBL has splintered and grown to the point where there are over 200 DNSBLs available.
He talks about automatically e-mailing users that he has "detected" are running zombies. Right, because that's a good idea and isn't spam.
Domains commonly associated with phishing (e.g. Paypal.com, Citibank.com)
As if there's a way to create a comprehensive, or even useful, list of "domains commonly associated with phishing".
with the passage of time it will become difficult for spammers to purport that all of their spam is sent via increasingly obsolete or esoteric brands of software.
Of course it won't. I still get spam from "The Bat!". Before, he forgot about the big guys; now he's forgetting about the long tail. Spammers can make up any number of X-Mailer names.