Slashdot Mirror
Novel Method for Universal Email Authentication
MKaplan writes "Most spam is sent using spoofed domains. Email authentication schemes such as SPF attempt to foil spoofing by having domain administrators publish a list of their approved outgoing mail servers. SPF is sharply limited by incomplete domain participation and failure to authenticate forwarded email. A paper describes a novel method to rapidly generate a near-perfect global SPF database independent of the participation of domain administrators. A single email from an unauthenticated domain is bounced and then resent — this previously unauthenticated domain and the server listed in the return path of the resent bounce are entered into a globally accessible database. All future emails sent from this domain via this server will be authenticated after checking this new database. Mechanisms to authenticate forwarded email and to nullify subversion of this anti-spam system are also described."
Is MS windows boxes that are comprised and doing this - you can see this where the spam mails get 'chinese whispered from one box to another and end up incoherent (to say the least).
Any ISP should/could get suspicious of thousands of mails sent from one 'home user' source at anytime. But when you have thousands of 'users' doing the same thing, it gets lost in the noise.
One simple solution is:
if account == home user & running MS
if mails sent > 10 per minute
block it
fi
fi
etc.
Very easy.
As I stood at a kiosk at a trade show this week, and waded through my spam-filled email on a few services (work email, hotmail, and gmail), the young woman at the kiosk next to me accessed her myspace and facebook accounts and responded to friends only.
She turned and said that only old people use email. And she was a VENDOR at the conference.... Things that make you go hmmmmmmmm......
Many if not most mail servers now drop messages to invalid recipients at SMTP time and don't send bounces any more. I've had to implement this on every mail server I set up to keep the mail queues from backing up to several thousand messages to invalid "bounce" addresses.
It would work if bounce messages were still sent.
My ISP uses it. It frequently bounces my Debian mail. I'm moving my mail to Newsguy where I can turn the damn RBLs off and filter my mail myself.
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
This is just an additional layer over automatic whitelisting of addresses using tagged responses.
Some years ago I set up for my family a pretty simple set of procmail rules and scripts that bounced messages that hadn't otherwise been classified as spam or been whitelisted with requests that they be resent with a certain keyword in the subject line. For example:
"Hello, you just sent me the following message. Could you send me the message again with the word 'leisure' in the subject line? You can reply to this message if you like, just be sure to add 'leisure' to the subject line."
Over a period of several years the only spam that's gotten through this has been from a 419er.
The advantage of a subject line token like this is that you can tell people the token to use, or put the token in the subject line when you send the message so it's usually there when the recipient replies.
Whether you take the resulting message and whitelist the sender address, or some other information in the header that you consider reasonable, that's up to you. It's not really the same thing as the SPF database, though, even if you choose to make the same kind of information the key you use for whitelisting. The point of SPF is that it's supposed to be authoritative for the organizations involved, and doesn't include things like "I sent something with my work address from Earthlink and now you're accepting mail from my work domain through Earthlink's servers".
And using this to whitelist the sender rather than their whole domain gives you a lot finer control.
So? They don't care. They have, effectively, limitless bandwidth and limitless processor power.
Greylisting WAS effective
No. It fails when they implement (as they have) a process to resend any temp rejections after X minutes.
Greylisting had THREE features:
#1. It could temp reject spam and if the spammer never tried again
#2. It could temp reject spam and if the spammer randomized the "From:" username/domain
#3. It could temp reject spam and if the IP addresses was listed in a blacklist within the temp reject time frame
Now all that is left is #3. It costs the spammers NOTHING to upgrade the zombies. And if they get the spam through, the spammer wins.
Now, the zombie can appear MORE legit than a lot of the real mail servers out there.
The spam problem is not that any new anti-spam idea is "unproven" or "untested" or "novel." The true underlying problem is that the mail protocols in place are not securable, never having been designed to be secured. Any true anti-spam change will require a protocol change and the total securing of every box on the net, and like the form letter also points out, that's not going to happen.
Spam filters are the modern equivalent of patent applications for perpetual motion. Eventually, the patent office realized that it had to reject them out of hand, because they were claiming to solve an unsolvable problem. Spam filters fall in that same category, and this form letter is a nerdy way of rejecting them out of hand. "We gotta try new things" is a fine attitude that solves a lot of problems that do have solutions. But it can't solve a problem whose only solution can not exist -- a new protocol relying on perfectly secured endpoint computers.
There is one place where spam filtering does work, and it's part of why I dislike these ideas: private filters can be very effective. That means if you invent a novel spam filter and want to enjoy its benefits for as long as possible, the best way to do it is to keep it quiet. My old simple perl scripts full of regexps worked great up until someone decided to mass-market spam blockers and apply the same principles at the ISP level. At that point my scripts were useless. Spammers really don't care if a handful of nerds block their stuff, but they are out of business if the blocking process can be automated and applied at the ISP or corporate levels. At that point they invent a workaround, rendering the previously working ideas useless.
John
I've been using greylisting. For me, it really hasn't become less effective, but I have noticed that the mix of the spam has changed dramastically.
I'm getting ready to switch to two methods.
First, on one specific account that has become inundated with spam (probably because it is on just about every web page with registered IANA port assignments), I'm in the process of switching it over to the point where it will only accept unencrypted e-mail from a select list of whitelisted sources. If someone is not on that list of whitelisted sources, they are going to have to encrypt the e-mail using my public PGP key for the e-mail to be delivered.
Second, our mail server has something in the range of 100 to 200 users. I am generated thousands of additional e-mail addresses and aliasing them on the server to a single account. Those thousands of new e-mail addresses, initially 8,192 e-mail addresses, will be listed on various web pages for the spammers to harvest.
As e-mail starts to be delivered to those addresses, I will opt-out of all the e-mail so that they know the e-mail address is real and gets read. Once the spam reaches a certain level, I will then start blacklisting every incoming server delivering e-mail to one of those 8,192 addresses.
The length of time on the blacklist will vary. No IP address will be removed until a reverse DNS lookup for it exists.
If the reverse DNS lookup gives any idea that it may be a dialup, dhcp, or anything else that makes it look like it is probably a home computer (e.g. dialup-10-1-1-99.example.com), the IP address will be blocked for a month or more.
If the reverse DNS indicates that it is an smtp server (e.g. mta09.example.com), it will be blacklisted for maybe 24 or 48 hours.
Anything else will be blacklisted for one to two weeks. If additional e-mails arrive from a blacklisted IP address, the clock will start over.
I figure that with 8,192 spamtrap addresses and 100-200 user addresses, most spam zombies will be far more likely to hit the spamtrap addresses first where they may be automagically blacklisted.
Instead of greylisting, I have experimented with a system where my SMTP receiver would send '450 try again later' messages randomly to incoming connection attempts. It actually works. Since the vast majority of incoming mail is spam, a 50% rejection rate reduces spam by almost 50% since most spammers will not retry, while legitimate mailers will.
In the end, I settled for an even simpler approach, where I just throttle the receiver dramatically, so that it takes about 10 seconds to receive a message. Most spammers don't like slow receivers and go away. Real mailers don't have a problem with it. It reduces incoming spam by more than 90%. So this simple change keeps my mail server very well rested from the nice long sleeps...
Excuse me, but please get off my Pennisetum Clandestinum, eh!
I don't know about the GP, but for me greylisting is very effective. I have a personal domain for my wife and myself. I have a catchall mail address.
Here are some stats for part of last week:
Start Date 23/09/07 04:02
End Date 28/09/07 17:00
5.54 days
Total spam: 4624
Spam blocked with greylisting: 4478 (96.8%)
spam via backup MX: 69 (1.5%)
spam retried (got past greylisting): 77 (1.7%)
Total through to end user: 146
Identified as spam (SpamAssassin): 123 (84.2%)
backup MX marked as spam: 50 (72.5%)
direct marked as spam: 72 (93.5%)
Total to end user not marked as spam: 23 (0.5%)
NB. Up until about a month ago, ~25% of SPAM came via my backup MX, which doesn't have greylisting. I don't know why it dropped, but I'm happy it did.
Ever stop to think