Slashdot Mirror
Fight Spam With Nolisting
An anonymous reader writes with the technique of Nolisting, which fights spam by specifying a primary MX that is always unavailable. The page is an extensive FAQ and how-to guide that addressed the objections I immediately came up with. From the article: "It has been observed that when a domain has both a primary (high priority, low number) and a secondary (low priority, high number) MX record configured in DNS, overall SMTP connections will decrease when the primary MX is unavailable. This decrease is unexpected because RFC 2821 (Simple Mail Transfer Protocol) specifies that a client MUST try and retry each MX address in order, and SHOULD try at least two addresses. It turns out that nearly all violators of this specification exist for the purpose of sending spam or viruses. Nolisting takes advantage of this behavior by configuring a domain's primary MX record to use an IP address that does not have an active service listening on SMTP port 25. RFC-compliant clients will retry delivery to the secondary MX, which is configured to serve the role normally performed by the primary MX)."
YASIGFINFE (Yet Another Spam Idea Good For Individuals, Not For Everyone) - Spammers will change their techniques to be more RFC compliant as soon as (if) Yahoo, AOL, Hotmail, Gmail adopted this method.
Your post advocates a
(x) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
(x) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
(X) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
(X) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
( ) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!
There are shills on slashdot. Apparently, I'm one of them.
This strikes me as the ultimate in temporary solutions. If spam senders *tend* to use only the primary MX record, and people start fighting spam by listing bad primaries, won't the spam senders simply start using secondaries? It almost seems the only way that this approach might be valuable, is if it weren't publicized and posted on /., and one kept it to oneself :)
Love many, trust a few, do harm to none.
This is not a long term solution.
1) It's bad netiquette, and a lot of people don't like that, including myself and I'm sure many other administrators.
2) It's an artificial "defense" that is easily circumvented because the rule is obvious. It's security through obscurity with the added suck that there is no obscurity.
3) It's solving a symptom and not any of the actual problems (e.g. hosts being compromised to send spam).
Thanks, but I'll pass.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
An anonymous reader writes with the technique of Nolisting, which fights spam by specifying a primary MX that is always unavailable.
Funny, I fight afternoon meeting schedulings in almost the same way. Just specify a primary time that's always unavailable.
The theory of relativity doesn't work right in Arkansas.
We get stuff directed at our secondary all the time, despite having a highly available primary. Why? Our secondary is listed at another domain - they do our backup in the case of disaster. I can only assume that spammers hit it thinking that its a 'back door' into the network, perhaps we don't have the same rigorous anti-spam measures there.
Dumb idea. You're better sending all your domain mail to gmail, using their spam filtering, and then pulling it from there.
I want to delete my account but Slashdot doesn't allow it.
Most spam bots already send to the *lowest* priority MX (ie. the highest number), and work their way backwards, because it's common for the backup MX'es to have lower anti-spam rules.
However, this idea would have been *great* six years ago. Once the developer invents a time machine, he's got the spam problem licked for at least a week!
There is more spam than penises needing enlargement, dammit!
I cant believe this is allowed to go on. How long did it take for callerID and no-call lists to get here? How long before we start putting these people in jail!
No more bandaids, lock these fuckers up!
For some time a few years ago, spammers used to IGNORE the primary MX and send to secondary MXs preferentially.
Since in our case, the 2ndary MX was a dumb sendmail relay only without knowledge of the user DB, it shot the traffic load out thru the roof with bounces to junk spam that, because they couldn't be rejected during the actual delivery attempt, hammered our backup relay.
This is just a dumb idea.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Based on watching a few corporate spam sites and even stuff which reaches my private, never-posted addresses, *much* of the spam could be eliminated by moving non-Windows clients. I'm not just talking about zombies. Some of the spam I see hits lists of addresses which are valid and include very difficult to guess addresses inside the company. Once somebody inside your company, or a buddy of yours is rooted, your previously private address is out there; I've never had this happen via any route but a Windows user. Of course, people who CC: everybody they know with idiotic crap instead of BCC: make this problem much worse.
Oh, and please stop with the lame form letter responses to these articles. It was cute once, long ago. I know at least five people will have posted them by now. Damn spammers.
it makes sense as a spammer to hit the secondary MX anyway as *most* secondaries don't know anything about the mail accounts themselves, but rather just spool and relay the domain onto the primary. with this in mind the secondaries will nearly always accept mail for any account in the domain, say 'thankyou very much' to the SMTP client and go about managing its local queue for delivery, hammering away at delivery attempts on the primary and then filling up the secondary queues trying to send the bounces back to bogus return paths, so i'm not sure i understand how nolisting is anything *but* a band-aid solution.
as a spammer writing your own SMTP engine, why wouldn't you just write in basic queue management into your client to get around nolisting/greylisting/nastyhacklisting...?
"Greylisting" is where an SMTP server refuses messages for a certain amount of time. You set the criteria on why the message would be refused and how long the server would refuse to accept it.
It's been pretty much defeated now because so many spammers have their machines try to hammer the message through until it does go through.
I'm using greylisting right now and the only advantage is that many times a spammer will end up on an RBL during the 15 minutes that I'm refusing his messages.
Remember, the spammers have, effectively, unlimted bandwidth and unlimited processing power at their disposal.
Spammers will often try secondary (and lower) MX's because there's a good chance that the anti-spam AND ANTI-VIRUS systems on those machines are weaker (read "outdated") than on the primary MX.
The more machines you have to maintain, the more likely you are to focus your efforts on the most critical ones and just let the other slide. Spammers are happy to exploit this.
Set the primary MX to 127.0.0.1 . That should keep those buggers busy for a few days. Have fun with those feedback loops, sucka!
;p
Of course, the same might be true of legitimate senders, as well....
Funny, I fight afternoon meeting schedulings in almost the same way. Just specify a primary time that's always unavailable.
When I worked overnights, I had a similar system.
Boss: We need to talk.
Me: Great. What night would you like to come in?
Boss: No, I mean you should stay late.
Me: But you don't come in until 9, and my shift ends at 7.
Boss: But it's important!
Me: Why is it always about your needs. Your need to have a meeting. Your need to get a decent night's sleep. What about my need not to sit around for two hours on the clock waiting for you to show up, surfing the web, all the while getting paid one-and-a-half my regular pa...okay, fine, you win.
Then, when I became the boss years later, I would always show up at the beginning of the night shift to talk to the employees, and then go to the bar. It made the employees feel noticed and made my superiors think I was motivated. Turns out my best defense against assholes like me is actually having been me.
The only surefire protection against Microsoft infections is abstinence. - The Onion
Standard Smartass Form for Comments on SPAM
.... you! Kind of joke
1. Please select format:
( ) In soviet Russia
(x) The same old form on spam subject we're tired to see here
( ) Some comment on female parts
( ) Suggesting you/slashdot_readers are virgins
( ) Will it run Linux?
( ) Cowboy Neal
2. Are you:
(x) Meant to be funny
( ) In a bad day, trolling
(x) Being authoritative on this subject
(x) Expecting to be modded up
( ) Agreeing with the news
(x) Trying to piss over something people might think it's interesting or relevant
3. Include "I'll be modded down for this but...."? (Y/N)
No
Thank you for submitting your message to the Slashdot forum.
Slashdot Quick'n'simple Form: The easy way to show people how smart your are!
Gmail's filtering is, well, badass. I'd think a large number of companies would be willing to pay them to handle email for their domains and forward to a company mail server which only accepts messages via gmail. You'd get a very nice web interface, but could still have the speed and power of a local POP/IMAP server. And virtually no spam. That would be worth a few bucks a month per account for a lot of people. Me, I'd be a little creeped out by them having that much access to my personal emails. Which is why I only use gmail for stuff that I don't want lost in a spam filter, like job searching, financial transactions, attorneys, my friends traveling in the Middle East, etc. But nothing personal!
I run a mail system that pushes ~3million messages per day. Not huge, not small.
We have thousands of domains pointed to our mail servers and secondary MX servers. Looking at the long run stats, I'd be tempted to completely disregard this technique.
When we take a primary down for maintenance, the secondaries and alternate primaries (same weight MX) see the load almost immediately.
I second the opinion that if this has any effect, it's only for low volume applications, with few/one domain.
We generally see more hits straight to the secondaries by spammers hoping for less rigorous checking. It would be interesting to profile IPs connecting to secondaries without being seen at the primary assuming a primary is always available - I bet that a very high percentage of these connections to secondaries could be viewed as spam.
The problem remains that most tricks of this sort - including greylisting - are eventually circumvented by spammers once the trick gains critical mass. Lets not forget that there are a lot of broken, yet not open relay, mail servers out there. Good engineers and administrators quickly find that Jon Postel's words ring true with their customers "Be liberal in what you accept, and conservative in what you send." - don't let your RFC enforcing configuration be responsible for delaying/blocking the delivery of that big contract your PHB was waiting for!
How hard would it be for Yahoo, Google and other internet mail services to simply have two inboxes?
One for mail addressed to someone in your mailbox.
One for everyone else.
90% of my spam problem would be solved by this simple recipe.
It is your personal duty to fight for what is right on a daily basis. Ignoring injustice is identical to approving
This probably works in many cases, but as a mail system admin I can tell you that it can fail and will cause problems for legitimate mail delivery. Over the past few months I remember seeing a few messages stuck in my Postfix mail queue, that didn't ever seem to make it out to the recipient's MX. These were domains with deliberately non-functioning MX, and I could not figure out why Postfix was not trying the other MX even though it was up and running. In one case I also tried mailing the recipient domain through gmail, which ALSO failed after many days of retrying. Again I am not sure why the scheme failed to work, but it did fail through both Postfix and gmail which are two very legitimate mail servers.
Sorry, this isn't going to work. It won't even help a little bit. As a long-time email administrator and the author of an email server I can tell you, with absolute certainty, that spammers ignore the priority of your MX records. In fact, they exploit multiple MX's much of the time, by sending spam to your secondary server(s) even if the primary one is up. In addition to extra target capacity, they often manage to take advantage of badly configured secondaries that might not have spam filtering that's as good as the primary, and in many cases the primary has its secondaries whitelisted to make sure no mail gets accidentally dropped.
Tired of FB/Google censorship? Visit UNCENSORED!
How comes everyone tries to fight spam by breaking infrastructure? Wikipedia neuters links, email server admins delay mails (graylisting) or even reject connections (unlisting), users turn off Flash and Javascript to avoid ads. IMHO, if we have to break our own toys to keep the spammers from playing with them, we're heading for dull times.
But they're often slow to respond. Hell, I changed a DNS record when I moved servers once and spammers will still going after the other server, with no DNS record pointing to it, for 6 months because they use static caches.
Many people were already using this trick, probably hoping it wouldn't show up as lead story on slashdot.
In some ways, selfish ways, it's like the story of the two hikers who face a bear. The first hiker immediately sits down and starts putting on his running shoes. The other says, "What are you doing? You can't outrun the bear!" The first hiker says, "I don't have to outrun the bear. I just have to outrun you."
Many spammers, faced with a failed attempt at sending mail, do not bother to retry or try other MX. Instead, they just move on to the next target in the list, since trying a new target is just as easy as retrying an old target. No real difference to them. But it means you just push your spam attempts onto other people who haven't elected to bend the standards to divert the spammers.
The "good" spam sending programs run many threads, timeouts don't punish them, their limit is more the bandwidth. Attempts to divert spammers onto others who have not tried the tricks should create an ethical question. Are we just arranging for the bear to eat our friend?
Has it been over a year since you last donated to the Electronic Frontier Foundation
Like it or not, these spammers run extremely profitable businesses. You may not realize it, but they can only continue doing what they're doing because enough people actually do happen to buy the products that they advertise via spam. If people stopped buying items advertised in that way, then the spammers would have no market to sell to, they wouldn't make money, and thus would have virtually no reason to send out spam.
A number of recent studies have shown that most of the major purchasers of goods advertised via spam are from the United States. One particular report offered statistics showing that most spam-advertised goods were bought by people in the Oklahoma, Arkansas, Mississippi, Alabama, Tennessee and Missouri region of the US. Another major area for the purchasers of spam-advertised items was London, England.
If anyone is responsible for spam, it is all the people who actively go forth and continually buy the items that are advertised via email spam.
ISPs must restrict clients to 'n' emails (ie free minutes) per day based on their type of account. If they want to send more they have to pay.
Undetectable Steganography? Yep, there's an app fo
I was reading the article, and suddenly port knocking came to mind. It wouldn't be a far stretch to modify an SMTP server to only reject connections on the lower priority IP address if the source had not tried to first connect to the higher priority IP address.
Instead of blocking the connection to the primary at a firewall or using an "unused" IP address, the primary SMTP server could give a greeting banner and then immediately return a "temporarily unavailable" status code (and cache who was connecting there).
In other words, an RFC compliant MTA should be connecting to the higher priority host as defined by DNS first, then fail over to the lower priorty host, in order. If an MTA tried to connect directly to the secondary MX first it could be rejected with a temporary failure status code which a spammer is likely to ignore. It would require the SMTP receiver to keep a cache of who had connected to what IP addresses within a certain time period which would eat up some memory depending on traffic load. We already cache reverse DNS lookups and RBL lookups, so it could probably be done.
With this setup you would have two MX records for your primary mail server that your SMTP server would be active and listen on. It would just track the order of connections to ensure that the remote MTA was following the rules before it allowed the source to get past the greeting banner.
I for one welcome our soon-to-be-RFC-compliant spammer overlords. I mean, we want standards compliance, right? Right??
This post expresses my opinion, not that of my employer. And yes, IAAL.
I have read some truly terrible ideas on this website. (Usually followed by a chorus of inexperienced idiots blindly saying how great they are, while all the skilled and experienced people rolled their eyes.)
This is one of the worst ideas I have ever read. Intentionally introducing a large and unpredictable delay into the receipt of all e-mail.
What's next, a recommendation to cut down on telemarketing by setting your PBX to automatically disconnect 50% of all incoming calls?
I run a fairly low-key server, which I only use for my family, so I am not sure how relevant my data is.
I remember at one point last year checking on the usage my backup MX gets and was surprised to see a lot of mail coming through it. Surprised because my primary server is (almost) always available. Upon a closer inspection I was astounded by what I found: all the email that came through the backup MX was spam for the past year was spam. No exceptions!
Certainly, mine is an extreme case, but I think the trend is very clear.
I use qconfirm myself but there's also tmda and others.
*If* you are serious about getting rid of the spam then just do it. The technical part is readily available.
I deployed that almost a year ago and never looked back. I still see the occassional spam in a
mailing list folder because those go through unfiltered for obvious reasons but I couldn't care less.
My inbox has been spam-free since then and that's what matters.
I don't quite get why people are still bothering with greylisting, spamassassin, razor, dcc, bayes and
the ilk. I tried them all and they're more trouble than it's worth. You get false positives, false negatives,
it's a stupid game that you can't win.
The first time I ever saw one of those "forms", I thought it was interesting.
The second time, I thought it was "ho-hum".
After hundreds, maybe even thousands, they are just plain lame.
The only good thing about them is that you instantly know that you can skip over them and not miss anything at all.
Actually, as long as it is correctly filled in, I find that form consistently insightful.
The reason is that a lot of people preach some new approach to fighting spam, and in reality there are a finite set of reasons which defeat every single one of these ideas to date. When someone comes up with an approach that passes this form, then we'll have something to talk about. If it can't pass this form, then further discussion isn't really merited since it's not even novel enough to get past the standard set of objections that have so far been raised against and successfully predicted the downfall of every failed anti-spam solution to date.
Ideas that can't pass the form are not worth more effort to respond to than putting an X at the appropriate spots on the form.
Slay a dragon... over lunch!
I can't believe someone that claims to have anti-spam knowledge is suggesting this when in fact the opposite is true. Spammers frequently forgo opening an SMTP connection to the MX with the highest priority (lowest numeric value) and instead opt for the ones with the lowest priority. They do this hoping that the secondary MX doesn't have the same spam-fighting abilities as the primary MX. They're hoping that it's a simple backup or that it only queues for the recipient domain in question and doesn't validate recipient userids. The spammers hope that the primary MX will accept all mail blindly from the secondary, as is usually the case. This has been a long-standing theory that hasn't ever been disproven. This jives with what I've always seen on all my MXs.
Except it wasn't filled in consistently.
These are incorrectly checked:
(X) Many email users cannot afford to lose business or alienate potential employers
(x) Dishonesty on the part of spammers themselves
(x) It will stop spam for two weeks and then we'll be stuck with it
(x) Asshats
The plan loses no email that is distributed by an actual mail server. Even the crappiest actual mail server out there follows the rules by checking another MX server, and if it doesn't it's going to lose a lot of mail anyway. Supporting multiple MX records isn't some obscure part of the standard, it's a major requirement, and all actual mail servers do.
And spammers can't 'lie' their way around it. They can use software that operates correctly in the first place, but the years have demonstrated exactly how long it takes them to switch. I have no idea how long the spam software pipeline is, but spammers have operated software that is broken in many ways, and people have been consistently using that brokenness to block spam for years.
If this reduces spam for a time and then stops reducing spam, I'm failing to see what the problem is. I'm still checking that the MAIL FROM domain is a real domain, and it's astonishing how much spam doesn't even bother to do that. Or checking that the HELO is not a negative number. (I have no idea what that's about.)
And we won't be 'stuck' with it. It doesn't change anything. People can point to a fake MX server for however as long as they want, and then switch back to just having their one real one, whenever they want.
And the 'asshats' check box is used to mean people can abuse or break the system. I have no idea why it was checked.
About the only one correctly checked complain is:
(X) Eternal arms race involved in all filtering approaches
Yes, it's an arms race, and, yes, it will lose power over time as spammer's crapware adapts. Aaaand? At the very least we cost spammers money, and upgrading spam software is insanely expensive. We didn't hurt ourself in the slightest.
If corporations are people, aren't stockholders guilty of slavery?
...and encourage readers to RTFA, where I've addressed many of the issues brought up in these comments. I also encourage people to try the technique, if they are in the position to do so (admins only, this is not a solution for endusers), and evaluate it for themselves. Or not. It's true that most new antispam solutions are dreamed up by crackpots. I might be a crackpot. If this possibility concerns you, don't be an early adopter. Wait and see.
It's true, in my experience, that Nolisting stops some spam with no false positives (in my experience). And that's a Good Thing. But it doesn't stop significantly more spam than a combination of other techniques, which I also implement. Some of those techniques use a lot of resources, such as content filters (often powered by perl) and virus scanners. Nolisting provides a way to free up some of those resources, possibly resulting in better performance and even hardware savings. These savings can be significant at large sites that currently scan each and every message that arrives.
Nolisting can be bypassed. I don't make any wild claims. Spammers can get past it easily by going directly to the secondary MX. Guess what? They already do that, and have been doing that well before greylisting was introduced. Nolisting significantly reduces the percentage of spam my MX processes, thereby freeing up resources. It's just one part of a layered solution.
I've limited secondary MX access by extending Nolisting into Unlisting (Port Knocking for SMTP): http://www.joreybump.com/code/howto/unlisting.html . It's wildly effective, except for one serious problem: A retry might originate from a different IP. This appears to be legal, and seems to be the result of load balancing strategies adopted by some important sites. For that reason I don't recommend it. It will randomly block messages from gmail, for example. You can't reasonably predict the IP a multihomed host will use for a retry, so be very skeptical of any approach that claims to have solved this problem.
Unwanted email is annoying. When it carries a payload, it is potentially dangerous. But I don't really view this as a security issue. I don't buy the argument that Nolisting is security by obscurity, and therefore bad. It's a form of access control, a gatekeeper, a prophylactic. It's an apple a day, not a cure for cancer. It's not addicting, fattening, or life-threatening. Try it, if you're looking for ways to improve the health of your mail system. Discontinue use immediately at the first sign of complications. Side effects include more sleep and time spent with your kids.
Nolisting rarely introduces delays. As I point out in the article, most relays retry immediately. Any relay that cannot get beyond Nolisting is seriously, seriously noncompliant. While I don't suggest Nolisting as a complete replacement for Greylisting, it is a viable alternative for sites that experience problems with Greylisting and find the delays it introduces to be unacceptable. As the name implies, Nolisting is meant to used without dependence on whitelists. Wider adoption and testing will determine if this ideal has been realized.
Like Greylisting, Nolisting breaks infrastructure to some degree. Many admins find this distasteful. I know I do. If Nolisting becomes widely adopted, logs will become fatter with "Connection refused" errors when the primary MX doesn't respond. I'm sorry for that. But our logs are already fat with 45x errors from Greylisting, RBL disconnections, SpamAssassin scores, etc. Nolisting might even help to make logs smaller, if you currently see a lot of these messages. Time will tell. Keep an open mind, and remember that we often make concessions to improve a system's overall health. Just reducing the possibility of another zombie being created on the Internet creates benefits for everyone.
Try it before you draw a c
For now I'll stick with SPF and old fashioned spamassassin (milter).
And whats with the anti SPF sentiment? Its not like we've got a lot of more effective alternatives on the market and the only real argument I read is the rejection of real email, when softfail pretty much takes care of that (then leaving it to spamassassin to decide if the mail is legit).
We send an receive a good deal of email and I certainly wish SPF was more common. I'm tired of forged bounces and the *slew* of undeliverable responses 'dumb' servers return to our system every day.
Yet instead of taking any real action we bicker while spammers laugh all the way to the bank. Their is no magic bullet, but from my POV SPF is the closest thing yet (unless my DNS gets hi-jacked, but then I'm fucked anyway).
Quack, quack.
Instead of rejecting connections to the third MX record, you could teergrube them, so the spammer's machine ends up dogged out on tiny TCP windows talking to a mail server that's going very slowly and will eventually reject their message. If you want to get fancy, you could also have it feed blacklists, or at least adjust greylist timers, but just being passive-aggressive toward spammers is a good start.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks