Slashdot Mirror
Ask Slashdot: Is Reverse DNS a Worthy Standard For Fighting Spam?
drmartin66 makes it to the front page with this question: "Last weekend I installed a new spam filter server for a client, and enabled connection rejection if the sending server did not have a Reverse DNS record. Since then, I have had a number of emails rejected from regulator bodies that do not have a Reverse DNS record, and are refusing to have one created for their email server. What is your opinion of Reverse DNS records? Are they (or should they be) a standard, and required? Or are they useless for spam fighting?"
Like all things spam, marking the message as bad automatically is generally discouraged. If you simply increase the SCL value by some reasonable number, and continue to raise SCL based on other soft violations (like spamhaus, surbl, etc), you will rarely put good senders in the junk email folder, and very frequently be able to reject most spam content.
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Yes, it may cut down on the "infected" PC spam that hits a mail server, but all it will do is drive that traffic to the numerous spam farms that are opening operating on ISPs
Yes. Absolutely. It works.
In all but the most closed groups, having a system that generates lots of false positives is in most cases going to be a bad move in my opinion.
What reason would anyone have to be running an SMTP server without a PTR record?
We (~10K users, ~1.5M incoming emails a month) used to do this, but there was a constant stream of complaints from users due to email bouncing from organisations that refused to set up the records. It got worse every year too. So we gave up and went for a commercial software solution for a while, and eventually outsourced our mailfiltering completely to a hosted solution. I would NEVER do filtering in-house again, unless for a very small number of technically literate users.
Bob.
Spammers can choose to use reverse DNS, many of the people you really want email from can't. All reverse DNS does is create more frustration for legitimate users.
Reverse DNS record
As with most spam fighting metrics it's up to you. Mail from a server without reverse DNS that doesn't trigger any of your other flags generally shouldn't be treated as spam if you care about false positives, if it's borderline then maybe the lack of reverse DNS will be enough to justify tagging it as spam. The decision of how heavily to weight the lack of reverse DNS is yours, personally I don't give it much weight but it does add a little to the score. Some people go hardcore and reject anything that doesn't have come from a machine with reverse DNS, they accept the significant false positive rate usually for idealogical reasons (while I like a properly configured system I'm not going to bite my nose off to spite my face).
Since an ip can map to multiple domains
Use multiple checks. Use scoring, and a threshold. voila.
Buanzo Consulting - 15 Years of GNU/Linux experience, for you.
I've set up a few mail relay and spam filtering server and I can tell you it helps a lot to reduce the number of spam arriving to them.
I am a Postfix kind of admin (hell with sendmail!) and I know you can set some filter just before the reverse DNS check to accept the connection if it comes from a particular host/IP address, bypassing the reverse DNS check. Or you could add that reverse DNS to your local/client DNS server but it seems not that a good solution.
Anyway following standards is always the best solution.
--- Bouh !!! ---
If your email server does not have rDNS records then it's very likely half your mail is not getting delivered. aol.com, gmail, hotmail, etc all require rDNS.
Blocking on invalid rDNS, invalid or missing A records and not following proper smtp protocol is helpful on a email gateway. However, if you are a relay for clients you'll have problems.
Unless you are one of the lucky few who have a full class address space, you are stuck with the will of the ISP to either setup reverse entries for you or to delegate resolution to you. Alphatel has it right. Use it if you choose, and grade along with other tests.
Denying all wrong rDNS is a bit harsh, however, denying what's a DSL and the rDNS declaring as such is a good idea. I've yet found very very few cases where it was an issue. For such case, just white-list the entry your customer is complaining about. Here's how to do with postfix:
/etc/postfix/maps/relaying_stoplist
/^dsl.*\..*/i 553 AUTO_DSL We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server. -dsl-
/[a|x]dsl.*\..*\..*/i 553 AUTO_[A|X]DSL We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/cable.*\..*\..*/i 553 AUTO_CABLE We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/.*dial(\.|-).*\..*\..*/i 553 AUTO_DIAL We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/ppp.*\..*/i 553 AUTO_PPP We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/dslam.*\..*\..*/i 553 AUTO_DSLAM We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/dslb.*\..*\..*/i 553 AUTO_DSLB We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/dynamicIP.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/fbx.*\..*\..*/i 553 AUTO_fbx We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/abo.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/socal.res.*\..*\..*/i 553 AUTO_REV We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/.dhcp.*\..*\..*/i 553 AUTO_dhcp We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
smtpd_client_restrictions = permit_mynetworks,
permit_sasl_authenticated,
check_client_access regexp:/etc/postfix/maps/relaying_stoplist,
permit
And here's the content of my
#/.*\.dsl\..*/i 553 AUTO_DSL2 We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/client.*\..*\..*/i 553 AUTO_CLIENT We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/pool\..*/i 553 AUTO_POOL We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/dynamic.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/staticIP.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/dip.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
It is possible to configure your mailer to require all sorts of things, like rDNS. If you configure all of them you will get almost no spam, but you'll also not get 50% of your legitimate e-mail. Perhaps that's ok with you, you're willing to only talk to the "clueful".
Most people though want to get mail. The old Internet axiom "Be conservative with what you send, be liberal with what you accept" applies. WIth spam fighting this generally means use every mechanism at your disposal (including rDNS existence, or matching with forward DNS); but use it only to affect the score of a message. That way the guy who doesn't have rDNS right, but does everything else right will still get through.
In an organization operating a mailserver without a PTR record for their SMTP, the users should be having so much difficulty sending outbound mail that they know something is wrong. I know this from experience, having set up an SMTP without reverse DNS, and then observing that half my test messages bounced back.
I run a pretty decent volume mail platform (80k mailboxes, 2.5M attempted messages/day) and when I first inherited it we required reverse DNS records. It seemed great, it demonstrably cut spam volume, and it really screwed a lot of our customers on a semi-regular basis. Unfortunately lots of legitimate mail senders simply don't have access to proper reverse dns, maybe because their ISP won't delegate or maybe because they're barely technically literate and too small to hire a competent consultant.
Bottom line I found was there are better ways to get the same results without having all the collateral damage. For a personal mail server I might still have the requirement, but the instant people are depending on my platform for their email I'm going to go with the stuff that doesn't have such strong downsides.
Many email servers do not have rDNS, therefore it is not advisable to filter based on a lack of rDNS alone.
It can be argued that it should have an rDNS, but if they don't, you have no control over that since it's their system. Then you'll be spending way too much of your time tweaking spam filters and creating white lists, contacting the sending company's administrators...
It's just a bad idea, don't do it.
That said, I prefer SaaS email spam filtering like Symantec's Messagelabs. (Disclosure, I am an ML partner)). I like this service because I don't have to worry about managing it. It saves me a lot of time.
A bit over a year ago, I performed a reverse dns scan of the entire internet. It took around 4 months and amounted to about 62GB of data which I haven't sorted through just yet, but I'd guess based on what I've seen so far that only 20-30% of the utilized Internet has reverse DNS entries. This is kinda what I suspected all along, but I now have data to back that up. How can anything properly use reverse DNS with that low of an adoption rate. Its sad because so many more services could be offered if it was properly adopted.
You got it. Today, unless you have a dedicated Email server guy it's retarded to run an Exchange server in house. Outside companies do it better and when the Company Fiber goes down YET AGAIN (Thanks AT&T) you dont lose 24 hours of emails, we actually lost 72 hours worth as AT&T decided that working over the weekend was not important.
Do not look at laser with remaining good eye.
You know....I hate spam. It made usenet useless for years, it continues to degrade the usefulness of email, spamers steal resources and are underhanded dickwads.
All that said, some of the anti-spam people are ridiculous zealots who don't care who gets caught in the crossfire.
I have a server in colo. Its my mail server, but it also does a number of other things. Until recently, it ran a tor node. Why? Because i had sooo much more allocated bandwidth than I was using on a monthly basis that it cost me nothing extra to run. Ran it for at least 6 years on the same node.
Its now shut off, why? Because some idiots at Spamhaus decided that running a tor server was suspect. Never mind that it was disallowed from exiting on port 25, which is publically posted info in its service descriptor....no... Of course, I think they are also fooled by the fact that several windows users have shell accounts and use it as a web proxy.... so somehow my box also was infected with a Windows trojan according to these geniuses.
We got it cleared up, but still are not able to donate excess bandwidth allowance to the tor network.... which is bad enough, but this isn't the first time I have had my server blacklisted for no good reason at all. I don't even remember what BS it was last time, just that it was... BS.
Now will this kill me? No.... I have reverse DNS setup and have for years but...come on.... seriously? Bouncing mail sucks, especially when you suddenly start doing it to whole domains.
If it were just me, my opinion is that anyone using one of these RBLs has a misconfigured mail server, I wouldn't have "fixed it".... but I host other peoeple's email domains, so the black ball tactics worked.
"I opened my eyes, and everything went dark again"
Being fed up with postfix and exim, I recently wrote a simple e-mail server using python. I followed the RFC standard as well as I could, but to my surprise, I noticed there are numerous special undocumented tricks one needs to know to get mail through to the recipient in a reliable way (whitelists, blacklists, reverse dns, etc). I am wondering if anybody here knows if there is a place on the net where such tricks are documented.
PS: IANAS (I am not a spammer, honestly)
If Pandora's box is destined to be opened, *I* want to be the one to open it.
Just whitelist any email that has a verified digital signature. Everything else you can't trust.
Good luck getting anyone to actually set up and use digital signatures/encryption, though :-P But if you make it a matter of policy and give them the tools to use it, there's no better way.
As somebody who used to work in the email industry, I can assure you that rejecting based on the presence of RDNS is useless against entities that run their own infrastructure. Everybody working legitimately ('opt-in', though I refer to it as 'suckered in' because the person didn't read the privacy policy that said people who filled out the form were going to get emailed and their info sold/traded) in the email industry creates RDNS as a part of their SOP. When I worked in it, we never sent out email without first verifying that reverse DNS was set up and set up properly. Granted, you'll still likely catch spam from botnets.
Your post advocates a
(X) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
(X) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
(X) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
(X) Susceptibility of protocols other than SMTP to attack
(X) Willingness of users to install OS patches received by email
(X) Armies of worm riddled broadband-connected Windows boxes
(X) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
(X) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(X) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
(X) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
(X) Why should we have to trust you and your servers?
( ) Incompatibility with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(X) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!
Remember that not every non-spam email originates from a perfectly-configured self-hosted SMTP server. Many organizations outsource their email, spam filtering, compliance filtering, notice / statement delivery, etc. While it's easy to posit that the IT departments in such organizations have a duty to maintain reverse DNS records for all their partners' servers, don't fall into the trap of thinking that every organization has a fully-staffed, knowledgeable IT department... or an IT department at all.
Having a reverse DNS is a good practice, and anyone with a mail server should be doing it. That said, a lot of small businesses don't have reverse DNS set up, don't know what you mean when you tell them to do it, or have ISPs that are a pain to deal with. I'd mark up the spam score on a message without reverse DNS on the sending server (and I do on my own server) but I wouldn't block it entirely unless it sets off a lot more flags than just that one.
I use Kerio Connect on my server - I add 2 points for lack of reverse DNS. 3.5 points drops you into the junk folder, 5 blocks you completely. Doing that I get pretty much no false blocks, a false positive every few days, and about 3-5 spams that make it to the junk folder per day. I block a few hundred.
-- Josh Turiel
"2. Do not eat iPod Shuffle."
Your post advocates a
(x) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(x) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(x) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
(x) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
(x) Lack of centrally controlling authority for email
(x) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
(x) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
(x) Armies of worm riddled broadband-connected Windows boxes
(x) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
(x) Joe jobs and/or identity theft
( ) Technically illiterate politicians
(x) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(x) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
(x) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
(x) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!
REM Old programmers don't die. They just GOSUB without RETURN.
The windows admin where I work hates reverse DNS. He thinks it's stupid and refuses to keep it updated. So... if there are more people like him out there, you might have issues getting email from them.
Making fun of dumb people since 2009
For short term outages, sending servers will queue messages and try again later. You can avoid long term outages like this one by having redundant Internet connections from different providers.
Uhh...not to nitpick, but that is what backup MX servers are for. When your primary server is not available, mail is delivered to one of the others. If your e-mail is that critical then you need to have a store-and-forward server somewhere else, just in case your link goes down.
There are lots of services that provide this, if you don't want to do it yourself. But setting up a simple store-and-forward server isn't all that complicated and doesn't need a full Exchange deployment.
Learning HOW to think is more important than learning WHAT to think.
Reverse DNS makes absolute sense as a means to counter spam, but the reality I've experienced is that there are too many IT admins that don't even know what a PTR record is, and since they don't experience an issue with all of their mail recipients, they assume it's an unnecessary step, even when you try to explain that it with a sock puppet show.
The requirement for reverse dns is in hindsight a part of the "security theater" where various claims are made, and remedies suggested against perceived ills. The suggestion for reverse DNS comes solidly from the era of TCP wrappers, another supposed saviour of ill maintained systems from outside evils.
In reality, there is no actual increase of security from checking if some address has reverse dns as for ages most of the dial up and broadband lines all have reverse dns. Also, as reverse dns zones are by and large often unmaintained, esp. when it comes to removing entries, you neither can rely on the data returned, nor assign any significance to what is returned.
Sounds like a plan, care to explain that to the CTO? I tried several times and was told that it is a "zero payback expense"
Do not look at laser with remaining good eye.
Unless you are one of the lucky few who have a full class address space, you are stuck with the will of the ISP to either setup reverse entries for you or to delegate resolution to you. Alphatel has it right. Use it if you choose, and grade along with other tests.
Any ISP will setup rDNS entries if you have a business account. The only time this is an issue is if you are trying to run a server from a home account. Most of the ISPs I have used or looked into prohibited running servers with a consumer account. Those that didn't were also happy to provide a rDNS entry if you paid for a static IP.
Trying to run a server when your ISP is opposed to you doing so is inherently problematic, rDNS just being one of your many concerns. It is fine for experimenting and learning, but not for servers that do anything important. The only excuse for a business or government agency not having a rDNS entry is incompetence.
The real question is how tolerant should you be of the incompetent, and from a business point of view the unfortunate answer is "very tolerant".
Yes, reverse DNS records should be standard. No, they should not be required. Yes, they are useless as a toll for fighting spam.
"Convictions are more dangerous enemies of truth than lies."
Besides, since when did every MTA stop following the SMTP RFC that says you MUST not discard the message until delivery has failed for over 72 hours? Even if your server was down over a weekend, you should still have 24 hours to get it back up without losing any mail.
Gamingmuseum.com: Give your 3D accelerator a rest.
The real problem is that while this would really help fight spam, there's collateral damage. Just like the judicial systems in civilized countries tend to operate on the principle that it's better to set 100 guilty people free rather than imprison 1 innocent person, most people who receive email would rather receive and delete 100 spam messages than miss one legit email inquiry from a potential customer or long-lost friend.
Sender Policy Framework seems even better than simple reverse DNS in theory, but it doesn't seem to get much traction because it causes more serious problems than spam in general causes. Until a critical mass of sysadmins basically tell the domain owners who are too stupid or lazy to add the appropriate DNS records to fuck off, lazy and stupid sysadmins will continue to not add those records. But until then, customers will cry and rebel if any of the good sysadmins who host them try to apply a passive spam filter that relies on such records. That's just how it goes, it's a Catch-22 which is preventing widespread adoption. The only potential solution would be to stage an SPF or rDNS record adoption day, get some big names on board, and hope for the best.
This is simple stuff here. Firstly, it verifies ownership of the domain. I will never accept email from a host that does not resolve. Doing so will of course allow a ridiculous amount of spam from infected computers around the globe from regular IP addresses. The email address needs to match the host in which it is sending from as well. It requires hardly *any* work. Why are we even talking about this?
HELO, ELO, wants the hostname as well. Are we expecting millions of mail servers to simple change the way they're doing things? More importantly, if spam is a problem for you, I've had great luck with filtering services such as Postini and SpamSoap. Both excellent providers and have better resources than anything you can whip together on your little email server. It costs pennies as well.
*plays the Apogee theme song music*
From my experience and that of our other sysadmins here, rDNS never works and you'd constantly have users come and complain that they are missing a large amount of their incoming emails. The best defenses against spam are keeping your primary email as private as possible, using a good statistical spam filter, and ideally mail encryption since that makes filters more easily recognize spam since it'll stand out against the noise of legitimate emails. Also, in terms email clients the only one we specifically recommend is Thunderbird, even in combination with gmail (except for tagging they integrate very well now).
If email is important to your organization then the cost of a correctly configured mail server is insignificant.
Seriously, your email server can be anywhere in the world. There's no reason that you have to go through a specific ISP. Even if they're blocking port 25, you can get a different ISP to accept mail from you on a different port. Even Google offers that option.
I've had this same fight with multiple vendors and organizations over the years. Like many others, rDNS should be implement properly. It's just the way to keep a clean house. However, the RFC doesn't require rDNS validation checking, and do to so break mail delivery for many, many legitimate services. Spammers will (actually, have--this idea is old) found new avenues of attack. Reputation scoring, token analysis, and other statistical measures are a far better set of solutions to work with. You can go on the cheap with some sendmail RBL and dspam type solutions all the way up to what I consider best in class right now--Ironport. A Microsoft shop would probably like FOPE--it integrates well and does a damn nice job of proper spam filtering too. The larger MS shops with EA agreements can roll than into the package for decent pricing.
The question "Is Reverse DNS a Worthy Standard For Fighting Spam?" is incorrect. Spam is not the reason; using it as a measure of clue is. Servers that emit spam and and clue level can be related, though. If someone is clueful enough to set up a mail server properly they're going to make sure it has reverse DNS. A mail server run by a less than clueful individual (or set-and-forget with no admin) is more likely to be a problem source either now or in the future than the ones that are cluefully configured and actively maintained.
Of course you are going to have spammers that are clueful mail admins and will set up their servers properly. That's why you can't pigeonhole reverse DNS as some kind of spam fighting method alone. But it can always be used as a measure of cluefulness.
this is my sig
A lot of system admins do not create rDNS causing rejections. Most of the time it's a headache so we don't do it.
The only truly effective spam prevention technique I've seen is greylisting.
Unselfish actions pay back better
Sounds like a good reason to go job hunting before getting fired for his incompetence.
Its right, its not fair; but its needed. Legitimate sites should have no problems setting up reverse records or getting their provider to do if for them.
Anyone who is not in a position get PTR records in place for their mail server is not actually in a position to be running a mail server anyway. Sorry that is just the way it is. PTR records are nice to have for any number of mail delivery troubleshooting and validation issues outside of SPAM.
As a mail admin I'd kinda consider them a requirement anyway. Its not easy to work transmission problems when I can't figure out who the admin of the other server is and how to get in touch with them.
I know its not within the standards, but I say no PTR record no, mail accepted.
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
Simple. Ask him if any of the email sent or received has value to the company. If the answer is "Yes", ask if he insures his car/home/dog. When he says yes point out the backup MX cost is insurance against the value of the email. If he says "No", point down you can save more money by closing down all email. Then go back to the first question, or close it down and get out of there.
just set up postini properly - all mail to you is filtered by likely the biggest most up to date filter in the world (and dont forget to set up inbound rule in your firewall narrowed to your mailserver only receiving SMTP from postini)- dont try and make your own spam filters -- postini is the best.
i do not work for google.
It should not be an automatic ban, instead part of the overall score applied by the filter(s).
Real men would scan the IPv6 space too.... :)
-Charlie
We pay our ISP to provide 'queue service' in the event our connection/Exchange box goes down. When 'selling' this service to the bosses I put it this way - How much will it cost the firm to not only be without email during an outage, but to lose all emails sent to the firm during that time period?
I feel your frustration though... IT is often seen as a 'zero payback expense'.
Please stop APK.. you're only hurting yourself.
I work for an ISP and we require rDNS records for all incoming mail. You will filter out a TON of spam email with that simple rule. It's much easier on the CPU load to filter on a simple reverse DNS check than to run spam assassin on that message. There are the occasional (not as many as you'd think) misconfigured servers that don't have rDNS. In those rare cases we contact the other end and let them know they're incorrectly setup, and usually add a temporary allow until they get the issue fixed.
I highly recommend requiring rDNS for incoming mail. 99.9% of legit mail servers will have those records, and only about 30% of spam servers will. We process over a million email messages a day with this method, it works.
My mail servers require PTR records and require that they match A records. This has stopped a huge amount of spam. It has also stopped some ham and as a result I now use this in combination with a whitelist.
My sig is wonderful. I love my sig.
Your post advocates a
(X) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
(X) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
(X) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
( ) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
(X) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
(X) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
( ) Sorry dude, but I don't think it would work.
(X) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!
You're going to read a lot of comments a long the lines of "Yes, because only a legitimate emailer will know {long list of obscure hoops you apparently have to jump through to send email, most of which are not documented in any RFC.}"
In reality, the answer is no. It's a fucking stupid idea, and the fact that there are a lot of fucking stupid idiots choosing things like this as a way to block spam doesn't justify it. In reality, it's something a spammer (generally a person who knows a little more about sending email than the average office network administrator) will know how to get around, but something your users and the people they want to communicate with will not.
Reject spam because it's spam, not because it doesn't comply to a made-up, easily circumvented, rule.
You are not alone. This is not normal. None of this is normal.
Sounds like a plan, care to explain that to the CTO? I tried several times and was told that it is a "zero payback expense"
Just out of curiosity, but did you submit a Business Impact Assessment/Risk Assessment detailing what the likelihood of a mail server failure would be and if it failed, how much money it would cost to re-establish services? You honestly outline what resources you have to bring to bear in solving an emergency and then plop that puppy right on his desk. If he then does a cost/benefit analysis and then wishes to ignore it, that's fine. You're covered. Of course, the GP said if a mail server is important enough, then... That's why these kinds of reports were invented; CBFs tie into DRPs which are subsets of BCPs.
Anyway, sounds like the CTO doesn't understand the proper role of support services for your company. IT is becoming more and more an essential feature of business (Critical Business Function) and though it may not be the prime income generator (web commerce), it still can be quite disastrous to the company if the mail server goes down.
"A government is a body of people usually -- notably -- ungoverned." -Shepherd Book
Use other things as well, like, SPF, whether the connecting machine is an MX for the domain, greylisting of various kinds (possibly), and weighting on lists like SBL and XBL. PBL will block hobbiests like me, fortunately gmail doesn't do that.
And then once you accept it you can do more thorough bayesian checks on headers and content.
In any case, absolute reliance on any of 'em is a problem. Spam Assassin is awesome at doing weightings, IMO.
-- perl -e'print pack"H*","6e656d6f406d38792e6f7267"'
Any ISP will setup rDNS entries if you have a business account
For certain subsets of "any", that is. I'm with two rather large national-scale ISPs, on business class accounts with both of them, and asking for rDNS makes their eyes glaze over and that's the end of that. Our previous ISP, also a large company, was just as inept...
A successful API design takes a mixture of software design and pedagogy.
It depends on how much money you want/expect the world to spend trying to revamp the entire outdated system, for starters.
The fact is, spam filtering these days is pretty doggone good. I know people complain about Google and privacy issues, but I pipe all my POP3 mail into my Gmail inbox, and I rarely ever get unwanted emails. But I get literally dozens of spams a day in the spam folder.
I also have my "catch all" account from a server I administer directed to a different Gmail account. And considering 99% of that mail is also spam, it catches hundreds a day, and passes through the very few random emails which were legitimately (and sometimes accidentally) addressed to someone at the domain. In fact, I've also witnessed emails come in from companies directed at people who are apparently engaged in fraudulent purchases. Receipts for hotels and airlines and such. It's interesting to see what shows up from time to time.
The only way you're going to really change the system is if you come up with an entirely new one which is inherently secure but also exciting and different enough to encourage people to switch. Trying to make people enable particular settings in cryptic configuration files of current software when there are countless sysadmins who barely know how to make email work to begin with is a waste of time and effort.
My ISP has a two-phase spam filter.
Phase 1 checks the domain from which the E-mail message was sent. If a DNS lookup indicates that no such domain has been registered, the message is trashed. This phase resulted in a very significant reduction in spam, both spam caught in Phase 2 and spam reaching my inbox.
Phase 2 uses a scored filtering system. It gives me the ability to add filters with my own scores. Any message caught in this phase is routed into a spam file on my ISP's server. I can review the messages in that spam file, marking some for acceptance into my inbox and some for deletion. In that review, I can also instruct the ISP's basic filter to "learn" what is really spam after a false negative or really not spam after a false positive.
Note well that I CAN REVIEW ANY MESSAGE THAT HAS BEEN MARKED AS SPAM in Phase 2. No matter how spam is detected, this is most important; the human element in determining spam is absolutely required. On the other hand, there is no human element in Phase 1, which does not perturb me. After all, I cannot reply to a message from a non-existent domain or or send a new message to it; I really do not want to receive any message that prevents an exchange of messages.
Use policyd-weight. It checks for DNS records and other stuff, and assigns a score to each test. I've been using it for a while and find it's a good compromise.
You do realize that your ISP doesn't have to also be your DNS provider? You can pay for any domain name registrar to do it for you, assuming that you have your own personal domain. You can even add your own DNS aliases. This still allows you to host your email server in-house but also have proper DNS entries. I use Dotster for my personal domains, but there are many out there.
...so it's not something valid to deny mail on, unless you don't mind not receiving certain mail (Which is fine, but don't count on the person sending it caring about it either).
Reverse DNS is messed up for my mail server, but that's because it's just too much effort to get the VPS provider to fix it (I don't have any control over it). People that reject mail based on things outside of RFC don't get my mail. Personally, I don't care, because, again, those people aren't worth the effort to satisfy. I can communicate with them by pen and paper (In other words, unless they're the government, they don't exist).
If the PTR request results in NXDOMAIN:
then add a X-Warning-no-RDNS header.
Customers are informed of this header. If they wish to make a client-side quarantine rule, they can. Customers are advised not to make rules to automatically delete such emails, as rDNS can get overloaded.
Also - if the rDNS resolves, and the answer is a KNOWN "dynamic or residential" rdns type name, then graylist that sender for 15 minutes. Most spam bots will not queue and retry their spam... they just move on and attack an easier mailserver. Note this is better than trying to use a "dynamic IP block list" because there isn't any, really (not anymore). Spamhaus PBL is not a dynamic IP list.
I use DynaStop http://dynastop.tanaya.net/ for this.
Note this is far better than maintaining your own IP blacklist. You're only graylisting IF the rDNS resolves to say 123-123-123-123-comcastbusiness.net for example. There are PLENTY of legitimate servers in domain space like that (albiet, IT running Exchange and not having a clue..). You would not want to block/refuse based on the rDNS. But the graylisting does thwart spam... in a very hands-off maintenance method. The sender can get through after 15 minutes OR after fixing their reverse DNS. (Once the sender gets through, their IP is considered green/OK for 4 days.. this eliminates the graylisting for frequent senders with bad rDNS)
I've been an Internet mail server administrator for over 20 years. About 10 years ago I experimented with a rule that would block all mail from any mail exchanger without a matching/existing reverse DNS record. I experienced the same problems that you are now experiencing. I tried to enhance awareness of this problem but didn't get very far. There are lots of small businesses (including mine) who run their own mail exchangers. Many administrators simply do not understand the technology enough, and they feel that they do not need to because it's working "good enough" already. You can either make a "white list" of the problematic servers, or just drop the rule as I did.
Filtering based on lack of rDNS is an old technique that actually did a good job of detecting spam without an excess of false positives for about a week in the late '90s. It has for some reason become enshrined as policy by a great many people now. These days it is occasionally a better indicator of NOTspam since the spammers all make sure they have rDNS set up and have done so since that week or so in the '90s.
Consider, if someone in a striped shirt wrote your business a bad check a decade ago, would you maintain a policy of not doing business with people who wear striped shirts?
I didn't read every comment, but the general theme seems to be that it's not absolutely required to have a reverse DNS entry. While this is true per the RFC - it's incredibly bad in practice. Google MX check, run the checker and if you don't have reverse DNS it'll point it out. Also people that say not one mail has bounced because of this must simply be wrong. Many blacklists will auto-add you if they notice you don't have reverse dns, then many companies will pick that up. Last time i had to move my companies mail server, the reverse was inadvertently not setup properly - not only did this cause problems fairly quickly it was slow to fix because while you'll be added to blacklists instantly, getting back off them is a manual process - you have to find every one you are on and then the companies that have picked up this info then have to get the new info - and some don't do this in a timely manner.
Running a real mail server for a real company without a correct functioning PTR record would be something that should get you fired.
The reasoning is simple, anyone running a real mail server will easily be able to set it up, if you don't have the PTR it likely means you are a spammer or you are running a server at home. Not that there's anything wrong with running your own SMTP server, but that's basically how botnets send spam...so there's a heavy correlation to that and spam.
Telnet smtp server on port 25. HELO pcname - MAIL FROM: address_you_want - RCPT TO: destination_email - DATA message .
There, someone just wrote a bot to spamm you 25 times per minute.
Spam originates from botnets and mailservers like gmail and hotmail. Both usually have correct, matching forward and reverse dns entrys. But you can filter the few retards that activly refuse to add reverse dns records, with a lot of collateral damage (people who don't know better or *can't* add them 'cause they don't control reverse dns for their IP...)
Glad you brought up the RFCs - isn't a proper rDNS entry for a server a requirement?
We aren't accept direct connection
why does your server talk chinglish?
10 ?"Hello World" life was simple then
Expecting rDNS is pretty common. Expecting PROPER rDNS, on the other hand, is another thing altogether.
If a machine doesn't have rDNS, then it can't send email to anyone at AOL, for instance. It'd be quite disingenuous to say that people who send email through a machine without rDNS would be surprised if they couldn't contact you.
On the other hand, there are too many ISPs who have rDNS, but broken rDNS (doesn't resolve in the forward direction, uses names which don't belong to them, et cetera). I block email from all connecting machines which have rDNS (or HELO/EHLO strings) which say yahoo.com, hotmail.com, gmail.com, or google.com, which cuts down on a LOT of spam. The real services always have blahblah.something.yahoo.com, for instance.
I also block HELO/EHLO names which don't resolve in DNS, and on my backup MX I also block when the HELO/EHLO doesn't resolve back to the connecting IP. This, IMHO, is much more effective than only rDNS checking. People don't always control their own rDNS, but they damned well better control whether their mail server is lying or not.
The bottom line is this: are you expecting email from just anyone? If so, you can't block it but you can increase its spam score. If you generally correspond with the same people and occasionally start corresponding with someone new, you could take the time when someone new has a broken mail server. This is what I've done for years (with HELO/EHLO) and most people thank me once I explain why it's in their best interest to fix it.
I run a small ISP and my email server is set to block illegitimate connections. Why should I allow an IP address with a PTR record of something like "generic_dumbass_user_IP123.233.domain.com" to freely connect to my email server and start spewing data? That's just stupid. Granted, a "fake" record can easily be created, but it is not unreasonable to setup or have your ISP setup a simple PTR record for your email server. It doesn't even have to match your domain's MX record. It just has to NOT be an obvious generic entry, or worse, no record at all. I understand the headaches when dealing with other email "admins" out there who may or may not know how to configure things, or may or may not care when you give them "friendly" advice on how things should be configured. Network admins are usually governed 90% by their ego, and nobody who is "king of his own little hill" appreciates being told by some stranger that his network is messed up. But there are "industry best practices" and RFCs to guide you. So IMHO, RDNS will not stop all SPAM, but it is simple enough to implement and does not create a lot of "false positives" when integrated with a comprehensive security profile.
Get Postini and be done with spam.
Well, that should be fairly simple now. Point to the circuit failure and explain that is what you're talking about.
Do it in writing (e-mail), so if someone above him screams at you, you can say "I expressed my concern but it was judged to be an acceptable risk by the powers-that-be."
I understand the problem you have. I've just resigned myself to the security mentality of "my job isn't make the decisions, it is to make sure the people who make the decisions have the best information". It means I get to say "I told you so" quite a bit.
Learning HOW to think is more important than learning WHAT to think.
No. Absolutely not.
This signature has Super Cow Powers
I wrote a script to check a bunch of popular domains (or any domain) and see if they have a reverse DNS entry.
http://www.perturb.org/code/rdns_check.pl
The summary is that every one (100%) of the domains I checked had reverse DNS.
digg.com has 7 MX records
** (Good) aspmx.l.google.com = 74.125.65.27 / gx-in-f27.1e100.net
** (Good) alt1.aspmx.l.google.com = 74.125.113.27 / vw-in-f27.1e100.net
** (Good) alt2.aspmx.l.google.com = 209.85.143.27 / dy-in-f27.1e100.net
** (Good) aspmx2.googlemail.com = 74.125.43.27 / bw-in-f27.1e100.net
** (Good) aspmx3.googlemail.com = 74.125.127.27 / pz-in-f27.1e100.net
** (Good) mail.digg.com = 74.125.127.121 / pz-in-f121.1e100.net
** (Good) diggstage01.digg.com = 64.191.203.34 / diggstage01.digg.com
nytimes.com has 4 MX records
** (Good) NYTIMES.COM.S7A1.PSMTP.com = 64.18.6.14 / s7a1.psmtp.com
** (Good) NYTIMES.COM.S7A2.PSMTP.com = 64.18.6.13 / s7a2.psmtp.com
** (Good) NYTIMES.COM.S7B1.PSMTP.com = 64.18.6.11 / s7b1.psmtp.com
** (Good) NYTIMES.COM.S7B2.PSMTP.com = 64.18.6.10 / s7b2.psmtp.com
gmail.com has 5 MX records
** (Good) gmail-smtp-in.l.google.com = 74.125.65.26 / gx-in-f26.1e100.net
** (Good) alt1.gmail-smtp-in.l.google.com = 74.125.113.26 / vw-in-f26.1e100.net
** (Good) alt2.gmail-smtp-in.l.google.com = 209.85.143.26 / dy-in-f26.1e100.net
** (Good) alt3.gmail-smtp-in.l.google.com = 209.85.229.26 / ww-in-f26.1e100.net
** (Good) alt4.gmail-smtp-in.l.google.com = 74.125.79.26 / ey-in-f26.1e100.net
hotmail.com has 4 MX records
** (Good) mx3.hotmail.com = 65.54.188.110 / bay0-mc3-f.bay0.hotmail.com
** (Good) mx4.hotmail.com = 65.55.92.168 / mx4.hotmail.com
** (Good) mx1.hotmail.com = 65.54.188.72 / bay0-mc1-f.bay0.hotmail.com
** (Good) mx2.hotmail.com = 65.55.37.72 / col0-mc1-f.col0.hotmail.com
yahoo.com has 3 MX records
** (Good) mta6.am0.yahoodns.net = 98.137.54.238 / mta-v2.mail.vip.sp2.yahoo.com
** (Good) mta7.am0.yahoodns.net = 74.6.136.244 / mta-v3.mail.vip.sk1.yahoo.com
** (Good) mta5.am0.yahoodns.net = 98.139.175.224 / mta-v1.mail.vip.bf1.yahoo.com
fastmail.fm has 2 MX records
** (Good) in1.smtp.messagingengine.com = 66.111.4.71 / mx2.messagingengine.com
** (Good) in2.smtp.messagingengine.com = 82.145.212.142 / tmx2.messagingengine.com
comcast.net has 2 MX records
** (Good) mx1.comcast.net = 76.96.62.116 / imta.westchester.pa.mail.comcast.net
** (Good) mx2.comcast.net = 76.96.30.116 / imta.emeryville.ca.mail.comcast.net
apple.com has 7 MX records
** (Good) mail-in13.apple.com = 17.254.13.11 / mail-in.apple.com
** (Good) mail-in14.apple.com = 17.254.13.13 / mail-in.apple.com
** (Good) mail-in11.apple.com = 17.254.13.7 / mail-in.apple.com
** (Good) mail-in12.apple.com = 17.254.13.10 / mail-in.apple.com
** (Good) mail-in2.apple.com = 17.254.13.5 / mail-in2.apple.com
** (Good) mail-in6.apple.com = 17.254.13.9 / mail-in6.apple.com
** (Good) mail-in3.apple.com = 17.254.13.8 / mail-in3.apple.com
google.com has 5 MX records
** (Good) aspmx.l.google.com = 74.125.65.27 / gx-in-f27.1e100.net
** (Good) alt1.aspmx.l.google.com = 74.125.113.27 / vw-in-f27.1e100.net
** (Good) alt2.aspmx.l.google.com = 209.85.143.27 / dy-in-f27.1e100.net
** (Good) alt3.aspmx.l.google.com = 209.85.229.26 / ww-in-f26.1e100.net
** (Good) alt4.aspmx.l.google.com = 74.125.79.26 / ey-in-f26.1e100.net
aol.com has 4 MX records
The spammers will find a way around it, no matter what you do. It isn't any better than filtering in terms of a real long-term solution.
And the reason why both suck and fail to bring a long-term solution is they are both reactionary measures to an economic problem. They are both rooted at least partially in the notion that spam is sent out to piss you off and waste your time, which could hardly be a less accurate analysis of the situation.
Spam is sent out to make money, period. It will continue to be sent as long as money can be made by sending it out.
Hence the solution is to disconnect the spammers from the people who pay them. Work has been done to do this already and it has been shown very effective because there is no incentive for spammers to send spam if they aren't getting paid for it. Anything else just encourages them to find ways to get around anti-spam measures and will inevitably make spam a more costly situation for everyone.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
I manage a large volume email installation (many mil+ daily). I block ips that do not have reverse DNS with little to no problem. It is a common requirement, and those mailing without reverse dns won't be able to mail a lot of people. This hasn't been a problem, but as a large local provider it may just be I'm in a strong position to bully people into fixing their dns.
I don't block those who have _broken_ reverse dns. That's common, even among some big sites. By that I mean their ip address resolves to a hostname, but that hostname does not resolve back to that ip. If you block based on this you will loose legitimate mail.
Make the world a better place for everyone else who rejects inbound email from IPs with no reverse DNS by doing the same. The only people who really have no way to provide reverse DNS for the IPs they're mailing from have no business running mail servers.
Glad you brought up the RFCs - isn't a proper rDNS entry for a server a requirement?
It's a "SHOULD", not a "MUST", as it's perfectly legal for an RFC-compliant email system to not have a hostname. Such a system would introduce itself (in its greeting and in HELO to other servers) as "[aaa.bbb.ccc.ddd]" with the actual IP address replacing the placeholders.
Likewise, it is a "MUST NOT" for rejecting based on not having reverse DNS.
The way I handle this is I greylist any email that comes in if:
1) it has no reverse dns
2) or the reverse dns matches certain patterns that indicate dynamic ip's
The rest comes in undelayed and is exposed to further checks. Note that it's ludicrous to require forward and reverse dns to resolve to the same thing. In today's age of hosting multiple domains in one server/cluster, it's just not reasonable to have PTR records defined that way.
I've been running my own mail server since my first 'domain' ended in .uucp (and also didn't have rDNS but whatever)... I likely have greater clue than most mail server admins who reject mail as spam for not having rDNS. I continue to run my own mail server and refuse to relay through my ISP's mail server. This way I can check my logs and know that my e-mail was accepted by a remote MX which is a useful debugging tool when someone says "I didn't get that e-mail"... ISP's tend to change their email policies at the drop of a hat and suddenly, without prior knowledge, your mails stop getting through because someone decided to disallow .tar.gz attachments, or whatever. My ability to add a PTR record to my subnets doesn't tell you anything about my ability to run a mail server nor does it say anything about my e-mail policies. I know lots of retard windows admins who know how to add a PTR record but I wouldn't trust them to reboot my son's computer, let alone run a reliable mail server. So if you're using the presence of rDNS to detect clue, then you are without clue. Given that PTR records are largely free-form, there is no valuable information in the content of a PTR record. Just because many sizable companies have jumped on the bandwagon, doesn't mean they're intelligent either. I only recently had to relent and add a PTR to my mail server IP because e-mails to my customer were bouncing due to Brightmail arbitrarily deciding that my site was a spam site due to the lack of rDNS. Since my customer is directly tied to my revenue and the smartypants admins at Brightmail won't accept a logic and fact based argument, I had no choice but to relent.
sigh. I hate people.
Like so many others I use multiple layers of anti-spam methods ..lazy admins..backscatter..virus hosts and all teh rest of teh crap excuses to spam me with ..of that 1 out of 10 is legit..but still mostly automated
lowest cost to higest , starting with
-- valid elho
-- to a valid user in my domain?
-- MX lookup, is this host the MX for the sending domain
-- does the RNDS match the MX
-- RBLs
-- body and header checks
-- a/v and ocr checks
Cuz I am sick of spam
and I dump about 1% into quar/teen that I need to check daily
crap..dinkdin..facialbook..titter..crap
Nowadays most of the spam comes from infected pcs, connecting to the internet with residential ISPs, and they all (well, at least 90%) have a rDNS for their customers IPs.
This article advocates a
(x) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(x) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
(x) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
(x) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(x) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
(x) Blacklists suck
(x) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(x) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!
While you have my deepest sympathies regarding your encounter with spamhaus, I would like to take the opportunity to suggest that you give freenet a try. The project is still making significant progress in terms of security, anonymity, and performance.
My approach, using Exim4, is not to reject messages outright based on single issues, such as not having a proper reverse DNS entry, but to reject based on combinations of them. This is a great way to limit false positives.
For instance, an incoming message may also have a bad HELO, a bad sender domain, be blacklisted locally or by a DNSBL service, or not have a working callout so that the existence of the sender's account can't be verified. There are more issues like these to look for. My systems count the number of these transgressions per message and reject when a certain value is reached, say three, while dumping messages that score one or two end in the recipient's spambox folder. With Exim, this kind of solution is surprisingly easy to construct using ACL statements with user-defined variables that include arithmetic statements. The last checks that are performed involve Clamd and SpamAssassin, because they are so resource-intensive.
I should also mention that my systems also perform a number of checks up front for obvious spam that is rejected immediately, e.g. if the sender address domain is gmail.com, but the sender HELO name is not part of the google.com domain.
it would be really hard to have one's own email server if reverse DNS is used to filter spam
I have a web hosting service that provides me a few Email addresses for my domain and the ability to server a web site. I do not have a dedicated IP for this. The result is my forward MX record does not match the reverse DNS (which lists my ISP as the owner). I have problems from time to time sending Email to clients because of spam filters that find such a situation nefarious. The truth is that the web hosting service has dozens of domains usign the same server and that it is not a unique situation.
You got it. Today, unless you have a dedicated Email server guy it's retarded to run an Exchange server in house.
I only agree on the point Exchange server, run exim or postfix for something stable and managable.
It is not really very hard to do, you need a bit of clue, but that is all. If no one at your company has clue then either outsource it or pay someone external to maintain it for you, neither should be expensive.
Oh: yes a rDNS record check is good, the problem is a numpty administrator who does not understand the issues and expects to force his brokenness onto the rest of the world. It should not be hard to whitelist that domain from the rDNS check.
It seems like an SPF record would serve a similar purpose, and not require your ISP to get involved.
Ok, so people can make lofty statements like there's no RFC requiring XYZ, or that it's hard to get PTR records from an ISP. But, here's some actual experience from my in-production system.
I increase an email's spam score to the threshold of what would be the cut-off for HAM when I receive an email from a system with no reverse record. This allows the content of the email to "prove" itself to SpamAssassin and my other tools while not rejecting the item out of hand. I can produce hard numbers by looking back in my SQL table for emails that triggered this rule that were marked as SPAM and ones that triggered the rule but were eventually marked as HAM.
Numbers from 2011-10-01 to 2011-10-13
Spam with RDNS: 6,350
Spam without RDNS: 11,899
Ham with RDNS: 14,609
HAM without RDNS: 486
TOTAL: 33,357
So, you can see from those numbers that not having an RDNS record may not be a direct indicator of SMAP, but it's pretty darn good. Naturally, every environment is going to see different criteria, so your best bet is to log what your SPAM system is doing and pour over that data for a month or two. I found that for my organization, it's extremely unlikely that a legitimate email would be routed through another country, so I automatically set email to the SpamAssassin threshold score like I do with missing RDNS.
With regard to the claim that ISP's will not create a reverse record, that's simply bunk. I've run IT groups for several companies in several locations and I've worked as a consultant/contractor for even more. The simple fact is that if you're using a business account, your ISP will have a process in place to create a PTR record for the IP address you've assigned to your email server, or the IP address being used for NAT/PAT. If you're too small to manage this on your own, don't have the knowledge to ask for a PTR record, or you're running on a consumer-level service, you should NOT be running your own email server. Of the small and unskilled organizations I've found doing this, 9 out of 10 of them had open relays or compromised email servers. Running a production service on the internet actually takes a level of knowledge, and not having it will cause you MUCH more harm than good.
Cisco ironports configured for strict checking want the HELO/EHLO to match the DNS name of the IP. They are reasonably intelligent about CNAMES and multiply-named hosts, though.
Ironport set to strict is the new standard, so get used to it.
While spammers certainly don't worry about conforming with RFCs, it should be noted that rejecting email based on the DNS is a specific violation of the relevant RFC, RFC-5321, section 4.1.4, which states:
An SMTP server MAY verify that the domain name argument in the EHLO command actually corresponds to the IP address of the client. However, if the verification fails, the server MUST NOT refuse to accept a message on that basis.
That said, it is common practice to do this anyway. The reality is that you will not have reliable email delivery unless the hostname your server issues in the EHLO command resolves to the servers IP address and the IP address resolves back to that same name.
Hey, who else could go for some flapjacks right now?
Speakeasy never had a problem setting my rdns records to whatever I asked them to when I was using them as an ISP with static IPs. Have you considered the possibility that your ISP sucks?
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Grey listing is much better.
- I've got bad karma because I won't parrot everyone else's opinion
I do this at a small business with a global customer base and have never (~5 years) had a single reported problem regarding rDNS.
Th reject rate is high (sorry don't have the stats here at home).
Another big spam killer is valid HELO hostnames.
Please use your internet provider SMTP Server
So what should a customer do if the only cable or DSL ISP in an area has an unreliable SMTP server?
Does "clueful" also mean "happens to be located in an area with at least one clueful business-class ISP"? Based on other comments posted to this story, it appears not all ISPs offering a static IP are clueful enough to make RDNS easy for customers to configure.
I worked with a postmaster at an ISP that blocked mail from anyone without rDNS, busted HELO hostnames and a whole host of other tiny things. We got virtually no spam at all. We also had thousands of people complaining that they couldn't receive e-mail from "grandma" in the Philippines because they had no rDNS. I think there are more effective ways of combatting spam while minimizing impact on end users.
I use it for years now and if a sender has a problem, phone in with error code, and I can add their server to our whitelist. (After trying to convince their admin to use it)
Best spam solution IMHO.
Rather than pay for a relay host, why not move your business to an ISP who will provide what you want?
If you want your email server to actual deliver mail, rDNS is required as many major email providers like AOL and MSN/Hotmail now require them to even accept incoming connections. It doesn't matter what anyone thinks about it, it's already a standard because of this.
I would not configure any servers to reject mail outright originating from servers without rDNS though. As the OP notes there are some legit servers whose admins apparently did not get the memo or do not care about delivery rates (if your ISP will not allow valid rDNS switch hosts or sign up for a free Google Apps for your domain, your mail deliver rates will be absurdly low without rDNS). Like someone else suggested a SpamAssassin/SCL rule to put it in a Junk folder instead would allow the one off legit message to be retrieved.
Although still imperfect, DNSBLs like Spamhaus and Barracuda seem to be a safer method to block spammy servers outright.
Agreed, agreed. We support mail servers for many companies on, idk, maybe a dozen ISPs and a few data centers. Bottom line, you have to be persistent. Someone (probably with DNS admin in their title) can do it if you ask nicely :)
Chance favors the prepared mind.
Perfect is the enemy of good.
1. Comcast, AOL, and a lot of other providers are rejecting mail for a non-existent PTR record. If you're requiring the existence of a PTR record, you aren't the only one. But don't confuse whether they should have a PTR with whether they are required to have one. This is fodder for spam filtering arguments going way back before me (I was a mail admin back in 2002-2004, and this was a recurring flame war then too).
2. The existence of a PTR record does not always correlate to a forward A record (ie, if host.example.com has the IP address 1.2.3.4, and 4.3.2.1.in-addr.arpa has the host as 4-3-2-1.some-isp.net, this is perfectly legit and very common. 4.3.2.1.in-addr.arpa does NOT have to point back to host.example.com, though it's still a good idea).
3. Any time you filter for spam based on any given criteria, you will encounter false positives. Period. Are there few enough false positives to be worth dealing with? Then configure mail server to accept mail from legitimate (but problematic) hosts on your whitelist before you blocking based on their DNS. Are there so many false positives that you are dealing with user complaints and exempting mail servers more often than getting real work done? Then drop that particular filter and try others.