Slashdot Mirror
Ask Slashdot: Is Reverse DNS a Worthy Standard For Fighting Spam?
drmartin66 makes it to the front page with this question: "Last weekend I installed a new spam filter server for a client, and enabled connection rejection if the sending server did not have a Reverse DNS record. Since then, I have had a number of emails rejected from regulator bodies that do not have a Reverse DNS record, and are refusing to have one created for their email server. What is your opinion of Reverse DNS records? Are they (or should they be) a standard, and required? Or are they useless for spam fighting?"
Like all things spam, marking the message as bad automatically is generally discouraged. If you simply increase the SCL value by some reasonable number, and continue to raise SCL based on other soft violations (like spamhaus, surbl, etc), you will rarely put good senders in the junk email folder, and very frequently be able to reject most spam content.
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Yes, it may cut down on the "infected" PC spam that hits a mail server, but all it will do is drive that traffic to the numerous spam farms that are opening operating on ISPs
Yes. Absolutely. It works.
In all but the most closed groups, having a system that generates lots of false positives is in most cases going to be a bad move in my opinion.
What reason would anyone have to be running an SMTP server without a PTR record?
We (~10K users, ~1.5M incoming emails a month) used to do this, but there was a constant stream of complaints from users due to email bouncing from organisations that refused to set up the records. It got worse every year too. So we gave up and went for a commercial software solution for a while, and eventually outsourced our mailfiltering completely to a hosted solution. I would NEVER do filtering in-house again, unless for a very small number of technically literate users.
Bob.
As with most spam fighting metrics it's up to you. Mail from a server without reverse DNS that doesn't trigger any of your other flags generally shouldn't be treated as spam if you care about false positives, if it's borderline then maybe the lack of reverse DNS will be enough to justify tagging it as spam. The decision of how heavily to weight the lack of reverse DNS is yours, personally I don't give it much weight but it does add a little to the score. Some people go hardcore and reject anything that doesn't have come from a machine with reverse DNS, they accept the significant false positive rate usually for idealogical reasons (while I like a properly configured system I'm not going to bite my nose off to spite my face).
Use multiple checks. Use scoring, and a threshold. voila.
Buanzo Consulting - 15 Years of GNU/Linux experience, for you.
I've set up a few mail relay and spam filtering server and I can tell you it helps a lot to reduce the number of spam arriving to them.
I am a Postfix kind of admin (hell with sendmail!) and I know you can set some filter just before the reverse DNS check to accept the connection if it comes from a particular host/IP address, bypassing the reverse DNS check. Or you could add that reverse DNS to your local/client DNS server but it seems not that a good solution.
Anyway following standards is always the best solution.
--- Bouh !!! ---
If your email server does not have rDNS records then it's very likely half your mail is not getting delivered. aol.com, gmail, hotmail, etc all require rDNS.
Blocking on invalid rDNS, invalid or missing A records and not following proper smtp protocol is helpful on a email gateway. However, if you are a relay for clients you'll have problems.
Denying all wrong rDNS is a bit harsh, however, denying what's a DSL and the rDNS declaring as such is a good idea. I've yet found very very few cases where it was an issue. For such case, just white-list the entry your customer is complaining about. Here's how to do with postfix:
/etc/postfix/maps/relaying_stoplist
/^dsl.*\..*/i 553 AUTO_DSL We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server. -dsl-
/[a|x]dsl.*\..*\..*/i 553 AUTO_[A|X]DSL We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/cable.*\..*\..*/i 553 AUTO_CABLE We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/.*dial(\.|-).*\..*\..*/i 553 AUTO_DIAL We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/ppp.*\..*/i 553 AUTO_PPP We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/dslam.*\..*\..*/i 553 AUTO_DSLAM We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/dslb.*\..*\..*/i 553 AUTO_DSLB We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/dynamicIP.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/fbx.*\..*\..*/i 553 AUTO_fbx We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/abo.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/socal.res.*\..*\..*/i 553 AUTO_REV We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
/.dhcp.*\..*\..*/i 553 AUTO_dhcp We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
smtpd_client_restrictions = permit_mynetworks,
permit_sasl_authenticated,
check_client_access regexp:/etc/postfix/maps/relaying_stoplist,
permit
And here's the content of my
#/.*\.dsl\..*/i 553 AUTO_DSL2 We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/client.*\..*\..*/i 553 AUTO_CLIENT We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/pool\..*/i 553 AUTO_POOL We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/dynamic.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/staticIP.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
#/dip.*\..*\..*/i 553 AUTO_ABO We aren't accept direct connection not from dedicated SMTP servers. Please use your internet provider SMTP Server.
It is possible to configure your mailer to require all sorts of things, like rDNS. If you configure all of them you will get almost no spam, but you'll also not get 50% of your legitimate e-mail. Perhaps that's ok with you, you're willing to only talk to the "clueful".
Most people though want to get mail. The old Internet axiom "Be conservative with what you send, be liberal with what you accept" applies. WIth spam fighting this generally means use every mechanism at your disposal (including rDNS existence, or matching with forward DNS); but use it only to affect the score of a message. That way the guy who doesn't have rDNS right, but does everything else right will still get through.
In an organization operating a mailserver without a PTR record for their SMTP, the users should be having so much difficulty sending outbound mail that they know something is wrong. I know this from experience, having set up an SMTP without reverse DNS, and then observing that half my test messages bounced back.
Many email servers do not have rDNS, therefore it is not advisable to filter based on a lack of rDNS alone.
It can be argued that it should have an rDNS, but if they don't, you have no control over that since it's their system. Then you'll be spending way too much of your time tweaking spam filters and creating white lists, contacting the sending company's administrators...
It's just a bad idea, don't do it.
That said, I prefer SaaS email spam filtering like Symantec's Messagelabs. (Disclosure, I am an ML partner)). I like this service because I don't have to worry about managing it. It saves me a lot of time.
A bit over a year ago, I performed a reverse dns scan of the entire internet. It took around 4 months and amounted to about 62GB of data which I haven't sorted through just yet, but I'd guess based on what I've seen so far that only 20-30% of the utilized Internet has reverse DNS entries. This is kinda what I suspected all along, but I now have data to back that up. How can anything properly use reverse DNS with that low of an adoption rate. Its sad because so many more services could be offered if it was properly adopted.
You got it. Today, unless you have a dedicated Email server guy it's retarded to run an Exchange server in house. Outside companies do it better and when the Company Fiber goes down YET AGAIN (Thanks AT&T) you dont lose 24 hours of emails, we actually lost 72 hours worth as AT&T decided that working over the weekend was not important.
Do not look at laser with remaining good eye.
You know....I hate spam. It made usenet useless for years, it continues to degrade the usefulness of email, spamers steal resources and are underhanded dickwads.
All that said, some of the anti-spam people are ridiculous zealots who don't care who gets caught in the crossfire.
I have a server in colo. Its my mail server, but it also does a number of other things. Until recently, it ran a tor node. Why? Because i had sooo much more allocated bandwidth than I was using on a monthly basis that it cost me nothing extra to run. Ran it for at least 6 years on the same node.
Its now shut off, why? Because some idiots at Spamhaus decided that running a tor server was suspect. Never mind that it was disallowed from exiting on port 25, which is publically posted info in its service descriptor....no... Of course, I think they are also fooled by the fact that several windows users have shell accounts and use it as a web proxy.... so somehow my box also was infected with a Windows trojan according to these geniuses.
We got it cleared up, but still are not able to donate excess bandwidth allowance to the tor network.... which is bad enough, but this isn't the first time I have had my server blacklisted for no good reason at all. I don't even remember what BS it was last time, just that it was... BS.
Now will this kill me? No.... I have reverse DNS setup and have for years but...come on.... seriously? Bouncing mail sucks, especially when you suddenly start doing it to whole domains.
If it were just me, my opinion is that anyone using one of these RBLs has a misconfigured mail server, I wouldn't have "fixed it".... but I host other peoeple's email domains, so the black ball tactics worked.
"I opened my eyes, and everything went dark again"
Being fed up with postfix and exim, I recently wrote a simple e-mail server using python. I followed the RFC standard as well as I could, but to my surprise, I noticed there are numerous special undocumented tricks one needs to know to get mail through to the recipient in a reliable way (whitelists, blacklists, reverse dns, etc). I am wondering if anybody here knows if there is a place on the net where such tricks are documented.
PS: IANAS (I am not a spammer, honestly)
If Pandora's box is destined to be opened, *I* want to be the one to open it.
Just whitelist any email that has a verified digital signature. Everything else you can't trust.
Good luck getting anyone to actually set up and use digital signatures/encryption, though :-P But if you make it a matter of policy and give them the tools to use it, there's no better way.
As somebody who used to work in the email industry, I can assure you that rejecting based on the presence of RDNS is useless against entities that run their own infrastructure. Everybody working legitimately ('opt-in', though I refer to it as 'suckered in' because the person didn't read the privacy policy that said people who filled out the form were going to get emailed and their info sold/traded) in the email industry creates RDNS as a part of their SOP. When I worked in it, we never sent out email without first verifying that reverse DNS was set up and set up properly. Granted, you'll still likely catch spam from botnets.
Remember that not every non-spam email originates from a perfectly-configured self-hosted SMTP server. Many organizations outsource their email, spam filtering, compliance filtering, notice / statement delivery, etc. While it's easy to posit that the IT departments in such organizations have a duty to maintain reverse DNS records for all their partners' servers, don't fall into the trap of thinking that every organization has a fully-staffed, knowledgeable IT department... or an IT department at all.
Having a reverse DNS is a good practice, and anyone with a mail server should be doing it. That said, a lot of small businesses don't have reverse DNS set up, don't know what you mean when you tell them to do it, or have ISPs that are a pain to deal with. I'd mark up the spam score on a message without reverse DNS on the sending server (and I do on my own server) but I wouldn't block it entirely unless it sets off a lot more flags than just that one.
I use Kerio Connect on my server - I add 2 points for lack of reverse DNS. 3.5 points drops you into the junk folder, 5 blocks you completely. Doing that I get pretty much no false blocks, a false positive every few days, and about 3-5 spams that make it to the junk folder per day. I block a few hundred.
-- Josh Turiel
"2. Do not eat iPod Shuffle."
The windows admin where I work hates reverse DNS. He thinks it's stupid and refuses to keep it updated. So... if there are more people like him out there, you might have issues getting email from them.
Making fun of dumb people since 2009
For short term outages, sending servers will queue messages and try again later. You can avoid long term outages like this one by having redundant Internet connections from different providers.
Uhh...not to nitpick, but that is what backup MX servers are for. When your primary server is not available, mail is delivered to one of the others. If your e-mail is that critical then you need to have a store-and-forward server somewhere else, just in case your link goes down.
There are lots of services that provide this, if you don't want to do it yourself. But setting up a simple store-and-forward server isn't all that complicated and doesn't need a full Exchange deployment.
Learning HOW to think is more important than learning WHAT to think.
Reverse DNS makes absolute sense as a means to counter spam, but the reality I've experienced is that there are too many IT admins that don't even know what a PTR record is, and since they don't experience an issue with all of their mail recipients, they assume it's an unnecessary step, even when you try to explain that it with a sock puppet show.
The requirement for reverse dns is in hindsight a part of the "security theater" where various claims are made, and remedies suggested against perceived ills. The suggestion for reverse DNS comes solidly from the era of TCP wrappers, another supposed saviour of ill maintained systems from outside evils.
In reality, there is no actual increase of security from checking if some address has reverse dns as for ages most of the dial up and broadband lines all have reverse dns. Also, as reverse dns zones are by and large often unmaintained, esp. when it comes to removing entries, you neither can rely on the data returned, nor assign any significance to what is returned.
Sounds like a plan, care to explain that to the CTO? I tried several times and was told that it is a "zero payback expense"
Do not look at laser with remaining good eye.
Unless you are one of the lucky few who have a full class address space, you are stuck with the will of the ISP to either setup reverse entries for you or to delegate resolution to you. Alphatel has it right. Use it if you choose, and grade along with other tests.
Any ISP will setup rDNS entries if you have a business account. The only time this is an issue is if you are trying to run a server from a home account. Most of the ISPs I have used or looked into prohibited running servers with a consumer account. Those that didn't were also happy to provide a rDNS entry if you paid for a static IP.
Trying to run a server when your ISP is opposed to you doing so is inherently problematic, rDNS just being one of your many concerns. It is fine for experimenting and learning, but not for servers that do anything important. The only excuse for a business or government agency not having a rDNS entry is incompetence.
The real question is how tolerant should you be of the incompetent, and from a business point of view the unfortunate answer is "very tolerant".
Yes, reverse DNS records should be standard. No, they should not be required. Yes, they are useless as a toll for fighting spam.
"Convictions are more dangerous enemies of truth than lies."
Besides, since when did every MTA stop following the SMTP RFC that says you MUST not discard the message until delivery has failed for over 72 hours? Even if your server was down over a weekend, you should still have 24 hours to get it back up without losing any mail.
Gamingmuseum.com: Give your 3D accelerator a rest.
The real problem is that while this would really help fight spam, there's collateral damage. Just like the judicial systems in civilized countries tend to operate on the principle that it's better to set 100 guilty people free rather than imprison 1 innocent person, most people who receive email would rather receive and delete 100 spam messages than miss one legit email inquiry from a potential customer or long-lost friend.
Sender Policy Framework seems even better than simple reverse DNS in theory, but it doesn't seem to get much traction because it causes more serious problems than spam in general causes. Until a critical mass of sysadmins basically tell the domain owners who are too stupid or lazy to add the appropriate DNS records to fuck off, lazy and stupid sysadmins will continue to not add those records. But until then, customers will cry and rebel if any of the good sysadmins who host them try to apply a passive spam filter that relies on such records. That's just how it goes, it's a Catch-22 which is preventing widespread adoption. The only potential solution would be to stage an SPF or rDNS record adoption day, get some big names on board, and hope for the best.
This is simple stuff here. Firstly, it verifies ownership of the domain. I will never accept email from a host that does not resolve. Doing so will of course allow a ridiculous amount of spam from infected computers around the globe from regular IP addresses. The email address needs to match the host in which it is sending from as well. It requires hardly *any* work. Why are we even talking about this?
HELO, ELO, wants the hostname as well. Are we expecting millions of mail servers to simple change the way they're doing things? More importantly, if spam is a problem for you, I've had great luck with filtering services such as Postini and SpamSoap. Both excellent providers and have better resources than anything you can whip together on your little email server. It costs pennies as well.
*plays the Apogee theme song music*
If email is important to your organization then the cost of a correctly configured mail server is insignificant.
Seriously, your email server can be anywhere in the world. There's no reason that you have to go through a specific ISP. Even if they're blocking port 25, you can get a different ISP to accept mail from you on a different port. Even Google offers that option.
I've had this same fight with multiple vendors and organizations over the years. Like many others, rDNS should be implement properly. It's just the way to keep a clean house. However, the RFC doesn't require rDNS validation checking, and do to so break mail delivery for many, many legitimate services. Spammers will (actually, have--this idea is old) found new avenues of attack. Reputation scoring, token analysis, and other statistical measures are a far better set of solutions to work with. You can go on the cheap with some sendmail RBL and dspam type solutions all the way up to what I consider best in class right now--Ironport. A Microsoft shop would probably like FOPE--it integrates well and does a damn nice job of proper spam filtering too. The larger MS shops with EA agreements can roll than into the package for decent pricing.
The question "Is Reverse DNS a Worthy Standard For Fighting Spam?" is incorrect. Spam is not the reason; using it as a measure of clue is. Servers that emit spam and and clue level can be related, though. If someone is clueful enough to set up a mail server properly they're going to make sure it has reverse DNS. A mail server run by a less than clueful individual (or set-and-forget with no admin) is more likely to be a problem source either now or in the future than the ones that are cluefully configured and actively maintained.
Of course you are going to have spammers that are clueful mail admins and will set up their servers properly. That's why you can't pigeonhole reverse DNS as some kind of spam fighting method alone. But it can always be used as a measure of cluefulness.
this is my sig
The only truly effective spam prevention technique I've seen is greylisting.
Unselfish actions pay back better
Sounds like a good reason to go job hunting before getting fired for his incompetence.
Its right, its not fair; but its needed. Legitimate sites should have no problems setting up reverse records or getting their provider to do if for them.
Anyone who is not in a position get PTR records in place for their mail server is not actually in a position to be running a mail server anyway. Sorry that is just the way it is. PTR records are nice to have for any number of mail delivery troubleshooting and validation issues outside of SPAM.
As a mail admin I'd kinda consider them a requirement anyway. Its not easy to work transmission problems when I can't figure out who the admin of the other server is and how to get in touch with them.
I know its not within the standards, but I say no PTR record no, mail accepted.
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
Simple. Ask him if any of the email sent or received has value to the company. If the answer is "Yes", ask if he insures his car/home/dog. When he says yes point out the backup MX cost is insurance against the value of the email. If he says "No", point down you can save more money by closing down all email. Then go back to the first question, or close it down and get out of there.
Real men would scan the IPv6 space too.... :)
-Charlie
We pay our ISP to provide 'queue service' in the event our connection/Exchange box goes down. When 'selling' this service to the bosses I put it this way - How much will it cost the firm to not only be without email during an outage, but to lose all emails sent to the firm during that time period?
I feel your frustration though... IT is often seen as a 'zero payback expense'.
Please stop APK.. you're only hurting yourself.
I work for an ISP and we require rDNS records for all incoming mail. You will filter out a TON of spam email with that simple rule. It's much easier on the CPU load to filter on a simple reverse DNS check than to run spam assassin on that message. There are the occasional (not as many as you'd think) misconfigured servers that don't have rDNS. In those rare cases we contact the other end and let them know they're incorrectly setup, and usually add a temporary allow until they get the issue fixed.
I highly recommend requiring rDNS for incoming mail. 99.9% of legit mail servers will have those records, and only about 30% of spam servers will. We process over a million email messages a day with this method, it works.
My mail servers require PTR records and require that they match A records. This has stopped a huge amount of spam. It has also stopped some ham and as a result I now use this in combination with a whitelist.
My sig is wonderful. I love my sig.
You're going to read a lot of comments a long the lines of "Yes, because only a legitimate emailer will know {long list of obscure hoops you apparently have to jump through to send email, most of which are not documented in any RFC.}"
In reality, the answer is no. It's a fucking stupid idea, and the fact that there are a lot of fucking stupid idiots choosing things like this as a way to block spam doesn't justify it. In reality, it's something a spammer (generally a person who knows a little more about sending email than the average office network administrator) will know how to get around, but something your users and the people they want to communicate with will not.
Reject spam because it's spam, not because it doesn't comply to a made-up, easily circumvented, rule.
You are not alone. This is not normal. None of this is normal.
Sounds like a plan, care to explain that to the CTO? I tried several times and was told that it is a "zero payback expense"
Just out of curiosity, but did you submit a Business Impact Assessment/Risk Assessment detailing what the likelihood of a mail server failure would be and if it failed, how much money it would cost to re-establish services? You honestly outline what resources you have to bring to bear in solving an emergency and then plop that puppy right on his desk. If he then does a cost/benefit analysis and then wishes to ignore it, that's fine. You're covered. Of course, the GP said if a mail server is important enough, then... That's why these kinds of reports were invented; CBFs tie into DRPs which are subsets of BCPs.
Anyway, sounds like the CTO doesn't understand the proper role of support services for your company. IT is becoming more and more an essential feature of business (Critical Business Function) and though it may not be the prime income generator (web commerce), it still can be quite disastrous to the company if the mail server goes down.
"A government is a body of people usually -- notably -- ungoverned." -Shepherd Book
Use other things as well, like, SPF, whether the connecting machine is an MX for the domain, greylisting of various kinds (possibly), and weighting on lists like SBL and XBL. PBL will block hobbiests like me, fortunately gmail doesn't do that.
And then once you accept it you can do more thorough bayesian checks on headers and content.
In any case, absolute reliance on any of 'em is a problem. Spam Assassin is awesome at doing weightings, IMO.
-- perl -e'print pack"H*","6e656d6f406d38792e6f7267"'
Any ISP will setup rDNS entries if you have a business account
For certain subsets of "any", that is. I'm with two rather large national-scale ISPs, on business class accounts with both of them, and asking for rDNS makes their eyes glaze over and that's the end of that. Our previous ISP, also a large company, was just as inept...
A successful API design takes a mixture of software design and pedagogy.
It depends on how much money you want/expect the world to spend trying to revamp the entire outdated system, for starters.
The fact is, spam filtering these days is pretty doggone good. I know people complain about Google and privacy issues, but I pipe all my POP3 mail into my Gmail inbox, and I rarely ever get unwanted emails. But I get literally dozens of spams a day in the spam folder.
I also have my "catch all" account from a server I administer directed to a different Gmail account. And considering 99% of that mail is also spam, it catches hundreds a day, and passes through the very few random emails which were legitimately (and sometimes accidentally) addressed to someone at the domain. In fact, I've also witnessed emails come in from companies directed at people who are apparently engaged in fraudulent purchases. Receipts for hotels and airlines and such. It's interesting to see what shows up from time to time.
The only way you're going to really change the system is if you come up with an entirely new one which is inherently secure but also exciting and different enough to encourage people to switch. Trying to make people enable particular settings in cryptic configuration files of current software when there are countless sysadmins who barely know how to make email work to begin with is a waste of time and effort.
They can only use reverse DNS for IP's they own if they're using their own mail servers.
My ISP has a two-phase spam filter.
Phase 1 checks the domain from which the E-mail message was sent. If a DNS lookup indicates that no such domain has been registered, the message is trashed. This phase resulted in a very significant reduction in spam, both spam caught in Phase 2 and spam reaching my inbox.
Phase 2 uses a scored filtering system. It gives me the ability to add filters with my own scores. Any message caught in this phase is routed into a spam file on my ISP's server. I can review the messages in that spam file, marking some for acceptance into my inbox and some for deletion. In that review, I can also instruct the ISP's basic filter to "learn" what is really spam after a false negative or really not spam after a false positive.
Note well that I CAN REVIEW ANY MESSAGE THAT HAS BEEN MARKED AS SPAM in Phase 2. No matter how spam is detected, this is most important; the human element in determining spam is absolutely required. On the other hand, there is no human element in Phase 1, which does not perturb me. After all, I cannot reply to a message from a non-existent domain or or send a new message to it; I really do not want to receive any message that prevents an exchange of messages.
This checklist is longer than when I last saw it. Is there a centrally-managed spot for the meme?
Use policyd-weight. It checks for DNS records and other stuff, and assigns a score to each test. I've been using it for a while and find it's a good compromise.
If the PTR request results in NXDOMAIN:
then add a X-Warning-no-RDNS header.
Customers are informed of this header. If they wish to make a client-side quarantine rule, they can. Customers are advised not to make rules to automatically delete such emails, as rDNS can get overloaded.
Also - if the rDNS resolves, and the answer is a KNOWN "dynamic or residential" rdns type name, then graylist that sender for 15 minutes. Most spam bots will not queue and retry their spam... they just move on and attack an easier mailserver. Note this is better than trying to use a "dynamic IP block list" because there isn't any, really (not anymore). Spamhaus PBL is not a dynamic IP list.
I use DynaStop http://dynastop.tanaya.net/ for this.
Note this is far better than maintaining your own IP blacklist. You're only graylisting IF the rDNS resolves to say 123-123-123-123-comcastbusiness.net for example. There are PLENTY of legitimate servers in domain space like that (albiet, IT running Exchange and not having a clue..). You would not want to block/refuse based on the rDNS. But the graylisting does thwart spam... in a very hands-off maintenance method. The sender can get through after 15 minutes OR after fixing their reverse DNS. (Once the sender gets through, their IP is considered green/OK for 4 days.. this eliminates the graylisting for frequent senders with bad rDNS)
I've been an Internet mail server administrator for over 20 years. About 10 years ago I experimented with a rule that would block all mail from any mail exchanger without a matching/existing reverse DNS record. I experienced the same problems that you are now experiencing. I tried to enhance awareness of this problem but didn't get very far. There are lots of small businesses (including mine) who run their own mail exchangers. Many administrators simply do not understand the technology enough, and they feel that they do not need to because it's working "good enough" already. You can either make a "white list" of the problematic servers, or just drop the rule as I did.
Filtering based on lack of rDNS is an old technique that actually did a good job of detecting spam without an excess of false positives for about a week in the late '90s. It has for some reason become enshrined as policy by a great many people now. These days it is occasionally a better indicator of NOTspam since the spammers all make sure they have rDNS set up and have done so since that week or so in the '90s.
Consider, if someone in a striped shirt wrote your business a bad check a decade ago, would you maintain a policy of not doing business with people who wear striped shirts?
I didn't read every comment, but the general theme seems to be that it's not absolutely required to have a reverse DNS entry. While this is true per the RFC - it's incredibly bad in practice. Google MX check, run the checker and if you don't have reverse DNS it'll point it out. Also people that say not one mail has bounced because of this must simply be wrong. Many blacklists will auto-add you if they notice you don't have reverse dns, then many companies will pick that up. Last time i had to move my companies mail server, the reverse was inadvertently not setup properly - not only did this cause problems fairly quickly it was slow to fix because while you'll be added to blacklists instantly, getting back off them is a manual process - you have to find every one you are on and then the companies that have picked up this info then have to get the new info - and some don't do this in a timely manner.
Running a real mail server for a real company without a correct functioning PTR record would be something that should get you fired.
The reasoning is simple, anyone running a real mail server will easily be able to set it up, if you don't have the PTR it likely means you are a spammer or you are running a server at home. Not that there's anything wrong with running your own SMTP server, but that's basically how botnets send spam...so there's a heavy correlation to that and spam.
Telnet smtp server on port 25. HELO pcname - MAIL FROM: address_you_want - RCPT TO: destination_email - DATA message .
There, someone just wrote a bot to spamm you 25 times per minute.
Glad you brought up the RFCs - isn't a proper rDNS entry for a server a requirement?
We aren't accept direct connection
why does your server talk chinglish?
10 ?"Hello World" life was simple then
Expecting rDNS is pretty common. Expecting PROPER rDNS, on the other hand, is another thing altogether.
If a machine doesn't have rDNS, then it can't send email to anyone at AOL, for instance. It'd be quite disingenuous to say that people who send email through a machine without rDNS would be surprised if they couldn't contact you.
On the other hand, there are too many ISPs who have rDNS, but broken rDNS (doesn't resolve in the forward direction, uses names which don't belong to them, et cetera). I block email from all connecting machines which have rDNS (or HELO/EHLO strings) which say yahoo.com, hotmail.com, gmail.com, or google.com, which cuts down on a LOT of spam. The real services always have blahblah.something.yahoo.com, for instance.
I also block HELO/EHLO names which don't resolve in DNS, and on my backup MX I also block when the HELO/EHLO doesn't resolve back to the connecting IP. This, IMHO, is much more effective than only rDNS checking. People don't always control their own rDNS, but they damned well better control whether their mail server is lying or not.
The bottom line is this: are you expecting email from just anyone? If so, you can't block it but you can increase its spam score. If you generally correspond with the same people and occasionally start corresponding with someone new, you could take the time when someone new has a broken mail server. This is what I've done for years (with HELO/EHLO) and most people thank me once I explain why it's in their best interest to fix it.
I run a small ISP and my email server is set to block illegitimate connections. Why should I allow an IP address with a PTR record of something like "generic_dumbass_user_IP123.233.domain.com" to freely connect to my email server and start spewing data? That's just stupid. Granted, a "fake" record can easily be created, but it is not unreasonable to setup or have your ISP setup a simple PTR record for your email server. It doesn't even have to match your domain's MX record. It just has to NOT be an obvious generic entry, or worse, no record at all. I understand the headaches when dealing with other email "admins" out there who may or may not know how to configure things, or may or may not care when you give them "friendly" advice on how things should be configured. Network admins are usually governed 90% by their ego, and nobody who is "king of his own little hill" appreciates being told by some stranger that his network is messed up. But there are "industry best practices" and RFCs to guide you. So IMHO, RDNS will not stop all SPAM, but it is simple enough to implement and does not create a lot of "false positives" when integrated with a comprehensive security profile.
Well, that should be fairly simple now. Point to the circuit failure and explain that is what you're talking about.
Do it in writing (e-mail), so if someone above him screams at you, you can say "I expressed my concern but it was judged to be an acceptable risk by the powers-that-be."
I understand the problem you have. I've just resigned myself to the security mentality of "my job isn't make the decisions, it is to make sure the people who make the decisions have the best information". It means I get to say "I told you so" quite a bit.
Learning HOW to think is more important than learning WHAT to think.
No. Absolutely not.
This signature has Super Cow Powers
I wrote a script to check a bunch of popular domains (or any domain) and see if they have a reverse DNS entry.
http://www.perturb.org/code/rdns_check.pl
The summary is that every one (100%) of the domains I checked had reverse DNS.
digg.com has 7 MX records
** (Good) aspmx.l.google.com = 74.125.65.27 / gx-in-f27.1e100.net
** (Good) alt1.aspmx.l.google.com = 74.125.113.27 / vw-in-f27.1e100.net
** (Good) alt2.aspmx.l.google.com = 209.85.143.27 / dy-in-f27.1e100.net
** (Good) aspmx2.googlemail.com = 74.125.43.27 / bw-in-f27.1e100.net
** (Good) aspmx3.googlemail.com = 74.125.127.27 / pz-in-f27.1e100.net
** (Good) mail.digg.com = 74.125.127.121 / pz-in-f121.1e100.net
** (Good) diggstage01.digg.com = 64.191.203.34 / diggstage01.digg.com
nytimes.com has 4 MX records
** (Good) NYTIMES.COM.S7A1.PSMTP.com = 64.18.6.14 / s7a1.psmtp.com
** (Good) NYTIMES.COM.S7A2.PSMTP.com = 64.18.6.13 / s7a2.psmtp.com
** (Good) NYTIMES.COM.S7B1.PSMTP.com = 64.18.6.11 / s7b1.psmtp.com
** (Good) NYTIMES.COM.S7B2.PSMTP.com = 64.18.6.10 / s7b2.psmtp.com
gmail.com has 5 MX records
** (Good) gmail-smtp-in.l.google.com = 74.125.65.26 / gx-in-f26.1e100.net
** (Good) alt1.gmail-smtp-in.l.google.com = 74.125.113.26 / vw-in-f26.1e100.net
** (Good) alt2.gmail-smtp-in.l.google.com = 209.85.143.26 / dy-in-f26.1e100.net
** (Good) alt3.gmail-smtp-in.l.google.com = 209.85.229.26 / ww-in-f26.1e100.net
** (Good) alt4.gmail-smtp-in.l.google.com = 74.125.79.26 / ey-in-f26.1e100.net
hotmail.com has 4 MX records
** (Good) mx3.hotmail.com = 65.54.188.110 / bay0-mc3-f.bay0.hotmail.com
** (Good) mx4.hotmail.com = 65.55.92.168 / mx4.hotmail.com
** (Good) mx1.hotmail.com = 65.54.188.72 / bay0-mc1-f.bay0.hotmail.com
** (Good) mx2.hotmail.com = 65.55.37.72 / col0-mc1-f.col0.hotmail.com
yahoo.com has 3 MX records
** (Good) mta6.am0.yahoodns.net = 98.137.54.238 / mta-v2.mail.vip.sp2.yahoo.com
** (Good) mta7.am0.yahoodns.net = 74.6.136.244 / mta-v3.mail.vip.sk1.yahoo.com
** (Good) mta5.am0.yahoodns.net = 98.139.175.224 / mta-v1.mail.vip.bf1.yahoo.com
fastmail.fm has 2 MX records
** (Good) in1.smtp.messagingengine.com = 66.111.4.71 / mx2.messagingengine.com
** (Good) in2.smtp.messagingengine.com = 82.145.212.142 / tmx2.messagingengine.com
comcast.net has 2 MX records
** (Good) mx1.comcast.net = 76.96.62.116 / imta.westchester.pa.mail.comcast.net
** (Good) mx2.comcast.net = 76.96.30.116 / imta.emeryville.ca.mail.comcast.net
apple.com has 7 MX records
** (Good) mail-in13.apple.com = 17.254.13.11 / mail-in.apple.com
** (Good) mail-in14.apple.com = 17.254.13.13 / mail-in.apple.com
** (Good) mail-in11.apple.com = 17.254.13.7 / mail-in.apple.com
** (Good) mail-in12.apple.com = 17.254.13.10 / mail-in.apple.com
** (Good) mail-in2.apple.com = 17.254.13.5 / mail-in2.apple.com
** (Good) mail-in6.apple.com = 17.254.13.9 / mail-in6.apple.com
** (Good) mail-in3.apple.com = 17.254.13.8 / mail-in3.apple.com
google.com has 5 MX records
** (Good) aspmx.l.google.com = 74.125.65.27 / gx-in-f27.1e100.net
** (Good) alt1.aspmx.l.google.com = 74.125.113.27 / vw-in-f27.1e100.net
** (Good) alt2.aspmx.l.google.com = 209.85.143.27 / dy-in-f27.1e100.net
** (Good) alt3.aspmx.l.google.com = 209.85.229.26 / ww-in-f26.1e100.net
** (Good) alt4.aspmx.l.google.com = 74.125.79.26 / ey-in-f26.1e100.net
aol.com has 4 MX records
However, reverse dns is a completely different beast. Whoever has the ipv4 subnet controls the rdns for that subnet. If an isp is nice, they can delegate smaller subnets of their larger block to individuals, but this is rare.
This should help clarify
The spammers will find a way around it, no matter what you do. It isn't any better than filtering in terms of a real long-term solution.
And the reason why both suck and fail to bring a long-term solution is they are both reactionary measures to an economic problem. They are both rooted at least partially in the notion that spam is sent out to piss you off and waste your time, which could hardly be a less accurate analysis of the situation.
Spam is sent out to make money, period. It will continue to be sent as long as money can be made by sending it out.
Hence the solution is to disconnect the spammers from the people who pay them. Work has been done to do this already and it has been shown very effective because there is no incentive for spammers to send spam if they aren't getting paid for it. Anything else just encourages them to find ways to get around anti-spam measures and will inevitably make spam a more costly situation for everyone.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
Read RFC1912 - no, it's not for SMTP, but a requirement of being on the net in general.
Note the "loss of internet services" - of which smtp is.
Make sure your PTR and A records match. For every IP address, there
should be a matching PTR record in the in-addr.arpa domain. If a
host is multi-homed, (more than one IP address) make sure that all IP
addresses have a corresponding PTR record (not just the first one).
Failure to have matching PTR and A records can cause loss of Internet
services similar to not being registered in the DNS at all. Also,
PTR records must point back to a valid A record, not a alias defined
by a CNAME. It is highly recommended that you use some software
which automates this checking, or generate your DNS data from a
database which automatically creates consistent data.
Glad you brought up the RFCs - isn't a proper rDNS entry for a server a requirement?
It's a "SHOULD", not a "MUST", as it's perfectly legal for an RFC-compliant email system to not have a hostname. Such a system would introduce itself (in its greeting and in HELO to other servers) as "[aaa.bbb.ccc.ddd]" with the actual IP address replacing the placeholders.
Likewise, it is a "MUST NOT" for rejecting based on not having reverse DNS.
The way I handle this is I greylist any email that comes in if:
1) it has no reverse dns
2) or the reverse dns matches certain patterns that indicate dynamic ip's
The rest comes in undelayed and is exposed to further checks. Note that it's ludicrous to require forward and reverse dns to resolve to the same thing. In today's age of hosting multiple domains in one server/cluster, it's just not reasonable to have PTR records defined that way.
I've been running my own mail server since my first 'domain' ended in .uucp (and also didn't have rDNS but whatever)... I likely have greater clue than most mail server admins who reject mail as spam for not having rDNS. I continue to run my own mail server and refuse to relay through my ISP's mail server. This way I can check my logs and know that my e-mail was accepted by a remote MX which is a useful debugging tool when someone says "I didn't get that e-mail"... ISP's tend to change their email policies at the drop of a hat and suddenly, without prior knowledge, your mails stop getting through because someone decided to disallow .tar.gz attachments, or whatever. My ability to add a PTR record to my subnets doesn't tell you anything about my ability to run a mail server nor does it say anything about my e-mail policies. I know lots of retard windows admins who know how to add a PTR record but I wouldn't trust them to reboot my son's computer, let alone run a reliable mail server. So if you're using the presence of rDNS to detect clue, then you are without clue. Given that PTR records are largely free-form, there is no valuable information in the content of a PTR record. Just because many sizable companies have jumped on the bandwagon, doesn't mean they're intelligent either. I only recently had to relent and add a PTR to my mail server IP because e-mails to my customer were bouncing due to Brightmail arbitrarily deciding that my site was a spam site due to the lack of rDNS. Since my customer is directly tied to my revenue and the smartypants admins at Brightmail won't accept a logic and fact based argument, I had no choice but to relent.
sigh. I hate people.
Nowadays most of the spam comes from infected pcs, connecting to the internet with residential ISPs, and they all (well, at least 90%) have a rDNS for their customers IPs.
This article advocates a
(x) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(x) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
(x) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
(x) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(x) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
(x) Blacklists suck
(x) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(x) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!
you can friend me!
Darwin Enforcement Agent
My approach, using Exim4, is not to reject messages outright based on single issues, such as not having a proper reverse DNS entry, but to reject based on combinations of them. This is a great way to limit false positives.
For instance, an incoming message may also have a bad HELO, a bad sender domain, be blacklisted locally or by a DNSBL service, or not have a working callout so that the existence of the sender's account can't be verified. There are more issues like these to look for. My systems count the number of these transgressions per message and reject when a certain value is reached, say three, while dumping messages that score one or two end in the recipient's spambox folder. With Exim, this kind of solution is surprisingly easy to construct using ACL statements with user-defined variables that include arithmetic statements. The last checks that are performed involve Clamd and SpamAssassin, because they are so resource-intensive.
I should also mention that my systems also perform a number of checks up front for obvious spam that is rejected immediately, e.g. if the sender address domain is gmail.com, but the sender HELO name is not part of the google.com domain.
You got it. Today, unless you have a dedicated Email server guy it's retarded to run an Exchange server in house.
I only agree on the point Exchange server, run exim or postfix for something stable and managable.
It is not really very hard to do, you need a bit of clue, but that is all. If no one at your company has clue then either outsource it or pay someone external to maintain it for you, neither should be expensive.
Oh: yes a rDNS record check is good, the problem is a numpty administrator who does not understand the issues and expects to force his brokenness onto the rest of the world. It should not be hard to whitelist that domain from the rDNS check.
While spammers certainly don't worry about conforming with RFCs, it should be noted that rejecting email based on the DNS is a specific violation of the relevant RFC, RFC-5321, section 4.1.4, which states:
An SMTP server MAY verify that the domain name argument in the EHLO command actually corresponds to the IP address of the client. However, if the verification fails, the server MUST NOT refuse to accept a message on that basis.
That said, it is common practice to do this anyway. The reality is that you will not have reliable email delivery unless the hostname your server issues in the EHLO command resolves to the servers IP address and the IP address resolves back to that same name.
Hey, who else could go for some flapjacks right now?
Speakeasy never had a problem setting my rdns records to whatever I asked them to when I was using them as an ISP with static IPs. Have you considered the possibility that your ISP sucks?
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
You do realize the ISP owns the f'ing address. That means *THEY* control what PTRs are assigned. They are the only ones that can change them.
Please use your internet provider SMTP Server
So what should a customer do if the only cable or DSL ISP in an area has an unreliable SMTP server?
Does "clueful" also mean "happens to be located in an area with at least one clueful business-class ISP"? Based on other comments posted to this story, it appears not all ISPs offering a static IP are clueful enough to make RDNS easy for customers to configure.
If you want your email server to actual deliver mail, rDNS is required as many major email providers like AOL and MSN/Hotmail now require them to even accept incoming connections. It doesn't matter what anyone thinks about it, it's already a standard because of this.
I would not configure any servers to reject mail outright originating from servers without rDNS though. As the OP notes there are some legit servers whose admins apparently did not get the memo or do not care about delivery rates (if your ISP will not allow valid rDNS switch hosts or sign up for a free Google Apps for your domain, your mail deliver rates will be absurdly low without rDNS). Like someone else suggested a SpamAssassin/SCL rule to put it in a Junk folder instead would allow the one off legit message to be retrieved.
Although still imperfect, DNSBLs like Spamhaus and Barracuda seem to be a safer method to block spammy servers outright.
Agreed, agreed. We support mail servers for many companies on, idk, maybe a dozen ISPs and a few data centers. Bottom line, you have to be persistent. Someone (probably with DNS admin in their title) can do it if you ask nicely :)
Chance favors the prepared mind.
Perfect is the enemy of good.