Slashdot Mirror
The Hostile Email Landscape (liminality.xyz)
An anonymous reader writes: As we consolidate on just a few major email services, it becomes more and more difficult to launch your own mail server. From the article: "Email perfectly embodies the spirit of the internet: independent mail hosts exchanging messages, no host more or less important than any other. Joining the network is as easy as installing Sendmail and slapping on an MX record. At least, that used to be the case. If you were to launch a new mail server right now, many networks would simply refuse to speak to you. The problem: reputation. ... Earlier this year I moved my personal email from Google Apps to a self-hosted server, with hopes of launching a paid mail service à la Fastmail on the same infrastructure. ... I had no issues sending to other servers running Postfix or Exim; SpamAssassin happily gave me a 0.0 score, but most big services and corporate mail servers were rejecting my mail, or flagging it as spam: Outlook.com accepted my email, but discarded it. GMail flagged me as spam. MimeCast put my mail into a perpetual greylist. Corporate networks using Microsoft's Online Exchange Protection bounced my mail."
I run a small email system ~2500 users and don't have your problems...
A pox on web designers who feel that window.innerWidth == screen.availWidth
There's little more to the article than the summary.
How does the person in question solve their mail issue? They don't, they went back to Google Apps.
Now you don't have to read it.
... to this new Brave New Internet.
Fighting SPAM was easy since the beginning. In the early 2k years, most of the SPAM fighting techniques was already somewhat prototyped on the mailing lists I was following,
Now, 15 years later, I think I know why nobody did anything for a decade and a half - control. Now it's God Damn easy to drop someone from the mail system - you can render a company inoperative if it dare to run his own mail system.
And so, for "safety", you need to pay for some bug corporation to run it for you - while harvesting you mail on the process.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
Without any details I can tell you you had something misconfigured.
Have you tried SPF?
I've been running my own mail server for a year or two now, the only places I've had reject my mail have been small businesses/organizations that have more restrictive policies. I haven't been flagged as spam on outlook.com, gmail, or yahoo mail and even my workplace's server has accepted them. Perhaps this person got flagged early on as a spam source and didn't realize it?
If you are Gmail or Yahoo or somebody big, you get money by data-mining people's email (something that would have gotten you an internet death penalty 30 years ago, and would have your name mentioned alongside a lot of nasty words, but these days everybody accepts that with no complaints).
So if you're one of those big players, you want to create as many headwinds as possible for people to run their own servers. If Bob and Mary each run their own servers and send mail to each other, you don't get to search their email in transit. But if you can make it harder for Bob and Mary to run their servers - say, by making it hard for them to email the multitudes using GMail - then it's more likely Bob and/or Mary will just give in and use GMail, and you then get to spy on their email as it goes by.
The point is that they don't mind this at all. It's the same idea as large established companies in some market welcome intrusive regulation, because they can afford to cope with it, but it means small upstart competitors won't get any traction because they can't easily deal with the paperwork burden.
Sad really. The internet was once the domain of peers on equal footing (at least until they did something to prove they shouldn't be given that equal footing). That internet is dead now, and we've corporatized the entire thing. It's in the control of a few big companies now, who can do as they please and there's sweet, sweet FA you can do about it. People warned about letting the camel's nose into the tent, but nobody listened.
Have a paying Google Apps customer contact their support and complain that they are not receiving emails from you. They should be able to tell you why, and maybe then you can do something about it.
from TFA:
The tech info you provide is solid and good, but your logic is flawed - you assume the author don't know that he is doing, while in the text he says de does (and hints some third party services to validate his claims).
Do you have some reserves with mail-tester and Port25?
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
..and set up SPF entries and reverse DNS. Also make sure Postfix is locked down and not acting as an open relay. It really is not that hard, this article comes off as whiny "I can't do it, so the world is against me" at best.
Did you even read the article? There's not much more than the summary, but there he does make note that reverse DNS and SPF records, among other things, were setup:
I've done this before, ...: not on any blacklists, reverse DNS set up, SPF, DKIM and DMARC policies in place, etcetera. (Side note: mail-tester.com and Port25 are great for checking your setup.)
The near-conclusion quote is his real point:
...from Microsoft's Postmaster Troubleshooting page:
IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, Outlook.com will typically allow for a better email delivery experience.
I run my own mailserver, mostly "just because".
The reputation problem I encountered early on was because of a lack of a reverse DNS entry. Easily fixed; I simply asked my VPS provider to create one.
The next problem that started about 18 months ago was reputation: my little server simply wasn't a trusted service.
Because of the (unbelievable) amount of spam hitting my server, I had taken out a Comodo AntiSpam Gateway subscription about two years earlier. It was initially free, but after a year or so they wanted money. Since the service rocks, I happily pay my ~$30 annually.
What CASG also offers is outbound scanning: if I tell my server (an Exchange 2010 server) that the outbound smarthost is CASG, my email all of a sudden piggybacks Comodo's reputation. Voila, email flows without incident.
Problem solved.
If people reject my mail for no reason - their loss more than mine. It will go on of course, as corporations never notice the lost opportunities. But I can order stuff from someone who reads my mail - and blacklist (for business, not just for email) those who refuse mail for no better reason than being a newcomer.
She had no issues apparently.
-CM
doing it to me. web.de doesn't bounce until 24 hours have passed. A phony-baloney and vague reason is given: IP address being on a 'well-known RBL' but my mailer is clean according to two site that track RBLs.
Coming 3.. 2.. 1..
The issue here is your setup, not email in general. Our company sets up lots of small office based email systems. we use a combination of open source and propriety solutions from postfix/cyrus imap, zimbra, exchange and zentyal. None of our systems have your problems. your issue is likely that you havent set up your dns security records and spf correctly in combination with sending from an ip address that doesnt have an acceptable ptr record, i.e. it resolves to a home user adsl or cable pool. The second issue is easy to fix, use your isp's smarthost or another 3rd party service. The first issue requires you to figure out how modern email authentication and reputations systems work and set up your dns accordingly.
Your lack of knowledge lead you to the wrong conclusion, go back and research whats actually required.
it's not only that... setup reverse DNS on your mx ip and also setup the SPF record. I setup my mail server 1 month ago, and for now 100% of my mails were delivered.
I have a server at home. It's totally disconnected from the Internet. I mean, like, TOTALLY. Well, everybody in my household can get emails from everybody else in my household, no problems. And fast too! How about that huh? How about a little of that?
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
These two setting of DKIM and SPF are pretty important to have your mail get into more restrictive Email environments, just sayin'
IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, Outlook.com will typically allow for a better email delivery experience.
Sounds like a Catch-22: "We won't accept accept email from a server until the new server until the server has successfully delivered lots of email."
Well, there's spam egg sausage and spam, that's not got much spam in it.
I still run mutt + courier imapd + postfix on my home box (though I admit I don't use it much anymore since juggling a few gmail accounts worka very well now compared to the old days of yahoo / netscape / hotmail / etc.)
Simple way to boost your reputation is to simply configure a smarthost to send outgoing mail securely. There are plenty of tutorials on using gmail or several ISP smarthosts (like Verizon Business FIOS.
Yeah, it's not an ideal solution, compared to, say, making everyone use GnuPG signatures against a registry for automatic whitelisting. But it will get you out of the "open relay" mailhost automatic blacklist (which I assume is the real problem with your configuration.)
I run my own mail server on a dyndns connection. At first, Google would filter out my mails, but once I set up SPF and DKIM records, they became much more friendly. Haven't tried outlook.com, but hotmail.com (also owned by M$) works fine.
PTR, SPF, DKIM, a clean IP and a properly configured SMTP server will work just fine. You're doing something wrong. Slashdot please improve your quality.
Do your due dilligence...
...and you do yours by RTFA.
I've done this before, and this server was configured perfectly: not on any blacklists, reverse DNS set up, SPF, DKIM and DMARC policies in place, etcetera. (Side note: mail-tester.com and Port25 are great for checking your setup.)
This is absolutely a case of the big guys using strong-arm tactics to try and fight spam. Sure, it might be effective to block 100% of mail coming from mail servers which haven't existed for more than 6 months, but it's also completely unreasonable and goes totally against the ethos of the Internet.
Spam detection is just like any other kind of testing. You can have a very powerful test with almost no false negatives but if you start racking up the false positives then the usefulness of the spam filter becomes marginal. This is especially true of the big email providers like Google, Microsoft, and Yahoo. Have a problem with their servers? Good luck finding someone at one of those companies to help you. They don't care about you and your cute little mail server -- especially since you're trying to compete with them.
A faceless tyrant may be the most terrible kind. But get used to it, because that's what we've got with the Corporate Internet.
"What do you despise? By this are you truly known." --Princess Irulan, Manual of Muad'Dib
/)
0. Previous RBL history for the IP address and the block
1. Not being an open relay for any amount of time while setting up
2. Reverse DNS
3. SPF
4. SMTP server host name 5. Retry delay not less than 1 hour. And e-mail starts running.
I've been running my own mailserver since 2003, and I have seen my share of problems.
1: mailservers blocking mail based on spamhaus DUL. You can delist your IP. But still, blocking exclusively on that?
2: hotmail.com accepting emails and then discarding them silently. No trace of them. No bounce. Recipient did not have it in their spam folder or anything. This was several years ago, so perhaps it's better now. But discarding emails after promising to deliver them without any possibility for the recipient to control it: bad idea.
3: Various greylisting email servers. Not really a problem as my MTA will retry and the email is only delayed for a few minutes.
4: gmail.com rejecting emails sent over IPv6 but happily accepting them over IPv4. It turned out to be a problem with their parsing of SPF records, and apparently fixed now. But I did find out that there is no reasonable way to contact the gmail team.
5: outlook.com rejects emails due to FBLW15, whatever that means. It seems you can get whitelisted, but it appears that a lot of hosts are being hit by it for no reason.
6: office365 bouncing emails due to "protection" with no explanation given, and direction to contact the recipient by other means to get whitelisted. This was for a the official email address listen on a company website. I decided that my email wasn't important enough. Their loss.
Bottom line: If you run your own email server then expect to occasionally do some manual whitelisting etc. And expect some email servers to be uncooperative and/or RFC-clueless.
I generally do not have a problem. Obviously an outbound spam filtering service will deal with the issue.
Did you do a slow start? Most common cause of this in the hosting industry is some guy gets a domain setups up email on a VPS then spams his entire contact list with a hey this is my new email to watch it get blocked, bounced etc. Oddly all the big guys seeing a mass mailing as the first thing they get from an IP they flag it.
Fastmail frankly it sounds like you're a spammer er opt in marketing company. Your looking to startup a paid email service, what sets you apart from the market?
No sir I dont like it.
Different subject but now every time I try to send my dad an email with a link through Charter.net it comes back saying it's SPAM. Each time I just copy and paste to Yahoo mail and it goes through fine. PissedAtCharterDotNet
My guess is that the problem lies in the fact that the OP is using a garbage TLD. I've configured our mail server to silently drop all traffic from many of the new garbage TLDs, including .xyz. It does wonders for cutting down the spam levels. Sadly it's just a new version of Whack-a-Mole. Neither I, nor any of my users, appear to have gotten a legitimate email from any other these domains. I'll bet if the OP were to use a more traditional TLD, like .com, .uk, etc. there wouldn't be problems.
You can thank years and years of spammers. The sad part to me isn't really that independent hosts are consider spam by default, its the fact, even that being the case, my independent hosted email accounts are *STILL* getting hundreds of spam mails each day. Very annoying.
I have continuously run my own email server since around 1990 in one form or another. Established a vanity domain in the mid 1990's and started hosting email on my own domain. I must say that has been a more difficult task as time has gone on and has required I be more savvy about IP reputation and how to maintain it. Sometime last year I moved my email server from a VPS to a dedicated host and my wife began complaining over this past summer that she could no longer send email to Outlook.com and friends as well as Optonline. Given that she's a dog trainer trying to expand her client base this is pretty much a disaster for her. I attempted to work with the ISP hosting my dedicated server and they were not interested in fixing *their* IP reputation. Seems the above mentioned providers were blocking *all* of the IPS's IP addresses out of hand. They insisted they'd have to work with the ISP and while they were sympathetic they wanted to work the issues through the ISP. There's more to that I'm sure but after my ISP frustrating me by not being responsive I talked to some new folks. First question I asked before signing on was "how is your IP reputation?" After a lengthy explanation on how they have "high profile reputable clients" they assured me I'd have no problems. So I signed on the dotted line, installed a Puppet client on the box and set things up so the box would get configured as my mail exchanger. During shakedown I didn't notice problems right away so I edited DNS and pointed my MX records to the new box, added my SPF and DKIM records for the new host and powered off the old box. The very next day I composed an email to someone that I communicate via email on a fairly frequent basis and after hitting "send" got a bounce notification within minutes. Verizon was blocking the new server. New problem. It took many attempts and iterations I finally worked out between my new ISP (who graciously gave me a second IP address for outbound email) and Verizon I finally got whitelisted. So, yep, the Internet has become increasingly hostile to private email servers, but the problems can be worked out with some effort and tenacity.
Anonymous Coward wrote:
your issue is likely that you havent set up your dns security records and spf correctly
The featured article mentions already having set up "SPF, DKIM and DMARC". To which "dns security records" do you refer?
in combination with sending from an ip address that doesnt have an acceptable ptr record, i.e. it resolves to a home user adsl or cable pool.
The featured article mentions this: "not on any blacklists, reverse DNS set up".
much wisdom in fraxinus-tree's six-point plan.
Try out https://mailinabox.email/, a project I began a few years ago to make hosting your own mail much easier.
It includes comprehensive diagnostics to ensure everything is configured correctly, including reverse DNS, which is the most common issue that leads to mail not being deliverable / going into spam. This doesn't solve every problem, but lots of people have had good results with this project.
Really, the whole closing of the open internet is about control and power. Just as FB is not about "connecting people to their friends!", it's about control and power then leveraging that power into more power, more money more control.
The exact same thing is going on with Cloudflare which is about inducing site owners to select options which preclude anonymity.
I am doing email-servers since 1995. Including UUCP and bang path. But it's increasingly difficult to set up servers that are able to reliably send emails to the big guys. Sometimes emails get through, sometimes they end up in SPAM. Same server, same configuration, same sender. So - yes, I think Google and others are brutally disregarding the principles of email to present their users an artificially spam free environment. How can it be that my servers receive 100's of spams a week (filtered by spamassassin with almost zero false positives) but I don't see _any_ on my Googlemail account? Not even in my Spam folder? Google is most certainly discarding a lot of (spam-) mail without bothering to notify the user. What if I m interested in some of it? I guess I have to be thankful that Google graciously accepts some of our servers email as span and doesn't discard it right away. mm.
I had a very similar issue with Gmail when I started sending legitimate mail. Thankfully, it was pretty easy to resolve. Maybe look at their support page for ways to fix your sender-side issues. Make sure to have domain keys, SPF, opt out trailer links, etc..
https://support.google.com/mai...
Also make sure your host / server IP aren't black listed out of the gate. Generally speaking all ISP dynamic IP address blocks are marked potential spam since no customer-end's should be hosting their own mail servers. If this is an exception, most respectable RBL's will remove your listing if you follow their sensible take-down procedures.
http://www.anti-abuse.org/mult...
Like so many things, having great power now requires great responsibility. Since email has made every host a potential spam target, its your duty to make sure you smell clean to your peers.
Bye!
I'd ask the Hillary Clinton. Although her viewpoint on reputation is under attack, her email server definitely worked!
new garbage TLDs, including .xyz [...] Neither I, nor any of my users, appear to have gotten a legitimate email from any other these domains.
Let me guess: neither you nor your users owns any shares of Google's parent company Alphabet Inc., whose web site is https://abc.xyz/ .
I used to be a email systems developer, and run a small mail server. I have run across some reputational issues with Microsoft Hotmail after an upgrade/migration changed my IP address. If you have a static IP VPS, hang on to your IP as long as possible. It can takes months to purge a bad reputation.
But the big thing that jumps out at me... The XYZ domain is a complete wasteland. Fully one third of my spam rejections are from the XYZ TLD, another third are .eu, or .us. Basically, if the TLD is cheap to register, it's likely a spam haven for throwaway "burner" domains. They'll register zjfhruh.xyz and use it for a couple hours. Toss it and move to the next one. My Postfix config simply blocks the XYZ domain, and anything with a rDNS PTR listing any of the major VPS providers...
HELO
554 Goodbye Snowshoe...
Anything coming from .ru & .cn goes in the hold bucket, along with .co, and many other minor TLD's. But that's only because my userbase is so small I can monitor the queue. The great TLD expansion was a money grab for the registrars, but the email community is largely using it as a reputational filter.
Cheap & readily available = Spam.
So the big dominant e-mail providers are abusing their dominance to shut out independent competition, eh? Sounds like we should all set up private e-mail servers and then sue.
The XYZ domain is a complete wasteland.
Then why is Google's parent company in the wasteland?
I've had the same issue in the pass few years when I used domains other than .COM. Even my .ORG outbound emails were getting tagged a junk or deleted etc at the receivers end. Hard to do business when your own customers can not see emails they are expecting from you. Once the internet gets turned over to the useless United Nations (hehe), I fully expect this problem will get worse.
Dave
My gut reaction is to say... of course the landscape is hostile, because there are many people that abuse email. But, in practice, I'm at least a little skeptical of this article as my own experiences with email server installation differs (multiple datacenter Ironport cluster setups and my own private VPS mail server) . I am further critical as there was not really any specifics given in the linked article.
Even if the article is correct, and their mail server was setup to perfection (SPF, DKIM & DMARC setup, Forward confirmed DNS, etc). How you use the mail server is also important. There are still legit reasons that emails might be flagged beyond a correct setup:
* I could see new IP addresses being flagged if they started sending 10,000,000 emails overnight vs a new IP addresses that was sending 100 in a day to begin with.
* Maybe the sender was sending 50 MB attachments, which is pushing what is acceptable
* Even if Forward Confirmed DNS was setup, maybe the sender's mail server did not give a proper HELO/EHLO to match the connecting IP
* Maybe a few users at the destination had already mistakenly marked emails from that domain as a spammer.
* Maybe some expected headers were missing when sending emails initially during testing (telnet email tests vs emails from a client app).
* Maybe SPF is setup syntactically correct, but there are over 10 nested DNS A record lookups in the SPF record, so SPF still fails due to protocol limits. Some SPF tool checkers don't catch this in my experience.
All I'm trying to say is that how you use the mail server is just as important as having a properly installed mail server. Email is hard, regardless of what anyone says. There are too many gotchas.
I run my own email server as well. But it's not as simple as an MX record. I use domainkeys and spf as well. None of the major services flag me as spam.
Agree. I run my own e-mail servers for a few domains and have no trouble at all. You need to be absolutely 100% sure that you aren't operating an open relay, or you'll be blacklisted immediately. You also need correctly configured STARTTLS with a valid certificate signed by a widely accepted root. Most relays will reject mail if STARTTLS is not used. Reverse DNS helps but isn't 100% essential. You want reverse DNS to resolve to something in the same domain. For example if people connect to the server as mail.domain.com but reverse DNS calls it srv1.domain.com that will be accepted by the vast majority of relays. If you want Google/Yahoo/Outlook to accept your mail you need DKIM signing, which involves generating key pairs, putting the public keys in DNS and configuring your mail server to sign messages. Correctly configured SPF improves your reputation, too.
So what you need is some means of sending large amounts of email to outlook.com addresses to build reputation.
Make sure you include lots of keywords in the subject line to ensure delivery. Also ensure they are trending topics like "Jennifer Lawrence", "V1agra", "Hot", and "Money"
Well, there's spam egg sausage and spam, that's not got much spam in it.
I had similar problems when I first started using my own email server but since I setup DKIM and SPF I haven't had any problems.
Did you even read the article? There's not much more than the summary, but there he does make note that reverse DNS and SPF records, among other things, were setup:
I've done this before, ...: not on any blacklists, reverse DNS set up, SPF, DKIM and DMARC policies in place, etcetera. (Side note: mail-tester.com and Port25 are great for checking your setup.)
The near-conclusion quote is his real point:
...from Microsoft's Postmaster Troubleshooting page:[...]
They also want them to enroll in join Return Path's Sender Score Certified Email program, update your Junk Email Reporting Program (JMRP) account with the new IPs (this is a free Microsoft account you have to sign into in order to enter your IPs). They don't want them using dynamic IPs. They want them to go to the Sender and ISP Services (since they count as an ISP, given what they are attempting to do: clone fastmail).
It'd be useful if they provided the 421/550 error codes from attempts to send mail to Hotmail, in order to identify the subcodes to see where they've gone wrong.
P.S.: They may also be being screwed by their antivirus software inserting headers which Microsoft is not prepared to trust.
This is likely a side effect of recent spam attacks that come from neutral reputation hosts. Often times what I would see is e-mail coming from a host that had a default PTR record but it was still a full round robin DNS.
Still, most places won't out-right block you but will receive a small number of messages. Since I can't verify anything I have to take the summary's word for it that it's set up correctly but my instinct tells me there's probably more to it.
Classic Microsoft. Lots of happy sounding language meaning nothing, and miss the point entirely. How the hell is your IP supposed to build reputation if they assign zero trust to begin with? It's like they read about this groovy new group called NANAS and said that looks really cool, let's make outlook.com work like that!
Except of course that philosophy doesn't work anymore. We learned long ago that, practically speaking, over-aggressive blacklists are often worse than nothing at all. It's why Gmail works so well...assign some trust then let the source prove itself over a large sample of recipients.
A domain without information is untrusted.
SPF tells them that you're trying to combat spam from pretending to come from you.
Similarlt for DKIM, that also tells them that you are checking and explicitly marking every message you send out from your domain and absence of such signing should be treated as suspicious.
Put both of those on, in a decent static IP range (nobody sensible accepts email from dynamic IP's!), and you're good to go. How do I know? My own domains are ALL run by me, on Postfix. They even forward some mail addresses to providers like GMail as a matter of course.
The only problem I ever have for delivery is when *I* have accepted a spam message and try to forward it on to someone like GMail (harder to stop than it sounds, even with greylisting, etc.). They spot spam that my system can't, even on a re-forwarder.
Hell, I IPv6'd my domain too. So long as you have valid PTR records for your reverse, places like GMail are perfectly happy with that. Never had a problem. (But if you can't set your reverse for your IPv6, there's a way to turn off using IPv6 and fallback to IPv4 just for GMail, etc. when using postfix - google it).
My entire email for the last 5 years at least has been self-hosted. I've been using tiny startup services for about 10-15 years before that without issue. If anything, I have significantly more issues with the big-brand provider we use as smarthost for the Exchange servers in work, which are routinely blacklisted for spam and I have to fallback to manual sending from our leased lines, than anything to do with my self-hosting personal email domains.
Just don't expect your no-name mailer on a dynamic range without even the simplest of anti-spam measures to be accepted by places like Google, and you're golden.
Running your own e-mail server can turn into a full-time job fighting off spam. There are many organisations who don't want you to run your own e-mail server they will disrupt frustrate and irritate you as much as they can. If you bounce messages they will use that bounce to bounce spam off of you to target others. If you start blocking IP addresses some countries want you to do that. If you let somebody else take control you never know whether you are in contact with the people who you would like to be in contact with. Other people want control some because they earn money from providing that service and others because they are megalomaniacs. Running your own e-mail server is as frustrating as running your own forum but more so. E-mail addresses are gold dust government departments want to know it the taxman wants to know it everybody you purchase something from wants to know it. And government spies around the world want to read it. https://en.wikipedia.org/wiki/...
Maybe he got flagged as spam/spammer because it was spam? Wheres his images of the said emails? why hasn't he produced any data on how he configured his mail-server. Just stating the obvious.
Jack of all trades,master of none
It's valid reasoning, though. Spammers go through IP blocks routinely. And MS isn't saying they block mail. They likely mean that they defer it, or give it a higher spam score. If you continue sending to them, eventually a reputatiuon will be built up, and if it's positive, the mail will go through.
Then I cannot see why he sees these problems I'm not seeing with my own mail server. I can mail google users and other big providers.
Only thing he does not mention and I suspect is, he's behind a residential DSL/cable line and that is problematic nowadays. My server is at a VPS provider. Those do cost little and work acceptably well.
I run my own mail server on a dyndns connection.
I'm surprised you don't have issues with that. There are RBL's that specifically list IP address blocks that are thought to be dynamic address pools, and some servers will reject you for nothing more. Also, how do you handle reverse DNS with a dynamic IP?
Only thing he does not mention and I suspect is, he's behind a residential DSL/cable line and that is problematic nowadays. My server is at a VPS provider. Those do cost little and work acceptably well.
*Excellent point*. I didn't thought of that.
Home Internet provider's IP are probably blacklisted by default.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
Recently I migrated my server to a new one, and this hit me, too.
The new IP did not have any reputation, so microsoft servers were not accepting mail from me.
solution: register into microsoft mail services, use the procedure for de-listing your ip/server as spammer.
after 24 hours everything works again.
A big part of the problem is that ISPs, not end users, now do most of the spam filtering. Under the old scheme, each user trained her own email client by using Spam/Not Spam buttons until the program learned that user's specific patten of expected mail. The complaint I get from users now is "My spam buttons stopped working!" By which they mean that they are seeing spam for which the Spam button in their client stays grayed out because the ISP decided the message was spam. Worse, they are seeing an increasing number of valid messages land in their Junk folder, and with no Not Spam button available. So now that ISPs are "helping" kill spam, we have to get used to treating our Junk folders as a specially flagged part of our Inbox.
Most of the larger email providers are very, very fickle about receiving email from major domains that aren't already recognized (gmail, yahoo, outlook, hotmail, etc).
They either silently drop the mail I send to them or reject it outright, arbitrarily labeling it as "spam" or "suspicious".
Several of the sites I run (some which sell products and some which are just forums for groups of like-minded people) almost never manage to successfully get an email to the people who are signing up and/or who have just bought a product.
If I manually resend the confirmation email from a Yahoo account then it goes right through. Same thing for people who buy stuff from my sites- often times they don't get any email allowing them access to the service or product they just purchased unless I manually resend the confirmation or product access email.
And in case you're wondering, no, I *never* send spam or any kind of unsolicited emails, never, ever, period. I never have and never will, but that hasn't saved me from being lumped in with all the shitbags who do.
Thanks a lot, spammers...you've really managed to fuck it up for everyone. If I could hunt you fuckers down and snap your necks, I'd do it.
Just cruising through this digital world at 33 1/3 rpm...
I feel like this article was written by someone who hasn't been paying attention to the email landscape for the past twenty years. The checking services that the author lists don't make sure your DNS PTR records are correctly set up. They don't make sure that your server isn't an open relay. And they don't insure that your server is RFC compliant. They run your content through SpamAssassin and invert the score to rate your chances of successfully delivering a marketing message. I also run my own mail server. I'm not doing it to provide you with a medium for your marketing messages. I'm doing it to provide a noise free communications path for my users. I'm probably better than most because if my users complain about losing _any_ messages, I consider that my problem and use appropriate tools to fix it. But email has been under assault for the past twenty years by people who want to sell us things without regard for our interest in buying them. Anyone who expects that landscape to be a green pasture of ease rather than a battleground is sadly out of touch.
I setup my own mail server. Got DKIM configured and everything. OP is just shit at following directions to properly configure and vet an email server.
I'm surprised such a non-story about a misconfigured email server makes it to the front page of /.
You're just doing it incorrectly.
Kriston
Yes, other servers are now MUCH more picky today about who they accept mail from, compared to a couple of decades ago. Sometimes it seems like recieving servers are looking for any excuse at all to reject your email. Your only option is to find and cross every possible t, THEN find and dot every possible i, to show that you are unequivocally a Good Internet Citizen.
Off the top of my head:
-You need an SPF record
-Make sure your server is configured with forward domain AND a reverse domain. The only way to do the latter is to talk to your ISP and get them to do it for you.
-Make sure your server speaks all the appropriate SMTP lingo, such as making sure your HELO request provides your fully qualified server name, AND that server name matches your domain name/SPF records.
-Make sure your server doesn't give up on the first try, as some recipients *intentionally* reject the first attempt, which admittedly is a good way of blocking a great deal of spam.
Others have mentioned DKIM. I haven't run into this particular issue.... yet. Doesn't mean that won't change in the future, and if you have the impetus to take care of that during setup, you may as well do it to save yourself future headache.
And sometimes that STILL isn't enough. I had an issue where a recipient was using some obscure anti-spam provider, and it turned out they objected to some obscure aspect of the email I was sending, even though it wasn't even an official part of the SMTP protocol. I don't remember anymore what that exact thing was, but the point is that despite thinking that you've configured everything correctly, you'll still need to roll up your sleeves and do extravagant detective work just cause one particular recipient decides they can be anal-retentive douchebags.
Remember the wild west, when you could just pull off the Oregon Trail, build yourself a shack, and call it home? Nobody told you how to build your house, or how big your yard could be. But when you had a visit from a thief, there was no police to call, and if you had a fire, you lost everything. It was up to you to defend your own life at all times.
Sure, life might have been simpler back then. But who would want to go back there?
The Internet is the same story. In the good old days, everything was free for the taking, but it was the wild west. Now the city slickers want to put up fences, and the cowboys want to tear them down. Whether we like it or not, the Internet is changing, becoming more regulated, and some people aren't going to be happy about it.
I run a small ISP and have several email servers I maintain. What I have observed is a hair trigger on the part of the large networks like Comcast. Even a few spam complaints is enough to get you black listed. It's hard to run a 100% perfectly clean server as you must to some degree trust your users. 1 malware infection on one of their machines sending spam and boom your cooked.
I have taken to telling customers that have issues sending to Comcast, AT&T, etc to get a Gmail account to send to these. No matter how much SPAM these servers send out they would never get listed as this would cause mass complaints from their users.
Millions of small compaines host their own email without issue. Of course reputation is a factor. Letting just anyone and where send email is why we have such huge spam problems and taking reputation in to account helps cut down on spam. I'm 100% sure this is a configuration issue and not Google and MS blocking out 'the little guy'.
from TFA:
The tech info you provide is solid and good, but your logic is flawed - you assume the author don't know that he is doing, while in the text he says de does (and hints some third party services to validate his claims).
Do you have some reserves with mail-tester and Port25?
Well, I would read the article but his domain "liminality.xyz" is blocked by my work's Corporate filter as "Elevated Exposure Risk". So whatever his issue is, it's not specific to email, it's his actual domain causing him an issue. It's just that the only thing he's personally noticing is the email. .xyz and all his shady, bullshit tactics he was using last year to inflate his numbers. It very quickly became known as a domain full of sketchy sites, spammers, and unverified domain holders, and there are a lot of questions about how many registrations were actually sockpuppets he did himself.
But it's probably because ".xyz" itself has some issues, starting with the guy who owns the TLD for
In other words, if the author of the article would have taken a little bit of time to look into things, he would have figured out what the problem was, and been able to correct it pretty quickly.
The truth is: stop bleating and survive with rules. I suspect that's why the internet is going to ground to a halt, because I suspect that you and the internet assemblers (Vinton Cerf, Yogen Dalal, and Carl Sunshine et. al.) will avoid the internet to lessen irritants.
I ran into the same issue with new TLDs few times. Big players are rather suspicious about such domains. Basically the same story: static IP, properly set up DNS, SPF, DKIM, well-behaved Exim... Also, beware any email generating scripts, like shell/PHP/etc.. As a matter of fact, the heuristics that Google/Yahoo/Microsoft email services use will fine your letter for non-standard headers, which, in conjunction with new TLD, may cause your email to be marked as a spam.
I host my own email server. I have it on a domestic connection with dynamic IP so no reverse IP and that makes outbound problematic to some sites. So I signed up to an Outbound SMTP service for that piece ( I picked a mysmtp.eu), my Postfix TLS's to them and they TLS outbound so acceptable security, for my purposes. Inbound SMTP for me works fine with dyndns holding my A and MX records (IP changes pushed to dyndns automatically from my router), even with my self signed cert (mainly just want the link encrypted). I have a VPS on a fixed IP as a secondary MX, that can hold my email when my IP changes and people's DNS TTL's take a little while to catch up with my dyndns changes. Also I can flip the secondary to primary if my home server is due to be down for a while (e.g. moving house, ISP outage etc).
Using an outbound service makes it easier to set an SPF record for your domain. This and reverse DNS (as other people have said) cause the issues with other sites accepting.
I bet they would if he was using Microsoft Exchange instead of postfix.
If it does not then you have the same problems as you have outlined.
Might be better to post to a help forum than here?
If you setup all three you're taking steps to communicate to those mail servers exactly how strictly your messages are authenticated as being from you. All of those providers mentioned recognize DMARC and I'd be shocked if that didn't almost immediately get you through.
"Don't teach a man to fish, feed yourself. He's a grown man. Fishing's not that hard." - Ron Swanson
thank you
You are misrepresenting my staatement by giving me a choice between three items which I did not address, nor do I care about.
I included "Other (please specify)" to address exactly that possibility of misrepresentation.
The Canonical name of the machine (the forward address) and the IP address delegation (the reverse address MUST match. [...] a CNAME record
So in other words, your answer is "Other: MX can point at a CNAME record whose content matches the reverse DNS." Do I understand you correctly?
signing certs for DNSSEC
I've read that some domain registrars, including a very well-known one in the United States, charge extra for DNSSEC service.
One of the problems I've encountered was ipv6 problem with Google: my US-hosted VPS had a bunch of extra v6 addresses and *some* outbound connections happened over ipv6 -- without any reverse zone for that v6 address. Fixed that and now Google does not complain about emails from my domain anymore.