Slashdot Mirror
The Hostile Email Landscape (liminality.xyz)
An anonymous reader writes: As we consolidate on just a few major email services, it becomes more and more difficult to launch your own mail server. From the article: "Email perfectly embodies the spirit of the internet: independent mail hosts exchanging messages, no host more or less important than any other. Joining the network is as easy as installing Sendmail and slapping on an MX record. At least, that used to be the case. If you were to launch a new mail server right now, many networks would simply refuse to speak to you. The problem: reputation. ... Earlier this year I moved my personal email from Google Apps to a self-hosted server, with hopes of launching a paid mail service à la Fastmail on the same infrastructure. ... I had no issues sending to other servers running Postfix or Exim; SpamAssassin happily gave me a 0.0 score, but most big services and corporate mail servers were rejecting my mail, or flagging it as spam: Outlook.com accepted my email, but discarded it. GMail flagged me as spam. MimeCast put my mail into a perpetual greylist. Corporate networks using Microsoft's Online Exchange Protection bounced my mail."
I run a small email system ~2500 users and don't have your problems...
A pox on web designers who feel that window.innerWidth == screen.availWidth
There's little more to the article than the summary.
How does the person in question solve their mail issue? They don't, they went back to Google Apps.
Now you don't have to read it.
... to this new Brave New Internet.
Fighting SPAM was easy since the beginning. In the early 2k years, most of the SPAM fighting techniques was already somewhat prototyped on the mailing lists I was following,
Now, 15 years later, I think I know why nobody did anything for a decade and a half - control. Now it's God Damn easy to drop someone from the mail system - you can render a company inoperative if it dare to run his own mail system.
And so, for "safety", you need to pay for some bug corporation to run it for you - while harvesting you mail on the process.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
..and set up SPF entries and reverse DNS. Also make sure Postfix is locked down and not acting as an open relay. It really is not that hard, this article comes off as whiny "I can't do it, so the world is against me" at best.
Did you even read the article? There's not much more than the summary, but there he does make note that reverse DNS and SPF records, among other things, were setup:
I've done this before, ...: not on any blacklists, reverse DNS set up, SPF, DKIM and DMARC policies in place, etcetera. (Side note: mail-tester.com and Port25 are great for checking your setup.)
The near-conclusion quote is his real point:
...from Microsoft's Postmaster Troubleshooting page:
IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, Outlook.com will typically allow for a better email delivery experience.
I run my own mailserver, mostly "just because".
The reputation problem I encountered early on was because of a lack of a reverse DNS entry. Easily fixed; I simply asked my VPS provider to create one.
The next problem that started about 18 months ago was reputation: my little server simply wasn't a trusted service.
Because of the (unbelievable) amount of spam hitting my server, I had taken out a Comodo AntiSpam Gateway subscription about two years earlier. It was initially free, but after a year or so they wanted money. Since the service rocks, I happily pay my ~$30 annually.
What CASG also offers is outbound scanning: if I tell my server (an Exchange 2010 server) that the outbound smarthost is CASG, my email all of a sudden piggybacks Comodo's reputation. Voila, email flows without incident.
Problem solved.
"I've been running my own mail server for a year or two now,"
Unless I'm reading this wrong the article indicates that the problem is NEW email servers. From TFA:
Now, I've no idea if that is true or not. I hosted my own email until around 2010 (since the 1990s) then moved to google apps. The only issue I had is when I changed IPs when I moved and the static I got were previously "unallocated" space and most hosts marked them as spam for just being an IP in that group (never mind SPF records). Took about 1 or 2 months before 'filters' got the clue and fixed the rules.
So if you weren't having any issues maybe it's because you've been up and running for a while. Or maybe the user was getting flagged for some other reason and the only "info" they found as to why before they gave up was from Microsoft.
IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, Outlook.com will typically allow for a better email delivery experience.
Sounds like a Catch-22: "We won't accept accept email from a server until the new server until the server has successfully delivered lots of email."
Well, there's spam egg sausage and spam, that's not got much spam in it.
I run my own mail server on a dyndns connection. At first, Google would filter out my mails, but once I set up SPF and DKIM records, they became much more friendly. Haven't tried outlook.com, but hotmail.com (also owned by M$) works fine.
PTR, SPF, DKIM, a clean IP and a properly configured SMTP server will work just fine. You're doing something wrong. Slashdot please improve your quality.
0. Previous RBL history for the IP address and the block
1. Not being an open relay for any amount of time while setting up
2. Reverse DNS
3. SPF
4. SMTP server host name 5. Retry delay not less than 1 hour. And e-mail starts running.
I've been running my own mailserver since 2003, and I have seen my share of problems.
1: mailservers blocking mail based on spamhaus DUL. You can delist your IP. But still, blocking exclusively on that?
2: hotmail.com accepting emails and then discarding them silently. No trace of them. No bounce. Recipient did not have it in their spam folder or anything. This was several years ago, so perhaps it's better now. But discarding emails after promising to deliver them without any possibility for the recipient to control it: bad idea.
3: Various greylisting email servers. Not really a problem as my MTA will retry and the email is only delayed for a few minutes.
4: gmail.com rejecting emails sent over IPv6 but happily accepting them over IPv4. It turned out to be a problem with their parsing of SPF records, and apparently fixed now. But I did find out that there is no reasonable way to contact the gmail team.
5: outlook.com rejects emails due to FBLW15, whatever that means. It seems you can get whitelisted, but it appears that a lot of hosts are being hit by it for no reason.
6: office365 bouncing emails due to "protection" with no explanation given, and direction to contact the recipient by other means to get whitelisted. This was for a the official email address listen on a company website. I decided that my email wasn't important enough. Their loss.
Bottom line: If you run your own email server then expect to occasionally do some manual whitelisting etc. And expect some email servers to be uncooperative and/or RFC-clueless.
I generally do not have a problem. Obviously an outbound spam filtering service will deal with the issue.
Did you do a slow start? Most common cause of this in the hosting industry is some guy gets a domain setups up email on a VPS then spams his entire contact list with a hey this is my new email to watch it get blocked, bounced etc. Oddly all the big guys seeing a mass mailing as the first thing they get from an IP they flag it.
Fastmail frankly it sounds like you're a spammer er opt in marketing company. Your looking to startup a paid email service, what sets you apart from the market?
No sir I dont like it.
My guess is that the problem lies in the fact that the OP is using a garbage TLD. I've configured our mail server to silently drop all traffic from many of the new garbage TLDs, including .xyz. It does wonders for cutting down the spam levels. Sadly it's just a new version of Whack-a-Mole. Neither I, nor any of my users, appear to have gotten a legitimate email from any other these domains. I'll bet if the OP were to use a more traditional TLD, like .com, .uk, etc. there wouldn't be problems.
I have continuously run my own email server since around 1990 in one form or another. Established a vanity domain in the mid 1990's and started hosting email on my own domain. I must say that has been a more difficult task as time has gone on and has required I be more savvy about IP reputation and how to maintain it. Sometime last year I moved my email server from a VPS to a dedicated host and my wife began complaining over this past summer that she could no longer send email to Outlook.com and friends as well as Optonline. Given that she's a dog trainer trying to expand her client base this is pretty much a disaster for her. I attempted to work with the ISP hosting my dedicated server and they were not interested in fixing *their* IP reputation. Seems the above mentioned providers were blocking *all* of the IPS's IP addresses out of hand. They insisted they'd have to work with the ISP and while they were sympathetic they wanted to work the issues through the ISP. There's more to that I'm sure but after my ISP frustrating me by not being responsive I talked to some new folks. First question I asked before signing on was "how is your IP reputation?" After a lengthy explanation on how they have "high profile reputable clients" they assured me I'd have no problems. So I signed on the dotted line, installed a Puppet client on the box and set things up so the box would get configured as my mail exchanger. During shakedown I didn't notice problems right away so I edited DNS and pointed my MX records to the new box, added my SPF and DKIM records for the new host and powered off the old box. The very next day I composed an email to someone that I communicate via email on a fairly frequent basis and after hitting "send" got a bounce notification within minutes. Verizon was blocking the new server. New problem. It took many attempts and iterations I finally worked out between my new ISP (who graciously gave me a second IP address for outbound email) and Verizon I finally got whitelisted. So, yep, the Internet has become increasingly hostile to private email servers, but the problems can be worked out with some effort and tenacity.
Try out https://mailinabox.email/, a project I began a few years ago to make hosting your own mail much easier.
It includes comprehensive diagnostics to ensure everything is configured correctly, including reverse DNS, which is the most common issue that leads to mail not being deliverable / going into spam. This doesn't solve every problem, but lots of people have had good results with this project.
Simple way to boost your reputation is to simply configure a smarthost to send outgoing mail securely.
That boosts the smarthost's reputation, not yours, unless I'm missing something fundamental.
I run my own email server as well. But it's not as simple as an MX record. I use domainkeys and spf as well. None of the major services flag me as spam.
Agree. I run my own e-mail servers for a few domains and have no trouble at all. You need to be absolutely 100% sure that you aren't operating an open relay, or you'll be blacklisted immediately. You also need correctly configured STARTTLS with a valid certificate signed by a widely accepted root. Most relays will reject mail if STARTTLS is not used. Reverse DNS helps but isn't 100% essential. You want reverse DNS to resolve to something in the same domain. For example if people connect to the server as mail.domain.com but reverse DNS calls it srv1.domain.com that will be accepted by the vast majority of relays. If you want Google/Yahoo/Outlook to accept your mail you need DKIM signing, which involves generating key pairs, putting the public keys in DNS and configuring your mail server to sign messages. Correctly configured SPF improves your reputation, too.
So what you need is some means of sending large amounts of email to outlook.com addresses to build reputation.
A domain without information is untrusted.
SPF tells them that you're trying to combat spam from pretending to come from you.
Similarlt for DKIM, that also tells them that you are checking and explicitly marking every message you send out from your domain and absence of such signing should be treated as suspicious.
Put both of those on, in a decent static IP range (nobody sensible accepts email from dynamic IP's!), and you're good to go. How do I know? My own domains are ALL run by me, on Postfix. They even forward some mail addresses to providers like GMail as a matter of course.
The only problem I ever have for delivery is when *I* have accepted a spam message and try to forward it on to someone like GMail (harder to stop than it sounds, even with greylisting, etc.). They spot spam that my system can't, even on a re-forwarder.
Hell, I IPv6'd my domain too. So long as you have valid PTR records for your reverse, places like GMail are perfectly happy with that. Never had a problem. (But if you can't set your reverse for your IPv6, there's a way to turn off using IPv6 and fallback to IPv4 just for GMail, etc. when using postfix - google it).
My entire email for the last 5 years at least has been self-hosted. I've been using tiny startup services for about 10-15 years before that without issue. If anything, I have significantly more issues with the big-brand provider we use as smarthost for the Exchange servers in work, which are routinely blacklisted for spam and I have to fallback to manual sending from our leased lines, than anything to do with my self-hosting personal email domains.
Just don't expect your no-name mailer on a dynamic range without even the simplest of anti-spam measures to be accepted by places like Google, and you're golden.
Anonymous Coward wrote:
your issue is likely that you havent set up your dns security records and spf correctly
To which "dns security records" do you refer?
Most likely he's referring to the fact that The blog site identifies the person doing the "anonymous" complaining as Jody Ribton, and if we look through the cached articles where he's talking about setting up his mail server, we see he's calling his service "GeekMail", and he's futze uf the PTR record such that it doesn't match the SMTP banner:
host geekmail.io ... ...
Non-authoritative answer:
Name: mail.geekmail.io
Address: 139.162.197.129
host 139.162.197.129
129.197.162.139.in-addr.arpa domain name pointer geekmail.io.
Notice that the reverse record is not pointing to a reverse name of "mail.geekmail.io", but is instead pointing to "geekmail.io".
So his forward and reverse records do not match.
Further, looking up his IP address: http://www.anti-abuse.org/mult...
We can see that he isn't being RBL'ed, so it's just that he's screwed the pooch on his DNS setup.
I'd check the rest of the setup, but it's "game over" because of the inaddr.arpa entry being wrong.
Only thing he does not mention and I suspect is, he's behind a residential DSL/cable line and that is problematic nowadays. My server is at a VPS provider. Those do cost little and work acceptably well.
*Excellent point*. I didn't thought of that.
Home Internet provider's IP are probably blacklisted by default.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
Remember the wild west, when you could just pull off the Oregon Trail, build yourself a shack, and call it home? Nobody told you how to build your house, or how big your yard could be. But when you had a visit from a thief, there was no police to call, and if you had a fire, you lost everything. It was up to you to defend your own life at all times.
Sure, life might have been simpler back then. But who would want to go back there?
The Internet is the same story. In the good old days, everything was free for the taking, but it was the wild west. Now the city slickers want to put up fences, and the cowboys want to tear them down. Whether we like it or not, the Internet is changing, becoming more regulated, and some people aren't going to be happy about it.