Slashdot Mirror


The Hostile Email Landscape (liminality.xyz)

An anonymous reader writes: As we consolidate on just a few major email services, it becomes more and more difficult to launch your own mail server. From the article: "Email perfectly embodies the spirit of the internet: independent mail hosts exchanging messages, no host more or less important than any other. Joining the network is as easy as installing Sendmail and slapping on an MX record. At least, that used to be the case. If you were to launch a new mail server right now, many networks would simply refuse to speak to you. The problem: reputation. ... Earlier this year I moved my personal email from Google Apps to a self-hosted server, with hopes of launching a paid mail service à la Fastmail on the same infrastructure. ... I had no issues sending to other servers running Postfix or Exim; SpamAssassin happily gave me a 0.0 score, but most big services and corporate mail servers were rejecting my mail, or flagging it as spam: Outlook.com accepted my email, but discarded it. GMail flagged me as spam. MimeCast put my mail into a perpetual greylist. Corporate networks using Microsoft's Online Exchange Protection bounced my mail."

22 of 217 comments (clear)

  1. Don't Know How You Made That Conclusion by 0xG · · Score: 4, Informative

    I run a small email system ~2500 users and don't have your problems...

    --
    A pox on web designers who feel that window.innerWidth == screen.availWidth
    1. Re:Don't Know How You Made That Conclusion by beelsebob · · Score: 5, Informative

      More likely, the original poster simply has his DNS misconfigured in some weird way, and doesn't know it.

    2. Re:Don't Know How You Made That Conclusion by Anonymous Coward · · Score: 5, Informative

      I second that emotion. Current *big* players are trying to limit spam and phishing, and require a few ducks in a row before you stop getting caught in their filters. I suspect proper analysis of the configurations and logs would pinpoint the issue. DNS would be a quick start but the problem could be in a few places depending on what mail implementation he's using. On another note, is it possible OPs domain has been used for spam/phishing in the past? The UNI I work has dealt with blacklists in the past and it was merely a case of spoofing and those adding us to blacklists didn't do their diligence in tacking it down properly. *Posted anon as to not get fired*

    3. Re:Don't Know How You Made That Conclusion by acoustix · · Score: 4, Informative

      I run a small email system ~2500 users and don't have your problems...

      You probably have a dedicated/static IP and it isn't tainted from others who have used it before you.

      For people trying to run their own email server at home it can be a real pain. ISP's blocking 25 and 587. DHCP means that your IP pool has a bad reputation. Etc...

      --
      "A plan fiendishly clever in its intricacies"- Homer Simpson
    4. Re:Don't Know How You Made That Conclusion by Z00L00K · · Score: 5, Insightful

      It's usually the case when the reverse lookup don't point back to the same domain/name as the server identifies itself with.

      And it's the ISP that need to change the pointer from some generic name to a specific.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    5. Re: Don't Know How You Made That Conclusion by Anonymous Coward · · Score: 4, Informative

      Probably no SPF or TXT records

    6. Re:Don't Know How You Made That Conclusion by MyFirstNameIsPaul · · Score: 4, Informative

      You need to go to their stupid new Postmaster service and 'fix' the 'issues'. I observed the exact same behavior for mail servers that hadn't changed a DNS record or even IP address in years roughly around the same time they launched this new 'service'. Coincidence? I think not.

      --

      I once took an excursion to Reddit, and later HN. Unlimited up/down voting sucks when dealing with a hive-mind.

    7. Re:Don't Know How You Made That Conclusion by Anonymous Coward · · Score: 5, Informative

      +1
      Rejections in my experience have nearly always always been related to the PTR record needs to be pointing to the domain actually sending the email, not the domain name in the email address. My limited understanding is this:

      So if my email address matt@example.com uses mail.isp.com on port 25 to send email then the PTR needs for the ip address isp,com sends from needs to say mail.isp.com... not example.com as you might expect.

      when isp.com talks to another smtp server it will be asked to id itself. The server should reply with its FQDN and it is this that the PTR record for the servers id needs to point to . Even if that server hosts hundreds of websites and email accounts.

      I believe most VPS hosts allow this to be changed to whatever you want if you are given a fixed ip address. If they don't allow this to be changed then problems will occur and if you are handling emails you need to check before signing up. The PTR record is not applicable to a domain but to an IP address. You can only have one PTR record for an IP address.

      That is if my memory serves correctly. When I set up email servers, I always seem to forget this until I do sending tests to yahoo and other big boys. Then I set it properly and things behave.

      Other problems happen if using microsoft exchange and the srv fields in txt records for the dns are not set exactly right. Though I don't have to fiddle with this for obvious reasons.

       

    8. Re: Don't Know How You Made That Conclusion by slasher999 · · Score: 4, Informative

      Missing spf records were the first thing I thought of as well. That isn't a silver bullet by any means but can certainly help your ratings while you are new and building a reputation.

    9. Re: Don't Know How You Made That Conclusion by MightyMartian · · Score: 4, Informative

      Missing SPF and possibility of being on one of the RBLs. I had that problem when we switched to a new ISP, and the address block we were given had ended up on Spamcop. It took a bit of doing, but within a day it was cleared up.

      --
      The world's burning. Moped Jesus spotted on I50. Details at 11.
    10. Re: Don't Know How You Made That Conclusion by alphatel · · Score: 4, Insightful

      Missing spf records were the first thing I thought of as well. That isn't a silver bullet by any means but can certainly help your ratings while you are new and building a reputation.

      If his domain is the incredibly stupid http://liminality.xyz/ then yes, he is missing SPF records. Use mxtoolbox.com to check.

      --
      When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
    11. Re:Don't Know How You Made That Conclusion by Anonymous Coward · · Score: 4, Insightful

      He's doing it wrong. Most probably he's not using SPF nor signing with domainkeys. That's expected today by most providers.

      If he's especially naive he's operating an open relay, which will warrant him to be blacklisted FAST.

      Another cause is, he could be operating his mail server from a "dialup" IP range, one declared as being assaigned to residential connectivity, which are usually blacklisted. I disagree with this practice, but that's how things go.

      Also most providers now require TLS support. So you need to generate certificates(self signed is not enough, but your own unofficial CA is enough usually, but make sure you're not using SHA1).

      Also, I happened to configure a mail server on a newly acquired IP from an hosting company a year ago or so and the IP they gave me was already tainted as being on a few blacklists. This can be solved too. I took the pain to discover which blacklists and followed their procedures to be taken out. Sometimes It was some automated procedure which just requested the server to be scanned again to make sure it follows best practices(as stated above). OOther times I had to politely ask and in one case even have the provider confirm the IP was actually reassigned.

      After this I have not seen a single email being rejected as spam.

      Operating mailservers could have been easy in the '80s and first half of the '90s when most mail server really were open relays and nobody cared, just because nobody was taking advantage of that. Nowadays it's become complicated because even the slightest misconfiguration will be attacked and exploited. It's in the general interest to request mail servers to be configured to a minimum standard that is getting relatively high, or we could really loose control of the email system.

    12. Re:Don't Know How You Made That Conclusion by Anonymous Coward · · Score: 5, Insightful

      There are several factors that I've seen with my mail server.

      1) Do not try to work over a standard ISP service - one that assigns your IP dynamically - because most blacklists and major corporations blacklist dynamic IP pools
      2) Don't host in any of those cheap virtual hosting services - many of them are also blacklisted
      2) Setup DKIM signing (sendmail config and DNS record)
      3) Setup SPF DNS record

      Basically, one has to avoid running one's mail server someplace that is cheap because that is where the SPAMers put their mail servers as well (because they are cheap and easier to do anonymously).

    13. Re: Don't Know How You Made That Conclusion by ale2011 · · Score: 4, Interesting

      The OP wrote "this server was configured perfectly: not on any blacklists, reverse DNS set up, SPF, DKIM and DMARC policies in place, etcetera." Perhaps he deleted SPF and DKIM records after he gave up? However, the domain is registered by Contact Privacy Inc. Customer 0141536996, which I wouldn't deem a good start for a mail domain. The IP belongs to LINODE, a German Linux hosting place, and seems to be static. Only one black list, rbl.rbldns.ru, has it, which shouldn't be a major problem, but may suggest that some email problems did happen. He didn't subscribe to DNSWL.ORG either.

      All that said, that conclusion is correct, IMHO. Microsoft in particular files all mail to the spam folder unless the sender is too big to block (TBTB). Even if I subscribed to their feedback loop, mail from an address they never saw, such as yyyy-mm-dd@my.example.com, is considered spam, no matter how many times the recipient whitelisted messages from the same domain.

  2. Re:Do your due dilligence... by unrtst · · Score: 4, Informative

    ..and set up SPF entries and reverse DNS. Also make sure Postfix is locked down and not acting as an open relay. It really is not that hard, this article comes off as whiny "I can't do it, so the world is against me" at best.

    Did you even read the article? There's not much more than the summary, but there he does make note that reverse DNS and SPF records, among other things, were setup:

    I've done this before, ...: not on any blacklists, reverse DNS set up, SPF, DKIM and DMARC policies in place, etcetera. (Side note: mail-tester.com and Port25 are great for checking your setup.)

    The near-conclusion quote is his real point:

    ...from Microsoft's Postmaster Troubleshooting page:

    IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, Outlook.com will typically allow for a better email delivery experience.

  3. I solved this very problem. by neilo_1701D · · Score: 4, Interesting

    I run my own mailserver, mostly "just because".

    The reputation problem I encountered early on was because of a lack of a reverse DNS entry. Easily fixed; I simply asked my VPS provider to create one.

    The next problem that started about 18 months ago was reputation: my little server simply wasn't a trusted service.

    Because of the (unbelievable) amount of spam hitting my server, I had taken out a Comodo AntiSpam Gateway subscription about two years earlier. It was initially free, but after a year or so they wanted money. Since the service rocks, I happily pay my ~$30 annually.

    What CASG also offers is outbound scanning: if I tell my server (an Exchange 2010 server) that the outbound smarthost is CASG, my email all of a sudden piggybacks Comodo's reputation. Voila, email flows without incident.

    Problem solved.

  4. Re:Do your due dilligence... by UnknowingFool · · Score: 4, Insightful

    IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, Outlook.com will typically allow for a better email delivery experience.

    Sounds like a Catch-22: "We won't accept accept email from a server until the new server until the server has successfully delivered lots of email."

    --
    Well, there's spam egg sausage and spam, that's not got much spam in it.
  5. Re:Welcome... by Anonymous Coward · · Score: 4, Insightful

    Just because privacy is hard, doesn't mean it's dead, nor does it mean it's a goal not worth striving for. Some things should be private. Just because you're comfortable doesn't mean everyone is, or should be.

  6. Echoes my experience by isj · · Score: 5, Interesting

    I've been running my own mailserver since 2003, and I have seen my share of problems.
    1: mailservers blocking mail based on spamhaus DUL. You can delist your IP. But still, blocking exclusively on that?
    2: hotmail.com accepting emails and then discarding them silently. No trace of them. No bounce. Recipient did not have it in their spam folder or anything. This was several years ago, so perhaps it's better now. But discarding emails after promising to deliver them without any possibility for the recipient to control it: bad idea.
    3: Various greylisting email servers. Not really a problem as my MTA will retry and the email is only delayed for a few minutes.
    4: gmail.com rejecting emails sent over IPv6 but happily accepting them over IPv4. It turned out to be a problem with their parsing of SPF records, and apparently fixed now. But I did find out that there is no reasonable way to contact the gmail team.
    5: outlook.com rejects emails due to FBLW15, whatever that means. It seems you can get whitelisted, but it appears that a lot of hosts are being hit by it for no reason.
    6: office365 bouncing emails due to "protection" with no explanation given, and direction to contact the recipient by other means to get whitelisted. This was for a the official email address listen on a company website. I decided that my email wasn't important enough. Their loss.

    Bottom line: If you run your own email server then expect to occasionally do some manual whitelisting etc. And expect some email servers to be uncooperative and/or RFC-clueless.

  7. Loose the .xyz TLD by JimMcc · · Score: 4, Informative

    My guess is that the problem lies in the fact that the OP is using a garbage TLD. I've configured our mail server to silently drop all traffic from many of the new garbage TLDs, including .xyz. It does wonders for cutting down the spam levels. Sadly it's just a new version of Whack-a-Mole. Neither I, nor any of my users, appear to have gotten a legitimate email from any other these domains. I'll bet if the OP were to use a more traditional TLD, like .com, .uk, etc. there wouldn't be problems.

  8. Re:Do your due dilligence... by SuricouRaven · · Score: 5, Funny

    So what you need is some means of sending large amounts of email to outlook.com addresses to build reputation.

  9. Re:Welcome... by Lisias · · Score: 4, Insightful

    I'll bite. What is in your email that you don't want Google knowing?

    My son's phone number, that is not Android and I don't want nobody out of the family to know. Just for starters.

    Better question - What is in your email that you think Google doesn't already know?

    Only Google knows, and this is exactly why it is a problem.

    Everyone with a smartphone complaining about privacy in 2015 has lost their mind. Privacy is dead. Get over it.

    Being this the reason you posted as an Anonymous Coward? :-)

    You don't know my bank account. You don't know my social security number. You don't know my personal phone number. And this is how things need to be.

    --
    Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org