Slashdot Mirror
Yahoo DMARC Implementation Breaks Most Mailing Lists
pdclarry writes: "On April 8, Yahoo implemented a new DMARC policy that essentially bars any Yahoo user from accessing mailing lists hosted anywhere except on Yahoo and Google. While Yahoo is the initiator, it also affects Comcast, AT&T, Rogers, SBCGlobal, and several other ISPs. Internet Engineering Council expert John R. Levine, a specialist in email infrastructure and spam filtering, said, 'Yahoo breaks every mailing list in the world including the IETF's' on the Internet Engineering Task Force (IETF) list.
DMARC (Domain-based Message Authentication, Reporting & Conformance) is a two-year-old proposed standard previously discussed on Slashdot that is intended to curb email abuse, including spoofing and phishing. Unfortunately, as implemented by Yahoo, it claims most mailing list users as collateral damage. Messages posted to mailing lists (including listserv, mailman, majordomo, etc) by Yahoo subscribers are blocked when the list forwards them to other Yahoo (and other participating ISPs) subscribers. List members not using Yahoo or its partners are not affected and will receive posts from Yahoo users. Posts from non-Yahoo users are delivered to Yahoo members. So essentially those suffering the most are Yahoo's (and Comcast's, and AT&T's, etc) own customers. The Hacker News has details about why DMARC has this effect on mailing lists. Their best proposed solution is to ban Yahoo email users from mailing lists and encourage them to switch to other ISPs. Unfortunately, it isn't just Yahoo, although they are getting the most attention."
DMARC (Domain-based Message Authentication, Reporting & Conformance) is a two-year-old proposed standard previously discussed on Slashdot that is intended to curb email abuse, including spoofing and phishing. Unfortunately, as implemented by Yahoo, it claims most mailing list users as collateral damage. Messages posted to mailing lists (including listserv, mailman, majordomo, etc) by Yahoo subscribers are blocked when the list forwards them to other Yahoo (and other participating ISPs) subscribers. List members not using Yahoo or its partners are not affected and will receive posts from Yahoo users. Posts from non-Yahoo users are delivered to Yahoo members. So essentially those suffering the most are Yahoo's (and Comcast's, and AT&T's, etc) own customers. The Hacker News has details about why DMARC has this effect on mailing lists. Their best proposed solution is to ban Yahoo email users from mailing lists and encourage them to switch to other ISPs. Unfortunately, it isn't just Yahoo, although they are getting the most attention."
DMARC and SMTP at Yahoo, mail broken.
Implementing SPF can also do the same thing, the issue is that mailing lists don't rewrite the from headers so despite having been forwarded through the mailing list server the original sender is still shown in the headers, only the mailing list server isnt really supposed to be sending mail *from* other people's addresses...
So either you allow mail to come from anywhere with any sender address, which lets mailing lists and email forwarding work fine but also makes spoofed spam very easy...
Or you don't, and break the above...
Really legit mailing lists should be rewriting the sender headers to reflect that the mail has been redelivered by the mailing list, the only difficulty this would cause is when users try to reply directly to messages rather than forwarding their replies to the list itself.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
I really hate how everything forces you to use text messages. I do not want text messaging and refuse to add it to my contract. If someone has something important to talk to me about, well, they already have my phone number.
Microsoft does the same for Hotmail/Live/Outlook. They claim suspicious use of your account was detected, and that to return access to you, you must change password, with a supplied phone number for secondary account control.
Bullshit. I had this happen across 5 MS hosted mail accounts in the same week - each were purpose-specific accounts to legitimately isolate commercial activity.
Google? The bastards try to wheedle your mobile number out of you at every PW change or update. They practically hide the UI to bypass this request.
Needles to say, all three are used only as "burner" addresses, now.
"Flyin' in just a sweet place,
Never been known to fail..."
It's not blocking relayed mail in the usual sense. Most mailing lists use the original poster's email address as the FROM field so everyone on the list knows who posted the message. The SENDER field contains the actual list address. And that should match the sending server's IP address. So reverse DNS and SPF (and DKIM if enabled) will validate the SENDER as the list server software. The REPLY TO will be either the list or the original poster, depending on list policy. DMARC requires that the FROM field also match the sending server, and ignores SPF and DKIM.
What the #%^+? Since when is Yahoo an ISP?
Several ISPs outsource their customer email service to Yahoo. If you're with one of those, and especially if you use your ISP provided email address, then moving would fix it (or just move to gmail/outlook.com/whatever, you're mail is in the cloud now anyway, since your ISP moved it there)
I don't know if they still do, but AT&T DSL customers used Yahoo mail as recently as last year.
-- Sent from a computer.
Back when the Internet Mail Consortium was a thing, we established best common practices for mailing lists, and most of them were vehemently against mailing list servers rewriting mail headers. Some popular MLM software rewrites standard headers, which breaks DMARC SPF implementations.
The thing to do here is to fix the MLM software to use the correct additional headers, rather than rewriting the headers the DMARC policy feels are important; in addition, this would allow the DMARC policy to "whitelist" based on the attached headers, assuming everything else wasn't a black mark, and avoid the "greylisting" that would happen ordinarily with most SPAM filtering systems in "medium posture" rather than "low posture" (i.e. the ones that have the concept of "suspect email" as a middle ground).
The idea that this "breaks all the IETF mailing lists" is basically alarmist BS - the IETF mailing lists are run on an individual basis, they aren't all hosted on a single machine out there, which is why they have varying degrees of SPAM and signal/noise ratios. So to claim that e.g. Namedroppers (the IETF DNS Working Group) mailing list server is impacted the same way the one Levin is all upset about is, is disingenuous.
On the other hand, I would be almost 100% happy if I didn't have the voice part of my smartphone.
The thing to do here is to fix the MLM software to use the correct additional headers, rather than rewriting the headers the DMARC policy feels are important; in addition, this would allow the DMARC policy to "whitelist" based on the attached headers, assuming everything else wasn't a black mark, and avoid the "greylisting" that would happen ordinarily with most SPAM filtering systems in "medium posture" rather than "low posture" (i.e. the ones that have the concept of "suspect email" as a middle ground).
I think you will find that most MLM software uses correct additional headers. At least listserv and mailman (for the lists that I manage) do. We've been playing nicely with ISPs for years on our lists, we create no spam (once we fixed the bounceback spam problem 3 years ago) and generally are among the more well-behaved email users around. The problem is that Yahoo's implementation of DMARC is not using the additional headers. All it looks at is From.
I think you will find that most MLM software uses correct additional headers. At least listserv and mailman (for the lists that I manage) do. We've been playing nicely with ISPs for years on our lists, we create no spam (once we fixed the bounceback spam problem 3 years ago) and generally are among the more well-behaved email users around. The problem is that Yahoo's implementation of DMARC is not using the additional headers. All it looks at is From.
Not a problem, if you leave the "From:" line the hell alone, and only add new headers, per RFC 5322, and RFC 2919, etc.. It can look at the From line all it wants, and as far as it's concerned, as long as the rest of the headers are unadulterated, your list server is an intermediate relay server in the SMTP routing path.
..leads me to have sympathy for Yahoo. Over a decade ago, I was partly in charge of maintaining the mailing lists at the IETF Secretariat - so I remember what volume of email they were working with in 2001, and I would never want to manage a mailing list that big again (certainly not in 2014). In hind-sight it wasn't so bad then, I recall about 47,800 messages in 4 days @ roughly 85% spam for the whole IETF mailing list, but that was in 2001. We had to implement anti-spam filters for lists of people with very strong opinions regarding censorship, and rightly so - but present yourself with the thought of handling filters for the "Anti-Spam Research Group" mailing list. ..bweheheh .. heh ..pwfff. useless. Spam quickly discovered it could spam more easily through the anti-spam email list. ..so many penis pills.
.".
..but it's been 10 years, and so far email hasn't totally collapsed. Time will tell.
I believe it just isn't possible to fix the spam problem in email as it currently exists. All is not lost, because auxiliary communications (phonecalls, texting, Twitter, Dropbox, Facebook, Skype, etc) are better suited for specific types of communication and are self-partitioning. Email is often just as boring and disappointing as physical mail - mostly advertising junk. Because it is based on physical mail, we can't really complain - it's doing exactly what we designed it to do.
The digital world treats bots and brains the same. Captcha was useful for a little while, but seems to be meaningless these days. These days, if I have a form that's getting spammed I use interactive JavaScript operations (mostly option selections) to create the html form and omit a submit-typed button. That way it takes a real person looking at the page to figure out where the "send" action is.
Fundamentally, the problem with the current SMTP infrastructure is that it is based on Recipient-liability without any real Sender-liability. It is the recipient's responsibility to have some gargantuan "put junk here" box instead of a reasonably-small tray for other's to say: "I have something for you, encrypted with this secret key, find it on my server here __
That would handle the storage penalty (the message is waiting in their outbox or application, sent to your inbox only when you choose to accept it). If the message is SO important, and you're REALLY who you say you are, then I can get back to you when I want to read/download your message - making the sender easier to authenticate. And both parties would know when the message has been received, or if the message has been read before the intended recipient chose to accept it.
Sounds simple, but for some of us dropping all mail to/from Yahoo groups isn't feasible. Of six mailing lists that I receive messages from semi-regularly for work related purposes, three of them are on Yahoo groups. All are specific to niche software packages and for two of the three at least represent THE primary source of "support" for those packages. Taking your suggestion wouldn't necessarily be career suicide, but could hurt me.
Having said that, I receive those mailing lists at my work address and not through my Yahoo email account, so the DMARC issue isn't likely to affect me too much. I could probably move my personal mail from the latter, but a) inertia and b) I find the newest incarnations of Gmail not enough better than the abomination the Yahoo mail has become to force the issue.