Slashdot Mirror

My e-mail, my wife's, and the ex co-worker I share the server with all have our e-mail greylisted. I have it set up so that it skips the greylisting process if the e-mail server it receives mail from is properly listed using SPF, which helps make sure that e-mails from large entities (GMail for instance) are never delayed. Nonetheless, I'll hear occasional complaints from the wife when she signs up for an account at a new set of forums or something and doesn't receive her confirmation e-mail immediately.

I think it works best on an individual basis, really. You could let everyone in the domain know that there's an option available which would help cut down on spam but might occasionally delay e-mails. For some people this will be completely unacceptable, but others will jump at the chance to reduce spam.

I'm reasonably knowledgeable about working with CPanel (all I have to work with on a a reseller account. But it feels as though I need to RTFM for SMTP in order to decipher how to use SPF when most my domain's email originates via my home DSL (covad.net) and must be sent via smtpauth.earthlink.net as they filter port 25.

What would be a much better solution all the way around IMHO is if servers were set up only to generate bounce messages to local users, and if people would STOP using challenge-response systems to try and combat spam--they only create more spam for everyone else!

The moron moderator who rated "Domain owners: Set up SPF NOW!!!" as offtopic needs to get a clue. SPF: Sender Policy Framework is used so you can filter out forged mail. The recent flood of stock-pumping spam used many forged domains in the "from", and if you filtered on SPF, you wouldn't have seen as much spam.

I might add, it would be nice for people to REJECT spam rather than BOUNCE it. When you bounce it, innocent domains get an email complaining about the forged email. With these spambots, it adds up quick! Doing a reject also allows legitimate senders to discover their email was not delivered.

"Executive summary

SPF and Sender ID are not the same. They differ in what they validate and what "layer" of the e-mail system they are concerned with. Sender ID is not the latest version of SPF - it is a new and independent experiment. The "spf2.0" tag name is a historical accident. Neither is better because they address different problems. There is controversy because Sender ID is incompatible with existing specifications. Microsoft is aware of the problem and representatives of theirs have stated that they have no plans to fix it. There are practical work-arounds for SPF and Sender ID users."

"The Sender ID specification contains a recommendation to use SPF's v=spf1 policies -- which are originally defined to apply to MAIL FROM and HELO identities only -- and apply them to the PRA identity as well. Specifically, it says to consider v=spf1 as equivalent to spf2.0/mfrom,pra. This is technically wrong, as is explained in detail below. Sender ID implementors should correct this and treat v=spf1 records as equivalent to spf2.0/mfrom. Unfortunately this mistake in the Sender ID specification was not corrected prior to its publication despite an appeal from the SPF project."

Sorry to disappoint but SPF has nothing to do with spam, spam has traditionally forged the sender address and widespread SPF adoption would put an end to that.

I've also no idea why that old SPF web site is still up. Try here instead

This event may boost other spam fighting solutions like SPF.

In my opinion, it's a much better long term solution.

For more info see: http://www.openspf.org/

I also do a very quick check on a few sender domains such as yahoo.com A "from" yahoo.com must be from a machine with .yahoo.com. If not, it is rejected.

If by that you mean that you're using SPF or similar, that's a great idea. If it means that you wrote your own code that does something like "if fromdomain != helodomain: reject()", then your system is pathologically broken and needs to be updated.

You have just described SPF (Sender Policy Framework).

Check out OpenSPF for more details.

I'm sorry. When I think of system I think of daemons. Improvements to the DNS system would be appreciated. Someone to provide me with commercialized redirections and pay per use DNS service doesn't equate to improvement.

Sites providing free email without protecting their URIz with spf protection is what needs to be fixed. This would help to kill spammers pretending to be google, yahoo, aol, et al.

For a real improvement in DNS use spf http://www.openspf.org/ and urge others to use it too.

The main advantage appears to be that they will prevent you from opening known phising sites. In terms of being faster, I'm not sure how they would be faster than my ISP since my ISP's DNS servers are presumably much closer to my machine than theirs. Any idea how they could make claims like that? Also, though the summary mentions foiling spammers, I saw nothing about that in the article. From the sound of the post, I thought this was something like SPF even though that doesnt seem to be the case at all.

I've had this issue with Hotmail and AOL users. Once I put in the SPF record in the DNS, all mail went through.
http://www.openspf.org/

My domain has a SPF record and I never had issues sending email to anyone on hotmail or other services.

See:
http://www.microsoft.com/mscorp/safety/content/tec hnologies/senderid/wizard/

&
http://openspf.org/wizard.html

Lean on everyone you know to implement the "sender policy framework" (its just an entry added to the DNS entries that state what the valid mail servers are for that domain). Once enough people do this, we will be able to start just dropping emails that don't pass the sender policy check (which will be all those from bots).

The secondary spam is known as a form of backscatter. Hopefully the DNS provider for the domain part of your email address will allow publishing an spf1 TXT record indicating the legit sources a message claiming to be from that domain or else the message is treated as a forgery and dropped (assuming the record ends with the -all hardfail mechanism rather than ?all or ~all). See http://openspf.org/ and http://openspf.org/mechanisms.html

The secondary spam is known as a form of backscatter. Hopefully the DNS provider for the domain part of your email address will allow publishing an spf1 TXT record indicating the legit sources a message claiming to be from that domain or else the message is treated as a forgery and dropped (assuming the record ends with the -all hardfail mechanism rather than ?all or ~all). See http://openspf.org/ and http://openspf.org/mechanisms.html

Three words: Sender Policy Framework :)

If the email came from a server authorized to send for that domain, no problems, otherwise into the trash.

I know this isn't the final answer, but to me it is by far the most responsible and far reaching.

• No cost. You already have DNS servers for your MX record if you are a valid server.
• Using DNS means that we already have a great infrastructure.
• Doesn't stop emails from people like amazon.com if you want them, but adding @amazon.com to your block list is now valid.
• Faster and more reliable then content filtering.
• Makes phising a bit harder, as you can no longer send support@citigroup.com.

Will spammers register real domains, yes. Will they send emails with a fake from address that has at least a valid domain, yes. It makes it just that much harder, and makes it harder to use farms. If the SPF record has a huge subnet then the spam blockers can ignore it, and then put it on a watch list. At least we are adding some level of authentication to the process.

The cost of SPF is so little, I don't understand why their is not more push for it, and why we can't just give it a shot. I'd rather do that then go thru some authentication process with a company and then pay for some type of certicificate. Lastly, as a programmer I hate when all of the suden we have to do quadruple opt-outs, when the real problem is people sending gobs of rolex adds from their dorm room with or without their knowledge.

I've been through the same thing - I feel your pain. A couple of points. I found that adding SPF records to my domain helped somewhat. Secondly, it stopped in the end. In my case I now get no bounces (and no spam, thanks to greylisting) - all in all I was being swamped for probably 2 months.

However, SPF would give you the ability to blacklist/whitelist domains instead of IPs - domains change less often. You could whitelist yahoo.com and not have to update the database every time yahoo adds a server to their cluster.

I don't follow; can SPF really whitelist domains? Isn't the point of the record that domain names can't be trusted to be not spoofed/forged? I know SFP1 has several mechanisms such as the MX and A records, but I'm pretty sure those are resolved to their IP address so the SPF-checking application is comparing apples to apples, no?

http://openspf.org/mechanisms.html

Somebody like spamhous could run a DNS-based database that determines whether a domain is spammy, non-spammy, or unknown based on complaint data (much as they currently do for IPs now).

Such as IronPort's SenderBase.com Also, for domains contained within the message body, there are SURBL.org and URIBL.com

Individuals could obviously make up their own minds regarding unknown domains, but if a domain has a reputation for not sending spam, why block it just because they don't own a T1?

Yeah, I wouldn't block merely if the IP is dynamic, but I would consider scoring it spammy as long as that score alone doesn't reach the spam block or spam redirect thresholds.

One of my systems handles > 40K messages per day, and I can attest that Verizon email recently became a problem. I added an SPF record, and that instantly solved the problem. (I also filled out the online Verizon form, but haven't heard back yet). Summary: try SPF. http://www.openspf.org/wizard.html

There's a far more effective, far more efficient scheme against phishing and joe-jobs already in place: it's called SPF, it doesn't cost a cent, and it allows domains to list those hosts or domains allowed to send email allegedly from that domain. It helps cut worm traffic incredibly by catching forged email from your own domain sent from non-domain members, and by simply assuming that all mail from a domain should use the basic "only from A records or MX records" SPF rules, it provides a very powerful and cheap to implement filter rule.

Better yet, it acts on the first connection from the spammer and blocks the email before it wasts your time and bandwidth loading up the message. It was polluted by Microsoft trying to staple their own special form of "allow me to spam" signature, but SPF version 1 is still alive and kicking at http://www.openspf.org/

If I'm expecting emails from my bank, I'll be putting them on my safelist anyway!

And I can tell you that everyday we receive dozens of automated emails asking us to click a link to verify that we are human beings and not a spam bot.
So good for you if you manually manage your safelist, but other people don't bother with it.

That said, the idea of certified email to fight spam to some level is not a bad idea, afterall, that what other people have been trying to do and they were welcomed (spf). However, I'm not too hot on them charging for it because those who can't afford to pay may become second class citizens.

I work for a financial services company who has a clients who are supposed to receive emails from us related to trades. Since I manage our web presence, email deliverability is also my problem.

Here are the places to start:

Free Certification
AOL: http://postmaster.aol.com/whitelist/
Yahoo: http://add.yahoo.com/fast/help/us/mail/cgi_bulkmai l
Verizon: http://www2.verizon.net/micro/whitelist/request_fo rm.asp?id=isp

Reporting
Spamcop: http://www.spamcop.net/w3m?action=ispsignupform
Hotmail: http://postmaster.msn.com/snds/
Senderbase: http://www.senderbase.org/

Email Signing
SPF: http://www.openspf.org/
DomainKeys: http://domainkeys.sourceforge.net/

Paid Certification
Bonded Sender: http://www.bondedsender.com/
Habeas: http://www.habeas.com/
Goodmail: http://www.goodmailsystems.com/

A lot of providers outside the US have many of their own rules and regulations to follow, which makes it quite difficult to achieve deliverability. At the end of the day, we try to follow all the rules that have been laid out from existing companies and then deal with individual providers on a needs basis. The more users that use that ISP, the more we are willing to obey their individual rules.

Unfortunately, I see paid certification becoming the way of the future. If I can pay to guarantee to have my clients email delivered rather then negotiate with ISPs every other week based on their varying criteria, I'm pretty sure my company will pay for it. I don't like it, but results are the bottom line.

I think you're referring to spam as the consequence?

Well, the reason mail is the way it became is that a few universities, defense contractors, and government organizations needed to communicate, and given the reliability of network equipment of the time, open relays were a necessity to ensure that email got through. The reason that something along the lines of SPF didn't come into play from the beginning is multifold; DNS wasn't around (hosts were maintained in host files at each site), every organization on ARPANET was 100% trusted, and there was no incentive to forge emails nor to do what we now call "spamming" - in fact the few early advertisements which went out in targeted emails were heavily criticized.

When ARPANET became the Internet and DNS came into being due to the volume of hosts going online, open relays were still the standard, not due to network reliability (which had significantly improved) but due to legacy support. To maintain backwards compatibility SMTP stayed pretty much as-is from day one, and with the harsh criticisms that followed early email advertisemtns from trusted organizations, no one really anticipated a number of things:

  - Internet access becoming a commodity (Quantum Link and Compuserve were just coming into their own then, and dial-up to proprietary online services was the wave of the future beyond private BBSes)
  - Everyone having multiple, multiple email addresses
  - Commercial entities abusing the network

In hindsight it was quite obvious that things like SPF would be required but given the Internet's early history (and computer networking in general) it's clear why they didn't think of security and sender verification when first implementing an email solution.

What AOL, Hotmail, and others SHOULD do is not use that GoodMail crap (it's not good sense to do that!) but to make SPF required rather than optional. If you want to send email to AOL recipients, on your authoritative servers, you must list which hosts are actually allowed to send emails from your domain via an SPF record, and all emails from your host not meeting the SPF rules will be regarded as spam and not even make it to the receiver's inbox.

This puts the onus totally on the senders. Want your mailing lists to make it through to the receiver? Make sure your listserver is listed in your SPF rules.

This is why SPF was proposed in the first place; to overcome issues arising from legacy support, to work around open relay-originating spam without having to block legitimate email from open relays, and to avoid the need for whitelisting.

Want to learn more about SPF? Check out http://www.openspf.org/

Posting this reminds me: I need to update our SPF records. Oops! :-/

SPF assumes all email to be spam unless proven otherwise.. seems to reduce it by the ton from what I've seen. We should have more implementation of this.

"Microsoft never have had much to do with standards, other than to completely ignore them and create their own stuff regardless."

I am not so sure about that. They made a fine friggin mess of the SPF standard by introducing patents on several key parts of the standard while delaying and filibustering until the IETF working group (MARID) became defunct as a result. I am sure I could find other examples of MS strong-arming, delaying, and otherwise being a general pain in the ass to standards bodies.

SenderID is an attempt by Microsoft to hijack a working open standard called SPF. At this point it is effectively dead because of Microsoft's cynical manipulation of Meng Wong's altruistic attempt to help everyone.

You will note I'm not normally a MS-basher, but in this case it's well deserved. SPF was ramping up into a system that would make email forgery impractical for spammers and virii, but Microsoft (with help from Yahoo and AOL, I guess) muddied the waters to the point where the anti-forgery community couldn't get a clear message out. Now SPF is still going, but very slowly, which is a shame since it is a practical thing you can do today that makes a real difference. If comcast (for one example) took the five freakin' minutes that would be required to publish SPF in their DNS, the world would be a better place for it.

Implement SPF. Laugh at the rotting corpse of SenderID.

SenderID is an attempt by Microsoft to hijack a working open standard called SPF. At this point it is effectively dead because of Microsoft's cynical manipulation of Meng Wong's altruistic attempt to help everyone.

You will note I'm not normally a MS-basher, but in this case it's well deserved. SPF was ramping up into a system that would make email forgery impractical for spammers and virii, but Microsoft (with help from Yahoo and AOL, I guess) muddied the waters to the point where the anti-forgery community couldn't get a clear message out. Now SPF is still going, but very slowly, which is a shame since it is a practical thing you can do today that makes a real difference. If comcast (for one example) took the five freakin' minutes that would be required to publish SPF in their DNS, the world would be a better place for it.

Implement SPF. Laugh at the rotting corpse of SenderID.

Well, if you ask me IPv6 sounds like a potential golden oppurtunity to enforce use of systems such as DomainKeys or SPF. Seeing as everyone would have to recode their mail server software to work with IPv6, they may as well make sure it supports those sender validation techonologies, and seeing as everyone is going to have to do it around the same time, it sounds like it would be a good idea to make it compulsary on IPv6 rather than optional, this means that spammers will be unable to forge their from addresses over IPv6. Of course nothing stops them from registering lots of domain, but it sure makes considerable more work for the spammer.

IMO part of the reason why "from header" forgery is so prevalent is that either an administrator's mail server can't utilize this technology by default, or they have no inclination to do so, if it becomes necessary, maybe people will get off their asses and do the required work.

Sender policy framework is a system to prevent fake sender address in emails. it works by checking the claimed sender domain, in the email, against a TXT record in the DNS system. The TXT record contains information of ip's or hostnames, allowed to send email on behalf of the domain in question.
If the email have a faked sender address it can be bounced or labeled suspicious.

This works amazingly well, and stops all faked sender emails before it's accepted in the server. Effectivly blocking virus and spam sent with forged addresses. Non exsisting domains are allready blocked in the mail servers so if everyone owning a domain was to implement this. It would make me a very happy person. Ofcouse spammers can still send email from domains under their own control, but those go into online blacklists fairly quickly

Unfortunatly it does not have the widest accept yet, but growing all the time. After hotmail implemented it in their DNS records, spam is at an all time low around here. Not getting a single spam email from faked hotmail addresses in ages.
And only 6 months ago I had a dedicated "sent from hotmail" folder since it was 99% likly to be spam anyway...

sepski

Sender policy framework is a system to prevent fake sender address in emails. it works by checking the claimed sender domain, in the email, against a TXT record in the DNS system. The TXT record contains information of ip's or hostnames, allowed to send email on behalf of the domain in question.
If the email have a faked sender address it can be bounced or labeled suspicious.

This works amazingly well, and stops all faked sender emails before it's accepted in the server. Effectivly blocking virus and spam sent with forged addresses. Non exsisting domains are allready blocked in the mail servers so if everyone owning a domain was to implement this. It would make me a very happy person. Ofcouse spammers can still send email from domains under their own control, but those go into online blacklists fairly quickly

Unfortunatly it does not have the widest accept yet, but growing all the time. After hotmail implemented it in their DNS records, spam is at an all time low around here. Not getting a single spam email from faked hotmail addresses in ages.
And only 6 months ago I had a dedicated "sent from hotmail" folder since it was 99% likly to be spam anyway...

sepski

Sender policy framework is a system to prevent fake sender address in emails. it works by checking the claimed sender domain, in the email, against a TXT record in the DNS system. The TXT record contains information of ip's or hostnames, allowed to send email on behalf of the domain in question.
If the email have a faked sender address it can be bounced or labeled suspicious.

This works amazingly well, and stops all faked sender emails before it's accepted in the server. Effectivly blocking virus and spam sent with forged addresses. Non exsisting domains are allready blocked in the mail servers so if everyone owning a domain was to implement this. It would make me a very happy person. Ofcouse spammers can still send email from domains under their own control, but those go into online blacklists fairly quickly

Unfortunatly it does not have the widest accept yet, but growing all the time. After hotmail implemented it in their DNS records, spam is at an all time low around here. Not getting a single spam email from faked hotmail addresses in ages.
And only 6 months ago I had a dedicated "sent from hotmail" folder since it was 99% likly to be spam anyway...

sepski

It's ironic that in setting out to 'solve' spam, Microsoft all but destroyed the momentum around SPF, fracturing it into several different, incompatible implementations.

They've implemented the Sender Policy Framework which was a real pain in the ass for me as well. All AOL and compuserve addresses were affected by it. Instead of a normal bounce message, they did put in the RFC that was violated so I gotta hand it to them that they did explain why the message was bounced.
If you put in the SPF record in your DNS, AOL will accept your mail.
You can find out more about it here: http://www.openspf.org/