Slashdot Mirror

Hey, you could have put the IP addresses of the local DSL lines in your DNS SPF records, because everybody uses SPF, right? *sigh*

You bet. I disagree with your idea, but that's not a person indictment. :)

Hell, Joe Schmoe could be the root authority.

The problem is, what happens when Joe Schmoe becomes glacially unresponsive or starts making heinous new policies to abuse his position of power? Once the system is entrenched, it may not be easy to simply switch the root to a new entity. For example, I don't know of anyone who doesn't dislike Verisign, but we still accept their authority because the cost of switching is more than we want to pay, or than we can expect others to pay.

Have you looked at the SPF anti-forgery protocol? Basically, it's a way to reject all mail that can't be proven to originate from the claimed sender. I am a big fan of this approach since it forcers spammers to use their own domain names for sending, and even though they're able to buy new ones at will, suddenly they're forced to eat operating expenses that they didn't have before SPF (assuming it catches on). It's not perfect, but it's certainly a start.

You know, your proposal is similar to a DNS blackhole list in reverse, where known-good domains are whitelisted instead of having known-bad domains blacklisted. You could implement such a system today, if you wanted, where you would register "good" mailservers and return "spam positive" answers for all queries for servers not listed in your database. Make your clients use SPF so that spammers can't forge email as coming from one of your whitelist entries and voila!, your own personal web of trust with you as the root.

First of all, IP addresses don't have MX records.

OK, so maybe what you really meant was to see if the domain name on the right side of the @-sign in the sender's email address has an MX record. That's really not useful because most spam uses forged addresses that do, and it's actually valid for a domain name not to have one if it has an A record pointing to the appropriate mail server.

What you want to know is whether the mail really comes from a server it should come from, and SPF is available to do that. If the mail comes from a designated server, you can apply less spam testing, or possibly accept it with no testing at all. However, if the mail comes from somewhere else, you can apply more tests, or maybe even refuse it. But regardless of how you want to do it based on that information, please do not ever send a bounce message back for any reason unless the sending IP address is confirmed as valid for sending mail as from that email address.

This is what SPF is meant to provide: "reverse MX" of sorts. It's a decent idea, but suffers from the need to be evangelized and heavily adopted to make an impact.

Here's the goddamned standard... Make it ultra-easy so it's simple to hit critical mass where everyone uses it.

Take a look at this: Sender Policy Framework.

There is even a wizard that walks you through the creation of the appropriate TXT records for your DNS zone file.

220 Some Mail Server
HELO: massmailer.example.com
250 Some success code
MAIL FROM: <viagra@massmailer.example.com>
250 OK
RCPT TO: <end_user@example.net>
250 OK its for <end_user@example.net>
BULK MAIL: (one of <unsolicited>, <personal>,etc)
250 OK
or
5XX This user does not accept <unsolicited> messages

Bravo... you've just invented the Reverse-MX style anti-forgery proposals. Check out SPF

One proposed solution I would love to see getting more attention is SPF ("Sender Policy Framework"), which allows each domain admin to specify their email sending policy using existing infrastructure.

See the SPF site or read this month's Linux Journal to find out more.

Executive summary of SPF: Just use DNS to specify where mail from your domain may originate from. If everyone used this, we could have domain blacklists that actually work.

Do an "nslookup -type=txt psychogenic.com" to see an example entry. And if you manage any domains, please consider doing the same.

Charging for email doesn't discourage spam. It discourages mass email. But there are many legitimate uses of mass email, like discussion lists, automated order confirmation emails, etc. - and increasing the costs of sending this type of mail will hurt open-source developers and small businesses the most.

It's not surprising that Microsoft doesn't see the problem with this. They can afford to buy a few more mail servers to handle all of microsoft.com's outgoing mail, and they'd love it if people had to buy more servers (each running a copy of Windows, of course) just to handle all of the added computational costs of sending mail.

In the article, "Goodmail chief executive Richard Gingras said individuals might get to send a limited number for free, while mailing lists and nonprofit organizations might get price breaks." But how do you know who's a nonprofit? Someone with a .org? Yeah, right!

I believe that SPF currently has the potential to put the biggest dent in spam, since it directly addresses forged email addresses without needing to replace SMTP. It's not a complete solution, but it's a lot more realistic than Microsoft's idea.

Many of sendmail's problems are related to building an extremely general purpose mail forwarder that had to on the Unix of the mid-1980s, support _all_ the popular email protocols of the time (including relaying between UUCP, BITNET, and many other non-TCP/IP things) and providing end-user mail receiving and mail sending services on time-sharing machines, as opposed to running in a dedicated environment with a simpler set of features. On the other hand, the AT&T Bell Labs Research UPAS mailers that became SVR4's mailer was just about as powerful but much smaller, cleaner, and more modular, and even under System III, the mail delivery software didn't need to run as root, so it wasn't the same horrendous security hole.

Microsoft-bashing isn't appropriate here either. Sure, the Exchange MTA and Outlook clients have appalling security and reliability records and used to be pretty much the Mos Eisley of security nightmares, but these aren't the security problems you're looking for. This is about addressing the security problems that are inherent in SMTP when you implement it _correctly_, which allows the mailer to receive all kinds of mail that makes fraudulent, bogus, distracting, or otherwise inappropriate claims about its origin that gives the naive recipient no way to hunt down and kill the evil time-wasting perpetrator (or makes it easy for the naive recipient to hunt down and kill the often-innocent bystander whose name was forged, whether the naive recipient is a human or a mailbot.) The problems have a lot of synergy - the lack of even cursory sender validation makes email an attractive nuisance when delivered to the naive recipient, who can be trusted to click on the happy shiny icon promising a display of dancing pigs (especially since Outlook is friendly enough to hide the ugly details from the user), triggering all the appalling things that can happen when you tell Outlook to trust a message it just received. This work is fundamentally about interfaces and scalability, and Sendmail Inc. is the right group for Microsoft to work with.

The details of the system seem a bit baroque to me, but you knew it was XML within the first couple of paragraphs, and it said Microsoft in the first half-sentence. It's not as lean and mean as LMAP and it has broader goals than SPF (which was originally about joe-jobs, and has suffered from some limitations as its scope has expanded.) And it's not going to solve the entire problem, because nothing short of a worldwide moral transformation or the extinction of the species is going to eliminate human greed and gullibility, but that's ok - even if it only eliminates _half_ the spam, that'll buy us another year of being able to keep reading email, plus it will annoy the spammers out there.

Of course, the continuing success of viruses that depend on the naive recipient pressing the button to watch the dancing pigs means that if SPF/LMAP/RMX/etc becomes widespread, we'll see more exploitation of Mail User Agent bugs that send out spam pretending to be from the naive user (or coworkers of the naive user), and then you can go back to bashing Microsoft while the Enemy continues their side of the arms race.

People have been using the limitations of SMTP for many things, some are quite innocent (forwarding, creating "send this page to a friend" scripts on web sites, etc.), and some are quite abusive (spam, viruses, 419 fraud, etc.). To a certain degree, it is time to throw the baby out with the bathwater. There is quite a bit of spam that has been produced by abusing web sites that allow "send this page to a friend", where you type in your e-mail address that appears as the "From:". While it is a nifty idea, it is just too easy to abuse, and thus it has been abused. It should have never been possible in the first place-- that was a flaw in SMTP.

Forwarding is another thing. It looks like you use pobox.com for forwarding-- they are the folks that invented SPF, and they have come up with a way to deal with forwarding that SPF does not break. I think their point is valid-- "SPF does not break forwarding, forwarding breaks SPF".

People have been using the limitations of SMTP for many things, some are quite innocent (forwarding, creating "send this page to a friend" scripts on web sites, etc.), and some are quite abusive (spam, viruses, 419 fraud, etc.). To a certain degree, it is time to throw the baby out with the bathwater. There is quite a bit of spam that has been produced by abusing web sites that allow "send this page to a friend", where you type in your e-mail address that appears as the "From:". While it is a nifty idea, it is just too easy to abuse, and thus it has been abused. It should have never been possible in the first place-- that was a flaw in SMTP.

Forwarding is another thing. It looks like you use pobox.com for forwarding-- they are the folks that invented SPF, and they have come up with a way to deal with forwarding that SPF does not break. I think their point is valid-- "SPF does not break forwarding, forwarding breaks SPF".

The official SPF Objections page has several solutions for you. One simple option: put your local SMTP account on the From: line and your Yahoo address on the Reply-To: line.

If that was sarcasm, excuse my reply, but...

It seems it DOES have this mechanism. See here -- there's support for include: and mx: even ptr: ... However, I don't want to have to keep tabs on the various ISPs and their changing SMTP servers.. or if one of my users switches to a new ISP, I don't want to have to edit my zone..

Others have mentioned SMTP auth. I agree that in a perfect world, my users would use it.. reality, though, is that these same users should be signing/encrypting their mail.. -- I'm saying that it's difficult to get my users (moderate to none, in the tech-skill department) to comply.

Also, at least one of my users' ISPs blocks outgoing port 25 (to curb worm migration, I guess). (which, yes, can be worked around -- see the previous paragraph).

S

spf.pobox.com has pdf converted versions of the docs.. as apparently some versions of M$ Word can't open the documents..
Caller-ID in PDF

I use two SMTP servers, one for my normal every day use, and one for my phone (damned phone company forces you to use theirs). SPF allows me to add smtp.orange.net as a valid from address and I don't have to worry about what that resolves to. With the MS solution I either have to query all the IPs the FQDN belongs to, or pass control over using <indirect>orange.net</indirect> and rely on my mobile phone company having an up to date entry of their own

Of course all of this is useless unless your mail server supports it. Not everyone uses sendmail or Exchange.

SPF is already a IETF draft, the first stage towards RFC-style standardisation.

The skinny is: while spf on its own can't do prevent zombies from sending mail, if the upstream host routes port 25 through its own servers it can control this.

For example, my upstream hosts, Nildram, block all port 25 traffic outbound and inbound unless and until they have checked your (static) ip for open-relay-ness and then put you on a whitelist.

If all ISPs were like that, and spf were to become widely adopted, spam would be toast.

J.

The SPF guys have them: http://spf.pobox.com/caller-id/

This "plug-in" scheme look totally redundant with SPF to me.

SPF seems like such an easy, simple implementation, it's hard for me to understand why folks aren't jumping all over it. For example: I think that sendmail is NOT pushing this solution. My conspiracy theory for that is it's not an "email only" solution, and involves DNS, which sendmail has no finger in.

In fact, if you search the /. archives, you'll find a somewhat recent article.

For the average /. reader who can't be bothered to RTFA, the short of it is that works like a reverse MX record. Only hosts listed in your SPF (Sender Policy Framework) rules (published in DNS) are considered allowed senders of email from your domain. Recieving MTAs can then make an informed decision on whether to accept mail that has an envelope sender from you domain, based on whether the sending host is listed as permitted. This means that for any domain that is publishing SPF rules, spoofing the sender address while using an open relay/M$ zombie box becomes impossible, as long as the receiving MTA checks SPF.

It won't put an end to spam, but when enough domains have implemented both publishing SPF rules as well as checking them for inbound mail, it will cause severe headaches to the spammers, and cut down their arena significantly. Best of all, if there ever are any false positives that are rejected, it's due to the originating site policies, not the receiver's or middleman (as the case easily is with distributed blacklists)!

Sendmail is one of the vendors working on Sender Permited From or Sender Policy Framwork is it not? spf.pobox.com I have no clue, nor did the article, on what Microsoft might be doing.

SPF is basicly a reverse DNS lookup on SMTP servers if I understand it correctly. Basicly under the plan to send mail you have to have a registered SMTP server in DNS so that your mail can be traced back to the sending SMTP server. No SPF records then your mail is most likely spam and can be discarded at the client or even at the POP server. Heck I suppose even SMTP servers could refuse to forward such mail. Will not eliminate all spam but it would halt the span-in-can email virus like SoBig that makes every Winblows box into instant spam machine. It would also stop spoofed email that causes so much headache.

Very needed plan IMHO.

I'd call it both. Spammers lie about who they are. They send spam using hacked machines that they do not own. If we change the system to charge for email, they'll just force those charges onto innocent victims. Sorry, bud, but when a spammer forges my domain in the spam he sends you, I'm not paying you, him, or anyone else.

In order to charge for email, you have to be able to verify who actually sent the mail. Once you can verify who sent the mail, I don't think charging per email will be necessary.

One possible solution that I've been watching is SPF. Sender Permitted From. http://spf.pobox.com/.

Legislation will not work. Will never work. Email is too easy to fake, and to hard to track, and to entrenched to change. When you do track it down it's coming from zombie machines.

Current Anti-Spam laws are like passing a law that you can't fast forward through the FBI warnings on a VHS tape. You need new hardware and protocols like DVD before pointless laws like that can work.

SMTP+SPF will work much better. I have thousands of email spam for training my filters. Only *3* of them are not from forged addresses. If most sites on the internet published MX and SPF records, then we could be reasonably sure that the email came from a the correct site. I could block that whole domain, and then they would need a whole new domain for each batch of spam as filters get updated. Once major sites like MSN, Yahoo and AOL decide they will ONLY accept email from domains with SMTP+SPF records the rest of the world will be forced to add those records. Especially spammers because those moron to user ratio is generally high in those markets. ;)

See this Wikipedia article or
The SMTP+SPF site for more info

"When all you have is a hammer, everything looks like a nail". In other words the only way Government can think of to fix things is to pass laws, or apply taxes, or blow shit up.

Try passing a law in your house that sending you Junk Snail Mail is a crime. How do you intend to enforce it when the sender doesn't live by the same rules? That's the situation we have with email.

There is already something out there that's pretty similar to what you're suggesting. It's called Sender Policy Framework.

Basically, as part of your DNS entry, you have a record containing a list of all of the addresses that are allowed to send email on your domain's behalf. I think there was a story on Slashdot a few weeks ago about it as AOL has starting using it.

Simply : untrue. It's as easy to fake the envelope sender as it is the From: header. I think you're getting confused with "Received" headers, where each mail system inserts its own bit of tracking information. The envelope-sender is completely under the control of the sender, and (usually) propagates un-modified as an email is handed between systems (indeed, one of the criticisms of SPF is that by modifying the envelope sender you break forwarding).

we are going to have to have more overhead in a "new" SMTP protocol of some sort

SMTP has been extended to allow authentication and verification of senders. Combined with some simple firewall rules on the part of ISPs and businesses, we could have this spam problem under control. Here's what we need:

• If you have an SMTP server that external (to your network) clients need to use to send mail (ie initial mail submission), use SMTP+AUTH+SSL (how-to, how-to). Configure initial mail submission on a port other than port 25 (465 or 587).
• ISPs, businesses, free hotspots, block egress port 25 traffic! The only reasons not to are addressed by the previous item.
• Implement SPF:Sender, for your SMTP server as well as publishing the DNS records.
• Use reasonable blacklists (DNSBL). As systems start to adopt the first three points (and more and more are every day), blacklist those systems that don't. They will be the only places left people could effectively send spam from. ISPs not cutting off spammers will continue to end up on blacklists, which leads to an economic hit (see original article).

Once in place (and these are just not that tough, so no whining), the economics of spamming start to change. Spammers will find it harder to set up shop. The use of hijacked Windows workstations is eliminated through egress port 25 blocking and blacklists. Spammer friendly ISPs are blacklisted, so that no longer works. Inboxes throughout the world rejoice. The Russian mob surrenders. The world plunges into a thousand years of peace, prosperity, and happiness.

this is why it makes sense to have such a facility managed by "professionals" - and why you should look at SPF if you are considering such an installation. It strongly suggests that mail gateways add SASL (Simple Authentication and Security layer) outgoing mail - so you connect via a secure path for outgoing mail to the same place you pick up your incoming mail from.

So, I guess I'm not surprised that there's a "lazy programmer" style security flaw in their products today.

...you'll just cause the spammers to sign up for a one time domain name setup the SPF, spew their spam....

I don't believe we'll ever get rid of spam. However SPF seems a decent approach to reducing the amount of spam humans have to see. The FAQ and the objections answered do a good job of treating these issues.

DNS security does remain a problem, of course. That's an independent problem, one we have to fix at some point.

...you'll just cause the spammers to sign up for a one time domain name setup the SPF, spew their spam....

I don't believe we'll ever get rid of spam. However SPF seems a decent approach to reducing the amount of spam humans have to see. The FAQ and the objections answered do a good job of treating these issues.

DNS security does remain a problem, of course. That's an independent problem, one we have to fix at some point.

... I'm not fond of SPF, because all someone has to do is be able to forge an IP, which isn't particularly difficult.

From the SPF FAQ:

There is no question that for a brief time in the early 90's there was some risk of such attacks because switched networks had not replaced large broadcast segments in some significant destination networks (i.e. corporate and retail ISP), ISN prediction was easy, and security was generally so rotten that with skill, an attacker could pretty much guarantee the ability to crack devices in the necessary places to make a spoof work.

It is not that time. A decade of cracking, the complete triumph of switches even in shoestring networks like my home office, and the cleansing influence of y2k have made a TCP session spoof into the sort of trick that requires such significant setup that there is no point in doing the spoof in the classical manner of sequence number prediction.

Well, it is the receiving server that does the checking actually. The point is if a spammer sets up a domain that ignores SPF, then that particular domain can be known as spam-friendly and can be "safely" blocked. SPF would work well in sort of a weighed-domain scheme, where you keep track of which domains are known to spoof addresses and mass mail so you can take the necesary steps to avoid being spammed. You can see more info here

Why?

Doing a reverse DNS lookup of the name associated with the IP of the server connecting is common enough already, eg.:

$ telnet staff.cs.usyd.edu.au 25
Trying 129.78.8.1...
Connected to staff.cs.usyd.edu.au.
Escape character is '^]'.
220 staff.cs.usyd.edu.au. V1.2 ready at Tue, 03 Feb 2004 10:22:23 +1100
HELO foo.com
250 G'day foo.com, I'm staff.cs.usyd.edu.au., I thought you were [censored].swiftdsl.com.au.

That doesn't seem to have brought DNS to its knees...

And it would do lots for spoofed addresses from real domains - you couldn't send email from a domain except via the outgoing mail servers of that domain. Hence you couldn't spoof email from ebay.com (well you could spoof it in the From: header but not in the envelo), unless you managed to get one of ebay.com's outgoing mail servers to do it for you (which hopefully, they would be configured not to do).

spf.pobox.com is one implementation of the idea, not using MX records obviously since that would break existing mail senders, but using TXT records.

That'll be SPF you're talking about there then.

I wouldn't be so sure. If the e-mail address receiving those bounces is listed anywhere on the web, then it could easily be a random joe job. Let's say that infected-luser@a.b.c sends a virus-carrying message to poor-victim@x.y.z, but gives a phony From: address (joe@blow.com). x.y.z, noting that poor-victim's mailbox is full (due to umpteen thousand other copies of the same damn virus-carrying message), bounces the message back to... you guessed it, joe@blow.com

SPF (Sender Permitted From) hopes to eliminate most of this, as follows: x.y.z contacts bar.com and asks "is foo@bar.com allowed to send from (IP numbers of a.b.c)?" bar.com answers "no", and x.y.z trashes the message without even trying to deliver it to poor-victim, much less bouncing it back to joe@blow.com

And tell your friendly sysadmins that if they would adopt SPF (Sender Permitted From), mentioned on Slashdot quite a few times now, that we would no longer have the problem of From addresses not patching the "postmark."

Better option: block all Comcast IP addresses except their mail servers.

Even better option: deploy SPF::Sender and you won't need to deal with Comcast changing the IP address of their outgoing mail servers (I know-- not quite a working option today, but it will be).

I don't see it as an anti-spam system, I see it as an anti-forgeing system.

That's good, because it's not being positioned as an anti-spam system. It's being positioned as an anti-forging system.

http://spf.pobox.com/howithelps.html

I would rather not allow spammers to use my domains in forged emails. My 20k+ users will just have to use SMTP AUTH if they're not on the same network as our outbound mail relays. This isn't a problem, since we already require it for people who aren't directly dialled in. I don't see any reason why "power users" can't deal with an additional setting in their mail client.

Maybe I don't know SPF, but I think there is a critical flaw in their implementation. As I understand it, the 'ptr' mechanism allows (ie labels a message as 'acceptable') a message to come through if the PTR record of the IP address for the sending server matches.

If a spammer has her own class C, or at least something that she can publish her own PTR (or 'reverse lookup' records), she can label her own IP address as 'chinanet.mx.aol.com' or even just 'mx.aol.com'. My incoming SMTP server checks the SPF record for AOL, sees that if the IP address resolves to 'mx.aol.com', and accepts it as coming from AOL. I think the 'a' mechanism is much more spoof-proof.

Please correct me if I am wrong, I may be reading the docs incorrectly.

Once other people start using SPF to filter their mail, all you'll have to do is modify your DNS records so that your email will get through to them. That should be simple for you.

You won't have to patch postfix/exim/whatever unless you decide that you want to filter based on SPF right now. If you don't want to patch your mail server software, wait for a newer version that comes with it standard.

Really... this is the best solution I've ever heard of for the spam problem. It sounds neat and clean. No messy government legislation, no centralized handling of certificates [screw you, Verisign], no cumbersome approaches with crypto, no unweildy modifications to SMTP, no client-app changes, no silly "hard computational task" to waste CPU cycles, no lame micropayment systems. If SPF catches on, it will reduce spam to a manageable problem. The only real worry is that it will encourage spammers to try their hand at IP-spoofing and intrusive hacking.

So now AOL users are SOL if they want to use any of the large number of applications that send mail for you, such as all those "Mail this story to a friend" links, or tools like eVite which manage party invitations for you. And tons of other applications, many of them useful.

I've got 1 word for you: Sender Rewriting Scheme, well three words.

So, in essence, AOL has decided that it's customers can no longer send mail from their AOL email address, unless they're logged into AOL.

No, they haven't. Here's the current TXT record for aol.com.:

v=spf1 ip4:152.163.225.0/24 ip4:205.188.139.0/24 ip4:205.188.144.0/24 ip4:205.188.156.0/24 ip4:205.188.157.0/24 ip4:205.188.159.0/24 ip4:64.12.136.0/24 ip4:64.12.137.0/24 ip4:64.12.138.0/24 ptr:mx.aol.com ?all

Now, if you knew SPF, you would recognize that the last bit -- ?all -- means that AOL is not stating that AOL-user mail is only legitimate if sent from AOL mail servers. The ?all tag means that hosts that don't match the rest of the SPF record are taken as unknown -- not as failures. That would be -all.

• If you have a common first name, don't have an email address of the form firstname@domain, you are guaranteed to be hit by a dictionary attack
• Don't publish your email address on the web, make sure any websites you subscribe to hide your email address or use email address hiding technique
• If your on a mailing list make sure that if the archive is available on web that it hides your address
• Use a bayesian mail filter

That seems to be by design. (Not offering an opinion, merely commenting. Seems to me all these schemes will cause much more pain for the small guys than for the big ones.)

Using muscle to force the Internet into a standard isn't going to work. We need something that *is* a standard, rather than *pushing* a standard upon people.

• AOL.com
• AltaVista.com
• DynDNS.org
• LiveJournal.com
• OReilly.com
• Oxford.ac.uk
• PhilZimmermann.com
• Perl.org
• w3.org