Slashdot Mirror
IETF to Look at Spam
m00nun1t writes "CNET has an article about the Internet Engineering Task Force (IETF) looking at what they can do about spam. According to the article, many of the proposals seems to "require changes in basic e-mail technology", which presumably means SMTP (and about time!). Maybe they are looking beyond just SMTP - anyone have any insights here?"
If an alternative to SMTP were developed, the protocol would not be likely to disappear immediately subsequent to the creation of its successor. The transition would be gradual, as reverse-compatibility could remain necessary for several years afterward. As suggested by the release of Apache 2.0, for instance, not every server administrator adopts a "technological improvement" until it becomes an adequately proven and stable product.
Do you like German cars?
Someone posted a response to another spam story a few weeks ago, sadly I can't find it, but they described an interesting mail delivery system they'd created.. and it sounded, to me, as if it could certainly be the future of mail delivery.
They said that when someone sent a mail, it simply went to the local server, and no further.
It sounded like a 'reverse IMAP' style system to me. That is, your outgoing mail simply went to a folder on your server, which allowed you to edit and even delete mails BEFORE they were picked up by the recipient. The recipient's e-mail server would only receive a 'notice' that someone had mail for them.
When the recipient went to collect their mail, their own mail server would then have a basic list of where the e-mails for the recipient are, and then it'd go ask for them from the remote servers and feed them through.
So, how does this help spam?
It allows spam to be truly filtered on the OUTGOING rather than the incoming!
Why's that a great thing? Well, it means that if you're an AOL or MSN user, you're not going to lose 80% of your mail simply because of over-zealous filtering by your ISP. Instead, spam mail will not even be sent, let alone received!
Of course, bad eggs could always set up servers with no filtering systems on them and send their spam that way.. but BECAUSE e-mail will be picked up FROM the senders server with this system, it means blacklisting is a whole lot easier! You just ban a server and you know you've got rid of the bad eggs.. whereas the current SMTP system allows open relays and all sorts of 'trickery' to get around filtering systems.
So.. the conclusion is.. make e-mail stay on the sender's server until it's time for it to be collected. It allows you to edit or delete mail before the recipient collects it, it stops spam, and it reduces bandwidth(!) -- if someone never collects their mail, then the mail has never gone across the net.. it's still on the sender's server.
I hope the original poster of this idea will pop up here again and correct me if I got his ideas wrong, but he was certainly on to something.
mogorific carpentry experiments
The technology exists, off the shelf, today.
There is a SMTP command called STARTTLS which will enable SMTP over SSL. It's defined in RFC 2487. Sendmail supports it with a compile-time option, and so do most other MTAs. It's backwards compatible with normal SMTP.
You will need a certificate, of course.
This has 2 big effects:
- encryption of email in transit between SMTP servers (a nice bonus)
- authentication of SMTP servers
Since sending spam isn't illegal in most jurisdictions, knowing WHO sent the spam (or relayed it) allows you to contact them and complain, threaten and retaliate (mailbomb, portscans, DDOS, etc.)
If you receive email from a host authenticated by versign (or whoever), you apply little filtering.
If you receive email from a host not using ssl, it goes into a queue for maximum filtering.
Much of the spam I receive today is from DSL customers who spew directly.
Downside:
- there will be additional CPU load for all the email servers
- cost of certificates
The article notes that one of the major problems is the filtering of genuine mail due to agressive spam filters necessitated by cleverer spammers. Consider this analogous to dropping some packets at the network layer. Just as the transport layer handles this problem, we can build a higher level protocol to handle filtered mail.
Note that having a mechanism to handle dropped mail allows us to employ agressive filtering: one that is sure to stop 100% of spam.
What I have in mind is as follows: when Bob receives a mail from Alice (i.e, it has passed through Bob's filter) the client software sends a confirmation mail back to the Alice. This is not a regular mail that the Alice will see in her inbox; it has a special header flag that marks it as a confirmation. Alice's client software keeps track of the confirmation messages; by looking at her "sent-mail" folder she can see which of her messages have not been confirmed (and are hence likely to have been mistaken for spam).
Finding that Bob has filtered her mail, Alice can either re-word it and send it again or do something like (assuming that Bob knows Alice): "Hi Bob, this is me, Alice. Your filter blocked this so I've rot13'd it to get past the filter. rot13 what follows to read my mail." Another option is to encrypt the mail with Bob's public key (assuming that spammers' scripts won't be clever enough to get your public key from your web page). Note that 99% of the time the mail is going to get through. You have to make that little effort to prove you are a human only once in a long while.
There is minor problem with requiring the receiver to send a confirmation message: Bob might check his mail only after a couple of days, during which time Alice may assume that her mail was blocked. There are 2 solutions: either Bob runs a script to filter his mail regularly, or else has his ISP implement his filter for him.
Note that this won't work if you have the receiver send a reply whenever the message did get blocked: the reply could itself get blocked etc. (This is called the red army - blue army problem in networking).
Actually, it's the IRTF -- not the IETF -- that is undertaking this work. To quote from the IRTF home page - "[Mission] To promote research of importance to the evolution of the future Internet by creating focused, long-term and small Research Groups working on topics related to Internet protocols, applications, architecture and technology."
Don't expect a quick fix from this initiative.
From what I take from all this discussion is that the only "solution" to spam is to do the types of things that we have been doing for years, but to do more of it and quicker. Use well run DNS blacklists (Spamhaus SBL, ordb, dsbl, etc.), use good content filters (bayesian filters, etc.), use bulk mail detectors such as DCC or vipul's razor, etc.) and per-user whitelists and blacklists.
Or, combine all of the above techniques by using SpamAssassin
--
I've been subscribed to the list since near the beginning and have been following it fairly closely. Much of the discussion has been rehashes of old topics such as "what exactly is spam?", "make the sender pay something, either money or CPU", etc.
The most interesting discussions that I've seen so far are:
Most spam specific programs will not queue and retry, and thus the spam will be dropped.
Spammers that use real mail transfer programs or open relays will need to be able to hold all their outgoing spam for a while, increasing the spammer's costs and slowing down the delivery of spam. Legitimate email will not be thrown out, it will only be delayed and only for the first time.
Of course, you don't really want the databases to remember every sender-recipient pair forever, nor do you want to remember pairs that were added by spam so this really isn't a "first time" database, but it is close.
Apparently the "canit" program already does this, but I had not heard of this technique before.
If you filter during the email receive process, you can make the sending MTA do the bounce. This means that you will not have to deal with spammers forging "from" and "reply-to" headers. You won't have to clean up bounces that never succeed, nor will you be responsible for bouncing spam to another victim that the spammer selected for the "from" or "reply-to" headers.
Also, false positives will recieve a bounce message instead of just disappearing. This reduces the danger of important email being lost.
Right now, there are DNS records that tell you which IP addresses are valid to try and send email to for a given domain (the MX records), but many ISPs have different machines for sending and recieving email. There are currently no DNS records to tell you which tell you which IP addresses a domain will send email from.
The problem with this kind of proposal is that there are many people who think they have legitimate reasons to forge "from" or "reply-to" addresses. It also forces ISPs to make sure that every time they add a new outgoing mail server, they need to update the list of valid IP addresses. If they forget to do this, then only bleeding edge spam filters will detect a problem.
SPF support for most open source mail servers can be found at libspf2.
...allow binary transfers. I can't tell you how much CPU time has been wasted by base64 encoding binaries, sending them over an inefficient protocol, and decoding them on the other end. yEnc does a good job but the whole encoding shenanigan is a major pain for anyone trying to send family photos or the latest AFI album. Please, IETF, make a better 8-bit clean push protocol, because SMTP is the only one we have.
"The lesson to be learned is not to take the comments on slashdot too literally." --Vinnie Falco, BearShare