Slashdot Mirror
IETF to Look at Spam
m00nun1t writes "CNET has an article about the Internet Engineering Task Force (IETF) looking at what they can do about spam. According to the article, many of the proposals seems to "require changes in basic e-mail technology", which presumably means SMTP (and about time!). Maybe they are looking beyond just SMTP - anyone have any insights here?"
If an alternative to SMTP were developed, the protocol would not be likely to disappear immediately subsequent to the creation of its successor. The transition would be gradual, as reverse-compatibility could remain necessary for several years afterward. As suggested by the release of Apache 2.0, for instance, not every server administrator adopts a "technological improvement" until it becomes an adequately proven and stable product.
Do you like German cars?
Someone posted a response to another spam story a few weeks ago, sadly I can't find it, but they described an interesting mail delivery system they'd created.. and it sounded, to me, as if it could certainly be the future of mail delivery.
They said that when someone sent a mail, it simply went to the local server, and no further.
It sounded like a 'reverse IMAP' style system to me. That is, your outgoing mail simply went to a folder on your server, which allowed you to edit and even delete mails BEFORE they were picked up by the recipient. The recipient's e-mail server would only receive a 'notice' that someone had mail for them.
When the recipient went to collect their mail, their own mail server would then have a basic list of where the e-mails for the recipient are, and then it'd go ask for them from the remote servers and feed them through.
So, how does this help spam?
It allows spam to be truly filtered on the OUTGOING rather than the incoming!
Why's that a great thing? Well, it means that if you're an AOL or MSN user, you're not going to lose 80% of your mail simply because of over-zealous filtering by your ISP. Instead, spam mail will not even be sent, let alone received!
Of course, bad eggs could always set up servers with no filtering systems on them and send their spam that way.. but BECAUSE e-mail will be picked up FROM the senders server with this system, it means blacklisting is a whole lot easier! You just ban a server and you know you've got rid of the bad eggs.. whereas the current SMTP system allows open relays and all sorts of 'trickery' to get around filtering systems.
So.. the conclusion is.. make e-mail stay on the sender's server until it's time for it to be collected. It allows you to edit or delete mail before the recipient collects it, it stops spam, and it reduces bandwidth(!) -- if someone never collects their mail, then the mail has never gone across the net.. it's still on the sender's server.
I hope the original poster of this idea will pop up here again and correct me if I got his ideas wrong, but he was certainly on to something.
mogorific carpentry experiments
The fundemental diference that protects most communication systems from Spam-like abuse is that the sender is responsible for a majority of the costs of the message. Yeah, there are telemarketers and junk postal mail, but the are seriously limited by the fact that there is a noticiable cost assosciated with each additional message they send. The fact that it costs money to send such communications makes it impractical to bother people with offers with an extremely low reponse rate.
SMTP/POP3 e-mail presently leaves the cost of holding the message during the wait for the intended reader to be available on the receive-side server. The spammer doesn't even have to maintain a constant and consistant Internet connection.
Under the current system, a sender can send 100 MB of messages in an hour without penalty. However, a receiver who gets 100 MB of messages in an hour usually will find any other messages sent to them bouncing.
Requiring the message be held on a sender-side server instead would transfer the costs of sending a large volume of e-mail onto the sender, and therefore discurages the practice better than any law ever could.
The technology exists, off the shelf, today.
There is a SMTP command called STARTTLS which will enable SMTP over SSL. It's defined in RFC 2487. Sendmail supports it with a compile-time option, and so do most other MTAs. It's backwards compatible with normal SMTP.
You will need a certificate, of course.
This has 2 big effects:
- encryption of email in transit between SMTP servers (a nice bonus)
- authentication of SMTP servers
Since sending spam isn't illegal in most jurisdictions, knowing WHO sent the spam (or relayed it) allows you to contact them and complain, threaten and retaliate (mailbomb, portscans, DDOS, etc.)
If you receive email from a host authenticated by versign (or whoever), you apply little filtering.
If you receive email from a host not using ssl, it goes into a queue for maximum filtering.
Much of the spam I receive today is from DSL customers who spew directly.
Downside:
- there will be additional CPU load for all the email servers
- cost of certificates
If any mail infrastructure reorganization were done before the finding this mount of this sendmail hole , that would have been be a good way to have a mostly forced deploy of compliant mail servers around the world.
The article notes that one of the major problems is the filtering of genuine mail due to agressive spam filters necessitated by cleverer spammers. Consider this analogous to dropping some packets at the network layer. Just as the transport layer handles this problem, we can build a higher level protocol to handle filtered mail.
Note that having a mechanism to handle dropped mail allows us to employ agressive filtering: one that is sure to stop 100% of spam.
What I have in mind is as follows: when Bob receives a mail from Alice (i.e, it has passed through Bob's filter) the client software sends a confirmation mail back to the Alice. This is not a regular mail that the Alice will see in her inbox; it has a special header flag that marks it as a confirmation. Alice's client software keeps track of the confirmation messages; by looking at her "sent-mail" folder she can see which of her messages have not been confirmed (and are hence likely to have been mistaken for spam).
Finding that Bob has filtered her mail, Alice can either re-word it and send it again or do something like (assuming that Bob knows Alice): "Hi Bob, this is me, Alice. Your filter blocked this so I've rot13'd it to get past the filter. rot13 what follows to read my mail." Another option is to encrypt the mail with Bob's public key (assuming that spammers' scripts won't be clever enough to get your public key from your web page). Note that 99% of the time the mail is going to get through. You have to make that little effort to prove you are a human only once in a long while.
There is minor problem with requiring the receiver to send a confirmation message: Bob might check his mail only after a couple of days, during which time Alice may assume that her mail was blocked. There are 2 solutions: either Bob runs a script to filter his mail regularly, or else has his ISP implement his filter for him.
Note that this won't work if you have the receiver send a reply whenever the message did get blocked: the reply could itself get blocked etc. (This is called the red army - blue army problem in networking).
World of Ends, recently discussed on Slashdot, discusses why the simplicity (or stupidity) of the Internet is so useful. "The Internet isn't a thing. It's an agreement," they say.
That same argument applies to e-mail. Following their logic, it is best to leave SMTP alone. Simpler protocols are better. Leave the "value-added" pieces to the edge, and let the simple message transfer protocol alone.
J'aime mieux les méchants que les imbéciles, parce qu'ils se reposent. -- Alexandre Dumas
email is by far the most widely used of all Internet services. I belong to an organization many of whose members are retirees are on fixed incomes, and it is only within the last two years that the number of people with email has grown to a critical mass (about 2/3 of the membership).
Of members of the lay public who regularly use email as a means of communication do not have the level of technical comfort that most Slashdot readers take for granted.
Of people who use email, the percentage who know how to use a web browser is much less than 100%. The percentage who can google for information is much less than 100%. The percentage who can successful extract and decode an email attachment is much less than 100%. The percentage who can view a government form or a corporate brochure in PDF format and read it with Acrobat is much less than 100%.
And the average age of their computers and operating systems is much more than three years--and they're not likely to update their email programs.
Whatever is done needs to be 100% backward compatible with existing email clients, not requiring even simple upgrades, or an astonishing proportion of real-world Net users will be disenfranchised.
(And please, let's not have any facile expressions of contempt for AOL users or webtv clients or people who bought email appliances (that includes one of the retirees I mentioned).
"How to Do Nothing," kids activities, back in print!
The only problem with this is scalability. Sure SMTP has had its problems, but the nice thing about SMTP is you control the server. You control how fast mail comes in, from who it comes in, how fast people can give you e-mail, and how fast you give it out to subscribers/recipients. All of these schemes seem to remove that control from your machine.
Instead of adding a band-aid solution to spam, let's sit down and list what we need for an a-mail server. Scalability, reliability, fault tolerance, expandability and distributed servers top my list. I'm sure that there are other better ones out there too. If you're going to revamp the protocol, try to get everything in the first time, and let's try to get it right.
You think that I'm crazy, you should see this guy!
Actually, it's the IRTF -- not the IETF -- that is undertaking this work. To quote from the IRTF home page - "[Mission] To promote research of importance to the evolution of the future Internet by creating focused, long-term and small Research Groups working on topics related to Internet protocols, applications, architecture and technology."
Don't expect a quick fix from this initiative.
From what I take from all this discussion is that the only "solution" to spam is to do the types of things that we have been doing for years, but to do more of it and quicker. Use well run DNS blacklists (Spamhaus SBL, ordb, dsbl, etc.), use good content filters (bayesian filters, etc.), use bulk mail detectors such as DCC or vipul's razor, etc.) and per-user whitelists and blacklists.
Or, combine all of the above techniques by using SpamAssassin
--
I've been subscribed to the list since near the beginning and have been following it fairly closely. Much of the discussion has been rehashes of old topics such as "what exactly is spam?", "make the sender pay something, either money or CPU", etc.
The most interesting discussions that I've seen so far are:
Most spam specific programs will not queue and retry, and thus the spam will be dropped.
Spammers that use real mail transfer programs or open relays will need to be able to hold all their outgoing spam for a while, increasing the spammer's costs and slowing down the delivery of spam. Legitimate email will not be thrown out, it will only be delayed and only for the first time.
Of course, you don't really want the databases to remember every sender-recipient pair forever, nor do you want to remember pairs that were added by spam so this really isn't a "first time" database, but it is close.
Apparently the "canit" program already does this, but I had not heard of this technique before.
If you filter during the email receive process, you can make the sending MTA do the bounce. This means that you will not have to deal with spammers forging "from" and "reply-to" headers. You won't have to clean up bounces that never succeed, nor will you be responsible for bouncing spam to another victim that the spammer selected for the "from" or "reply-to" headers.
Also, false positives will recieve a bounce message instead of just disappearing. This reduces the danger of important email being lost.
Right now, there are DNS records that tell you which IP addresses are valid to try and send email to for a given domain (the MX records), but many ISPs have different machines for sending and recieving email. There are currently no DNS records to tell you which tell you which IP addresses a domain will send email from.
The problem with this kind of proposal is that there are many people who think they have legitimate reasons to forge "from" or "reply-to" addresses. It also forces ISPs to make sure that every time they add a new outgoing mail server, they need to update the list of valid IP addresses. If they forget to do this, then only bleeding edge spam filters will detect a problem.
SPF support for most open source mail servers can be found at libspf2.
...allow binary transfers. I can't tell you how much CPU time has been wasted by base64 encoding binaries, sending them over an inefficient protocol, and decoding them on the other end. yEnc does a good job but the whole encoding shenanigan is a major pain for anyone trying to send family photos or the latest AFI album. Please, IETF, make a better 8-bit clean push protocol, because SMTP is the only one we have.
"The lesson to be learned is not to take the comments on slashdot too literally." --Vinnie Falco, BearShare