Slashdot Mirror
Revising the Internet Email Infrastructure
Lauren Weinstein writes "People For Internet Responsibility (PFIR) today released a white paper aimed at starting discussion and work to fundamentally revamp Internet e-mail systems to control spam, forgeries, and a range of other problems, while empowering e-mail users rather than ISPs." Excellent start.
They may well come up with some "standard" for a new internet email system but, nobody is going to use it. Hell ESMTP has been out for years and it still isn't supported by more than half the systems that are on the net.
I know very little about this so correct me if I am wrong. The only way to really let each user have complete control over email, would be for each user to have there own mail server and/or domain. This is why most people let their ISP's handle their mail. And you would still get crap from bulk mailers, spammers.
SMTP is here to stay. We're going to have to live with it. Spam control filtering is getting better and there is a good chance that together with decent legislation, spam can be reigned in. A new system will ultimately just create new kinds of abuse, which wil lrequire the industry to take another two year cycle to address.
They bother because it *is* insecure. SMTP is going to be around for a while, but that shouldn't stop a better MTA protocol from being developed. Kinda like ipv6, although thats takeing a lot longer than anticipated :)
Driving backwards on the highway of life
It's unfortunate that it's so unsecure, but that's just the way it is.
I think it's great that it's not secure. Just like every other classic protocol that truly supports the net (tcp, ip, ftp, etc), it's not about what you put over it - it's about moving data as it's told. This distinction is what makes it so difficult to control or "own" the net. I don't believe we could build a "secure" protocol that retains the inbuilt freedom that we have today.
Yes, people abuse that freedom just like they do any other, and yes, spam is so annoying that many who normally fight for freedom now beg to take it away in this instance, but there are solutions that don't involve removing freedom for everyone.
The idea of challenge response is good.. as is baysian filtering.. as is pgp key signing, etc...
And the solution to the abuse of bandwidth on the servers is not to recreate the protocol. it's to make sending spam pointless in the first place - and that happens at the ends. The middle needs to be stupid in order to be smart.
And now my shameless (and probably inaccurate) retelling of "the world of ends" will itself end.
A revamping of the email technology is what needs to take place. Not an internet tax (good crap we are taxed enough already). Along those lines (better technology instead of more bureaucracy) two great technologies that already exist, that help in the email realm, are GnuPG and Bogofilter.
I'm sick of reading proposals (often from industry profit-seeking types) who want to put a paid-for "stamp" or similar "token" on email. (I'm talking generally, though---yes---I did read this paper)
It looks attractive logic:
1. Lots of people use email
2. We offer a system which will beat spam at a cost---our 'trusted 3rd party' or whatever---but only if people who use it can't talk to anyone else, so everyone has to use it
3. Profit.
This is NOT the way forward on spam. Nor, realistically, is anything which re-writes the rules for email. People like editing headers. In fact, if it weren't for spam, people like email as it is---period.
The way forward seems simple:
smtp servers should start requiring genuine users to log in. (though rarely used, there are smtp systems which allow this, and most major clients---yes even the MS ones---already talk to these servers and have done for years)
servers which don't should quickly find their way onto blacklists.
(I shall leave the exact way these blacklists should be used as an exercise for the reader)
Simple. Low cost. Not a business model; but a clear solution.
Anyone want to start writing to ISPs?
i read the paper, but i don't see what is so new with this. the suggestions it makes seem to be similar to methods for email encryption and spam filtering that are already in place.
joe emailer hasn't taken the time to figure these existing methods out, that's why it seems as though they're not working. i don't know what tripoli is going to offer that will get joe off his butt and get him signed up with a "Pit Certificate Authority".
aoeu
Those who would trade freedom for security will lose both, and deserve neither.
The current "hysteria" over spam is going to lead the Joe Sixpacks and the Mothers-protecting-their-children crowd to accept, indeed to beg for, restrictions on their liberties, all in the name of "stopping those spammers". For the rest of us, for whom "WWW" is NOT synonymous with "The Internet", this could have dire consequences. What if I run my own server, and I'm not "blessed" by the current Official AntiSpam Policy Du Jour ? Do I lose out?
Spammers suck, use your filters. DON'T give the government (and media giants, and Big ISPs) the authority to rewrite the way that the Internet works.
I want to delete my account but Slashdot doesn't allow it.
As bad as the spam problem is, it's unlikely that you can get sufficient momentum in the community to displace one of the primal IP application protocols anytime soon. The solution, for better or worse, is probably going to be a combination of filtering technology, $$ legal judgements $$, and Ghu help us, legislation.
(Though anyone taking up a collection to hire the Narn Bat Squad for re-educating spammers please let me know...)
5ms (sounds like an underestimate to me, but lets go with it) * 2 billion messages (AOL blocked it in one day) = ~116 days computer time.
Slowing spam by 2 orders of magnitude would still help.
It would seem so. Like any certification mechanism, you've got to trust the certifiers. And in practice, that means a few big ones.
I found the point especially odd considering the polemic in the beginning about how individuals need to have their own MTAs that can negotiate around port restrictions lest the evil ISPs control them.
A verbose article, which didn't seem very consistent. The kernel idea (don't allow forged headers) has been brought up a number of times. Not much value added here.
The problem with nearly every single encryption technology, or initiative for securing and improving Internet communication, is that it tries to solve too many problems at once. History has proven over and over again that it's the small, easy steps that move progress forward, not giant ones.
PGP, HTTPS, S/MIME and countless other 'standards' have all made the same mistake in trying to force users to adopt multiple new rules. What's wrong with just providing encryption, without any of the additional burdens of establishing identity? Countless transfers are sent unencrypted every day because the cost of a web server certificate - which is only expensive because it establishes identity - is so high. Anyone can make a server that provides encryption, but such a server is useless with today's browsers. And yet, I'm supposed to have faith that the people Microsoft, AOL and Opera choose to trust are the people that I want to trust?
It is obvious where email will change next, no matter how much money and time is spent on projects like this one. More and more people will use 'virtual receptionist' services that require you to return an auto-reply message to prove that you're real. Eventually, email clients will develop a way to autodetect and autoreply to these messages, until some sort of system is hammered out. You'll write your message, it will be delivered, the receiving server will connect back to you to verify that you're real, and your system will confirm it, all transparently. Someday, it'll happen in real-time, maybe. Spammers won't be able to use this, because of the increased load on a server that must stay online as long as they want their mail delivered.
That's how change happens. Not because of a bunch of idealists get together and tell me to start PGP-signing my mail. You know what? I started doing that 3 years ago. I haven't once found a single person who even knew how to verify my messages. Not to mention the pathetic state that the keyservers are in, full of expired and forgotten keys, and easily corrupted (again, I know from experience - I corrupted my own keys in an attempt to remove them permanently).
-Elentar
The wheel it turns, around and around, with an ancient rumbling sound.
Alas, we need to get the mega web-of-trust built first. And that is very, very hard to do, since people are so apathetic about PGP. (I couldn't even get Slashdot-Meetup and 2600-Meeting people to do it. Although maybe (I almost hope) the 2600 people thought I was a narc or something. ;-)
A good web-of-trust would have sooo many applications... what a waste. :(
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
I see this as a dangerous time. Many people have discussed going to an e-mail system that relies on encryption and security certificates. Are we going to end up with another debacle like we have now for secure websites, where Certificate Authorities like Verisign and Thawte charge hundreds of dollars every year for a certificate and free certificates set off more alarms than a than a Great White concert in a gasoline-soaked tent?
Will Microsoft make lucrative deals with high-roller Certificate Authorities to include them in the Microsoft Exchange e-mail server? Will you be unable to run a mail server without paying big bucks to some "trusted" Certificate Authority?
If we are not careful, the only e-mail servers that will exist will be commercial e-mail servers where the owners can afford hundreds of dollars every year for certificate renewals.
Why do I believe this? Because I follow the money. If Microsoft, Verisign/Thawte, Netscape, etc. think that there's a way to make money, they will push for a standard that ensures it.
Just because SMTP can't be fixed (it can't) doesn't mean it has to die - just that a better alternative has to emerge. I'll keep my SMTP servers running indefinitely and I'll keep SMTP mail, but as better systems emerge I'll be telling people that the more reliable way to contact me is with methods that I know aren't going to give me the experience of picking through the trash when I check my mail. I'll still check my SMTP mail, but probably with decreasing frequency as time passes.
For those of you saying "just improve your filters," (1) give me a filter that can parse an HTML message containing only an image to determine whether it's spam or not (no, you can't reject all HTML mail or mail with attachments, if my brother drags-n-drops a picture of my nephew and clicks "send," I want to receive it), and (2) figure a way to keep the message from being delivered until that determination is made. Post-delivery filtering doesn't solve the bandwidth/cost/traffic problems.
Be courageous, people. Nobody screamed that we didn't need the telephone because the telegraph worked fine. Protocols emerge from changing circumstances. SMTP had its use over the last 30 years, but its time is waning with the onset of the global public internet full of untrusted senders seeking to abuse the system. It's time for a better protocol, and I applaud everyone involved in making a serious effort at developing one instead of trying to fix the unfixable.
-- http://frobnosticate.com
True, if everyone was filtering their email to where noone ever saw any spam, the problem would die off from lack of demand.
But, IMHO, that's a pipe dream. There will always be a fair number of people who will receive spam against their will (with the current system), and there will always be a small (and idiotic) subset of those people who will fall for the scams and thus keep spamming alive as a business practice.
The kind of solution Tripoli proposes would keep spam from being delivered in the first place, and make it easier to discourage ISPs from tolerating spamming customers for short-term financial gain. Both of these will (IMHO, naturally) go a lot farther in containing or even eliminating the "spam problem".
Xentax
You shouldn't verb words.
So I would say that we simply use what we currently have to take on spam and encrypt e-mail. Just a few thoughts...
Of course it is possible but, the probability is very low, in my opinion. It is already possible for most modern mail clients to automatically encrypt and decrypt mail, making them secure. Yet very few people use PGP or S/MIME. It is already possible for most MTAs to use SSL and/or TLS to encrypt their communications, yet most still do not use this feature. It is already possible for most POP3 and IMAP4 servers to encrypt their communications using SSL and/or TLS as well as having four or more secure authentication options available, yet most still do not use this feature.
It is possible to redesign and rebuild the email infrastructure of the internet in such a way as to completely eliminate spam and forged addresses, it is howeber improbable that good old insecure and vulnerable SMTP will be abandoned. Prior to the internet and standardization on SMTP, there were many secure mail systems around the world. There was also an inability for them to communicate with each other. This is the problem with a new system. In order for it to work and for email to remain a useful tool, everyone will have to switch and everyone will have to do it at the same time. This is highly improbable.
-Begin Rant-
The problem with spam is simple: the old rule that we should be forgiving about what we accept and strict about what we send.
We could wipe spam out, or at least render it controllable, if we simple required proper DNS entries (A, MX, PTR) and proper server configuration (HELO information, etc.)
Unfortunately, every Tom, Dick and Harry feels it is his god-given right to run a mail server despite having ABSOLUTELY NO IDEA what is required to run one. The sheer number of people without postmaster and abuse accounts is astonishing and both are required. The sheer number of people without matching forward and reverse DNS entries is astonishing. The number of people who call their server "Blah" and then put in a DNS entry for "mail" without an entry for "Blah" is amazing. Although this last part is not required by the RFC's, why on earth should I have to look through my logs and see "Blah" when there is no DNS entry for it? How am I supposed to troubleshoot?
Oh well, I give up.
-End Rant-
Now we are told once more that the best cure against spam should be to reinvent something to replace the tried-and-true eMail system of decade-old reliability, just because some sociopaths apparently cannot learn to behave without getting a spanking (or jail time) and U.S. privacy laws are still too weak to stop the spam.
And after all the years that spam has plagued the networks, that's quite a poor achievement for a nation that managed to outlaw junk faxes, and had confirmation from the courts that regulating advertising does pass constitutional muster perfectly well:
Subsequently, numerous decisions have also made it crystal clear, over and over again, that neither the First Amendment nor the Dormant Commerce Clause are an obstacle to outlawing electronic spam, by fax or any kind of eMail.
Nor is it at the expense of any legitimate business. Industry itself can't stand the spam anymore.
This is not about "lawmakers never knowing enough about the Internet to regulate any aspect of it in a meaningful way", it's about doing something to prevent imposing compulsory changes to technology that keep fighting the symptoms rather than the cause.
Congress should get over such shameful cowardice and make the simple law that's needed and proven to work.
There is no need to re-engineer the Internet.
There is no justification for widespread surveillance and data retention under the poor excuse of trying to track down spammers.
There is no risk of banning mailing lists or commercial eMail.
There is no doubt what the sociopathic behavior is.
All that is needed is mandatory opt-in for unsolicited bulk eMail (encompassing all kinds of electronic messaging).
And yet some self-proclaimed "experts on electronic advertising" (whose only merit probably is that they know how to spam because they've done it a trillion times at everyone else's expense) keep pretending that opt-in wasn't legal, or feasible, or desirable.
Opt-in works, and it does not hurt anyone but the spammers.
Europe has adopted it, Australia is adopting it (how far behind do you want the U.S. to be, are we to wait for China to outlaw spam before the U.S. will?!), but most importantly the USA have successfully adopted it themselves against junk faxes.
There's probably something wrong in Washington D.C., and the news media in general, when the most insightful newspaper article on the issue comes from USA Today.
Be sure to fax or eMail it to your congress(wo)man though.
Don't spam them, but do attach some selected masterpieces of spam if you think they need an idea of what ends up in the inbox of their constituents, and of their children, 9 billion times, every single day.
Have the SMTP amended so that MTAs perform a DNS check on the previous server, and if it doesnt match correct the header. With guarenteed un-forged headers then at least reporting will be a hell of alot easier.
The 2.4.x kernel -- isn't that the "stable" kernel which had a complete VM subsystem change and two filesystem corruption bugs?
Stable trees might *theoretically* only include bug fixes, but in practice they tend to have rather more than that.
Tarsnap: Online backups for the truly paranoid
Bad example, IMO. In my experience, IPv6 hasn't caught on because IPv4 still works, and there's no major incentive for most big networks to upgrade because there's no features they need that can't be done in 4.
With spam, however, a new protocol for SMTP that could provide protection against virii and spam would provide a sufficient incentive for upgrading.
"Quoting famous computer scientists out of context is the root of all evil (or at least most of it) in programming." - K
I think there's a fundamental difference between the problems IPV6 is trying to solve and what any "SMTP2" solution is trying to solve.
IPV6 will solve the underlying problem of running out of IP space.
"SMTP2" would NOT solve the spam problem, because it's not a technical problem, IMHO. Spammers would move over to "SMTP2" eventually. They'd just have to find that one little flaw or feature and they'd be back exploiting it like they're exploiting weaknesses in SMTP now.
If widespread adoption of "SMTP2" takes anywhere near the amount IPV6 adoption is taken, it's not going to work. Spammers would have 5 years to study the new technology and develop solutions to get their crap across the new protocol.
By the time "SMTP2" is in place and used by everybody, the spam problem would no longer be what it is now and we'd be back in the cat-and-mouse game with spammers and their spamware techniques.
All the "SMTP2" solutions I've seen would make normal Email communication between non-spammers much more difficult. I think that's something that should be avoided, even at the cost of not solving the spam problem using technology solutions.
Proletariat of the world, unite to kill spammers. Remember to shoot knees first, so that they can't run away while you slowly torture them to death.
In Soviet Russia, I ruled you
Have you considered that DNS is often controlled by people who don't control SMTP?
Example: ISP ownes the IP and give you 1 IP for your SMTP server.
Or if you have multiple switched internet links for redundancy? Link goes down - you switch IP's to route around problem (switch providers)... but you can't force DNS cache to instantly update.
Also consider clusters. What if you have 3 machines, which need their own name for hardware management, but they are all acting as a single mail exchange host? Yes, I know about multiple MX records - clustering solves other issues. You can combine clustering + MX records.
Bottom line: It costs extra money and time to get your own block of IP Addresses and properly manage DNS. AOL can do it, but so what? Why lock out the small mail servers of the world just because they don't have reverse DNS?!
Mail comes into my USPS mailbox in front of my house. The "FROM:" server does not have to exist to come to the TO: location.
Yes, SPAM is a problem, but quit blaming protocols and technology with the issue is the small percentage of e-mail users who are _sending_ the spam.
RoundSparrow
Your example is a bad one. Microsoft did its best to avoid starting over with its operating systems. And when it did, it did so very carefully with as much backwards compability as possible.
Windows will still run MS-DOS binaries and Windows 1.0 through Windows ME all ran atop the MS-DOS code base in one way or another. They started over exactly once, when they build NT. And they gave it over 7 years to mature before they dumped the old MS-DOS/Windows code. And even with this one example, they ensured it was as compatible as possible to the old, which is why almost any program written for Windows 95 (and many written for earlier OSs, too) will still work with XP, 7+ years later.
Operating systems are a particularly good analogy, too because, like e-mail, it is a critical piece of infrastructure that depends heavily on interoperating with what else is out there.
LOTS of spam is passed through open relays. Closing Sendmail open relays has been easy for A LONG TIME now. Yet hundreds of open relays still exist. A new protocol is spiffy and all BUT WE CAN'T GET PEOPLE TO USE THE EXISTING TOOLS. A new 'magic bullet' ain't the answer, education is, boycott may be, and use of blacklists can help. Implementation of Tripoli is nice and all but if we can't get people to upgrade to a sendmail/qmail/... with closed relay support how do we get them to upgrade to Tripoli? Figure that out and then use the same method to get the open relay holes closed with the existing tools and save the Tripoli coding time. Hell, spammers that spam from their own address get blacklisted pretty quick, use the blacklist, and close the relays.
(Yeah, I know the blacklists aren't perfect but we can't even get that to work, a new tool isn't likely to work either.)
Bottom line, this is not REALLY a technology issue, it's a LUSER/Business issue. A new technology that penetrates 20-60% of the net still gets spam from the other 40-80% of the net. Tech answers work great IF you get 100% market penetration.
Techies like you and I do, and I would rather cast my lot with fellow techies who share in my pain.
Success comes from failure if you dare to try again, revise, adapt, and overcome. I don't see why we should continue to bend over for spammers if the possibility exists that they will exploit a new system for mail transfer.
Personally, the SMTP system has rendered e-mail useless. I'd accept a challenge system, whitelists, or whatever else someone comes up with if it meant I could communicate with my family and friends effectively. As it stands, 100-200 spam messages are jamming the transmission.
So uh, what's insightful about the parent comment?
"Tripoli -- nobody is going to use it" -- FreeLinux, 2003
"640K ought to be enough for anybody." -- Bill Gates, 1981
"There's no possible reason anyone would ever want to have a computer in their home" -- Ken Olson, 1977
"I think that there may be a world market for maybe five computers" -- Thomas J. Watson, 1943
"Stocks have reached what looks like a permanently high plateau" -- Irving Fischer, 1929
"This wireless music box [the radio] has no imaginable commercial value. Who would pay
to hear a message sent to nobody in particular?" -- RCA Executives, 1920
"Who the hell wants to hear actors talk?" -- H.M. Warner, Warner Brothers - 1927
"Heavier than air flying machines are impossible" -- Lord Kelvin, 1895
"I'm gonna get laid this year" -- FreeLinux, 1999, 2000, 2001, 2002, 2003
Just being a curmudgeon, just saying "feh", is not insightful.
If the user has set his headers, and generated a key, the key will match the email header. If the email header is valid or not, the key matches. The problem is one of authenticating credencials not just message integrity. In the end, who do you trust?
Depending on some signing authority to end spam is stupid. Spammers will just buy keys like they buy disposable AOL accounts unless the price is high enough to be a burden on small sites.
Expecting laws to stop people who already make hiding their true identity and crossing as many jurisdictions as possible because they are usually selling ILLEGAL products is insane.
In the end there is only ONE solution. It is the use of encryption/signing, but not the way most people think of using it. Mail User Agents need the following fixes, made so that the average AOL/Outlook user can handle it. By default they only accept mail from people already in the address book. All mail is sent GPG/PGP signed, with the public key attached and the clients grab keys automatically.
When an mail arrives from someone that isn't in the address book it sends them a challenge that only a human can answer (more on this below). If that test passes it allows the original message through and sticks the public key in the addressbook. If the message was not signed it stores the address of the SMTP server it came from as a backwards compatible fallback. The end result is that legit senders only get challenged once if their client signs, otherwise they get challenged once each time they send from a different server. Spammers have to have a human involved for each spam for each user which kills the attraction of the practice.
Now, about those challenge methods that only a human can solve. Make that a plugin architecture. Have modules that send a multiple choice question or two, some that send text as a graphic in some whacked way, etc. Allow people to express their personality through their choice of verification method.
This suggestion would kill spam dead, put only a minimal burden on legit traffic and require no laws or centralization of the Internet. Which is why Outlook will never implement it and therefore the problem will continue to fester.... until enough people become willing to trade liberty for what? In this case, mere convenience.
Democrat delenda est