Slashdot Mirror
Replacing SMTP?
dousette asks: "In reading over one of the RFC's governing the SMTP protocol, and other RFC's as well, it's interesting to note that you see some big names and big companies from time to time. With all the loopholes in the current SMTP specification, is it possible for the Slashdot collective to come up with another one? Would it stand a chance in making it into a standard, or do they just listen to Cisco, AT&T, etc? I realize that a lot of people have a lot of ideas how things should be done (and they haven't been shy about posting them to Slashdot), but has anyone tried to write the RFC for a replacement protocol? As a side note (where I won't be shy about posting how things should be done), if there were a replacement trusted protocol, one could have mail received via that protocol bypass spam filtering, id checking, or whatever checks might be in place (saving processor cycles, etc). The regular checks could still be done on other mail received via the 'older' SMTP protocol. If more and more ISP's make use of this, SMTP could be gradually phased out... or if you are one for a sudden cut-over, just cut to the new one at the same time as the IPv6 upgrade!"
This would work until the spammers start using the new protocol.
It's the implementations and default configurations that suck.
im2000
But there are protocols that needs upgrading aswell.
For an instance HTTP, it could use some on the fly compression (which would speed up things a bit).
GAAH! MY PRINTER IS ON FIRE!!! PUT IT OUT! PUT IT OUT!
Option to not even let e-mails that aren't on the list get in. White-listing is going to be very important.
-Libertarian secular transhumanist
Why doesnt the new implementation use the evil bit. It the server is written by m$, or running on an m$ platform it sets the evil bit. If its running under linux it doesnt set it and ignores all mail comming in using evil bit! :P
Simple really :P
Can't Jabber do a lot of what you're asking for?
No, the "Slashdot collective" has no realistic chance of replacing SMTP. But then, neither does Cisco or Microsoft or Sun. Not with umpteen trillion SMTP servers out there, all of which would need to be replaced en masse.
SMTP certainly isn't perfect, but I'm not sure what improvements need to made to the protocol. Mail *should* be open and unrestricted. While I realize many people have issues with the current system (such as spam), I think most of these should be corrected at the server or client level rather than at the protocol.
"is it possible for the Slashdot collective to come up with another one?"
SlashDot Transfer Protocol - Essentially, the way it works, is the information is posted on one single, easily crashed server. Then, this information is linked to by Slashdot. Then, said server is taken down. However, 1,000 other posters will have mirrored it by then, therby helping in the "transfer" of the information.
Jason Lotito
Dan Bernstein also created QMTP protocol for fast mail delivery. Check it out at http://cr.yp.to/proto/qmtp.txt
SPAM is a problem, but I think it can be fixed above SMTP by whitelisting or webs of trust. What are these "loopholes" in SMTP?
You're probably typing on a QWERTY keyboard, right? Why? Its function is to slow you down so that you don't jam the typewriter.
Moral: Just because one design is better than an already widespread yet inferior design does not mean that it can and will replace the current one. Change is not easy in the least.
In case the site (or routes to the site) get slashdotted. Here is a mirror. The mirror is HTTP, while the original link is FTP. So if you're browser isn't handling FTP so well, this mirror will work in pinch. It's a pretty big file: > 192k
--
Martin Studio Slashdot Effect Mirror Policy
Go to the ietf's application protocol division, and request a working group be formed. I'm pretty sure they'd be willing to let you form one. And i'm also pretty sure that alot of people would be interested. Or if you prefer; form an ID and submit it for approval. I personally prefer the working group approach, because it may take a bit longer, but allows for a broader range of input.
I don't think the name needs more work. We can just call the new one SMTP Hi-Speed and the old one SMTP Full-Speed. If the USB people can do it, so can we.
The latest revision of the SMTP protocol (rfc2821.txt) has support for IPv6.
If security is an issue, you can always use SMTP over SSL, and there is some support to allow authentication.
In a nutshell? I dont think SMTP is going anywhere in the next few years.
SMTP is so deep-rooted and pervasive already it will be a long, hard change to implement. Every little cellphone that comes with a mail-client. Every router that has smtp alerting. Every application that uses it for various tasks... they would all have to be updated!
Doesn't mean it shouldn't be done, but don't be fooled into thinking it's just a "propose a new spec, step 2?, profit" type of deal....
--D
>It would only cost me a couple of bucks a month too at the rate that I send email ...
Sure... what if you wanted to run the Linux Kernel Mailing List in your server?
http://arhuaco.org/
I mean, it would be nice to not have spam an all - but I've been having great success with POPFile so 100+ spams a day don't bug me anymore. And there are other programs and services that give other people similar relief. I like the idea of having a "wilderness" sort of Internet - it promotes innovation and new ideas (I use Baysean filtering for a lot more than just Spam now). We are losing this wilderness everyday - RIAA, Homeland Security, etc. Let's not kill off our electronic diversity!
YOU CAN'T RUN A PAY BY THE EMAIL SYSTEM UNLESS THERE IS *1* EMAIL SERVER (or network of servers run by the same entity) FOR THE WORLD.
you don't realize that IM is a form of email? you are just sending packets of text... the second someone charges for SMTP, i'll just run my own. you could just charge the end users for data transfered instead of flat monthly fee, but most wouldn't go for that.
MARIJUANA, SHROOMS, X: ONLINE?! - E
While I would gladly pay for a (reasonably priced) email system, I'm not sure the outbound payment scheme would work. In my daily use of email, i don't send more than 5 or 6 *internet* messages per day. Of course, lan messages to coworkers is quite different. Obviously this would impact telecommuters and independant contractors.
Where my main cause for concern is, is the use of email by corporations for notifications and account issues. Every time i make a purchase from amazon.com, i don't want to have a $0.30 or so in hidden costs factored into my shopping experience for the order confirmation and shipping notification messages i should get. When you factor the number of sales (and thus, the number of emails they send out) it can really eat into amazon's bottom line, which affects me.
I totally agree that the system needs to punish and prevent spammers though. I just think doing harm to a section of home and legitimate email users is not the answer.
I'd like to simply see SMTP updated to require work. On establishing a connection, a recipient should be able to give the sender a task to complete that takes a second or two. The recipient will only accept the mail once the work unit has been done.
This would make it too slow to send spam, by making it simply too processor intensive. Legitimate users would be unaffected.
IANAProgrammer, but my idea about the spam situation is relatively simple. Scan the text of all incoming mail and compare them. If more than $x ppl are receiving mail that is similar to within $y characters, block that e-mail. You'd probably want to mess with some numbers to come up with optimal values of $x and $y. This would definately increase delivery time though...
So, combine it with a whitelist of trusted senders. This trusted senders list could be implemented by using the receiver's address book and list of most recent received (int z;) messages from unique senders.
Now, the big problem is still false positives on valid e-mails. But, there should not be many of these, simply because of the first comparison check.
It is my understanding that IMAP(?) implements server-side mailboxes, so I believe this idea is possible.
Webmaster Wanted - Entropic Reactions
I agree that something ought to be done to cut down on the huge volume of spam that clogs most SMTP traffic.
On the surface of it, a white-listing system, perhaps based on public-key cryptography and endorsements might work.
But, as someone who values freedom and anonymity, I'd hate to have a system that closes off completely the opportunity for more anonymous communication via email.
Whistleblowers in the government and in the corporate sector, dissidents under a repressive political regime are some of the use cases for email that I'm not really inclined to sacrifice merely to eliminate spam.
"Provided by the management for your protection."
The "pay for email" approach would only work if it was possible to whitelist addresses who would then not have to pay. The mailing list problem then would not exist -- you simply require that anyone who signs up whitelists the mailing list address.
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
I got so tired of sending my bank account info to sons of exiled African presidents that want to put $35 Million (US) in my account that I just set up an auto-responder. Perhaps the new protocol could handle this for me.
With all the loopholes in the current SMTP specification,is it possible for the Slashdot collective to come up with another one?
To start with, I would suggest a detailed look at RFC 2549.
The Standard for the Transmission of IP Datagrams on Avian Carriers described therein is fairly broad and could prove a feasible alternative to current email delivery mechanisms, specifically SMTP.
The reason I think it hasn't taken off since 1999 is that it proposes to completely replace IPv4 (like IPv6). Maybe it would be easier to first phaseout SMTP over IPv4 for now, rather than the whole IP layer.
An Indian-American Hindu committed to non-violent thought/speech/action alarmed by the global explosion of radical Islam
The QWERTY-slow typewriter story has been debunked. QWERTY forever!
This is a rather silly article. If you want to create a new protocol - do it. If you want to create an RFC - do it. The IETF publishes instructions on the steps that must be followed to create an RFC - see RFC 2418. There is nothing stopping you and you don't need Slashdot approval to accomplish it.
Like IPv6? You mean most things will already be there but no one will support it, no one will care apart from a few and no one will implement regardless of how hopeless and disastrous the current implementation is?
Ah yes, like IPv6 indeed. You know, I'll send a shiny mail delivered by SMTP2* over an IPv6* internet about the release of Duke Nukem Forever* to my gaming-addicted girlfriend* on the day SCO coughs up some evidence*
Note:
* = May or may not require divine intervention.
Hate me!
With all the loopholes in the current SMTP specification, is it possible for the Slashdot collective to come up with another one?
Yes, I'm sure that Slashdot is up to the task of coming up with another loophole.
What's wrong with SMTP AUTH? If people would just enable that instead...
With all the loopholes in the current SMTP specification, is it possible for the Slashdot collective to come up with another one?
I'm sure the Slashdot collective will come up with another loophole in no time.
Mod the parent up. Another link about the qwerty myth is here.
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
Hrm, never seen that before, im2000 has some good idea for simplifiying things, but it seems like it would just be unreliable and unfeasible.
With the current system, an smtp server can go down, and no one would notice because no one was received their email yet, but with im2000, if the sending machine goes down, then no one can read their mail from there. This would create a lot of unknowns, "why can't I read my email?". Also what about people that don't have a full on connection, you don't want to require those people to be connected just to read their mail. Sure you can queue it for downloading offline somehow, but that's going to be much slower than normal because you have to connect to say 30 different servers where your email is hosted.
Also there's the case of somesmallcompany.com sending out a mailer/advertisement to millions of people, because the email is hosted on their machine, their connection/server might become overwhelmed, causing heaches for everyone wanting to read their mail. "Why does my mail load so slow?"
It's a nice try, but it'll never work.
Another thing, what happens when the message is done being read? Is it deleted on the sender's machine? If so, then how will the user remember that they sent the email to check if it's been read. If not, when will the message get deleted? Obviously it can't stay there forever.
The great thing about the current system, is that you just send and forget. If it bounces, you get a new email message saying hey, something went wrong. But with im2000, if the message hasn't been read yet, WHY? Did the user just not check their mail yet? Is there connection/routing problem where they suddenly occurred after the hosting server sent the notification, etc.
Maybe ISP's should charge users for each outbound SMTP connection they make? I'd happily pay 10 cents per email I sent if it would reduce the amount of SPAM I received. It would only cost me a couple of bucks a month too at the rate that I send email ..
Do you suppose the ISP that is owned by SpamCo, Inc. would actually charge its users the fee?
The price of freedom is eternal litigation.
What a lame question.
Can we, as a community, do that? No. Why not? Because it's really, really hard.
The IETF, as a community, is not dumb or lazy. (Some of us individuals are.) Many contributors to the IETF read Slashdot, as a matter of fact. But the reason spam is a problem is not that SMTP is flawed; it's because Internet email is successful. It's successful because it ignores the problems of authentication and authorization and just lets anyone send mail to anyone else.
Would a new protocol (or an SMTP extension) fix the problem? Of course not. Spam happens because you typically want people you don't know, with whom you don't have a relationship, to be able to send email. It's easy to solve the spam problem: it's a trivial special case of either a public key system, general text classification algorithms, or micropayments. We've been waiting for each of these for decades, but I'll leave it up to you as to which one you want to solve first.
The ideas mentioned for a "trusted" protocol do not require a new protocol. SMTP, perhaps with extensions, can be used to handle the vague ideas in this story.
> I wish people would stop inviting rate increases or new charges as an answer to spam. It's not the answer.
:)
:)
And the perfect example is regular junk snail mail. It costs them to send it, yet even in the Internet Age(tm), I still get a ton of it. Obviously that's NOT the answer, so "Don't Go There"(tm).
I think locking down SMTP servers and requiring verified & correct return addresses would go a long way toward curbing spam. Then when you disallow someone to send you mail, it could really work.
A combination of white lists/black lists, and Baysian filtering stops so close to 100% of spam that it's really silly for anyone to be bitching about spam these days. I don't GET any spam anymore - 0. Not 0.001%, 0 - the integer 0, as in none. If I ever get another piece of spam, then I'll change my email address (I can do that more easily than most as I have my own domain.), though this isn't the answer for everyone - lots of people have e-mail addresses printed up on lots of expensive cards & letterhead, etc. For them, the white list / black list / Baysian filtering solution should suffice way more than anyone should practically need.
Stop yer bitchin', people, and implement the technologies that are already out there and work great. Plus use yer freakin' brains for a change, and don't spew out your real e-mail address to everybody who asks for it. Use your friend's!
Working out the details of an appropriate certificate policy is not trivial, though.
C'mon now; the IPv6 upgrade will be spread out over at least several decades. And both Microsoft systems and many US Government installations will still be using it a century from now, because it's "standard".
After all, it's now past the death of typewriters, and we're still using the typewriter keyboard from nearly two centuries ago. And we use a ridiculous rail gauge, because the standard was set centuries ago.
And here in the US, we're still using inches and feet, measurements based on the lengths of the thumb and foot of a long-dead king. And we call them "standard".
We will be stuck with IPv4 for long past the final download of anyone reading this.
SMTP will probably be around even longer. But that's OK; it's fun to impress friends by a "telnet 25", followed by typing in a message directly to the server. I like to use "MAIL From: dubya@whitehouse.gov", and ask them if they'd be interested in a nice job in the TIA program. Then I challenge them to prove from the message they get who actually sent it.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
Technological fix to a social problem.
It's simple. Don't bother.
The problem will remain, it will just shift tactics. By 'fixing' SMTP you're not addressing the problem, you're addressing a symptom of the problem.
Anything we do on the technology side to fix this problem will ultimately do nothing.
That's not to say that SMTP can't be improved on... but improving on it purely to 'stop spam' is a waste.
Chicks suck.
Guys are ugly.
Pass the kleenex.
But if everyone were to use Bayesian I swear we wouldn't even have to propose a new protocol, talk about new legislation, etc.
Agreed, people often seem to miss the point. Spammers only exist for one reason, because some morons actually bought something advertised in spam. If people stop paying attention to them, nobody would pay for spam to be sent, and the problem would just go away. It does cost spammers money (not much, I know) to operate, and if they have to lower their rates, or can't attract enough customers because most people just ignore spam, they'll go out of business. But as long as your grandmother keeps buying every crappy photo frame she sees, and you keep looking at porn, spam will probably be here to stay. I don't like it, so I filter it.
--That's the point of being root, you can do anything you want, even if it's stupid.
IPv6 is going to be about as "sudden" of an occurance as the production of Duke Nukem 4ever.
People who think they know everything really piss off those of us that actually do.
An IPv6 like protocal for SMTP would be fantastic. Completely tracking and accountability..
Actually, I've been working on a broader based piece of infrastructure than a new mail protocol, but the first problem I intend to attack is mail.
RFC 822 is fine for messages, but the transport needs a big upgrade. Also, envelope senders and receivers are non-verifiable, and therefor broken. One day, spammers are going to start using mailing lists and message boards to construct a profile of people you talk to, and send you mail that appears to come from them, thereby making whitelists useless.
The basic premise of my general transport is that all messages are addressed to a public key and come from a public key. All messages are signed by their supposed source ID, and most messages are encrypted to the destination ID.
A public key ID plays a similar role to an IP address in an IP packet. There will be distributed databases that hold (signed) mappings between public key IDs and their locations using other networking mechanisms.
I'm trying to design this protocol and its implementation so its easy to encapsulate it in almost anything. My first connection to an outside protocol will be IMAP/SMTP.
It's far from being ready for even a public alpha yet, but I do have preliminary code for creating certain kinds of messages at https://svn.generalpresence.com:5131/repos/trunk/C ++/pract_crypto/. I'm borrowing heavily from Bruce Shcneier and Niels Ferguson's latest book, Practical Cryptography. The initial implementation is in a mix of Python and C++. It requires Swig and the GMP library. I haven't designed the implementation itself to be in the least robust against attacks by someone who has root on your machine.
I am calling the protocol 'CAKE' for now. CAKE stands for Key Addressed Crypto Encapsulation. It is a layered protocol, since I intend it to be layered on top of any other protocol you can think of. :-)
One intention of mine is to publish a hash collision problem along with information mapping a public key to a mailbox. First time senders will have to solve the hash collision problem to avoid having the mail thrown away. I'm planning on simply wrapping an RFC 822 message in a CAKE shell.
Need a Python, C++, Unix, Linux develop
From there it can be evaluated, a Working Group created to push it through engineering review to Last Call, to proposed standard.
Sounds easy, well you can expect to spend aproximately 20 hrs/wk on it for 3 years, and that is if it is a non-controversial idea. For something controversial like changing the SMTP protocol, expect it to effectively never happen, why you might ask... Well lets say the first problem is to define SPAM, we will move on from there (Do I have to mention the ANTI-SPAM BOF held in San Fransisco that was a complete waste of time)
I have mod points and I am not afraid to use them
I thought SMTP was Spam Mail Transfer Protocol!
Just kidding.
Seriously, because of spam issues, there have been many proposals for ways to replace SMTP or to modify it. Some of them are downright comical.
But it's going to take something a lot bigger than that to change anything.
Any replacement would have to be completely backwards compatible with SMTP for years to come. Many people would never switch. Others would switch only after seeing it in operation for a long time.
Since it would have to be completely backwards compatible for years, any spam getting through now would still get through for years to come.
I think that what might prod most to change would be if one of those crazy schemes of having the Post Office charge postage on e-mail were to be enacted. Then, you would see the creation of something else designed to skirt the definition of e-mail under the scheme.
What you can do is to come up with an optional private method that would not break SMTP. That way, those who didn't want to use it could get along just fine without it.
Such a scheme would probably work best if it were adopted by a large percentage of the Internet. For that reason, it should be usable for everything form small personal SMTP servers up to very large SMTP servers that handle millions of people.
For what it's worth, I've considered the idea of just rejecting all incoming e-mail to my accounts that is not encrypted.
You're an idiot. The article that you pointed to substanciates the story. I'd type more but this qwerty keyboard is hurting my hands...
Bah!
Why not P2P email? Then all mail is on the senders machine until the recipient comes online to check it. It seems P2P is perfect for such a system.
Is it possible for the "Slashdot collective" to come up with anything but a bunch of trolls, whiners, masturbators, and goat secx? What cave was I in when the "Slashdot collective" turned into a productive development community???
There is a fine line between being a cultivated citizen and being someone else's crop. - A. J. Patrick Liszkie
Yea, and the spammers would get the bulk rate of .001 per email.
This scheme is an important part of the old UUCP package. Part of its handshake protocol is a message that lists all the protocols that the caller understands, in the order the caller would prefer to use them. The recipient goes through the list, picks its favorite, and sends back a message saying "Let's use X."
The advantage to this is that you can introduce new protocols completely painlessly. You pick a new name (after asking around on the newsgroup if anyone is using it), link your new protocol module into the protocol tables on the systems where you want to use it, and start using it. If you connect to a machine that doesn't have your protocol, it will simply tell you to use one of the others on your list. If your protocol is good, it will spread and will be early in the table for a lot of software. It can then slowly supplant the older protocols.
And you stay compatible with older systems by merely keeping the old protocol modules in your tables.
This is 1970's technology. So I suppose we'll soon read that Microsoft has just patented it.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
I think coming up with a new way (protocols and the like) to handle email traffic is a noble idea. However, as long as there are people trying to devise a solution to the problem, there will be others (spammers) finding holes in the solutions. The only way I believe to take care of this is through ISP enforcement. There is no other way. Unfortunately, no one likes the idea of having any more freedoms taken away by ISP's and the police.
I would not like to have to pay any extra amount for any email that I send out. (I only send out around 10 a day on the average-- @ 10cents per each that would cost me an extra $30 a month! $360 a year) A solution may be to charge a tax (millage, fee, what have you) whenever a sender decides to send more than a certain amount of emails in a certain time period. Unfortunately, there will be ways around even this.
What it boils down to is freedom vs security and/or convenience. Personally, I don't see spam as a threat to me (I can delete them rather quickly--and I am getting many daily). I'd rather not see any new laws concerning internetworking if at all possible. Things look scary enough with the Patriot Act and those that are willing to give away freedoms for security and conveniences.
Because the cost becomes built in to their business model. it won't stop, it will only hurt regular users to charge for email/services. Sure, their profits may be cut a little bit, but that's not going to stop them. if anything, they'll do it more, because if their profit margin is smaller, they'll have to spam harder... right?
Non-email messaging systems have been thinking about virtually the same problem quite a bit, and have come up with a set of solutions that try to solve what are fundamentally the same issues: message integrity, message non-repudiation, and message authentication. And the surprising part of this is that nobody really focused on the protocol, because it doesn't provide the path to a meaningful solution to the problem.
Case in point: web services. While initially the people who were playing iwth web services started out doing security at the transport level (i.e. with SSL and various derivatives thereof), but realized that something like WS-Security (where the security of a message is a part of the message itself) is the more optimal approach.
Why not just force the issue into the realm of S/MIME (and similar extensions to rfc822) and handle it at MUA space? You can cover virtually all the problems with SPAM by following the example of the reliable messaging systems and doing more with the contents of the message itself, rather than trying to say that messages have to transmit over a particular protocol. For example, depending on your trust environment, S/MIME signatures solve the authentication, non-repudiation, and integrity problems perfectly. What more do you need/want?
Why not have a system where to send mail, you HAVE to have a digital certificate issued by a trusted third party. The certs expire, and you have to provide legit details to get one. All mail sent is signed with your cert. Get too many complaints, and your cert is revoked. Servers only accept mail with valid certs. Allow cert blocking to block all mail from the same company regardless of route.
What do others think? Could it work?
Maybe ISP's should charge users for each outbound SMTP connection they make? I'd happily pay 10 cents per email I sent if it would reduce the amount of SPAM I received. It would only cost me a couple of bucks a month too at the rate that I send email ...
John Dvorak suggested a scheme along these lines, and in theory, it's a good one, though I'd suggest a tenth of a cent, which would still make sending a million emails prohibitively expensive.
In practice, though, it's not workable. Spammers aren't using the SMTP server their ISP provides; they're using their own, just like most desktop Linux users are. As far as the ISP is concerned, Spammer X is making a bunch of outbound connections, but they're streaming out through the ISP's switches and routers, not through their SMTP server.
To impose a tax on certain kinds of TCP connections would require detailed inspection of outbound packets. This is because a single SMTP connection can involve the transfer of many messages. To be reliable, the ISP would have to parse every outbound packet bound for port 25 on a remote system in order to count the number of emails sent. I don't think most people want that level of attention paid to their private emails.
Moreover, this presumes that all ISPs participate honestly and thoroughly in such a system. All it would take is a few spam-friendly ISPs (and they exist, are legion, and jump around IP ranges like ferrets on a hot skillet) to render such a system useless.
The alternative would be to implement email billing at the recipient side. Maybe AOL and Earthlink can pull that kind of blockade off, but small companies and J. Random Luser cannot.
Bernstein's IM2000 proposal at least keeps the bandwidth consumption down, but that's primarily a cost issue for ISPs. (Don't try to convince me that if the amount of spam declined, ISPs would lower their prices.) The main hassle of spam for the user is that it takes time and energy to delete spam, and having to inspect the stuff with ambiguous could-be-from-someone-I-know subject lines would not be alleviated by IM2000; you'd still have to pick and choose what pending inbound email to read or delete.
The fundamental problem with email as a mail system is that it's open to anyone who wants to send mail -- which is part of the point of mail in the first place -- but there is no economic limiting factor for the sender as there is with paper mail. Since we can't eliminate the openness without destroying the utility of the system, the only possible strategy is to artificially impose a cost on the sender. Unfortunately, owing to the nature of public networking, the only remotely reliable way to do that would be to route all mail through a centralized clearing house. No one company will be able to establish such a monopoly, and I don't think anyone wants the alternative -- which is to have the government do it.
This may or may not be a soluble problem, but it is, as of today, still an unsolved problem. Personally, I think it's going to take national legislation and international agreements to stop it, and that will no doubt take a long time. Paper (actually clay tablet) mail existed for several millennia before the International Postal Union was finally established. Let's hope email is brought into line a little faster than that.
Proud member of the Weirdo-American community.
If the sender has to store the message, then suddenly they have to be accessible.
That's important. A reachable IP makes identification easier. Besides, the sender is also responsible for storage and bandwidth, and those are costs I can bear for my personal correspondents, but that would move spam a little further from free.
The question is a better one than I thought when I first read it. Although it doesn't sound like IM2000 can guarantee a low enough level of spam by itself, I would not only install a new client, but actually serve a new protocol if it was a way of delivering email without spam.
Assembly is the reverse of disassembly.
I think you have to completely drop SMTP and implement a new, more secure protocol. You can yell about backward compatibility all you want, but this is one case where I think we just have to say "too bad". If we do that, everyone will do what it takes to make the switch, since e-mail is so vital to most everyone.
SMTP really is the root of the spam problem, since (using the defaults) anyone can send mail through any mail server, using any address, with no authentication. Sure there are add-ons and options to tweak, but I think we need to implement a protocol that requires authentication (user/pass at minimum) to send, and can only send from the account that authenticated.
Another thought I have is to have a registry system for mail servers, like there is for IP address space and domain names. You have to register with some central authority to be allowed to send mail to other mail servers.
This would still allow for unregistered, internal type mail servers for LAN mail, but you'd have to register in order to send out of the network.
These are the thoughts I have. Granted, I'm not an expert and don't know the technical feasibility of my ideas, but logistically I think they make a lot of sense, and I think they shouldn't be too difficult to implement.
D. J. Bernstein, the author of the supremely reliable and secure qmail mail server, wrote a proposal for a new Internet mail system a couple of years ago.
The key words there are "a couple of years ago." Yes, a few years ago he published that web page. And you know what happened as a result of that?
Absolutely nothing.
Why?
Because it makes no sense whatsoever.
Bernstein came up with a lot of stuff over the years which did make sense, and which did take off, and became accepted. This isn't one of them.
So what if the mail's contents are now stored on the sender's system? How's that going to prevent a spammer from keeping a copy of the spam stashed somewhere, and then spamming a few hundred million mailboxes with a link to the spam. So, what have we accomplished here?
The problem with spam is not where the bits and bytes that make up the spam are physically stored. The traditional definition of spam is "unsolicited junk E-mail". Jiggering the bits around, and saving them in a different directory, or on a different server, will not magically turn unsolicited junk into solicited E-mail.
"...if there were a replacement trusted protocol, one could have mail received via that protocol bypass spam filtering, id checking..."
...and if money were free we'd all never have to work again. Let's get started right away on a project to make money free!
C'mon guys. SMTP actually works fine for what it is supposed to do. We have TLS and PEM which both work through it fine. These other "problems" being cited are things that can not be solved without resource-intensive calculations of one form or another, and as such are not likely to be solveable on the general internet for just any wanker to use anytime soon. On private networks, it might be possible, but then, private networks don't have to deal with the fact allowing just any wanker to use it is the biggest problem.
It is realistic to expect SMTP to mostly work even as users grow increasingly retarded, which is where we're at. It's not realistic to think one could even design a protocol or system that is both useful and able to properly function as stupidity approaches the maxima.
Simple solution: not let you run your own. I know you can for SMTP now, but if they changed protocols and implemented some kind of "trusted" or "registered" server list, you're "pirate" mail server would be an island.
SMTP works for it's intended purpose lets extend it to SMTPv3 (I think it's been extended once allready) So here is a general brainstorming.
If you exent it two servers with the extensions will send mail in the new more secured format.
Frst things first keep it friendly you should be able to do everything via telnet because it jsut makes testing easy.
So what do you need a little bit on how to get into this new mode once your connected to port 25.
The server must offer what encryption methods it allows as a list more perfered methods first. Unencrypted should be an option all senders and receivers MUST allow it as encryption is nice but CPU heavy and you can allays depreciate enencrypted senders.
There should be a DNS entry for the sending mail servier in the domain that the from address and the reply to address originate (some new DNS field well it's a nice big distributed DB with cache so why not?) This needs more work and it sorta outside of scope.
If the sender domain is part of the servers domain of responcibility the server must use the from and replyto addresses to authenticate the user(s) passwords via CHAP, Kerebros etc this MUST be done after the encrypted state is up and can NOT allow unencrypted passwords and perferable uses a CHAP like system where the password is never on the wire.
The receiving server must include all options specified by the sending server as message headers.
The server MUST only accept mail that is destined to it's domains or source from one of it's domains if accompinied with valid credentials.
A server to server intermediary authentication may be implemented.
OK thats no where near completed but it's a start.
No sir I dont like it.
Apparently you didn't read the whole thing. Dumbass.
I don't think this is a good idea. The **AA would probably just legislate some sort of DRM into the protocol, making it a crime to open your email because it would be violating the copyright of the sender.
SMTP already has authentication, and anyone who operates an SMTP server is free to accept or not accept mail from whomever he wants. You don't need a new protocol to require mail to be authenticated. If you can solve the trust problem, you can implement a trusted mail solution more quickly and easily with SMTP than by requiring deployment of an entirely new protocol.
AOL won't accept email from my mail server, which has never sent a single spam, because other sites on the subnet are spamming.
What we need is a system based upon asymmetric cryptography, whereby I can sign your public key to note that I want to receive email from you. Anything that looks like mass mailing could be tested for a signature.
You can't judge a book by the way it wears its hair.
Ever heard of it?
That is a very good point. Making spamming costly will not stop all spammers. Those with a legitimate business will still spam because there will still be enough legitimate customers out there who buy based on their spamming.
But making spam costly will indeed stop the majority of spam that is sent today, which is useless, annoying stuff that far less than 1/100 of 1 percent of people actually make a purchase based on.
That is the kind of spam that I really want to stop, and I think that making spamming cost money will have the desired effect.
In my mind I see a smooth graph of how much it costs to send junk email versus how many companies will send junk email. Right now we are at the end of the scale where the cost to send spam is almost nothing, and so the number of "companies" that will send out spam is HUGE, and not-concidentally the content of that spam is very poor.
We need to work our way to the end of the scale where spam costs real money to send, and only a moderate number of legitimate businesses then find sending spam worthwhile. And the kind of spam which is sent is then would be less annoying (to me anyway) as well. Spam would be more like advertising of actual products than make-money-rich schemes based on trying to sell snake oil.
That's not the answer either. Microsoft, Yahoo, et al have been lobbying for this approach, and for good reason. They want to function as the certificate authority (CA). They want to determine who can or cannot send email. They can use that power to literally sell the ability to send spam. They can also use that ability to censor their opponents.
Microsoft also wants a new patented standard that can't be legally implement with open source software.
While we're at it, why not PGP sign with the server as well? Signed mail from allowed server or recipient could pass through the spam filters while everything else could be sent through much more stringent filters. This way all of your family using hotmail could still reach you, which mail with spoofed hotmail source addresses could not. If nobody is willing to host the key servers, each mail server could provide its public key with an incoming connection. Also, clients sending mail should do so via an extension to POP3 or IMAP to end the nightmare of auth/relay configuration and clean things up further.
Whatever is implemented must be allowed to work with the current SMTP during a transition phase and must prioritize itself in some way (otherwise the spammers will just keep using SMTP).
People are working on projects to replace SMTP, for examplem, SMAP:
http://www.courier-mta.org/cone/smap1.html
Looks interesting,
A.
--
Adam Sherman
Freelance Geek
So... instead of a friendly hello, the mailserver should instead answer for an incoming connection request with a gruff "nuqneH!" (Klingon for "What do you want?")
Why not make an SMTP spec that is backward compatible with SMTP but offers new methods of user verification/authentication over SMTP before the DATA segment.. er wait.. they've already done that..
or SMTP Mail Transport Protocol in the GNU tradition..
I don't think the /. community could do this. Why? Too many idealists. Look at all the "successful" protocols (HTTP, POP3, etc) - they all are loaded with problems, but regardless, they get the jobs done and where appropriate, get fixed over time. A pragmatic approach is required IMHO - something that does the job and that a large group of people could agree on. Pragmatism & consensus are not things the /. community are renowned for.
Read reviews of shopping cart software
The focus of the linked article is that QWERTY should not be held out as a "market failure" (that is, an example of "locking in"). However, it clearly states its point: All the article is saying is that the cost of retraining is not worth paying, which is the same reason why Windows is better than Linux. Remember that this is the opinion written in The Economist, not a study by ergonomists.
DJB's Internet Mail 2000 spec includes a method for interoperating with the existing mail infrastructure, by using low preference MX records as described below:
MXPS: the Mail Exchanger Protocol Switch
Normally, when a client has responsibility to deliver a message to a remote recipient, it looks up MX records and attempts to make SMTP connections to each of the addresses it finds.However, an MXPS client that supports QMTP and sees the MX record 12801 mailin-01.mx.aol.com will try a QMTP connection to mailin-01.mx.aol.com before trying an SMTP connection to mailin-01.mx.aol.com. The point is that QMTP is faster than SMTP.
Here is the general rule. Particular MX distances are assigned to protocols as follows:
- 12801, 12817, 12833, 12849, 12865,
..., 13041: QMTP.
If the client supports MXPS and QMTP,
it tries a QMTP connection to port 209.
If the client does not support MXPS or QMTP,
or if the QMTP connection attempt fails,
the client tries an SMTP connection to port 25 as usual.
The client does not try SMTP
if the QMTP connection attempt succeeds
but mail delivery through that connection fails.
Of course, servers are obliged to continue supporting SMTP for non-MXPS clients, and clients are obliged to continue supporting SMTP.In the far future, if all clients are upgraded to support QMTP, it will be possible for servers to switch from SMTP to QMTP, turning off SMTP. In the farther future, if all clients support QMTP and all servers have switched from SMTP to QMTP, it will be possible for clients to drop support for SMTP. In the short term, none of these simplifications are possible, but clients and servers can benefit from the speed of QMTP. History I proposed MXPS in the documentation for the first qmail release in 1996. In the original design, distances 12801 etc. were assigned to QMTP without an SMTP fallback; this type of assignment makes sense for potential future QMTP-only servers but makes current QMTP+SMTP servers unnecessarily difficult to set up.
Look up hashcash, then extend it to the point where the recipient gets to specify the problem in some kind of bytecode. The sender must run the program and come up with a numeric result which is sent back to the recipient's server. That result effectively proves that the sender "did the math", so to speak.
If everyone used this, unsolicited bulk mail would be extremely hard to spew, since the computational power required on the sending side would be extreme.
Now to connect this with the payment stuff you mentioned - ISPs would charge their own users for the privilege of running these little programs. Maybe you'd get so many free CPU seconds per month, but after that they charge you.
The ISP already has an arrangement for charging their users, so this can actually work. No money changes hands between the sender and the recipient.
By the way, by bytecode, I'm talking about some hypothetical language that only allows mathematical constructs. Maybe you specify some kind of formula and say that it must be run through an iterative process 50K times. When the spammers catch up in terms of computing power, write new challenge programs.
Final note: you'd obviously want to whitelist your friends, or they'd have a hard time mailing you.
Among other things, I do use Bayes, I've tried several different implementations of it (currently the Moz Thunderbird one seems to work best).
Most do a very good job, but the problem is they don't work perfectly. For those of us who use email for work, *one* missed email can cause a lot of trouble and, occasionally, risk of job loss.
I get maybe three messages a week that look like spam from the header, get classified as spam by the filter, but which are important messages that I can get in a lot of trouble for if I don't read.
Mail from an semi-literate customer using a random hotmail account really has to be looked at by a person.
The other thing I've noticed is that some spammers are obviously trying to reword their junk so it can pass the filters. They're not very good at it yet, but I can see they'll get there.
I really can't see any solution that keeps the spam flowing working.
I do like the idea of storing the sent mail on the senders (ISPs?) mail server. It's simple, practical, and consumes the senders resources. And if the ISP cuts off a spammers account, then all the unread spam is *gone*.
Those who host big mailing lists are just in the same situation as those who host big web sites. I'm sure a system of mirror feeds could be implemented to help out.
- Muggins the MadI would agree with this, except I get spam that is not an apparent advertisement for anything. I suppose it is just a test to see if the address bounces, but still, I can't understand why I get so many like that...
Nerd rage is the funniest rage.
A better solution would surely be to add an additional layer of functionality to ESMTP via another HELO/EHLO varient. The additional layer could then add whatever additional security and validation was required by the new RFC to help prevent, or at the very least, filter spam. Also, because it's an RFC, it becomes possible to require, or at least recommend, some of the things people keep moaning about being broken in the current ESMTP.
Besides, (E)SMTP isn't *broken*, it just wasn't designed to do what is now being asked of it because at the time it was designed such things were simply not an issue.
UNIX? They're not even circumcised! Savages!
As one of the posters in this thread pointed out, it would go a long way towards helping establish accountability on the part of spammers. If you have to be accessible in order for "recipients" of your mail to be able to read the mail, then I suppose you'd have to be pretty easy to track down and also sue under the current anti-spam laws.
... I've been using it on my server for years and really love it. I'd be particularly interested in hearing what your ideas are about the ways that we can solve the spam problem, given that you are the author of a complete modern mail system.
Anyway, it's just an idea. I think that the whole question of "should we come up with a new mail protocol" is kind of misguided because obviously the hard problems here are not technical, they're in dealing with the huge momentum built up around the current mail technologies. It seems to me that we already have all the technology that we need to solve this problem, it's more a matter of enforcement by ISPs. I was just posting the link because it was relevent.
If you're the same Mr. Sam that made Courier, then thanks
I've been working on a proposal to replace SMTP for a few weeks now. It's called AMTP and it addresses two concerns in tandem: Authentication and Classification/Policies. I have a web site set up at http://amtp.bw.org/ for the project. There's an announcement list available if you want to find out when the draft is ready for comment.
home
gimme back my
Keep using SMTP... But after the message is recieved, and the connection dropped, have another mechanism connect to the senders server (from the MX in DNS), asking if it sent this MD5SUM(message), on this date/time? If so, let it through. In not, check the user's preferences to see if they allow non-checked emails, and processs accordingly (place it in a users subfolder, forward to admin, whatever...)..
This doesn't break anything as it stands now, users/admin can choose how its handled, and should be fairly simple to implement.. There would be an overhead cost of keeping track of the MD5's... But it could be done...
Just an idea... Waiting to be shot down...
Slashdot is like Playboy: I read it for the articles
The simplest such policy is a whitelist -- but this means you can't just give your email address to a friend, and expect her to be able to send you mail. It means that if your friends change email addresses, they can't just send you email saying "This is my new address -- my old ISP stinks!"
A more complex policy might include some public key infrastrucure, where a user needs to have a valid key to sign their messages, in order to send mail. This brings up "who do you trust" in terms of who can sign message-signing keys, but more importaintly, who is going to have more trouble with this, a spammer that wants 100 throw-away accounts with keys per week, or someone's Mom who just wants to type a message and click "send?"
I think that it will be little use in trying to come up with a protocol solution to spam, without first defining what security policy you need to enforce. Once you do that, you can design exactly what you need to make your new policy work. (You might even find that SMTP is not at fault -- you might just need a layer above that transport to secure your mailbox.)
Also there's the case of somesmallcompany.com sending out a mailer/advertisement to millions of people, because the email is hosted on their machine, their connection/server might become overwhelmed
I think that's the whole point -- make it infeasable to send a mail to a ludicrous amount of people.
Can't we use Karma?
Simple premise - everyone in the world signs onto the 'KarmaMail' service, and get to send mail at "1". Once enough KarmaMail users validate the user's email as being legitimate, their Karma goes up. Registered users can also complain about a spammer, thus making their Karma go down. Marking email messages as 'urgent' requires a higher Karma. Users with a negative Karma (>= -5?) can only send at '0'. Users with a very negative Karma get booted off the system.
Then individual users can use Karma plus Whitelists to decide who to read mail from. Whenever a server receives mail, it checks with the central KarmaMail repository and inserts the user's Karma into the mail headers (optionally, Karma can be assigned to the *server* as well, eliminating the open relay problem). The header can then be processed by the mail reader.
Maybe someone would care to expand this idea further to clear up the many loopholes I've left?
You can accomplish anything you set your mind to. The impossible just takes a little longer.
People wouldn't have to stay connected for their sent mail to be accessed by others with IM2000 ... it's the sender's ISP that stores his pending sent mail ... not the sender himself.
... they will still have to back-up their users' pending outgoing mail that hasn't been picked up by recipients, and they will still have to back-up their user's 'notification' records that there is mail waiting for them somewhere else. Because of the aforementioned cloning delay, these back-ups will gobble up a lot less space, but they must still be performed. After all, if they lose my notification, that is equivalent to them losing my mail from my end, since I will still have to ask the original sender to send it again ... that's if I was even expecting it!
Also, as I understand it, there would be no need for extra bandwidth strain on the sender's ISP: after all, when you send a mass mailing to say 1000 people (like if you run a mailing list), your ISP's SMTP server has to relay that one email 1000 times. How is that different from 1000 people each connecting to your ISP's IM2000 server to read that email 1 time? If anything, it's better, because the load will be more distributed over time.
The advantage is that the message is cloned at the moment of retrieval, rather than at the moment of sending. This delay in the cloning of the message for each recipient on the list will greatly decrease the overall resources required to cache all the email that is out there on the internet waiting for people who haven't read it yet.
This is all good. You can summarise the IM2000 idea as converting email from the USENET-style distribution model to the HTTP-style server model. Your ISP maintains a 'web-site-equivalent' of local mail whose membership consists of all the valid email addresses in the world.
However, I strongly disagree with the IM2000 document where it claims that ISPs will no longer need to back-up user mail
Have you ever tried to subscribe to a mailing list on an account that doesn't admit anything that is not white-listed?
For what it's worth, I use procmail to implement a white-list on an account that I use only for mailing list subscriptions.
It is nearly impossible to subscribe to a mailing list on that account without shutting down the white list for a while.
SMTP is not the problem. Open Mail Relays are the problem. As long as you can drop an SMTP box on the net and have it spew Spam to other mail servers you will have Spam. Any new Mail Transport Protocol will have to be backwards compatible. i.e., receive mail from old SMTP servers. You can't switch all machines over at one time, you need to roll out one box at a time and keep it all working all the time.
:-)
In the country where I live there is a general rule for farm animals, the farmer is not responsible for fencing them in, it is your job to fence them out. Mail is the same, its not my job to stop spam being sent, but to stop it being delivered (to my users) There are many ways to do this, a combination of a few can be very effective.
As for the home user, well stop buying into the "submit your e-mail and we will send you porn" forms on the sites you wife does not want you to look at.
What about people that like to forward their mail from one account to another, or even accounts that only allow forwarding, so they just bounce your mail to a different address(convenient when you switch isp's). I wouldn't be real happy if I had to pay extra for each message, and I really like that sort of setup.
Nerd rage is the funniest rage.
Maybe a root whitelist (similar to the root nameservers) could be used. Instead of RBL's trying to keep up with spammers people would have to register their IP (which might also be confirmed NOT an open relay) with a centralized whitelist, and servers could be setup to only accept mail from registered IP's.
When enough complaints are submitted about a specific IP it would be removed.
It's just a thought, but with some refinement it could probably eliminate a great deal of spam.
SMTP isn't all that broken, it just needs some sort of authentication added to it. I think the best way to do this would be to tie it to DNS by adding in an SMTP record, much like the MX record exists today. Actually, it could even use the MX record we have now, the reason I specify a new record is because some companies like to split up their outgoing and incoming mail services (I know my university does), and that would also make it easier to identify which SMTP servers support the new authentication protocol.
An SMTP server using this new form of authentication would then just connect to the machine listed as the SMTP machine for the domain of the email (which comes from the From: header of the email), and confirms that this mail was sent from that domain. If there is a response, the message is delivered. If there is no response (i.e. a server not compatible with authentication), then an email is sent to the address in the From: header asking for manual confirmation with a unique code. The sending user then replies to this mail, and the message is delivered.
How does this stop spammers? If the From: address was faked (like spammers usually do), you get no confirmation, and the message is dropped. If the From: address isn't faked, then you know it is tied to a real domain name, the spam is reported by a user, and then you can notify the ISP, blacklist the domain, or use NSLOOKUP to find and sue the spammer. Blacklisting is much more effective in this case as well, as it is not tied to a shifting IP address, but rather a domain name that is the same regardless of IP, which will also provide a way to track the spammer down. A spammer could try buying up a ton of domain names, but given how low the return rate is and how much more effective the blacklisting would be, it would be cost prohibitive for the number of domain names they would have to buy on a weekly basis to keep operating.
The authentication scheme is annoying enough that should one or two big providers (read AOL, Earthlink) pick it up, the little guys will probably follow suit, yet at the same time, doesn't inconvenience anyone too terribly much and is compatible with existing SMTP protocols. It provides a way for states to enforce their spam laws while allowing ISPs to more easily blacklist out of country spammers, and should cut down on spam considerably without requiring users to make any changes to their software. It's simple, it's backwards compatible, and it can be phased in as slowly as it needs to be.
Any thoughts?
P.C.
XMTP is a mapping of MIME to XML/RDF. Not a protocol replacement, but a piece of the puzzle perhaps.
http://www.openhealth.org/xmtp/
--------- Matt
right now if the company wanted to send an email out to the world it would take a while... with this system they are just sending headers so it goes much faster. the whole point of this is to reduce spam by not letting people send out tons and tons of mail at once, and if they want to THEY have to suffer the bandwidth and storage instead of the ISP.
that is what natural selection said about your family, but you are here, so never count anything out.
MARIJUANA, SHROOMS, X: ONLINE?! - E
So what if the mail's contents are now stored on the sender's system?
You don't think every single spam being forced to have a verifiable source address does anything? With im2000, missing/invalid/incorrect source address == impossible to receive said spam. It's at least a small improvement.
You do have a point...
However, if a spammer sends me a message I don't ever want to see it. This means I don't want to see the message in my inbox, but marked as "unsigned email". It shouldn't ever make it to my MUA. In fact, it shouldn't ever use up my bandwidth so that it makes it to my MUA which hides it from me.
I think the focus is on the protocol, because if my MUA throws out 500 spams a day and I only ever see the 10 real messages I'll be happier than I am now. On the other hand, I'll still have spammers wasting my bandwidth. To stop that bandwidth waste, we need to tackle the problem before it hits the MUA.
With all the loopholes in the current SMTP specification, is it possible for the Slashdot collective to come up with another one?
Loopholes? Which loopholes? It's a SIMPLE protocol. There are no loopholes. It works as designed, no better, no worse. There's a reason it's not called "Complex Secure Encrypted Spam-Free Authenticated Mail Protocol".
People used to use UUCP, BITNET, BBSs (including popular ones like Prodigy, GEine, CompuServe), FidoNET, and X.400 to talk with each other. People on one network might even know how to send e-mail to person on another network. There was a short time in the late 1980's when people thought X.400 (messaging) and X.500 (directory services) would glue everyone together. SMTP, though, was mush easier for e-mail software programmers to implement and became the de facto standard when the Internet was commercialized. It was easy for large user communities like CompuServe and AOL to gateway their e-mail using SMTP. It was also easy for cell phone and pager companies to install SMTP gateways for their products.
SMTP is the (current) least common denominator.
New user communities are developing in Instant Mesaging (AIM, MSN IM, Yahoo IM, IRC, Jabber). Why e-mail someone when you can send an Instant Message? Why use an e-mail mailing list when people have easy access to a threaded web-based forum (like SlashDot) or a BLOG? Why send people e-mail invitations when you can use an Evite?
If you give people your instant messaging ID instead of your e-mail address, that's one less person you will need to communicate with via e-mail. If you give people your cell phone number, they'll call you there or e-mail your text messaging feature. If you tell people, "go to myname.myblog.com" and enter a message, they'll do that. They might bookmark the method they'd use to contact you and use it.
Examples of the future might include:
"blog://myblog.com/myname/yourname"
"google://messages&from=myname&to=yourname&date-g
"http://im2000.yahoo.com/myname/messageid"
"vtext:212-867-5309"
No one company or standards board is going to be able to define "the new standard" (X.400 proponents tried and failed here). The people themselves will decide what it is (with a little help from commercial online services to make it easy for them to use).
... and the spammers will use the new protocols when the number of users using a new messaging protocol number more than a million. If there are technical ways of defeating spam in new protocols, then they will be more desirable for people to use than other protocols (including SMTP).
...!uunet!ziegast)
YIM://ez_sd
(aka
(aka ziegast@umdd.BITNET)
There's not a lot wrong with SMTP. The trouble is that SMTP is always implemented so it delivers mail as fast as possible. And that's the problem.
Judicious teergrubing (intentional slowing of responses; teergrube is German for tarpit) can alleviate many problems.
For example, let's examine the Rumplestiltskin attack (a form of dictionary attack to guess e-mail addresses). The trouble here is that most mail servers send back their "No such account" response immediately, so an attacker can try about 5-15 addresses a second. If the mail server was programmed to wait 5 seconds before sending back the response, then the Rumplestiltskin attack would be slowed down by about 50 times. Even better would be to make the delay longer and longer for repeated attempts from the same IP. This way, a normal user with a couple of dud e-mail addresses is not harmed much, but the Rumplestiltskin attack eventually gets bogged down in the tarpit. We have a 3 second delay at the login prompt if we enter the wrong password, so why not a delay at the mail server for incorrect e-mail addresses?
Another way to slow the spam is to teergrube *all* e-mail connections so all email takes a few minutes to send. Legitimate users aren't harmed much by this, but spammers are hurt a lot. Spammers rely on speed to send all their e-mail, and if we slow them down we can hurt them.
Then there's the question of what happens if a spammer sends another RCPT or other similar packet before receiving the response from the first? SMTP can legally drop the connection because such command buffering may be "unsupported". So the spammer must be teergrubed or must experience a *lot* of dropped connections.
There's no need to replace SMTP yet. Instead, we use the tools we have in a slightly different way, and the spammer can be inconvenienced a lot.
For more information on teergrubing, go here.
The only thing necessary for the triumph of evil is for good men to do nothing. - Edmund Burke
This way spammers can be found and taken to court if they send to someone on a do not call list (like phone spammers). The details about signing can be taken care of by ISPs. The problem is that it would have to be a worldwide thing.
One advantage is that people could actually sign up for certain types of spam (something for the money people to chew on)
At least this way one could decide to receive email only from people that allow themselves to be identified by the receiver.
Personally the way that the telephone system works isn't so bad, I don't get many sales calls, but maybe that's just me.
here siggy siggy, here boy
Here is the response to the paper which the page you linked is based on.
Take FTP as an example. It is an ugly, ancient, hard to firewall, and totally incecure protocol that refuses to die. There are many great replacements for it, but people refuse to even try them. BTW isn't cy.yp.to the coolest domain name you've ever seen?
I come to this discussion as an expert, albeit a bit dated, as I spent a number of years as the lone software developer supporting ALL email software at Apollo Computer (before it was bought by HP).
There once was a very interesting competing standard from OSI, the X.400 standard. Most people now think of X.400 as an interconnect standard for bridging the various email systems out there. Yet, it actually is a specification for a very robust email system in and of itself. It is based on a self-describing data representation... no, not XML since XML wasn't even a twinkle in someone's eye at that time, but ASN.1. That standard has been somewhat successful as used in X.500, which has become somewhat popular through its exposure via LDAP.
SMTP has never been a particularly strong standard. First, it is not the specification for a complete email system. It mearly describes a protocol for exchanging messages between two processes via the network. This is not sufficient to build an email system. Thus we also get POP and IMAP, and any number of supplimental bits that are not necessarily standards. Even sticking to exchanging email between two processes, SMTP has always been rather loosely specified. Sendmail has served as the reference implementation. Supporting sendmail was more a matter of figuring out what it was doing than reading the SMTP specification since sendmail used a far richer protocol for exchanging email than described in the specification. Thus, the question of what comprised a compliant implementation was more like (does it interoperate fully with sendmail) than going through a specification and checking off each element it described.
Apollo started a project to produce a native X.400 email system. It had a very rich set of features that go far beyond what we see today in Unix and Windows email systems. The project was put on hold when I was reassigned to a higher priority task, I was a member of a strategic technology team given the task of determining what "everyone" meant by the term "CASE Integration" with the goal of producing a corporate strategy and piloting and/or prototyping some initial products. Given the state of the CASE community, it sure seems like pursuing the email strategy would have had better long term success. Of course the CASE Integration project died a painful and horrible death when HP bought the company. Surely "SoftBench" did everything and more...
Check out RF2821.
Why should users have to pay to send e-mail? Spammers don't send their mail from normal ISP e-mail accounts, so all that would accomplish is a decrease in the amount of legitimate mail that gets sent. A signal-to-noise ratio problem can't be fixed by filtering out the signal!
Number 1 changing SMTP breaking clients nothing really to gain here bar more work total.
The faults can be simple to list and to fix.
Login on all SMTP and login sets from: removing user set from:.
Login adds password protection.
Finally have servers only deal with a regiseted list of Email servers that out law spam and have protection installed.
Network tracking to IP and so on will make this a lot hard to spam. And do every thing users want it too.
Not to mention all the idiots who misspell "teeming". ;-)
Spammers can afford to sign up their "pirate" mail servers but as a normal person just trying to run my own small domain, I can not.
That's not an answer to anything.
This from a guy whose .sig is only one step above spam.
-a
If any of the sender, reciever and message don't support TSMTP, then the message would simply be delivered using regular SMTP. (unless the messsage was somehow flagged as requiring TSMTP transmission).
Of course, defining this protocol is still left as an exercise for the reader... I'm expecting something with public key signatures and possibly distributed via DNS (all sorts of record types in DNS that are getting little use these days that could be used for that purpose).
Once the protocol gained support of a few of the big players (both servers and user agents), it could start to snowball. In the world of open standards it's almost all about momentum.
Free Software: Like love, it grows best when given away.
Actually, IM2000 has zero effect on the bandwidth requirements of SPAM ... a 1Kb message with 1000 recipients is 1000 Kb of bandwidth for the sender's ISP, no matter how you slice it, because it must be cloned by the sender's server *before* it is passed on to its various destinations.
... thereby erasing the need for *any* ISP to store thousands of copies of an identical message. As we get more and more mailings stuffed with media, this could become very important.
Nor does IM2000 impose a lot of storage requirements on the spammer's ISP, since that 1Kb message, even with 100,000 recipients, is still stored by that ISP as a single 1Kb message. It is only duplicated when picked up by the recipient. So the storage space required for a mass mailing is reduced to such a miniscule level, that it really doesn't significantly impact the spammer's own ISP to accept this duty.
In fact, the only way in which I can think of that IM2000 makes things tougher on spammers, is that when their ISPs are notified of their shady activity, the ISPs can terminate their accounts and nip all of their pending spam in the bud before all of the recipients have picked them up. Whereas with SMTP you can't close the barn door after the horses have left, with IM2000 you possibly could. But this is not that important an advantage. By the time enough users have complained for the ISP to actually take this action, the overwhelming majority of recipients will have already checked their mail.
So the verdict: IM2000 is essentially spam-neutral. But it is very ISP-friendly (in terms of storage not bandwidth) because as I mentioned in an earlier post it delays the cloning of an email for all its recipients until the moment each one retrieves it
TO RECAP: IM2000 does lift the storage burden of all that spam from the recipient's ISP. But only a very tiny percentage of that burden is shifted to the sender's ISP. The rest evapourates due to the advantages of the new paradigm.
Layouts like Dvorak address the top-row overloading and unbalanced left-right loads.
However, Dvorak is absolutely dreadful where the alternating left-right hands is concerned, which accounts for an awful lot of QWERTY's speed. The link I gave is a really just a more readable summary of the original, extremely lengthy paper. The point is addressed in depth in the original. (I believe that the original is linked in the straightdope article from the parent of my post.)
To add some real info (rather than just handwaving about the reasons), several studies have been done comparing qwerty to dvorak. For the most part, they tend to show that typing speed is pretty much equal between the two. Most of the anecdotal evidence says the same thing.
There is some evidence that the dvorak layout is better ergonomically -- people using it seem to have fewer problems with carpal tunnel and other typing-related injuries. I'm unaware of any controlled studies of this, however.
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
there has been an article in slashdot i read earlier (if someone could retrieve it, please post it :)
:)
it is about putting a list of smtp outbound (aside from the current inbound) servers in the dns zone file for a domain (something like mxout records.) when another server receives it, it will verify the dns if the server sending is indeed "authorized" by the company for that domain. this will prevent spoofing. the server rejects messages that come from a different server with a different ip address.
if you are using bayesian filtering, it will lower the score if it comes from an authorized server.
on another point, since smtp is a widely used protocol, maybe new commands may be added in it instead of replacing it. right now there are already authentication options for sending mail and this is a good way of validating a sender if they are "authorized" to use the mail server.
probably more effective way of combating spam (if that will be the main concern) is to be able to do multiple techniques together (bayesian, reverse dns resolution, blackhole lists)
Live your life each day as if it was your last.
Another mail transfer protocol that used to be THE future mail protocol was the x400 protocol (and the network protocol should be OSI), but I guess it was just too complex too get the average mail user to use it. Quite a few mail servers supported it, but the support has been abonded within the last few years even if the closely related x500 (directory services) has become more popular.
It seems tcpip and smtp was real life examples of that simple solutions was best.
"Bernstein's IM2000 proposal at least keeps the bandwidth consumption down, but that's primarily a cost issue for ISPs."
Sigh. No it doesn't. It reduces the storage space. Bandwidth remains the same. See my posts above called 'IM2000 Strengths and Weaknesses' and 'IM2000 fights SPAM? Not Really.'
Plus if you think about it no matter how you organise it a message bound for 1000 different recipients always takes the same bandwidth.
I know, we'll propose this new protocol and call anyone who objects a 'pro-spammer'. Then we can begin identifying them and hunting them down....of course then they will be able to get 'minority' designation and beat me out of my next civil service job application with extra 'minority' points on the interview.....ah heck with it, lets just all go back to world wide open relays.....
>I do like the idea of storing the sent mail on
>the senders (ISPs?) mail server. It's simple,
>practical, and consumes the senders resources.
>And if the ISP cuts off a spammers account,
>then all the unread spam is *gone*.
The problem is spammers can write their own software to give them the ability to store 1 email and billions of aliases to that one email used for spamming.
You can't get people to upgrade their mail servers now, or any other part of their server, so how can you expect them to upgrade to something completely different?
Maybe ISP's should charge users for each outbound SMTP connection they make? I'd happily pay 10 cents per email I sent if it would reduce the amount of SPAM I received.
Maybe I'm just a bit daft, but I think people are seriously overthinking the whole SPAM prevention issue. SPAM could be virtually eliminated with one simple measure implemented by mail servers...sleep.
Yep, that's right, the sleep function. At some point in the SMTP process (after mail from, or before data), the mail server should sleep for 2-3 seconds. To the average user, this is just a 2 or 3 second wait while their message is being sent (no big deal). But to a SPAMer sending 1,000,000 messages, this adds 2-3 million seconds to the send time (approx 3 weeks). If we make sending 1,000,000 message a non-starter for any individual user (Big ISPs could white list each other), SPAM would virtually disappear.
Oh, and if you need a whiz-bang techie solution, you could use HashCash instead of sleep (though it would have to be part of a new protocol).
1) As long as people keep buying from SPAM it will not stop. Beleive it or not some one out there is buying those pen1s enlargements.
2) Even if no one out there buys anything you will still have the idiots who will buy into these lists.
3) I still do not understand why people beleive that they can outsmart everyone. As fast as you add filters some one will pass it throught those same filters and see what gets through.
We need to be able to identify the sending server, no an IP is NOT an ID. It needs to be part of a TLD, and maybe the registrar need to be held accountable for the content of a registration.
DRM? No thanks, I'll just get it somewhere else...
The key to solving the spam problem is to attack the definition of spam: "unsolicited junk E-mail." All that's needed -- really -- is a simple convention that may be used to designate whether a given E-mail message was solicited by its recipient, or not. That's it.
For example, you see one of your own message IDs in the References: header of an incoming message. That tells you that it's a solicited response to one of your own messages. Unless you're a troll, presumably each one of your messages merits a response from an interested party. Additionally, by the virtue of sending the original message you've implicitly solicited appropriate replies.
Now, I don't think anyone needs to keep track of every message they send. This is just a conceptual example of identifying solicited versus unsolicited E-mail. Once a method for doing so can be worked out, and put in widespread use, the spam problem will be gone.
"Not with umpteen trillion SMTP servers out there, all of which would need to be replaced en masse."
... but not alongside IPv6 or any lame idea like that, it would be its own movement made necessary by the strain on the infrastructure of multiple-recipient, media-rich mailings.
No they wouldn't. The IM2000 specification could simply be written to double as an SMTP server, in case the sender's ISP is not IM2000 compliant, and to identify which sort of server is present @recipientsisp.com, and to interact with it in the way that it expects. Once the software was there and was reliable and accepted as a standard, widespread adoption by ISPs would quickly follow, because of the enormous storage savings to be had. Even if no one else on the internet had yet adopted it, a large ISP would benefit from going IM2000 just to handle the mail bouncing back and forth between its own users without needlessly storing multiple copies for multiple recipients.
I could really see IM2000 or something like it, taking off and being phased in to replace SMTP (and POP, by the way)
This is currently being discussed on NANOG (where it's an offtopic favorite). I highly recommend this list for peeks and views into the people who keep this Internet thing working.
In the discussions yesterday and today, there's been a lot of talk about how to "bootstrap" this new protocol. There are interesting discussions of the business ramifications of being an early adopter of something like this -- very sililar to those for IPv6.
It's been said by far wiser people than me: spam is a social problem, and it must have a social cure. Any solution which does not respect these two facts is doomed to failure.
>How's that going to prevent a spammer from keeping a copy of the spam stashed somewhere, and then spamming a few hundred million mailboxes with a link to the spam.
It won't, it won't at all. Then again, that's not how most spams I've received work.
>So, what have we accomplished here?
A lot. Tell me, of all the spam you've received, how much of it came from a 100% legitimate source? A machine that wasn't hacked? A non-open relay? The spammer's box itself? Was it still up when you checked?
We'd push spam back to 1997, before spammers were using open relays and hacked boxes.
Now, while it would stop the open relay abuse dead in its tracks, it wouldn't stop hacked boxes. It would just mean that the box would have to remain hacked for a lot longer. Hopefully a competent admin would notice it the next morning and pull the plug, thereby preventing most of the spam from reaching its destination. Otherwise, the admin gets to pay a hefty bandwidth fee, and possibly get kicked off their ISP. Either way, I guess I'm happy.
If you could be told what you can see or read, then it follows that you could be told what to say or think - BoC
Writing and publishing a new electronic mail transfer protocol sounds good to me, but it won't do anything unless people use it. Just having an RFC you think is better than SMTP is not good enough, if you want it to be used by anyone you've got to have client and server software which uses the new protocol. And even then people will be slow to switch from their usually mail program.
But regardless, I agree with those who say SMTP is not the problem. Maybe the average Joe should learn about PGP, or is that too much to ask?
"And there be unix which have made themselves unix for the kingdom of heaven's sake." - Matt. 19:12
This may be total flamebait, but I've been "out of the ISP business" for a few years now and I don't remember (honestly) anyone suggesting this:
Facts:
1. Email comes in from an IP address.
That's all you need to take into consideration for this.
Solution:
1. Have the recipient mail server check the sending server for an open relay. If it exists, send an email to "root@, postmaster@, and/or administrator@" on the originating server letting them know they have an open relay and the message sent was disallowed.
2. Have the recipient server keep a compressed (or otherwise optimally-indexed) log of IP address mail is sent from, recipient, and some sort of searchable indicator of the body content of the message.
3. Configure recipient mail server to deny messages from any particular IP with a body and/or subject matching or closely resembling the indexed bodies if the message is sent from the originating server at a particular rate (e.g. 25 or more emails in 5 minutes' time) to any recipient address on the system.
4. Configure recipient mail server to deny messages originating from a unique mail server that appear to be dictionary-based scans (e.g. aaa@, aab@, aac@).
5. Combined with the above or separately, configure recipient mail server to deny messages to (z) invalid addresses at a rate of (x per y minutes) with identical bodies or nearly identical bodies and/or subjects.
6. If a user on the recipeint system/network forwards an email to spam@recipeint.domain, automatically quarrantine all messages from sending server for a period of (x) minutes to allow the recipient server to "collect more evidence" in its incoming spam scans without actually delivering more messages to the recipients' inboxes.
I'm truly looking for feedback. I don't want "you're an idiot because you didn't think about a, b, and c" type replies. I obviously haven't thought of every possible loophole or abuse case, so I'd like constructive comments. I believe, at a high level, that this could work.
but with im2000, if the sending machine goes down, then no one can read their mail from there.
Any serious mail user should have a 99.9% uptime/redundancy plan in place for their outgoing mail server.
Also what about people that don't have a full on connection, you don't want to require those people to be connected just to read their mail. Sure you can queue it for downloading offline somehow, but that's going to be much slower than normal because you have to connect to say 30 different servers where your email is hosted.
I believe mail would always be downloaded to the local system (or ISP) when it is read by the client. It could use concurrent connections to connect to each of the 30 servers simultaneously (or at least several at a time), to minimize the total delay.
Also there's the case of somesmallcompany.com sending out a mailer/advertisement to millions of people, because the email is hosted on their machine, their connection/server might become overwhelmed, causing heaches for everyone wanting to read their mail. "Why does my mail load so slow?"
Someone who wanted to do that, and had a legitimate reason to do that, had better have a beefy mail server. But that's the case with the current system too. A million messages would swamp a typical sendmail server!
It's a nice try, but it'll never work.
Sure it would. There are issues to be resolved, but I think it's fundamentally a great idea.
It doesn't matter what new protocol you introduce to enable servers to exchange mail, at the end of the day someone is always going to be able to get a legit webmail account somewhere and shove a ton of mail out through it.
The real problem is being able to make people accountable for the traffic that comes from their IP address, regardless of the protocol it comes via. Of course whether that is desirable or not is a different matter.
It's just a simple fact that if you use an unregulated commmunication channel there's going to be data coming down it you might not like.
How did you ever get that link posted without the [straightdope.com]?
Enquiring minds wish to know!
If you could be told what you can see or read, then it follows that you could be told what to say or think - BoC
The primary problem with Spam is its automated, bulk, impersonal nature. It wouldn't be so bad if a sales rep personally wrote you a letter. That would be almost tolerable, and would protect against delugest of spam. To prevent the former, could it help to add an extension to SMTP making it impossible to send more than, say, 10 e-mails of identical (or very similar) content per server per day. Mailing lists? These are formalised on an opt-in basis, and dealt with specially.
((lambda x ((x))) (lambda x ((x))))
Who modded this up?
Has the thought be given to improving email client software so that it filters SPAM rather then having ISPs and Network Administrators do it. It's the same as filtering junk Mail I get weekly. It's never going to go away 100% but you eventually notice a pattern of what it looks like and apply rules in your head when you use it. If an easy system like a "This is Spam" button and it asked what makes it spam you could say the subject, part of the body, or who it was from and your smart client could remember that. Combine this with all the wonderful spam technologies out there then we've got something.
If we were going to get rid of SMTP then something that provided assured delivery would be very nice to investigate. Standards that would allow you to make sure you email gets there and if it was tampered with on the way (some hashing comes to mind). I know there are ways to do some of this now, but if we're looking towards the future that's a place I would start, security and delivery assurance. In that you could easily provide better definitions of the sender and require them. Then this would require "SPAMMERS" to have that information so your smart client could block them more easily.
Unfortunately somebody would find a way to send spam out even if there is a change. There's always a way, and somebody always finds it, it is just a matter of time...
What a terrible implementation you've described.
The amount at risk should be set by the receiver, and they should tell the sender what it will cost.
Friends fly free, and so should subscribed mailing lists.
A mailing list wouldn't pay a dime,
because it wouldn't send email to people who demanded it put money at risk.
-- this is not a
There is a solution on the table and US law that requires the US government to use it. Its called X.400 and it is a mess. For a start you have to register your server and that used to cost something like $25,000 or maybe $40,000 for businesses. The Gossip program for gov email requires all email systems to migrate to this x.400 nonsense but I manged to get them to allow a migration path through SMTP (the others were worse and the only two that were even consididered that worked were SMTP and UUCP). The only encrytion addon for sendmail happens to be a result of work that started from encrypting x.400 stuff.
If you want to fire up your own X400 server to play with, grab isode and try to get it to compile on your machine without gagging if you can. Its one nasty bit of bad code.
SMTP isn't that broken. It works for about a billion people. Any attempt to "fix" it will break it for way too many of them.
After looking through the posts here (most of the +5 should be -5 Stupid), its clear that most of the experts don't understand email in the real world.
Encryption:
The 1st tings is email must be interceptable. Many governments won't allow high level encryption that isn't full of holes that allow them to play pack recorded streams. Most large email servers can't deal with the CPU load of full encryption anyway so 100% solid encryption is out.
Authentication:
Authenticating the server is very importaint to many sites. Once you start doing some level of encryption, you need to make sure you know who your connecting to.
Authenticating the client is the where spam issue comes from. There are many ways to do this but none of them are being done and none of them work 100% (which is why none of them work)
There is no way of knowing of a new business is a spamer or not. Therefore there is no way to filter out spamers that have enough cash to hook up to new ISPs all the time. (there are some stupid ideas like charging--my isp is rich enough, forcing all email out--my isp's mail server is up 100% doesn't understand MX,I can run my own server and it works so why chnage?)
reverse MX record checks only work if you can trust the ISP to get reverse dns working correctly and they won't deligate it to a spam house. The other choice is a verisgn like company to whitelist everyone or some sort of distributed whitelist (which the spamers will try to hack into)
As far as fixes:
The solution is patch sendmail, qmail, postfix, exim to understand email on port 26 (pointed to by a srv record) and if mail comes in on the new port, then it must be checked with a reverse MX record or its dropped. Get the clients to stop handing off email on port 25 (sendmail allows port 587 for that) Use something like the SSH transport layer to encrypt (i.e. set up the encrypted channel 1st and then figure out whos talking). Add a new smtp verify_message command so I can ask another server "did you send me messages Xcxczxczqweczx?". Patches for all 4 systems must come out at the same time but be tested aginst each other. The when an ISP figures enough of its mail comes in on something other than 25, kill port 25 forever. That will kill all the proxies and all the old email gateways that haven't been updated in years.
Or save up your money and buy your self an X.400 gateway license adn tell all your friends about your cool new email address with all thouse nice slashes and no @.
A combination of white lists/black lists, and Baysian filtering...
Stop yer bitchin', people, and implement the technologies that are already out there and work great.
This doesn't sound like a general solution for J. Random Homeowner.
It sounds alot like "Well, shoot! Quit yer bitchin' an just put in some tuned ports and performance cam! Hell, my grandmother could do that!"
My other car is a 1984 Nark Avenger.
I don't risk job loss (since I'm self-employed), but I do risk missed contracts. I use Bayesian. I've had an occasional false positive, mostly during the first couple months when I was building my corpus.
BUT: I'm currently receiving about 140 spams per day. Do you really think I'm going to do any better than Bayesian regarding false positives when I'm mass-deleting 140 messages? I think not. I'm almost positive I'm going to incorrectly delete more good messages than Bayesian's false positive rate when I'm dealing with such a huge number of emails manually.
Plus if your Bayesian system is half-decent you should be able to adjust its sensitivity. On mine 0-49% is almost always good email and 50%-99% is almost always spam. When I look for false positives I look for anything unusually low, such as 65% or so. If you want, just adjust your threshold to 65%. I've never had a valid email with a Bayesian score of, say, 90%.
> They either can't figure out the tools or don't think they should have to.
:-)
And this is the thing - they really _shouldn't_ have to. Bad UI really ticks me off.
> I agree, however, that people are generally naive/dumb when it comes to common sense issues like sending out email addresses at will or even worse...
The thing is - I intended that for this particular audience. _SLASHDOT_ users, of all people, should know by now how to avoid getting spam. I mean _really_.
> clicking on the "Remove" links from spam! VERY DUMB!
Actually, I've proven to myself this is a myth.
Here's my story:
Last year, around, say, September or October, I was getting, on average, about 200-250 pieces of spam PER DAY. This, I realized, just Would Not Do(tm).
So, since it was obvious I was going to have to shut down all my existing e-mail addresses, generate new ones, and be ultra-selective about giving them out in the future, I realized it was time to test that "Don't click on the remove me links" piece of advice. I'd given it out myself many times, even in an article I once wrote. Time to put it to the test! So, for the period of one month, I followed all the instructions on each piece of spam, every day, to see what would happen to my flow of spam. I kept track of who was sending me spam (the company/product/service, not the 'return address'). I found out that you WILL get LESS spam if you actually follow the advice in the spam, in general. My spam reception went from the 200-250 per day to around 20 per day, in the span of about a month. Obviously, this was still way too much frigging spam, but let me say this: the spam I kept getting was almost entirely from the sources that didn't have a (working) removal method, not from the ones that did. Many of the ones that did have an (apparently) working method DO indeed take a few weeks to start working. But, surprise of surprises, it CAN indeed lessen your spam, when it's offered and is working. Bizarre, I know, but I swear it's true.
I'd still rather make spam technically impossible than rely on that, though.
I propose TMTP - the Trusted Mail Transfer Protocol.
For the Slashdot crowd that IS a solution. I didn't post that on usatoday.com you know. :)
Is there any kind of a test implementation of IM2000 yet? I for one would like to see it in action.
If not, I bet one could be hacked up using Python and Twisted in a couple days.
I have a technical website where users can subscribe to a nightly mailing list that sends them a single email containing all the messages posted to the forum in the last 24 hours. The users subscribe by signing-up with their email address. A single email is then sent to that user who is then asked to click a link to confirm they want to receive the nightly mailing list.
Now, I've heard people propose that there should be a system whereby an "unknown" email is charged 10 cents (or whatever) unless the receiver subsequently tells the system "Yeah, I wanted that" at which point the 10 cents is effectively refunded and, presumably, all future emails from that source have the charge waived.
The problem is if someone starts signing up random email addresses on my mailing list. Each time the system sends out the single confirmation email I will be dinged 10 cents with the expectation that everyone will "refund" that back to me since they asked for it. But what if someone with a grudge and a lot of extra time goes to my site and starts signing up lots of people--people that have never heard of my site. The "confirmation email" is precisely to protect users from getting on my mailing list without asking for it, but I'm not at risk of having to pay to have that confirmation email delivered.
If such a system were imposed then how would a mailing list be able to send out confirmation emails without the risk of someone maliciously signing up random users just so that I get hit with a bunch of email charges?
Just add an element to the SMTP protocol. If the message is so riddled with gramatical mistakes that Strong Bad wouldn't know where to start, drop it. That would take care of every piece of spam (and email from people I wouldn't mind hearing from when they're sober) I've ever gotten.
The masses are the crack whores of religion.
As well as going to IPv6 I've also installed this new wonder smtp replacement protocol that sorts out the spam problem!!
La la la la la!
Oooh look, flying piggies!
Actually, it might even save some bandwidth. Most SMTP implementations are so braindead that the recipient's server tells the sender's server that there is no space left etc AFTER all message data is sent (since there is no way for the server to tell the client that the DATA failed in the middle of the transfer). So instead of getting this temporary error a few times, the header (which could very well be less than 100 bytes) is sent and the mail is sent only once, when the recipient wants it.
D. J. Bernstein, the author of the supremely reliable and secure qmail mail server....
1) Bernstein is also a renowned asshole, so you consider any of his proposals Dead On Arrival as far as the Internet Gnomes are concerned.
2) As the author of a widely used MTA, he is in the unique position to push a new protocol implementation. Which he has not done. Which pretty much speaks for itself.
From the draft document:
Big advantages of this proposal are
If you do trust the applet (man, I should really verify the results to be sure
1. You stay on the home row more.
2. You alternate hands more.
3. You change fingers more.
4. Your fingers don't travel as far.
Well, whatever the reason, I feel more comfortable typing on a Dvorak layout
I agree. There are stupid people out there, and us smart people can't change that. But if Bayesian was universally implemented, especially where stupid people congregate (Hotmail, Yahoo, etc.) then even the stupid people would see the spam less often which would cause spamming to be less attractive.
Even if no one out there buys anything you will still have the idiots who will buy into these lists.
Only if they know about them. And if Bayesian were universally implemented then fewer stupid people would even read about the availability of the lists so fewer would buy them.
Those that do buy them would generate email that would still be caught by the Bayesian filter. If all we have spam-wise is stupid people getting taken advantage of by buying spam lists we'll be order of magnitudes better than we are today. Unfortunately, people ARE making money today with spam and that's why there's so much of it. If we can turn $100 of spam profit into $1 of spam profit then I think we'll see the vast majority of spam disappear; if a spamhause making $40k per month today can only make $400/month they'll look for another line of work.
I still do not understand why people beleive that they can outsmart everyone. As fast as you add filters some one will pass it throught those same filters and see what gets through.
That's been true in the past. And there might be a few tricks left for the spammers to try. But Bayesian is different than previous filters in that it learns about YOUR email and spam. It's not just looking for keywords or adding up some "spam score" that no-one really knows what it means. It's looking at your past email and past spam and making a statistical judgement as to whether this new message is spam.
The only way for spammers to get around Bayesian is for them to make their spam look exactly like YOUR good email. That's almost impossible, but even if they managed to make their spam look like your good email doesn't mean it's going to look like my good email and certainly isn't going to look like the good email of millions of other users.
The only way a spammer can really get around Bayesian is to have access to your corpus and use a bunch of words that your corpus indicates are "innocent" for you. Spammers don't have access to your corpus (or shouldn't) so it's virtually impossible to get around it.
And, no, contrary to what some people believe you CANNOT get around Bayesian by just inserting some random, non-related paragraph(s) into the message. Unless that random paragraph contains quite a few of my VERY innocent words (i.e. more innocent than the spammy words are spammy) then that paragraph just makes the spam a bit bigger--it doesn't have any effect whatsoever in the probability that it'll get through a Bayesian filter.
We need to be able to identify the sending server, no an IP is NOT an ID. It needs to be part of a TLD, and maybe the registrar need to be held accountable for the content of a registration.
I'd rather keep control on the side of the user and receiver rather than some registrar that will basically have to decide if I'm "authorized" or "trusted" to send email. I think Bayesian is a good approach to the current problem since it doesn't require any radical protocol changes and can be implemented today, whenever the receiver wants to. If we're talking protocol shift, PGP signing should be sufficient--but nothing that requires centrally authorized certificates or anything like that. The last thing we want to do is give control of email to central authorities to get rid of spam.
That's the prob!
The config file is a joke, who the hell made that?
No one in his right mind could figure that out, and if he did, a week later he would have no clue what is going on and have to study the manual all over again
take the geek out of linux you geeks!
Wouldn't that be GNU/SMTP?
The only reason we have the rights we have is that people just like us died to gain those rights. -- Cheerio Boy
If it sleeps after MAIL FROM it is painfully slow even for, say, a small company w/o their own network/mail server/blah setup to send a small information mail to maybe 30-40 customers.
If it dosen't sleep after MAIL FROM, then what difference does it make? Just add 1000 recipients at the time (or 100,000 if the server allows it) and then sleep 2-3 seconds. Then repeat. Now we're down to like 0.002 seconds per mail.
And uh yeah, why use the ISPs mail server to send 1,000,000 messages when I can use my own bulk-mail optimized SMTP server?
Yes, for every packet ack send a wack to sco.com.
and also send a copy of the linux kernel to sco.com for every mail transmission.
(to return "their" property)
hence what i said about one "entity" running everything. a global list of who is paying fees to run one to a single entity.
= bad.
MARIJUANA, SHROOMS, X: ONLINE?! - E
I'm even opposed to the "pay a dime, but I'll give it back if I wanted to hear from you" approach. Those of us running a mailing list would run the risk of having some idiot sign-up a bunch of accounts only to have that person say "No, I didn't want that" and collect the money.
Just require someone to send you an email to sign up, like in the olden days. Of course, this would require something in place to only be able to "claim" against a person only once in a given time limit.
I agree though that there should be a trusted protocol. Especially something that requires authentication to send a mail over SMTP like POP does, since most spam comes from open relays.
bananas like monkeys.
Hmm... The name is fine, it just needs a minor adjustment to clear up possible consumer confusion.
How about Slashdot Mail Transfer protocol High Speed
and
Slasdhot Mail Transfer Protocol Full Speed.
There! No more naming confusion.
Part of the subscription process is that they whitelist the mailing list so it doesn't pay.
If, when you go to send them the confirmation, they try to charge you 10 cents, that's a clear indication of no confirmation.
If everybody charged strangers, then we could do away with confirmation emails altogether,
since the fact that they've whitelisted you should be confirmation enough.
We could probably do away with unsubscribe too.
-- this is not a
IM2000 may reduce spam somewhat, because now the sender must provide a valid path to retrieve the spam.
The downside is the privacy issue: The sender knows precisely if and when you've opened your mail. I don't like that. I prefer to receive my mail and anonymously decide whether I'll open it or discard it.
Personally, I'd like to see a web-of-trust system between mail servers, similar to how PGP public key trust systems are set up. Would prevent the open relay problem. Mail servers would authenticate each other via a challenge-response system. If mail-server A wants to connect to some other mail server B that doesn't know it, it needs to provide a list of 'recommendations' from other servers that B does trust. This would lead to some central points in the web, but if cached intelligently, it should scale pretty well--something like Freenet.
I think such a system of white-listing could work pretty well, and overlay our SMTP system reasonably well. And if an mail server 'goes rogue,' you could treat it like a PGP key revocation. --Joe
Program Intellivision!
It's not so much the storage, it's the bandwidth that costs, I think.
Sure, they can store one email and send out millions of notifications from "different" addresses.
But as soon as people start reading it, those networks are going to start getting really busy.
A self-slashdotting type effect.
- Muggins the Mad
There already is such an extension. See RFC 2487. Postfix (and probably others) already allow you to use key IDs in ACLs.
Comment removed based on user account deletion
masses of idiots find your comment insightfull.
"Nothing in education is so astonishing as the amount of ignorance it accumulates in the form of inert facts." - Henry A
I'm surprised nobody has mentionned TRIPOLI yet. (although who knows with the "discussion search")
http://www.pfir.org/tripoli-overview
One of the people behind this is PGN, whom you might know from comp.risks...
Actually, what I'd do is just setup a system whereby you only accept mail (whitelist) from people who have registered with a certain service whereby they have pre-paid for a certain amount of email they will use. People registering would also have to be authorized for sending large bulk mails to ensure they are not SPAM.
OK, I agree with you in principle, but realistically I think we need one entity in charge of e-mail, like we have ARIN and ICANN for other web services. Ideally I'd like to see things stay the way they are, but we NEED to get rid of spam, and we may need to make sacrifices to do so. Among them being a central entity in charge of e-mail. I'm not saying that every e-mail would go through a single server, just like all web traffic doesn't go through a single server, but I think we need some accountability, both on the user end and on the server end.
Assuming this is how IM2000 would work, it seems like one of those ideas thats great for security but not in practicality. Email is high latency. However users want to power up their MUA and get their email now not wait for a bit then get their email. And I happen to agree with them.
Currently it's the servers that have to pay for the delays but it's not a big deal since they're always on. And since the user is downlowding from a local machine they don't experience the delay.
However is the email is retrieved from the sender when the user requests it. It measn that every morning at the same time all your users are going to get on and and screw your connection. Not to mention they'll still complain to me when it's someone else's server which is down.
I wasn't thinking of a simple "pay the money and get a registered server" system. There would be something more to it than that. And it wouldn't be a lot of money, either, for situations like yours. And if too much spam comes from your server, it's gone. It's not a great solution, I know that, but we need to change SOMETHING. I'm just throwing out the ideas I can think of.
But it doesn't help the sending-from-lots-of-places problem much - you won't be able to send mail from a random cybercafe that way, so travelling users get screwed. (This happens increasingly often anyway - see some of John Gilmore's rants about having SMTP sent by his laptop get blocked by dialup-user blacklists.) On the other hand, you _could_ make things work more often by using tunnels to the ISP (ipsec, or SSL tunnels to the ISP's outbound mail relay) which would let your laptop use a consistent IP address from the ISP's space.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Because, you can come up with a solution that solves all of these problems for all of the different protocols if you create something that lives beneath them that does those things.
It shouldn't be the responsibility of each individual application to implement this kind of thing. This should be a function of some layer of your protocol stack that's shared by multiple applications.
Need a Python, C++, Unix, Linux develop
All that's needed -- really -- is a simple convention that may be used to designate whether a given E-mail message was solicited by its recipient, or not. That's it.
For most people, unsolicited email isn't the problem. Unsolicited bulk email is the problem. Generally, unsolicited, personalized, individually sent emails aren't sufficient to bother people.
I've been using the Internet since 1987. My current main email address is <deven@ties.org>. This is my real email address that I use every day, and I refuse to hide it, even on a high-profile site like Slashdot. I know it gets harvested routinely, but I don't care. I've had this email address since 1994 and I intend to keep it permanently. I also have it associated with several domains I own.
Unsurprisingly, I get tons of spam. Hundreds of junk mail messages daily. I've got some custom filters I wrote that do a reasonable job, but I'm planning to work on more effective solutions. Any solution which depends on security through obscurity (hiding my email address) or refusing unsolicited personal email from strangers, is unacceptable to me.
Back in 1987, the ability to spoof email addresses via SMTP was well known. As a rule, everyone knew about it, but nobody abused it. When people forged email addresses, it was as a joke, such as sending email from <president@whitehouse.gov>, years before that address actually existed! But there was no spam problem, because commercial interests were mostly kept off the Internet, and Internet users at large were responsible and valued the community. Random unsolicited email wasn't a nuisance -- more often, it added some variety to the day, because such messages were few and far between.
I don't consider the spam problem solved yet, but not all of the proposed solutions are desirable. Electronic "stamps" to charge real money for all email is undesirable. Having to hide your real email address and change it when discovered is undesirable. Blocking unsolicited personal email is undesirable, but many have become so sensitized to unsolicited bulk email that they come to hate all unsolicited email.
Let's not throw out the baby with the bathwater. UBE is the problem, and the solution needs to leave communication paths open between people, but closed to spammers...
Deven
"Simple things should be simple, and complex things should be possible." - Alan Kay
Bernstein's IM2000 proposal at least keeps the bandwidth consumption down, but that's primarily a cost issue for ISPs."
Sigh. No it doesn't. It reduces the storage space. Bandwidth remains the same. See my posts above called 'IM2000 Strengths and Weaknesses' and 'IM2000 fights SPAM? Not Really.'
Plus if you think about it no matter how you organise it a message bound for 1000 different recipients always takes the same bandwidth.
Both of you missed the point completely. First of all, if you group recipients by domain, then you can dramatically reduce the bandwidth 90% or more, btw. Check out the SMTP protocol.
Also, of *course* ISPs won't lower their prices, but spammers would have to buy more bandwidth to spam.
Ok, so that requires additional steps. Before the user signs up the user must be made aware of the address from which emails will be sent so that it can be whitelisted BEFORE the verification email is sent. A new business plan for spammers wold be to cruise or spider the net and gather the email addresses of common mailing lists and use that as the "From" when sending spam to increase the probability of using a whitelisted, toll-free address.
If you're talking about whitelisting an email address AND an email server that's fine for tech people but would probably a bit complicated for normal users... and for lazy technies.
Let's present a new protocol. I suggest we name it Slashdot Mail Transfer Protocol.
I hope I get to metamoderate the morons who modded this up as funny.
The hashcash calculation has to be done per recipient to make the scaling work. This could be done in the message headers, or could be done in the envelope, but if you do it in the message headers, your mail system can't tell it's been done until it's accepted the message, while if you require it in the envelope or enforce it in the transfer protocol, it forces the sender to calculate it for every message. Adam mostly made the right choices.
However, if spammers can abuse open relays into doing their hashcash for them, then the scaling doesn't bother the spammers' machines, just the relay owners, and they're using hundreds of them in parallel so it's no problem. On the other hand, open relays aren't likely to implement new hashcash protocols, and mailers that do implement new hashcash protocols can be configured not to do relay by default.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
there's no protocol changes you can make that will disallow this. period.
/dev/null. else append to mail spool"
however, I do agree that it would be best solved on the server and not the client. However... guess what? The parent post can be done on the server rather than the client. No reason it couldn't be...
"message not signed? send to
procmail anyone?
But I do get important mail from people I dont know.
Maybe we could improve spam filters by building some kind of probabilistic web-of-nospam-trust.
Or how about we all start to only accept email containing a timestamp and encrypted using our very long public keys. Obviously it will not stop all spam since the public key is public.
But collecting say a million public keys and encrypting a spam mail with every key would cost the spammers more time and money.
And whereas the 1/10 cent per email tactic has some bad side-effects, having all mails sent to you encrypted would increase your privacy.
http://www.defendmail.sunet.com.au/
Implementing it on a recipient-controlled basis is not only politically correct anarcho-capitalism (as opposed to central-planner-based and doomed to failure), it matches the main problem with spam, which is that it wastes the recipient's time, and only the recipient knows what that time is worth., and the recipient of the mail is the one who gets the money, not some middleman whose time isn't being wasted. If I dislike spam more than you do and am more willing to reject mail from people who might have interesting things to say, I can charge more money to read email than you do. Yes, email-provider ISPs also have higher workloads because ~50% of email is spam, but that's a much smaller amount of money per recipient than the wasted time, and they can charge the users accordingly.
Of course, if the recipient of the email gets paid for receiving the email, there's an obvious product for spammers to sell "Get paid receiving email" ;-)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
A combination of white lists/black lists, and Baysian filtering stops so close to 100% of spam that it's really silly for anyone to be bitching about spam these days. I don't GET any spam anymore - 0. Not 0.001%, 0 - the integer 0, as in none.
Have you done the power-curve analysis on that? My mother works at a law firm, and they once tried to install a spam filter. It was state-of-the-art, with Bayesian filtering, and white/black lists, and additional whitefilters on top. It blocked most (not all) of their spam. But it also blocked some tiny fraction of legitimate messages.
Even if you have the (extremely impressive) power curves of Paul Graham's Plan for Spam -- and that was on a very well-trained Bayesian filter written by a coding genious -- it is Not Good Enough when missing a legit email could get you sued for millions. Either you risk blocking legit email, or else you have to wade through a pile of spam bigger than that legit email... either way, another protocol would be nice.
I hereby place the above post in the public domain.
It's not the only reason, but one of the main reasons that most Unix email programs did relaying was the need to connect between different kinds of email systems - if your machine wasn't smart enough to reach BITNET:user%foo@berknet , you'd send it to somebody smarter or better connected or likely to be closer in a protocol-hop way to the recipient, and they'd try to deliver it. Since everybody was friendly back then, there wasn't a major abuse problem, and email systems that were expensive (like Fidonet) didn't relay for unregistered people, unless they were sponging resources off of large companies (like several Bell Labs uucp relays ihnp4 and allegra, for whom it was mostly funny-money.)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
The new system will of course have built in redundancy, ala the CRAP (Cmdrtaco Re-post Automatically Protocol) which automatically sends you two copies of the message whether you needed it or not.
Riker: Some jazz, Captain?
Picard: Make it soul, Number One.
No problem. The user's ISP could have a IM2k -> POP3/IMAP gateway. The gateway will periodically grab the emails from the IM2K servers from the user's notification, and store it locally, and the user will download the emails via POP/IMAP from the gateway.
The pros:
- Extra source of income for user's ISP.
- ISP's gateway can optimally download emails in bulk, esp. if it's addressed to a few users in the ISP, and it needs to download it only once.
- More or less transparent to users: they don't have to switch from their Lookout.
- ISPs can implement spam controls.
- The senders get the features many non-technical people been wanting: the assurance that the email has been downloaded by the receipient (if not using gateway agent), or the ability to "unsend" emails. (Oops, I didn't mean to send that email to my boss!).
The cons:
- How would the IM2k server know that the ISP's gateway is the correct receipient? Get it wrong and it'll be too easy for crackers to read other's emails.
oxen is the plural of ox.
I admit that boxen is a stupid plural of box (should be boxes IMHO) but as a DEC VAX owner, I am quite use to reading "vaxen"
The people who design a new mail protocol should, at the very least, all know the current one backwards and forwards.
Another thing I'd really like to see is a mechanism for MTAs and MUAs (and their users) to negotiate policy agreemeents for the content that they carry. It would work a little bit like content negotiation on the web. For instance, if I'm sending a piece of personal non-anonymous noncommercial email to a single individual I already know, that message would meet a very large number of policy agreements. (Some of them might be agreements I or the recipient had written, but most of them would be standard agreements written by third parties.) So my mail software could present the recipient's mail software with (at least) four pieces of information: sender, recipient, the location (on my server) of the message, and a list of the locations of a bunch of the agreements that I promise the message is certified to meet, probably with checksums so it can be proved that I'm talking about the same set of agreements the recipient is reading.
The recipient's software (and the distinction between MTA and MUA might be fuzzier than it is now) would similarly have a list of policy agreements that it was configured to accept. If any of the list of policy agreements the recipient was willing to accept matched any of the list of policy agreements the sender certified the message against, the message would be "accepted" -- presented to the recipient by the MUA. Presumably the recipient would have a choice of copying the message to local storage or leaving it on the sender's server (where it might expire if the sender decided not to keep hosting it, although some policy agreements might promise to keep messages for a certain amount of time). If none of the policy agreements intersected, the message would not be presented to the user (and probably the sender would be so informed).
This mechanism would allow recipients to choose what kind of mail they accept, and therefore it would not (in and of itself) prevent important things like anonymous mail, or mail sent from unreplyable addresses, or mailing lists, or mail from unknown persons. It would just be a mechanism for recipients to say what kind of mail they're willing to accept, and senders to describe (in the form of an agreement or "contract") the characteristics of their mail. And that needn't be limited to just spam prevention -- if I don't want any mail longer than 40k, or I don't want any mail with images or animation, or I don't want any mail with more quoted text than original text, I can express that.
Now, that's still infrastructure that depends on some degree of honesty on the part of the sender, so it's not a complete solution by itself. The people who forge mail headers so their spam appears to come from real (innocent) people wouldn't be deterred by this; they'd just lie as they do now. But a large amount of spam is ultimately traceable if the victim cares enough, and those spam senders probably would just let their spam fall into the bit-bucket. It would actually make spam *cheaper* for them, because they weren't transmitting as much data (the meta-information for everybody, but the message itself only to people who choose to read it), and that might give them less incentive to force their spam on people who didn't want to read it. And as a recipient, I might choose only to accept mail that certifies compliance with policy agreements that have enforcement clauses ("...and if this message shall not originate from the Authenticated Sender, Actual Sender shall be liable for damages of..."), and then anybody who successfully sends me spam had to agree to that contract and is liable for breaking it, if I can track them down. The fact that lots of spam actually does get sent out with "ADV:" or the Korean or Chinese equivalent in the subject line suggests that at least a noticeable fraction of spammers would just not list agreements fraudulently and live with the smaller market.
Well, it's half-baked, but it's an idea.
No, because there is no "slashdot collecitve"; there's only a bunch of discordant banshees screaming stridently at each other.
Okay, so maybe this is sort of flame-bait-ie or trollish, but it reads well. I think being a banshee would be fun.
Furry cows moo and decompress.
A lot. Tell me, of all the spam you've received, how much of it came from a 100% legitimate source? A machine that wasn't hacked? A non-open relay? The spammer's box itself? Was it still up when you checked?
Yes, as a matter of fact today I blacklisted a spamhaus that began crapping into my INBOX from a new IP address range. I've had their primary spam spewer blacklisted for over six months now, they've been happily hammering away from their Exodus IP address space for a long, long time. But I guess that after pretty much everyone had them blacklisted they decided to start spewing from a new IP address range.
Granted, this kind of a situation comprises a small minority of my incoming spam. But that's only because there are other, easier spam delivery mechanisms available today. But I see nothing that would prevent everyone else from switching to this modus operandi once those other spam delivery mechanisms are eliminated by "son of SMTP": all the spamhausen will simply churn through "100% legitimate source IP addresses" at a much faster rate than they are doing now.
In the end, switching to "100% legitimate source IP addresses" would've bought very little benefit in the grand scheme of things, and we're back to square one: instead of thousands of hacked boxes and open relays, we'd have thousands of "100% legitimate source IP addresses" spamming away into the night...
Heres an idea , it also helps make it a bit more secure . :-) .
All communications of the new protocol must be initiated using IPSEC (using more processor) and then the server gives the client a randomly generate GPG public key each time they connect and the entire conversation must be encrypted with the public key. Of course this will become less effective as processors get faster , but just increase the key size
For most people, unsolicited email isn't the problem. Unsolicited bulk email is the problem. Generally, unsolicited, personalized, individually sent emails aren't sufficient to bother people. ... Any solution which depends on security through obscurity (hiding my email address) or refusing unsolicited personal email from strangers, is unacceptable to me.
Ahh, but if your E-mail address is out there on some web site -- together with some arbitrary content -- and someone chooses to send you E-mail after reading said contents, and wanting to respond to you for some reason; then, well, from where I'm standing it looks to me like you've solicited that response, didn't you? You certainly didn't solicit some spambot sucking it out of the HTML, and then feeding it to a Viagra spam engine, of course. But, by the virtue of signing your name+E-mail address to some article, I would argue that you've solicited individual readers to drop you a note after they've read the article, if they felt like it. If you didn't want, or solicit, people to reply to your Slashdot posts, then you simply leave out your address, that's it. What good would posting your E-mail address here if you don't want people replying to it?
I mean that's what an E-mail address is for, I think: for people to know how to get in touch with the author, regarding the written subject matter.
Similarly, by signing your name to your Slashdot posts, I would argue that you've solicited, or invited, E-mails from anyone who've read them and wanted to give you a piece of their mind, for some reason.
I frequent other web discussion forums, where I do NOT use my E-mail address. That's because I really don't care for any replies on the subject matter over there. Here, if someone wants to flame me away for something -- go right ahead. I'll bitbucket it, of course, but I wouldn't say that it was unsolicited.
So, I think, it all comes down to "solicited" vs. "unsolicited". It's rather impossible to give a precise definition of "bulk". What is "bulk"? If "bulk" means a certain amount of substantially similar messages, then there has to be some number X where X is bulk, but X-1 is not bulk.
So, what is X, then, and can you provide a valid, cogent argument why X is bulk, and X-1 is not bulk?
I don't think this is going to work. I don't think you can make a common-sense based argument for bulk vs non-bulk. But I think an argument on solicited vs. unsolicited can be made, based on a common-sense definition of "solicited."
It appears that your definition of "solicited" means something that's exclusively a response to some previous written screed of yours. I think that a more liberal definition of "solicited" works better: meaning "did you reasonably expect to receive this kind of a message." Obviously, from the fact that you've posted a message of your own to a discussion group, with a valid E-mail address, it can be reasonably inferred that you've asked -- hence solicited -- replies. But, at the same time, if you put up an average home page, where you wrote that you're a graduate of East Side Nerd High School, and if you've provided your E-mail address, then it can be reasonably said that if an old buddy of your from East Side Nerd High accidentally stumbled across it, his E-mail wasn't exactly unsolicited. He didn't just sent something, addressed to your E-mail address, by random. It was in direct response to what he read -- hence it was solicited.
On the other hand, if some spambot lifted your address, and started sending you Viagra spam, then it can't be reasonably argued that you've solicited Viagra spam simply by the virtue of describing your high school follies.
Now, you might think that this approach is not going to work because "reasonable" is in the eye of the beholder. Something may be reasonable to one person, but not reasonable to another person. Spammers will argue that using your E-mail address on slashdot can be inferred to reasonbly means that you want
A combination of white lists/black lists, and Baysian filtering stops so close to 100% of spam that it's really silly for anyone to be bitching about spam these days. I don't GET any spam anymore - 0. Not 0.001%, 0 - the integer 0, as in none. If I ever get another piece of spam, then I'll change my email address
[...]
Stop yer bitchin', people, and implement the technologies that are already out there and work great. Plus use yer freakin' brains for a change, and don't spew out your real e-mail address to everybody who asks for it.
Don't make claims that the current technologies are "so close to 100%" if you're not willing to put your money where your mouth is. Post your email address far and wide, then see if you still don't get any spam. Otherwise, don't get up on your high horse and tell people they shouldn't worry about spam.
Security through obscurity sucks. We need a better solution for spam.
Deven
"Simple things should be simple, and complex things should be possible." - Alan Kay
YASMTP (Yet another)
SINS (SINS is not SMTP, but spam is.)
GNU/Mail (Because if it's not crafted by a moose, it's not really free.)
I've run out of GNU rip-offs, and I can't think of dependant recursive acronyms to make fun of HURD. I personally like SINS, because it makes a word.
Now all you have to do is require signatures and keys to be on a trusted, non-profit, key-server. We all pay 1-5$/mo for our key so the keyserver will stay up. It's non-profit, so no selling out to spammers. Contracts won't let you send mail that the recieving party didn't solicit. If you get a few verified reports on you (log only source and destination for a month so that it can be verified within a reasonable amount of time), you get booted for breach of contract.
You have to charge for two reasons: 1) paper trail to spammers bank account. 2) pay for key servers and bandwidth. If you can solve both these problems without a VISA card, I would be thrilled.
The end of spam is near. People will set up SMTP bridges at the source that will automatically sign email before putting it on the new network if the target is the new network. This will let legacy applications that clients are attached to still take advantage of some of the features of the new network until the software catches up. (Could do something like Norton does to catch mail under windows.)
Also, it will be nice to get rid of the cracker fodder that is Sendmail. VHosts will work right. Most of the reasons for using all the weird sendmail hacking will be eliminated.
Please make the world a better place! I will help if it's needed. I would love to implement a search API for IMAP-ish stores. If anyone gets serious about making something like this, email me.
Karma Clown
For a measley $20 or so a year, you can register your own domain name, get all e-mail to that domain forwarded to your "real" address (which you make unguessable and never give out), using a service like Zone-Edit. I've done this and effectively cut off ALL spam, since I give out a different e-mail address to each entity (usually something like, "[company-name]@[mydomain].com").
When I discover a piece of spam, I check the sent-to field, and set a rule in my mail program to color it a certain color using the sent-to field as criteria (or transfer to trash), since I know that this single address has been compromised somehow. Since you can clearly know WHICH e-mail addresses are compromised, it makes it very easy to filter these out.
Don't replace SMTP! Then I'll have to learn a completely new set of commands to type into telnet every time I want to send an email!
"Freedom means freedom for everybody" -- Dick Cheney
To address the problem specifically with bogus reply addresses.
A message is sent by the mail server - add a random key value to the message
The receiver sends back an authentication request to the reverse dns lookup with the
Random message Key
Sent Date
Addressee
Addressor
Subject
The smtp receiver opens a UDP request on a new well known port back to the sender reverse lookup address with the above information. The Sender then either sends back a an authentication or denial.
If the authentication is confirmed, the message is accepted. The random number only has to be kept with the information for 180 seconds after the message completes sending. After which the number is purged.
If e-mail receiver is not compliant and e-mail sender is:
The message is automatically accepted
If e-mail sender is not compliant and e-mail receiver is:
The message is flagged as suspect and placed in a non-authenticated mailbox similar to a spam mailbox. Alternatively, the client can flag the message as a different color or similar.
This does nothing if a legitimate email address but his makes spam blocking systems much more effective as spam senders will be quickly known forcing more frequent domain changes.
I would agree with this, except I get spam that is not an apparent advertisement for anything. I suppose it is just a test to see if the address bounces, but still, I can't understand why I get so many like that...
Are you sure you are not viewing an e-mail with multi-part/alternate and a blank text and a real html part?
http://www.ietf.org/internet-drafts/draft-danisch- dns-rr-smtp-02.txt
This is a _very_ simple mechanism: when receiving mail from domain "slashdot.org", simply lookup the nameservers for "slashdot.org", and check if the sending server is authorized to send _on behalf_ of slashdot.org. Reverse MX.
i.e.:
slashdot.org. MX 10 inbound.smtp.slashdot.org
slashdot.org. RMX outbound.smtp.slashdot.org
This will cut massive amounts of spam posing as other domains (aol, yahoo, hotmail, and everything else of course).
Use SMTP over TLS, authenticated at both ends by a certificate. It's not like we all need to go buy expensive VeriSign certificates, either. For private users, you could set up a PHP-style "web of trust" using certificates. For corporations, you could use traditional hierarchical PKI.
Once all email traffic is encrypted, nonrepudiatable and origin-authenticated, where will we be?
We'll be in privacy hell. Because it will be effectively impossible to send email without leaving an unerasable trail. So, if you plan on securing email -- be it through SMTP or another protocol -- you'd better add plenty of support for anonymous pseudonyms!
When was the last time you got a PGP-encrypted spam?
Stop accepting unencrypted mail and your problem
is solved.
-I like my women like I like my tea: green-
yes, those "remove me" links actually work. The webpages themselves seem like total BS, but I mean, it's just a CGI that adds your email to a text file that is processed later to do the actual removing.
When I stop doing the "remove me" thing and let Mozilla just delete my spam, I find the numbers gradually increasing back to a level of 100+ a day. And then, I go through and submit my email to the remove addresses. Which settles it down to a normal level.
My main problem with spam is that it's taking up a whole lot of bandwith. Mozilla keeps 99% of it out of my inbox, but I'd rather the message were never sent.
It breaks my pluginses, my precious!
E.g. opt-in requests, mailing lists and anything else that can be spammed? Oh so I added 500 fake hotmail addresses to that list. Now that server is tarpitted to hell and back on hotmail, and noone of the real people will get their email.
Tarpitting is *not* a good solution...
Kjella
Live today, because you never know what tomorrow brings
You gotta love how the poster feels that the Slashdot community is capable of producing such sweeping change. The only thing that might happen is a +4, Interesting.
Great, now I'm a -1, Troll. But I calls them like I sees them.
Bayesian spam detection is too late. The main problem with spam is not what the end user sees, but what the ISPs and mail providers see: bandwidth consumed by waste, and so made unavailable to 'real' users, but still being
payed for by the ISPs.
Paying is a fairly simple solution: it limits the amount of mail a spammeister can send to what money he can pay, whereas today it's basically free. Someone with better insight into spam economy than I can say if that would do the trick.
Signing email won't help the ISPs with their problem: they can't verify every signature:
email forwarding would grind to a stop.
As for the end user, it *may* help in finding out who the sender claims to be. If the certificate is signed, it may even indicate that at one time a certain mail address worked, but it says nothing about what the current state of affair is.
In order to be really useful, you'd have to accept only signatures attributable to a real person, not just an email account. And for that that person would have to run through some real hoops, and few would do that.
Which means that the spammeisters would probably pay for private keys, and so virus and other malware would be created to find badly protected private keys. And so the problem shifts elsewhere.
Spam meisters consume 30% of the bandwidth of the Internet with waste mail. Should they not at least take the costs for that?
"Maybe ISP's should charge users for each outbound SMTP connection they make? I'd happily pay 10 cents per email I sent if it would reduce the amount of SPAM I received. It would only cost me a couple of bucks a month too at the rate that I send email ..."
Spammers already used stolen credit-cards to sign up to ISPs to spam from them, and highjack other people's connections.
What makes you think charging will bother them when it'll be mugs like you and I paying it, rather than the spammers?
Plus, charging for email will just give rise to a large and diverse set of "new" forms of communication, none of which cost anything to use, rather than the standarized email we have today.
Charging for email is NOT the answer, and people like you suggesting it are just going to play into the hands of the "corporates" who are "looking for new revenue streams".
Here's a synopsis of the ideas that I've seen touted here over the years:
We're a bunch of mediocre thinkers who sit around belching and farting and thinking that our opinions matter. They don't. What matters is that we get served ads in return for that belching and farting. But let's not get delusional and actually start believing that our opinions do matter, even here, and especially not in the real world.
If you were blocking sigs, you wouldn't have to read this.
How about having an XML based system which converts SMTP into XML on the local server, sends it via HTTP/HTTPS to a similarly enabled server which then re-converts it back to SMTP and pipes it to the local mail server?
;-)
Ok, I know this is a bit heavy weight but this would allow legacy servers to be secured fairly easily and remain part of the sending system until mail server software is updated to a new version which supports XML/HTTPS natively.
Oh, and the name? I quite like KMTP
Actually I'd say that this could be done already with current technologies, it's just that ISPs and large network providers are not being responsible in ensuring that the users of their networks pay the appropriate price for sending out SPAM.
It may be possible, but it's completely unworkable. What if the spammers were to run their own servers and somehow bypass the ISPs, I think they would be willing to pay the surcharge to charities!
Even if some ISPs implemented this small charge market forces and differences between different countries would mean that there would be plenty more places left for spammers to shovel their shit.
You've probably noticed that people's noses get bigger as they get older. That's because old people are huge liars.
Yeah we're one big beowulf cl... ah forget it :(
Also it's a lot harder to hide where the email came from. If you don't supply the correct details then the mail can't ever be collected.
On the down-side; it'd let spammers know which addresses are live.. and it'd enable any message sender to know when their message got read and from where, which raises some fairly big privacy issues.
455fe10422ca29c4933f95052b792ab2
I've always thought that ISPs should just effectively put a massive tax on spam into their user agreement. Basically, if your IP gets reported as sending spam, they investigate you, and if you are, charge you $50 per email sent, and permanently disconnect you, possibly blacklist you as well.
maybe spammers would just move overseas at that point. i dunno.
It seems that with todays VPN and Proxy technologies it would be a simple matter to start setting up virtual X.400 mail relays acrosss the Internet. There are already plenty of pay services for this, but it would seem a coordinated group of people could provide a similiar service for free, or a much reduced cost. By cutting out the 45 minute garaunteed delivery time, it would become even easier.
And I dont suppose some of the bigger service providers would mind letting secure virtual X.400 services connect with thier "real" X.400 services.
X.400 is an old technology, but a reliable one. One that could be used give spammers a really hard time.
Or perhaps X.400 needs a facelift?
McDoobie
If you get a few verified reports on you (log only source and destination for a month so that it can be verified within a reasonable amount of time), you get booted for breach of contract.
As someone who runs a solicited advertising mailing list on behalf of a client I can state that being solicited (we only send to people who have signed up at the web site, we confirm addresses) and offering a working unsubscribe facility does _not_ guarantee that they won't complain about the messages anyway.
People just forget that they subscribed, never bother unsubscribing and complain for the sake of it.
I've heard this argument thrown around alot. My reply: has anyone tried in earnest? I didn't install Gentoo Linux until I heard other people's success with it. I would not have been able to install Gentoo unless someone put together the distribution.
I think we underestimate the power of word of mouth. If a new mail protocol and system were devised, an ISP installed it, and that ISP found that it was da bomb, then I suspect that other ISP's would get on board real fast. No it won't be easy, but you subtract 100+ spam mails a day from your average user's inbox, and you might have a customer for all time.
Okay, so it's a great idea to have old-new idea's like mail2000 and some variations on it.
What about a mailclient that would issue certificates with a one-time-key encryption back and forth between sender-receiver. You would receive my encrypted key and issue back an authorisation with your encrypted key. Based on your email adress and the key we both issued a somewhat advanced checksum is compiled and the email adresses are authorised to send or/and receive messages to one another. This would require no changes to SMTP or SMTP servers (wich took long time to become stable AND secure). AND autorisation based spam-fighting would enable users to simply ERASE all other mail in their box or keep it for review. Authentication spamming could probably easily be countered by some limits built into the mailclients on both sides. Receiver would automatically erase all double requests etc.
So it's a bit of a pgp-thing mixed with icq, shaken not stirred. And probably you'll laugh, but it will not cost fortunes and a whole new infrastructure would it ?
free dom(inion) - free energy - free your mind - whee!
Maybe ISP's should charge users for each outbound SMTP connection they make?
I a world of zombie computers forwarding SPAM this won't hit spammers at all. It might just make people secure their computers better (about the time pigs learn to fly).
> I wish people would stop inviting rate increases or new charges as an answer to spam. It's not the answer.
But wait. What if ISPs simply charge each other for traffic depending not on the direction of traffic but depending on which side initiated the TCP connection. That way the person downloading from a web site will be the one paying (because he made the HTTP connection) and not the web site host. And the person sending the email will be the one paying (because he made the SMTP connection) and not the recipient.
If only a few big ISPs agree to work like this others will follow and soon even small ISPs will start charging their customers for traffic based on this method.
Could this help to put an end to spam?
That was the sound of the point going over your head.
The transaction should be encrypted so that a simple packet sniffer can't see your password as you log into your mail server. Have you heard the one about Telnet being insecure, because your password is sent as plaintext? Well that applies to FTP, POP3 and in some cases SMTP Auth (It depends on the AUTH method). Anyone who can see the packets between you and your server can see your password.
If, when you go to send them the confirmation, they try to charge you 10 cents, that's a clear indication of no confirmation.
well, yes, thats obvious, but what about the poor mailling list operator who's stung for 10 cents... per person signed up by j random loser? if they signed 1000 ppl up then our mailling list owner is $100 out of pocket.
dave
ARIN and ICANN aren't in chanre of these "web services" you speak of. ARIN (and RIPE and APNIC) are in charge of allocating IP addresses out in their relative areas and ICANN rules int gTLD's. there is no-one in charge of "web services" and neither do I think there should be one, nor of email.
while I see spam as a problem, I'm not sure I see a technical solution in the near future. most technical solutions are either too unweildy for most (ie, TDMA), too problematic to instill (pay per email) or too against the original ethos of the internet (every sender has to be traceable to a unique digital ID).
dave
Since you just look at emails tagged as spam with low scores, you probably mean that you haven't _found_ a valid email with a high score :-)
But I do get important mail from people I dont know.
As do I. Come to that, I thought that was the whole point of email in the first place. If I just want messages from people I already know then I can use IM.
It seems that almost every "solution" here wants to turn email into a closed, black boxed system where you have to go through three circles of hell just to get your identity verified, and then you're treated with suspicion and fear whenever you actually send a damn email.
Yeah, thats just what we need, a closed worldwide communications system! Just try it; I'll start running an RFC2821 SMTP server and peering to every other right-minded individual out there, and I'll gateway to your "trusted" system, too! Email needs to be open and simple. I bet not a single one of the people here talking about PGP signatures, encrypted channels or trusted networks has ever needed to write an SMTP client.
very good!
...
:)
:-)
finally!
first off remind ourselfs about the network.
there's the OSI. there are a few layers.
i view the lowest layer NOT open source because
we cannot just make hardware. so here we have
a standard so a NIC or whatever hardware
gives us network functionality is compatible
accross the platform
but then what we want to send over this physical layer is complitely OPEN SOURCE. it doesn't have to be TCP/IP or anything like that. we can just make our own protocol!!
but since we are lazy (and want to play games too) and not code the whole day long, we'll use TCP/IP.
so there's RFCs for that. finish.
but now we want a service running on that protocol, say sending messages. we have SMTP (should acctually be called SMTS (Simple Mail Transport Service)) but nevermind. This ancient service should acctually be dead already (AIM, Netmeeting, ICQ, IRC etc.) but since spamers are
using it so exessivly it is alive and thriving ; )
but we want to receive messages too, even if we are not online!
so somebody on-line all the time (24/7) has to cache it for us.
SMTP (it should be called SMTS) does this for us.
i guess one big flaw is, that SMTP (should be called SMTS)does not honour bandwidth.
my SMTP (should be called SMTS) can send as many e-mails as it wants, if the destination/target (this is a military term)is listening, ready to receive. this is the flaw.
somebody has already mentioned, that there's is better way to do this. by caching the content on the source computer. this way the bandwidth will be honoured. the receiver/target will just get a notification ("You have mail" (sic)) but not
the real content/text/message/payload(military term).
the intended receiver will have to go pick it up at the source(if he wants to).
this is also smart because the spamer cannot be procecuted because the receiver/user/consumer was dumb enough to go pick it up
-side note-start
if you drive drunk or to fast and you have an accident this is not the problem of the goverment who built the road or the comapny that build the cars.
we have to stop making laws to protect stupid people. if they don't grow up and get into what they are doing nothing will change. if you want to drive a car you have to pass a driving-test and understand street-signs. maybe we should make a law, that requires internet user to first
visit a internet-driving school, to not always bump into does street-signs and other user.
-side-note-end
if this were implemented, a spamer would have to cache all the spam on his computer. it is not a great problem for him, since it's just one message (mostly same for all targets) but
he cannot saturate the bandwidth of the public internet.
if the dumb user (sorry but an old SYSADMIN wisdom is that all the secrurity function cannot help a blind man see, e.g. a dumb user will still click the spam and/or open the *.exe/vbs file.)
still wants the spam, he can go get it.
my main concern is bandwidth, harddisk-capacity, CPU-use-time. one should keep this in mind for future improvements of e-mail.
some philosphical thoughts. when the internet was released publically it was mostly is/academis/science/labaratories/ etc. using it. the protocols/services defined where for people who wanted to be productive. but the internet today is for momas, popas, kids etc.
SAY CONSUMERS!
they do not know the history of the internet etc.
i do not communicate with these people since i know they are a security threat. if i send them an e-mail they will have outlook, etc. configured
to add me to their address-book. if they get spam or a email virus (*.vbs,etc.) i will be a target of their zombie system. these people do not
rely on their computer. if it drops off-line or their HDD gets formated it's a day of bitching and the next day they pay the PC-repairman 100 bucks to reinstall windows and off they go write to their friends how mean the internet
was
No to mention all the idiots who use words like boxen.
boxen is a perfectly cromulent word.
Idiot!
so use snail-mail, dumb ass!
"it possible for the Slashdot collective to come up with another one?"
10 monkeys could come up with a better specification than this one. 2003 and we still can't have an e-mail standard that natively supports binary data.....
It is probably the case that SPAM is a real "tragedy of the commons". However there are two separate issues. One is the end recipients problem, our problem, the pain of dealing with SPAM, and the cost in terms of the extent to which it costs us bandwidth money or degrading our "service". There are a number of different more or less successful methods for dealing with this and we can all do it on a case by case basis to scratch our respective itches.
But the more important problem is the impact that the problem has on the overall infrastructure of the net. For example LINX, the (one of?) the major link points into the UK, recently reported some metric about how much spam was coming down the lines. (Sorry can't find the reference). It is when the concentration of this rubbish gets so higha s to affect this level of infrastructure that we all have a problem.
Now the tricks required by the spammers to try and get around the filters may preclude some of these ideas, but the idea that I should receive 'a copy' of Jane, or Sarahy or Ken's Viagra offer, ie a copy of the same mail that was sent to all the people at my mailhost (this is particularly important for big providers like AOL or BT where they have millions of customers) rather than an individualised one would certainly reduce the strain on the underlying infrastrucutre. How would one achieve that? Good question, I haven't really thought about it enough, but I am not suggesting an economic answer, I don't want to try and coerce the spammers, let's try and make the solutions in their own interests before we use the price tool.
However having rasied the price question, it is critical to remember that the reason why the junk snail mail, SPAM comparison falls down is that the marginal cost of increasing the distribution list of a SPAM is zero and the marginal cost of increasing the distribution list of JUNK is (in addition to postage) the cost of the JUNK. This provides a natural price imperative to the mass mailer. Can we introduce such a cost to the spammers. Er, well um no. Not that it is hard, ot is just impossible. Think as hard as you like and anything you come up with will be flawed in some way. So don't try. Just accept it.
So, we must return to the politics of SPAM. Clearly the sender of the spam must be identifiable. Perhaps it must be identifiable from the "International register of spammers" what ever that might be. SPAM not sent by one of these servers is automatically dropped. Then one must decide how one determines what is SPAM and there is probably the rub. I don't really know maybe a global bayesian filter?
Anyway. The point is that the first problem is to contain SPAM, make it in the spammers interest to identify their mail as SPAM. From then we can manage the SPAM bit by bit. Of course the obvious solution is _don't buy from spamvertisers_ but then it only takes a few idiots to make that strategy infeasible.
"The first thing to do when you find yourself in a hole is stop digging."
and all the idiots who call other people idiots for mispelling "teeming" when in fact it is the correct spelling.
When SPAM hists the wire, it costs resources. If it has to go all the way to a POP server, or even to the recipients own system before it is rejected then it is one big waste of space.
On your last note about volunteering yourslef to receive spam, well sorry that doesn't work. I have seen enough Emails come through with generated name combinations, i.e., Adam.Smith, Adrian.Smith until they hit something. VFY from SMTP will even help.
See my journal, I write things there
RFC-822 is about ASCII. Anything else causes hiccups - even foreign characters such as German umlauts. The messages are sent as text, which means anything that isn't must be encoded. MIME and so on is handled transparently, but it shouldn't have to be. Each byte I send down the wire is 8 bits, using a subset of seven of them is a waste unless all I am sending is US/UK text.
See my journal, I write things there
I believe any change to existing email protocols would also incorporate features for billing customers, much like SMS
yes...
Nerd rage is the funniest rage.
Hey,
But if everyone were to use Bayesian I swear we wouldn't even have to propose a new protocol, talk about new legislation, etc.
Why not put filtering on the outgoing SMTP servers? Sender authentication could be turned on to tell ISP customers apart, and people could be allowed to send, say, 3 e-mails that set off the filters per day. After that, thier SMTP server bounces thier e-mails for rewriting.
As we're putting sender authentication on, we could mark message headers with an abuse address and a message number. The ISP could keep a table equating message number with username (could throw away/recycle message numbers after 48 hours). Mail readers could have a 'report spam' button which forwarded the message and headers to the abuse address. The ISP could confirm it is spam, and automatically penalise the sender.
Just my $0.02,
Michael
"Goodness me, how unlike the FBI to abuse the trust of the American public." -- The Onion
The "pay for email" approach would only work if it was possible to whitelist addresses who would then not have to pay. The mailing list problem then would not exist -- you simply require that anyone who signs up whitelists the mailing list address.
So why not just stick with the whitelist, and not worry about the 'pay for mail' part?
autopr0n is like, down and stuff.
I was getting all pissed about spam untill I got the baysian filter for Outlook. God that rocks. I still get spam, but now I can watch it all being deleted right before my eyes :P
The few things that do get through either have obvious subjects that I can just delete, or obviously designed to get around the filters.
I've been getting a few (like one or two) 'image' spams which contain a bunch of random text and an 'image' which is an advertisement. All we need to do is code Bayesian filters to comprehend the text embedded in images, and we should be on our way.
Seriously people, don't propose some draconian anti-Spam measures until you have a Bayesian filter installed. For most people the false-positive rate is better then if you delete Spam manually. That's right, you are more likely to falsely identify Spam then the filter.
I also think the whole sender-verification thing is good, where a person needs to reply to a letter to 'prove' they are a real person. (there's another term that's in vogue these days for that but I can't remember it)
I think Bayesian filters, sender-verification for high-scored msg, and reverse MX together will probably solve %99.99 of the spam problem. This is a lot better then charging for email (which is idiotic) and blocking off huge parts of the net in a sort of 'let god sort 'em out' attempt to keep people from spamming.(note that somethingawful just got there entire class B blocked by spews.org)
The trick isn't to try to stop people from spamming, but rather to prevent yourself from getting it.
autopr0n is like, down and stuff.
I may have read something about what Internet mail 2000 is years ago.
.. Basically its Parcel pickup. But its simpler since servers don't need to talk to other servers to handle clients etc. Instead it would a cross between an ftp server and pop. Perhaps could even be an extentsion to pop. .. it doesn't go to them. /better behaved/ if they care to be...
But here's an idea I add a few monthes ago. Leave email as is -- mainly.
But add user settable rules for max size email. Like I'd take a 4k email
from anyone, but not 4mb from anyone. Really if its actually an email then 60-100k
is generous. And not in pop server, thats just downloading it.. I mean user settable rules (filters ) that truncate the mail to a size in the inbox.
Add another protocol that works like reverse email where the "parcel " stays on the server until you pick it up, to supplement, not replace email. The protocol would not itself notify you, it would be strickly pull, user initiated, so email would still be needed. ( or a phone call, or snail mail..
Suppose you had a email-bulletin, the email could go out plain text ( maybe html-- you can't stop them), but no attached megabyte pdfs... instead that goes to the parcel pickup server were it stays until everyone in the to: list picks it up ( or a certain amount time ) then it's deleted. If the user doesn't care to pickit up
And may add some basic encryption for a parcel-id. You could do this with a webserver or ftpserver.. but it would require a db and scripting or you'd have to do manually.
The problem with email is all the extra things its having to do. Like file distribution.
Really there needs to more alternatives.
None of this kills spam.. but it would minimize the effects, and allow users to be
Remember that the original ethos of the Internet was to design mechanisms that allow the U.S. military to function in the midst of all-out nuclear war. The equalization of the masses was just a side benefit. :-)
I wish people would stop inviting rate increases or new charges as an answer to spam.
Then why do you advertize a spam-filtering pay service in your sig?
This may be total flamebait, but I've been "out of the ISP business" for a few years now and I don't remember (honestly) anyone suggesting this:
Facts:
1. Email comes in from an IP address.
That's all you need to take into consideration for this.
Solution:
1. Have the recipient mail server check the sending server for an open relay. If it exists, send an email to "root@, postmaster@, and/or administrator@" on the originating server letting them know they have an open relay and the message sent was disallowed.
2. Have the recipient server keep a compressed (or otherwise optimally-indexed) log of IP address mail is sent from, recipient, and some sort of searchable indicator of the body content of the message.
3. Configure recipient mail server to deny messages from any particular IP with a body and/or subject matching or closely resembling the indexed bodies if the message is sent from the originating server at a particular rate (e.g. 25 or more emails in 5 minutes' time) to any recipient address on the system.
4. Configure recipient mail server to deny messages originating from a unique mail server that appear to be dictionary-based scans (e.g. aaa@, aab@, aac@).
5. Combined with the above or separately, configure recipient mail server to deny messages to (z) invalid addresses at a rate of (x per y minutes) with identical bodies or nearly identical bodies and/or subjects.
6. If a user on the recipeint system/network forwards an email to spam@recipeint.domain, automatically quarrantine all messages from sending server for a period of (x) minutes to allow the recipient server to "collect more evidence" in its incoming spam scans without actually delivering more messages to the recipients' inboxes.
I'm truly looking for feedback. I don't want "you're an idiot because you didn't think about a, b, and c" type replies. I obviously haven't thought of every possible loophole or abuse case, so I'd like constructive comments. I believe, at a high level, that this could work.
How much legit email was being accidentally deleted by people along with the Spam? Did they ever check that? I'd be willing to bet that more legit mail was being deleted accidentally by people then was being deleted by the filters.
autopr0n is like, down and stuff.
What if someone sends you a message, then dies? You'd never get the message.
But seriously, sender-verification is great, and if these Bayesian filters stop working so well I'll probably get it setup. But really it's one tool in a toolbox. The best solution would be to use that only for messages that fail (or get an ambiguous score by) a Bayesian test or a reverse MX test.
autopr0n is like, down and stuff.
You would send out these emails without a 'stamp' so they couldn't get charged. People would need to open their mailboxes for a while while they wait for the conf mail, or add the sending address to their whitelist.
autopr0n is like, down and stuff.
It would also work if all people running mail servers agreed to a payment protocol. It wouldn't happen of course, because it's idiotic, and entirely to much work.
autopr0n is like, down and stuff.
The EMA - electronic messaging assocication (www.ema.org) helped build and advertise x400 10 years ago as a replacement for smtp mail. only a small amount of businesses bought the idea, and of those probably 5 are left. Now its too late, to much investment in smtp mail to change it.
START TLS already identifies the source MTA. No START TLS -> full filtering; certificate of known trusted source -> avoid unnecessary filtering.
One thing *is* needed -- I don't recall any standardized way to provide a trace of all certificates for relayed messages. We need something like Received: only for cert.s, or a standard way to insert/extract cert.s in Received: lines.
It's interesting to note that, without a certificate trace, current practice (using, in some cases *forcing*, relay through faceless ISPs' MUAs to sanitize outgoing mail) works *agains* real secure SMTP exchanges. The transport layer can't verify the identity of a sending system if it only knows the identity of the last relay. (But note that the application layer can still apply the model given above: if it's from someone known and trusted, don't bother to apply filters.)
All of the proposals I've seen for replacing SMTP want to replace it with something so cut-down and locked-down that it only serves a small sub-community's ideas about what email should do. The nice thing about SMTP is that it is general enough to encompass a lot of different models.
Trusted protocols abound. SMTP, for example, has a protocol within it called TLS. This allows key exchange, and once you've done key exchange, and are talking over a secure connection, the world is your oyster. You can give each identity a trust rating, or you can only alow mail from identities you've manually addded or you can just use a whitelist of valid identities.
It's not the trusted protocol that's hard, it's the infrastructure AROUND that protocol. You need a tool that a) allows (perhaps degraded) communication with non-conformant hosts b) everyone has an incentive to use and c) makes senders of valid mail more trusted over time.
Unfortunately it doesn't look like it would do much to stop spamming, which is the major problem with the current internet mail infrastructure. For that, we need some way to make sending bulk email costly to spammers.
It does a lot. Now spammers rely on ephemeral servers to blast mail to your mailbox and then they dissapear.
With the sending mail server having to store its outbound messages spammers have to keep a system alive and running for their spam to ever get to you. That's a big difference! Suddenly they're accountable and they can't get to you without your being able to track them down.
-Peter
== Just my opinion(s)
For example, you see one of your own message IDs in the References: header of an incoming message. That tells you that it's a solicited response to one of your own messages.
No, it doesn't. It simply tells you that the person sending the message managed to get hold of the message ID of something you wrote once upon a time. The spam (if it's spam) might be totally unrelated to the original topic of conversation.
I send messages to several different mailing lists which cover a variety of topics. Searching for my name or e-mail address in Google would generate hundreds, if not thousands, of Message IDs which a spammer could insert into the headers of their messages.
I cannot see any way in which your proposal differentiates genuine responses from automated bot-harvested Message ID spamming. Sure, it raises the bar for spamming by a small amount; but mainly it would only affect new mail addresses or people who never write anything (publically).
For most people, unsolicited email isn't the problem. Unsolicited bulk email is the problem.
I disagree. Unsolicited commercial email is the problem. I don't care how many other people received the same message, so bulk doesn't bother me. What bothers me is having commercials shoved in my face.
defecate!
$ grep ^[asdfghjkl]\*$ /usr/dict/american-english-large | wc -l
141
$ grep ^[aoeuidhtns]\*$ /usr/dict/american-english-large | wc -l
2103
$ grep -i ^[asdfghjkl]\*$ /usr/dict/american-english-large | wc -l
290
$ grep -i ^[aoeuidhtns]\*$ /usr/dict/american-english-large | wc -l
3575
Don't want to move your fingers at all?
$ grep -i ^[aoeuhtns]\*$ /usr/dict/american-english-large | wc -l
1118
$ grep -i ^[asdfjkl]\*$ /usr/dict/american-english-large | wc -l
138
If you just transpose ; and e you get:
$ grep -i ^[asdfghjkle]\*$ /usr/dict/american-english-large | wc -l
942
So having puncuation on the home row really hurts QWERTY.
I suspect that as spammers get smarter about Bayesian analysis that they will find tokens that register as non-spammy for a large percentage of the population. And as we implement measures to discriminate against those tokens, spammers will migrate to a new set, and so on and so on and we'll discover that Bayesian filtering is just another round in the fight between spammers and... well... everyone else.
Filters are -very- expensive (both for the computer, you and me), and a fee-per-email system is silly, and does nothing to actually control spam.
The only really effective way I can think of is another fscking registry. ISPs and companies large enough to really need external relays pay the fee to register their mail-server there, and the new implementation of the SMTP-protocol only accepts external mail from other listed servers.
The downside? A fee comparable to the price of a domain name for ISPs, companies and stubborn individuals. Don't give me the old crap about "having to run your own relay", because you still could, by in turn having it relay through your ISPs server. Your ISP doesn't provide you with a relay? After this, they would have to.
The upside? It would be a lot easier to blacklist spammers. No more hijacked boxes on broadband-connections flooding us with spam.
Oh, I know, it will be shot down because there's a fee involved, but keep in mind that I would be one of the people that would have to pay that fee, and it would be a very small price to pay to protect myself and my users from spam.
Right... you keep telling yourself that is an opt-in aproach, and unsubscribe is an unsubscribe option.
I'm only kidding, but there is some truth to it. 1) They wouldn't be complaining if it was obvious that they were going to be spammed, 2) They wouldn't be complaining if the remove function was prominantly displayed and worked. Most people that complain the way you do are those that try to hide the fact that they are going to be spammed, and have hoops you make them jump through in order to unsubscribe.
If that's not the case, keep records, and in every email say "You signed up on this site at this date/time, then you responded to our verification email at this time from this IP. If you've been signed up in error, click here and you will be removed from our list such that you will have to call tech support(#####) in order to be added back." See how many people complain then.
Karma Clown
I'm even opposed to the "pay a dime, but I'll give it back if I wanted to hear from you" approach. Those of us running a mailing list would run the risk of having some idiot sign-up a bunch of accounts only to have that person say "No, I didn't want that" and collect the money.
You are seeing a problem where none exists. There are two options.
Option (1) If someone wants to sign up for your mailing list they have to do so by sending "stamped" e-mail. You then use that stamp to send them each issue of the list. If they ever cancel that stamp the next issue to them simply bounces and you stop sending it. Running the mailing list never costs you a cent.
Option (2) If someone wants to subscribe to your mailing list they have to put you on their "approved list" and you simply send unstampped mail. If the mail gets bounced for not having a stamp then it is their own fault and you simply drop them from the list. Again, running the mailing list never costs you a cent.
When people hear the phrase "stamp" they usually mistakenly think it works like normal mail where you have to pay for every email. E-mail stamps are different, you buy a dozzen stamps for a dollar or two and you can keep reusing them. Virtually all legitimate mail remains perfectly free, but spammers wind up paying the people they spam.
-
- - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.
Not entirely, there are still holes in SMTP that can be used to relay mail even if your server is not an open relay.
Such as me sending a mail from spoofed user victim@box.com, to nonexistant@yourdomain.com. What does yourdomain.com's mailserver do? it replies to victim@box.com saying user nonexistant not found with the entire copy of the origonal mail underneath. You just relayed some spam.
This particular one is due to an issue with the SMTP protocol, both servers are acting correctly. Receipt verification should be moved into the client imo.
So why not just stick with the whitelist, and not worry about the 'pay for mail' part?
You've never wanted to e-mail somebody who doesn't know you personally?
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
No[t] to mention all the idiots who use words like boxen.
This particular plural construction probably dates back to before you were born, Anonymous Coward.
http://www.faqs.org/docs/jargon/B/boxen.html
Complaining about parts of the geek culture that existed before you came in brands you as a newbie, and by association, as clueless.
it will only hurt regular users to charge for email/services.
Actually there are "stamp" systems were e-mail remains almost 100% free for regular users. You pay a dollar up front for a dozzen stamps and you can pretty much keep re-using them forever. Signing up for a new ISP account would probably come with a few dozzen free stamps.
The system doesn't "hurt" regular users at all. The ISP gives them two dozen free stamps and that can be life-time supply.
the cost becomes built in to their business model. it won't stop, it will only hurt regular users to charge for email/services. Sure, their profits may be cut a little bit, but that's not going to stop them.
The profits on spam are often in the range of a tenth of a cent. If the cost of sending spam exceeds the profit margin then they can't "spam harder" because they just lose money even faster. Virtually all current spam would vanish instantly.
You are right that some spam would still exist, but it would be miniscule compared to current spam. It would also only be for actual valuable bussiness offers - things where there is a reasonable chance you would want. But the real beauty of the system is that you get paid for each spam you get.
Spam isn't a problem if there is very little of it, it is for valuable offers, and you collect a nickle on each spam you get.
These stamps aren't about paying for mail delivery service. There stamps are about paying the end recipient of the mail.
Perhaps the stamps would cost a dime. If someone gets a spam they would cancel the stamp and probably collect a nickle. The other nickle would pay for running the system.
-
- - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.
Don't forget DJB's replacement for SMTP, QMTP: http://cr.yp.to/proto/qmtp.txt
> But if everyone were to use Bayesian I swear
> we wouldn't even have to propose a new
> protocol, talk about new legislation, etc.
Wish that were true.... I've received a lot of spam recently that simply points one to a website:
e.g.
Hey Arun,
Take a look at www..com.
Cheers,
I've also received several variations on a theme. Given that I (and others I'm sure) often send out and receive notes which look VERY similar when I find an interesting site or tidbit on the web, it's unlikely that Bayesian filters will be good at picking these out.
the only possible strategy is to artificially impose a cost on the sender.
There are stamp systems where legitimate email remails almost 100% free and there is a cost imposed on spammers.
It is not an artificial fee for mail delivery. It is a real fee charged by the person reading the mail. The stamp is a deposit asserting that the mail is legitimate. If the mail is not legitimate the person reading the mail cancels the stamp collects the fee for their time.
owing to the nature of public networking, the only remotely reliable way to do that would be to route all mail through a centralized clearing house.
There can be multiple "stamp servers" and the mail itself doesn't have to be routed through them. When you send/receive stamped mail you simply send a single packet to the stamp server to verify that the stamp is valid. Actually it would be even more efficent for ISP's to validate stamps on incoming mail in bulk batches.
It's even possible for to preserve anonymous e-mail and still collect money from spammers. Cryptographic techniques used in the stamps makes it possible.
-
- - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.
But the second point is more important to cover, which is that if the marginal benefit of sending spam truly does drop to zero because all emails sent go into a global bit bucket, and nobody ever actually sees them, then there will be no point in sending email. The reason why spam happens is that it either works, or is perceived by morons to work (I think it's a little of both). If all spam disappeared before getting to its sender, then there would actually be no reason to send it at all. These guys aren't doing it for their health, they're doing it to make money. The moment that enough of their outbound email is dropped at the MUA that it's not profitable to send it anymore, and they have no technical workaround they can use, they'll stop.
Good points. But there still is that "moron factor". If a solution were available that stopped me from seeing all spam at the MUA level, I think there would still be vast volumes of spam coming at me for years. Why?
That's why I'd like to have a means of stopping spam before it hits my SMTP server.
Everyone knows SMTP has issues. However by SMTP, most people really mean RFC 2822 internet mail format, and RFC 821 SMTP, and thirdly implementation of those RFCs on the common Internet.
IM2000 is neat, but perhaps to overly invasive.
SMTP's issues are:
#1 Lack of strong identification for remote MTAs,
and users of the MTA.
#2 Lack of required transport encryption
#3 Access control is a long after thought, smtp auth is tacked on. strongly related to #1
#4 enables information gathering, dictionary user attacks etc.
#5 quality of service is poorly defined.
SMTP can be "fixed", but I think the author is correct, starting over is much better.
Mail format RFC 2822:
Hard to parse
Trace headers information is loosely defined, important information is often stuffed into comment fields
envalope headers are impossible to check for validity
very limited multi-language/encoding support (mime is a poor answer)
Potential solutions:
Make the mail format something easily machine readable, possibly XML. Require DNS support public keys for MX records, support a query protocol to verify mta and message path. Have MTA's sign the equivilant of trace headers. Have email encoding type support, Require authentication for relaying, no more POP/SMTP hacks.
Just a thought.
Please re-read my last two posts on the subject.
There is no such mailing list operator, because they don't send mail if it would cost them money.
-- this is not a
So, what, you thought forcing the sender to put money at risk was "step free"?
Compared to the problems of implementing a cash-based fee on internet mail,
standarizing mailing list subscriptions so they include automatic whitelisting would be a piece of cake.
(And if you're charging money, you'd better have that forgery problem licked too,
otherwise it won't just be mailing lists that get joe-jobbed.)
-- this is not a
I did read your mails, and I would say that losing mailling lists would be too high a cost for the new email system. saying that being charged is a "no confirmation" would the death of legitimate mailling lists and I think that would be a very sad thing.
dave
Nothing is lost, whitelisting mailing lists simply becomes an automatic part of the subscription process.
Only illegitimate mailing lists would be affected.
-- this is not a
So you are saying that the idiot who doesn't know how to spell "teeming" and who hasn't a clue about "boxen" may not be a total idiot?
That, in fact, he said "teaming" because he meant "teaming"?
Let me ask you, since you seem so up on conspiracy theory, do I really need a tinfoil hat, or can I get away with aluminum foil?
"I believe we need a trusted protocol. This might be as simple as having all emails PGP signed and everything else being sent to the bit-bucket (if you want to be aggressive) or only passed through to the user if the unsigned message had an extremely low spam score."
Ok, implement the protocol, software, system, and adminstrate it for free and I'm in.
But if you say "that's too expensive, I can't afford it", then what you are saying is that you should be allowed to act like a spammer and its the one receiving the mail that has to pay the costs.
If your mail isn't worth paying to send, then why are you sending it?
The marginal cost of paying for sending email should not be higher than the current marginal cost of receiving the email is today; the only difference is who is paying for the email. Actually, the marginal cost will be lower because there will be less traffic and that will reduce the cost of resources required to handle the email.
Note that if the marginal cost is $.001, the cost to someone sending one million spam messages to get 2 responses for $25 become unprofitable. Its unprofitable even at $.0001 per message.
The key is to offer multiple email classes which will allow priority mail to subsidize bulk mail. But with the bulk mail marked as such as it transits the system, user agents will be able to readily separate the bulk mail from the priority mail.
But in any case, decades of experience has shown that there isn't any chance of a public good being well utilized without a market to allocate it. Even communists have embrace markets to allocate resources. Yeah, the "free market" has its share of abuses, but when you dig deep you find that every free market failure is due to a known externality that is not applied because of political power. My current view is that political power is forcing the receiver of mail to pay the cost of spammers sending email "for the good of the community". And then they go a step further and basically say that the problem with spammers is because the victims of spam aren't spending enough (effort, dollars, whatever) to prevent spam from overwhelming the system. This ranks up there with the failures of communist command and control to produce an acceptable level of goods.
The challenge is to design a low cost system that allows for
OK, so Cisco and other big names are mentioned. Check how many of those names were big when they first contributed.
Ain't is in some dictionaries now too, but that doesn't make it a word.
I disagree. Unsolicited commercial email is the problem. I don't care how many other people received the same message, so bulk doesn't bother me. What bothers me is having commercials shoved in my face.
But it's only a problem because it happens in bulk. Non-commercial bulk email is just as much of a potential hassle as commercial email. I don't want religious fanatics, politicians or charities to get a free pass to spam me just because their bulk email is "non-commercial"...
On the other hand, some commercial email may actually be welcomed -- if you bought a piece of software, and registered it along with your email address, you might be happy to receive a notice that a new version of the software has been released. Of course, this might be considered "solicited" commercial email by some, but the point is that mass-mailings are the problem, moreso than any commercial intent of the email.
Even a cold-calling commercial email from a stranger might end up being welcomed if it was individualized. Suppose you posted online about how frustrated you were with a particular problem, and a company had a product/service which could solve your problem, and a representative crafted an individual message to you in direct response to your posted message, explaining what they offer and how it could help you. This is UCE, but not UBE, and you might even appreciate the information and choose to become a customer. Even if you don't desire a commercial solution to your problem, or feel the UCE is an intrusion, it's still not really a problem. How many personal, hand-crafted commercial emails do you really imagine you would get?
Deven
"Simple things should be simple, and complex things should be possible." - Alan Kay
Non-commercial bulk email is just as much of a potential hassle as commercial email. I don't want religious fanatics, politicians or charities to get a free pass to spam me just because their bulk email is "non-commercial"...
In theory, the potential may be just as bad. But in practice, I've only ever received one "religious spam", (probably because I have the word "agnostic" on my home page). It didn't bother me very much, because it was just one message. If I started getting as many religious spams as I do commercial spams, then I might revise my opinion.
Suppose you posted online about how frustrated you were with a particular problem, and a company had a product/service which could solve your problem, and a representative crafted an individual message to you in direct response to your posted message [...]
That's something of a grey area. Personally, I'd consider that "solicited", and I wouldn't be offended by the offer, as long as the message was done with a bit of taste and class. (Flashing red letters or scantily clad models would be a major turn-off in this case, unless I had been asking for "help" in meeting scantily clad models.)
I draw the line thus: if I actually ask for help in some area, then good-taste commercial offers are OK. If someone does a data mining operation and learns that I once worked with a product which resembles theirs, and then sends me commercials for their product, that's not OK. Other people may draw the line differently, which is what makes this an interesting field.
Well I read this post.... now u gotta keep your word...
You aren't bothered by "religious spam" because you don't get flooded with it constantly. So it's no big deal to you. You do get flooded with commercial spam, so you feel that the commercial nature is the problem. Actually, the problem is that you're getting flooded with the spam. The fact that it's all commercial is immaterial.
You don't want to be flooded with crap you're not interested in. Non-bulk email (even commercial) isn't likely to flood your mailbox, so there's no reason why it should annoy you any more than that single religious spam message did. That's why you should be concerned about UBE, not UCE.
If companies put telemarketers to work sending personalized, hand-crafted emails to individuals instead of calling you on the phone, then it might be time to worry about non-bulk commercial email, if it's piling up. However, that would raise the costs of such "spamming" enormously, eliminating a major motivation of spammers -- that it's virtually free.
Also, targeting UBE instead of UCE allows you to claim that your "censorship" is content-neutral, and therefore not infringing on any free speech rights a spammer thinks they might have. (Nevermind that "free speech" doesn't obligate you to accept email from spammers!)
Deven
"Simple things should be simple, and complex things should be possible." - Alan Kay
Here's what works for me: 1. Choose a good e-mail provider (e.g. yahoo.com) 2. Don't give out email address ever. 3. Don't participate in chain e-mails. 4. ??? 5. Profit.