Slashdot Mirror
IETF to Look at Spam
m00nun1t writes "CNET has an article about the Internet Engineering Task Force (IETF) looking at what they can do about spam. According to the article, many of the proposals seems to "require changes in basic e-mail technology", which presumably means SMTP (and about time!). Maybe they are looking beyond just SMTP - anyone have any insights here?"
If an alternative to SMTP were developed, the protocol would not be likely to disappear immediately subsequent to the creation of its successor. The transition would be gradual, as reverse-compatibility could remain necessary for several years afterward. As suggested by the release of Apache 2.0, for instance, not every server administrator adopts a "technological improvement" until it becomes an adequately proven and stable product.
Do you like German cars?
Maybe their address got used as a reply-to in the latest "$1000 per week from home" letter...
Have you seen my stapler?
Although I'd find it hard for the IETF to swallow djb's personality, there's always IM2000.
http://cr.yp.to/im2000.html
Replicants are like any other machine, they're either a benefit or a hazard. If they're a benefit, it's not my problem
Perhaps they are deleting "penis enlargement" spam.
Do you like German cars?
First Dupe!
This is another attempt to have technology police human behavior. It hasn't worked in the past, why should it work now? I agree that SMTP could be made a lot more secure for the commercial Internet
The article didn't say much beyond "gee whiz, there's a spam problem!" -- not exactly a revelation. It does hint at technical solutions to the spam problem, but I wonder if that's an approach that will ever work. I think at the heart this is a problem that can only be fixed with legislation (like it or not). Something along the lines of making it illegal to send spam with forged headers would help a lot.
Among many, many others, I saw Vernon Schryver, the guy behind Distributed Checksum Clearinghouse, on the list. It's been pretty high volume, though, and I haven't had a chance to really spend some time reading it yet.
Carousel is a lie!
Maybe you should change TCP, UDP, and IP while you're at it? Maybe we could switch every 6 months or so just to make sure developers are "keeping up."
For the sarcasm impaired that means... nevermind, you're perfect.
Take a look to Postfix Sender verification system. It can help little bit among other measures.
The really interesting thing about dupes is that they tend to suggest that there are large numbers of readers who pay more attention to the site than the guys running it.
If I was running slashdot, I'd probably push the people who had the power to approve stories to read each and every story that gets approved. It seems like a reasonable minimal committment to the community even for volunteers, and presumably some of these guys are drawing actual paychecks for the work they do here.
The dupes show that the guys approving the stories don't really care enough to take the time to do that.
Someone posted a response to another spam story a few weeks ago, sadly I can't find it, but they described an interesting mail delivery system they'd created.. and it sounded, to me, as if it could certainly be the future of mail delivery.
They said that when someone sent a mail, it simply went to the local server, and no further.
It sounded like a 'reverse IMAP' style system to me. That is, your outgoing mail simply went to a folder on your server, which allowed you to edit and even delete mails BEFORE they were picked up by the recipient. The recipient's e-mail server would only receive a 'notice' that someone had mail for them.
When the recipient went to collect their mail, their own mail server would then have a basic list of where the e-mails for the recipient are, and then it'd go ask for them from the remote servers and feed them through.
So, how does this help spam?
It allows spam to be truly filtered on the OUTGOING rather than the incoming!
Why's that a great thing? Well, it means that if you're an AOL or MSN user, you're not going to lose 80% of your mail simply because of over-zealous filtering by your ISP. Instead, spam mail will not even be sent, let alone received!
Of course, bad eggs could always set up servers with no filtering systems on them and send their spam that way.. but BECAUSE e-mail will be picked up FROM the senders server with this system, it means blacklisting is a whole lot easier! You just ban a server and you know you've got rid of the bad eggs.. whereas the current SMTP system allows open relays and all sorts of 'trickery' to get around filtering systems.
So.. the conclusion is.. make e-mail stay on the sender's server until it's time for it to be collected. It allows you to edit or delete mail before the recipient collects it, it stops spam, and it reduces bandwidth(!) -- if someone never collects their mail, then the mail has never gone across the net.. it's still on the sender's server.
I hope the original poster of this idea will pop up here again and correct me if I got his ideas wrong, but he was certainly on to something.
mogorific carpentry experiments
All the (legitmate ISP's in the world should come together and make a global blacklist of (rouge) isps. Along with distributed baylasien filtering, and reject emails with forged headers a different reply-to attribute (to stop joe jobs), should stop all most the most commited of spammers.
Linux users tend to get less spam because there are less security holes to "extract" your e-mail address, a lot of spammers deliberatley take advantage of microsofts flawed software selling "evidence erasers", "spam blockers" (ironicly), "pop up lockers", and "computer cleaners", of course these software packages are visual baisc applications which open up smtp relays on your computer.
Yet another dupe. Sigh. There have been way too many duplicate articles lately. Are /. editors not reading their own site?
Make the changes to what is wrong with the sytem that allows for things like Email parsing to occur, or random address creation, etc etc..
e a' option myself, but that doesn't fly in todays non eye-for-an-eye society.
Possibly, make those practices illegal, or more enforcable if they already are illegal.
I'm still one for the 'beat-spammers-severly-about-the-head-and-face-ar
a/s/l here. Sorry, adding domain tags to your s
The fundemental diference that protects most communication systems from Spam-like abuse is that the sender is responsible for a majority of the costs of the message. Yeah, there are telemarketers and junk postal mail, but the are seriously limited by the fact that there is a noticiable cost assosciated with each additional message they send. The fact that it costs money to send such communications makes it impractical to bother people with offers with an extremely low reponse rate.
SMTP/POP3 e-mail presently leaves the cost of holding the message during the wait for the intended reader to be available on the receive-side server. The spammer doesn't even have to maintain a constant and consistant Internet connection.
Under the current system, a sender can send 100 MB of messages in an hour without penalty. However, a receiver who gets 100 MB of messages in an hour usually will find any other messages sent to them bouncing.
Requiring the message be held on a sender-side server instead would transfer the costs of sending a large volume of e-mail onto the sender, and therefore discurages the practice better than any law ever could.
at last I will know to where in Nigeria I should go!
/solicited/ spam as being the only way you will be able to read salon. if it's still going by then.
seriously tho. even if it is all legal-ified and I'D correctly, there will still be such things as ticking the little box to say you dont want any spam from service X,Y, and Z.
In fact, the way online revenues are going i can see recieving
it would be nice(?) to have a better system but I never forget the age old adage of no system being tamperproof. Lots of enterprising folks enjoy anonymity for non-spam purposes, so naturally some form of workaround should emerge fairly quickly.
oh lord i'm sounding like Toffler.
<B>note to self:</B> <I>post as html</I>
Why don't we have a system deliver a kind of cookie for a permanent or reply only privileged contact? That would help a lot by ensuring no known important mail would get filtered.
Dan Bernstein has a project called Internet Mail 2000, ment to design a new Internet mail infrastructure which makes mail storage the sender's responsibility.
The adoption of another protocol is certainly a momentous proposition. However, considering the lethargic rate at which Apache 2.0 is being adopted, we can only hypothesize regarding a new email standard. Optimistically, I believe that it would require several years of dedicated effort on behalf of the Internet community.
Do you like German cars?
That statement is original and in no way related to a comment on another totally unrelated article. *grin*
-Lucas
Convincing "larger ISPs" to implement an alternative standard would also require prodigious effort.
Do you like German cars?
The technology exists, off the shelf, today.
There is a SMTP command called STARTTLS which will enable SMTP over SSL. It's defined in RFC 2487. Sendmail supports it with a compile-time option, and so do most other MTAs. It's backwards compatible with normal SMTP.
You will need a certificate, of course.
This has 2 big effects:
- encryption of email in transit between SMTP servers (a nice bonus)
- authentication of SMTP servers
Since sending spam isn't illegal in most jurisdictions, knowing WHO sent the spam (or relayed it) allows you to contact them and complain, threaten and retaliate (mailbomb, portscans, DDOS, etc.)
If you receive email from a host authenticated by versign (or whoever), you apply little filtering.
If you receive email from a host not using ssl, it goes into a queue for maximum filtering.
Much of the spam I receive today is from DSL customers who spew directly.
Downside:
- there will be additional CPU load for all the email servers
- cost of certificates
If any mail infrastructure reorganization were done before the finding this mount of this sendmail hole , that would have been be a good way to have a mostly forced deploy of compliant mail servers around the world.
Creating laws, regulations, and whatnot will come nowhere near solving the problems. Sure, if a spammer lives in the US then maybe this would work; but what about all these scams from Europe, Australia, Britain, etc. Just because laws exist in one jurisdication, it doesn't mean that others will play ball. And even having laws does nothing if they're not enforced. Why not have a group of IT police hunt down spammers? After all, they're already guilty of theft and fraud (think bandwidth people). Why not prosecute under existing laws and treat spammers like the theives they are. Even though you won't catch spammers outside your legal jurisdicition, you'll help. And every country that helps would quickly be eliminating the spam problem we live with.
The article notes that one of the major problems is the filtering of genuine mail due to agressive spam filters necessitated by cleverer spammers. Consider this analogous to dropping some packets at the network layer. Just as the transport layer handles this problem, we can build a higher level protocol to handle filtered mail.
Note that having a mechanism to handle dropped mail allows us to employ agressive filtering: one that is sure to stop 100% of spam.
What I have in mind is as follows: when Bob receives a mail from Alice (i.e, it has passed through Bob's filter) the client software sends a confirmation mail back to the Alice. This is not a regular mail that the Alice will see in her inbox; it has a special header flag that marks it as a confirmation. Alice's client software keeps track of the confirmation messages; by looking at her "sent-mail" folder she can see which of her messages have not been confirmed (and are hence likely to have been mistaken for spam).
Finding that Bob has filtered her mail, Alice can either re-word it and send it again or do something like (assuming that Bob knows Alice): "Hi Bob, this is me, Alice. Your filter blocked this so I've rot13'd it to get past the filter. rot13 what follows to read my mail." Another option is to encrypt the mail with Bob's public key (assuming that spammers' scripts won't be clever enough to get your public key from your web page). Note that 99% of the time the mail is going to get through. You have to make that little effort to prove you are a human only once in a long while.
There is minor problem with requiring the receiver to send a confirmation message: Bob might check his mail only after a couple of days, during which time Alice may assume that her mail was blocked. There are 2 solutions: either Bob runs a script to filter his mail regularly, or else has his ISP implement his filter for him.
Note that this won't work if you have the receiver send a reply whenever the message did get blocked: the reply could itself get blocked etc. (This is called the red army - blue army problem in networking).
World of Ends, recently discussed on Slashdot, discusses why the simplicity (or stupidity) of the Internet is so useful. "The Internet isn't a thing. It's an agreement," they say.
That same argument applies to e-mail. Following their logic, it is best to leave SMTP alone. Simpler protocols are better. Leave the "value-added" pieces to the edge, and let the simple message transfer protocol alone.
J'aime mieux les méchants que les imbéciles, parce qu'ils se reposent. -- Alexandre Dumas
I know - I don't write code - and this probably sounds stupid. But I don't really see any way of forcing someone to quit sending you email. SMTP is short and sweet - but it can't continue to just hop mail. It has to be checked somehow. And it would slow down the mass emailers a lot. Hopefully someone a lot smarter than I in this area can come up with something.
Duke
FreeBSD: Nothing runs like a daemon with a pitch fork.
Watch for any attempt to impose digital certificates or other revenue generating schemes--wouldn't Verisign love it if now not only those who wanted SSL to work without presenting dire warnings to customers but everyone who wanted to be able to send email at all had to pay Verisign's extortion money for a certificate recognized by MSIE.
CEE5210S The signal SIGHUP was received.
Yes, leaving the email on the sending server essentially makes the sender pay for outgoing emails, but how does it solve the spam problem?
If I recieve an email from Foo@bar.com, how do I know that it's not from one of my friends who are using the Bar cybercafe or from someone on a mailing list I'm on? Most spam filters rely on actually reading the contents of the email. It would be nice if I could tell the Bar.com server, "please filter out this spam using this filter" but it ain't going to happen and if it does there's no way I'm going to trust them to do it.
So basically, the only way to be sure it's not spam, is if I filter it in my own system.
email is by far the most widely used of all Internet services. I belong to an organization many of whose members are retirees are on fixed incomes, and it is only within the last two years that the number of people with email has grown to a critical mass (about 2/3 of the membership).
Of members of the lay public who regularly use email as a means of communication do not have the level of technical comfort that most Slashdot readers take for granted.
Of people who use email, the percentage who know how to use a web browser is much less than 100%. The percentage who can google for information is much less than 100%. The percentage who can successful extract and decode an email attachment is much less than 100%. The percentage who can view a government form or a corporate brochure in PDF format and read it with Acrobat is much less than 100%.
And the average age of their computers and operating systems is much more than three years--and they're not likely to update their email programs.
Whatever is done needs to be 100% backward compatible with existing email clients, not requiring even simple upgrades, or an astonishing proportion of real-world Net users will be disenfranchised.
(And please, let's not have any facile expressions of contempt for AOL users or webtv clients or people who bought email appliances (that includes one of the retirees I mentioned).
"How to Do Nothing," kids activities, back in print!
Same thing must be done in email business. All messages must have correct From field, and all must be signed with key identified by From field. All keys must be signed in a public (well known and trusted) CA. The router should relay only messages with verified signature keys.
No more anonymous email!
It is not a business of the router to watch the content of the message. But the header (let's think that the sig is the part of the header) must be checked. Specifically, the key of the sig must be verified in CA for being non revoked. Of course the revoke list can be cached on the router side for performance reasons.
Then it will be very easy to create a report gathering center (it could be a DB at the same CA), where you can send the fingerprints of bad guys. Once the key collects more than N (specified) bad reports, the key is revoked, and all messages with key won't be relayed anymore.
Of course, inside the interprise you can have "old-style" non-signed email. But in order to relay the message into outside of theintranet, it must be signed. That can be done automatically by the enterprise relay on behalf of corporate users. Even using the corporate key - remember NAT for IP?
Have this infrastructure on the place and it will often unnecessary to presecute spammers: their keys will be revoked after N bad reports.
By the way, how to send a bad report? Simple. Remember Y!Mail UI? They have a link in the message header area: "Report as a spam". Same button should be in every MUA UI: "Report the key a spammer's one". As well as another button: "Bleck messages with this key" - you still can do blocking (or any other filtering) of messages locally based on the key (ID).
Why a key is better than a return address? You must have a valid key. That's the key :)
Can you generate new keys with the same speed as you can generate your return addresses? I don't think so. And if you still can generate enough of key to keep your business profitable, then let's change your profit calculation forumal. From now on, the spammer has to pay big penalties for revoked keys.
Am I am asking too much?
Less is more !
All mail accounts should use whitelists enforced by the ISP.
The only way to get on a whitelist is to send an email:
Subject: Request to join whitelist ... ...(100 chars max)...
From:
Reason:
Any emails from people who are not on the whitelist must be of this format or they are automatically rejected at the ISP. Mail clients recognise request mails and notify the ISP of the user's choice.
Mail clients can also automate the process of requesting to be added: 'you haven't emailed ... before. Please say a few words to persuade them to accept your emails, e.g. introduce yourself.'
Unfortunatley the age-old problem crops up: this is only a good solution if everyone adopts it. (Then, for example, mailing lists can mail out requests to new members...)
Hi, arvindn, it's me, 1nv4d3r. Your spam filter blocked my mail, so rot-13 this to see what I sent you.
TEBJ LBHE CRAVF OL FVK VAPURF, VS LBH URYC ZR FZHTTYR ZBARL SEBZ AVTREVN.
(not to mention that return token let's them know that you at least look at your mail.)
Spam is highly redundant commercial advertisement. And we don't want it. So the basic approach would be to exploit this redundancy to filter from the original message streams. However highly localized approaches like personal mail filters will always fail due to the high variety of spam.
How would your personal mail filter know Japanese or Chinese spam ? Not at all, it would just let the spam mails through.
So we can only solve this problem by a worlwide highly sophisticed effort. And the key component will be again the identification of redundancy.
The idea is to build up redundancy signatures of email messages. The global network of SMTP servers exchange these signatures by a grid orientated P2P network. When the distribution of a special signature is worldwide too high then it's identificated as spam and destroyed everywhere [1]. Note that a distributed annealing algorithm can do this in O(log(log(n)))+theta(n) operations (theta(n) a blotzmann distribution due to the annealing process).
Building up these signatures is a little more complicated, something like MD5 checksum won't be sufficient. After some preparsing we have to project the text into a suitable banach space for further operations. Cleckerson and Allman have proposed an infinite dimensional Lie group for this task, surely any Diffeomorphism group over a Hardy space will do [2]. In this space we can build unique (!) singnatures using a simple frequency domain transformation. However we can't of course exchange an infinite amount of data, so an approximation will have to do it. Reseach any Valdpornik and Rosé has shown that there are decent approximations for this task [3]. So we can use these thing to build up our signatures and the whole stuff works indeed.
However the highly numerical approach requires DSP coprocessors in mail servers. But I think that Intel and AMD will throw in a DSP core to their server processors in a few years anyway, so that's not a big issue.
Owner of a Mensa membership card.
We've kind of taken that approach with our own MLM by storing the outgoing messages on our own server and delivering notifications via RSS. Subscribers can read the actual message by visiting our web server.
He uses the idea of emergent structure. To quote, " if all (or even most) players expect other players to act in a certain way, a predictable pattern of behavior emerges which becomes compelling for all players. This is the way all organizations work."
Replace "IETF" with "Microsoft" and you have thisslashdot story a whopping two weeks ago. Of course the slant then was how evil Microsoft was for daring to make people pay for email (which of course was not true... the article was about email accountability to reduce spam).
"It takes considerable knowledge just to realize the extent of your own ignorance." - Thomas Sowell
But you would have a valid email address of the
spammer. Which means it is much easier to shut
him down or sue him or whatever you think is
appropiate.
But there already exists something similar to
the idea the problem is that it isnt any
mail client that has somehting built-in.
I would like something like this:
1. You receive email. Mail client check if
this somebody you have in your addressbook
, you sent an email to him before or you accepted
an email from him before. If so pass it through
and happy ending. Other wise go to 2.
2. Mail client returns mail with some specific
format and ask for confirmation mail. It the
other client supports it can autogenerate a
confirmation. Client would only autogenerate
response if it knew it sent email for the
requested confirmation.
3. If you receive confirmation pass the mail
through.
I am sure this would reduce spam substantially
since they would have to have a valid email address. And all this without filtering.
When the protocols we all use now were developed, everybody trusted each other. There wasn't a real need for advanced security options. Nowadays, with the current commercialization of the net (which also provides me with my income) it looks as if the commercials are winning. By commercials I mean those who have absolutely no respect for other peoples right or bandwith. Let's not forget that spam isn't the only problem: dos attacks are a real threat too.
Due to the original designs being not real secure, I'm quite sure that the spam problem can not be solved without fundamental changes in the way we use email nowadays. Perhaps the policy regarding blacklisting can be changed: at this moment most people accept mail from everybody, but not from a few blacklisted sites. It's likely that this will be changed: we don't accept your mail unless we know who you are. Unfortunately, even then there will always be people who will abuse it. Hopping from one account to another, or sue-ing every single ISP that has the guts to disconnect their connection after spamming. In short: it's not simply a technical matter, their will be a need of *globally equal* legislation too. Legislation alone won't do the trick either. No, it's time for Mr Geek to marry Miss LawAndOrder.
Don't forget that the IETF is not the first to attempt to find a solution. RIPE has its anti-spam workgroup for example.
I'm not a complete idiot... Some parts are missing.
Why can't the current SMTP servers keep a list of trusted servers, and anything that is not trusted it severly limits the delivery rate (maybe 1 per second (minute?) and then increase the rate).
It wouldn't eliminate SPAM, but it would slow down its propagation.
What, me worry?
IETF to Look at Spam
I've been looking at spam for years. In fact, I can honestly say that I look at it more every year. Hasn't helped at all.
Come on, IETF, try something we haven't done yet! Like get congress to declare Spam an act of Terror.
Maybe with the knowledge that filtering is imporving and new protocols, proggies etc are on the way spammers will just stop spamming.. LOL
So in a nutshell you could handle it this way (i didn't think of this my self i read the story and links and other messages)...
Mails are not auto delivered they sit on the server so any fake emails looking like they came from someone else will not get delivered.
So.. Fag Spammer sends mail looking like it came from cowboyneal@slashdot.org and the client sees (automatically) that there is email to pick up from cowboynea@slashdot.org and then requests the mail, cowboyneal's postoffice responds no such message and then deletes the notification... boom spam stopped. At this point in time this would stop 90% of spam dont' you think? Its such a simple solution.. a big undertaking maybe but simple.
And this would cause email to come from real addresses only or atleast real servers. But then you have the same problem... blacklists, you'd have to either setup your own blacklist or *GASP* assume someone elses blacklist aint gonna fuck up your mail. On the other hand you know the real server so its easier to talk to the datacenter/isp the spammer is using and have their box shut down before that spam is picked up and since it was never delivered the whole time.. just ready for pickup i guess a lot of spam is still stopped.
Someone's server connects to yours and sends the MAIL TO and RCPT FROM commands. Your server checks this info, along with the originating IP address), against a list of pre-authenticated sites (your employer, your family, etc) to decide if it will accept the DATA command. If so, you get the mail; otherwise the connection is severed. The sender should then start retrying deliveries intermittently over the next few days.
Meanwhile, you get a message from your server telling you that mail is available from so-and-so addressed to such-and-such. You use a canned reply: DENY, to keep rejecting the mail until the server gives us, ACCEPT ONCE, to accept the next message with that description, or ACCEPT FOREVER, to add the description to your permanent acceptance list.
Note that "important" stuff gets fully delivered immediately, so you can check your email offline later, while potential spam is held at the sender's expense, not yours.
Also note a simple variation on this idea: your SMTP sever could decide to accept the DATA commnand, but then read the mail headers, evaluate them, and then decide whether to break the connection prior to accepting the body of the message. This would give you enough data to perform stochastic filtering on the message. (I've seen reports that indicate that the headers are the most useful data when applying spam filters.)
Nothing for 6-digit uids?
Actually, it's the IRTF -- not the IETF -- that is undertaking this work. To quote from the IRTF home page - "[Mission] To promote research of importance to the evolution of the future Internet by creating focused, long-term and small Research Groups working on topics related to Internet protocols, applications, architecture and technology."
Don't expect a quick fix from this initiative.
See for example hashcash and camram
Test 1 2 3 4
2. Mail client returns mail with some specific
format and ask for confirmation mail. It the
other client supports it can autogenerate a
confirmation. Client would only autogenerate
response if it knew it sent email for the
requested confirmation.
As an added benefit, it would also send a huge amount of extra traffic to spoofed aol.com return email addresses. A new DOS attack is born every day!
This sounds better than the first post, where Alice tries to send mail again manually. In this setup, it sounds like Alice's mail gets through after the confirmation step.
Couldn't this be built in to the internet mail servers? They could always do this step, and stop forwarding mail that the return addresses don't think they sent?
I thought about it one day and just rambled on on my weblog...
D =1 046736931
http://firsttube.com/weblog/viewcomments.php?cI
--
Adam
From what I take from all this discussion is that the only "solution" to spam is to do the types of things that we have been doing for years, but to do more of it and quicker. Use well run DNS blacklists (Spamhaus SBL, ordb, dsbl, etc.), use good content filters (bayesian filters, etc.), use bulk mail detectors such as DCC or vipul's razor, etc.) and per-user whitelists and blacklists.
Or, combine all of the above techniques by using SpamAssassin
--
I've been subscribed to the list since near the beginning and have been following it fairly closely. Much of the discussion has been rehashes of old topics such as "what exactly is spam?", "make the sender pay something, either money or CPU", etc.
The most interesting discussions that I've seen so far are:
Most spam specific programs will not queue and retry, and thus the spam will be dropped.
Spammers that use real mail transfer programs or open relays will need to be able to hold all their outgoing spam for a while, increasing the spammer's costs and slowing down the delivery of spam. Legitimate email will not be thrown out, it will only be delayed and only for the first time.
Of course, you don't really want the databases to remember every sender-recipient pair forever, nor do you want to remember pairs that were added by spam so this really isn't a "first time" database, but it is close.
Apparently the "canit" program already does this, but I had not heard of this technique before.
If you filter during the email receive process, you can make the sending MTA do the bounce. This means that you will not have to deal with spammers forging "from" and "reply-to" headers. You won't have to clean up bounces that never succeed, nor will you be responsible for bouncing spam to another victim that the spammer selected for the "from" or "reply-to" headers.
Also, false positives will recieve a bounce message instead of just disappearing. This reduces the danger of important email being lost.
Right now, there are DNS records that tell you which IP addresses are valid to try and send email to for a given domain (the MX records), but many ISPs have different machines for sending and recieving email. There are currently no DNS records to tell you which tell you which IP addresses a domain will send email from.
The problem with this kind of proposal is that there are many people who think they have legitimate reasons to forge "from" or "reply-to" addresses. It also forces ISPs to make sure that every time they add a new outgoing mail server, they need to update the list of valid IP addresses. If they forget to do this, then only bleeding edge spam filters will detect a problem.
SPF support for most open source mail servers can be found at libspf2.
> The server on which the the e-mail is stored
> according to the notice is info enough.
Is it?
> If you trust that server, you'll accept that
> they have already done the spam filtering and
> canceling for you. If you do not trust that
> server, you simply ignore all notices from that
> server.
Some of my friends use hotmail.com and yahoo.com. Filtering based on servers isn't enough. I'm on some mailing lists and I often recieve legitimate email from unknown servers.
This solution doesn't stop spam. We can already do the "I don't accept emails from this server" today without a special protocol. We can already "accept email from only a few people on your list". Several ISPs allow you to set your own spam blocking on the ISP server so you don't see spam.
Until the ultimate fix for spam is created why can't mail servers just limit the amount of mail that is sent by any user?
Is it possible to setup sendmail and such to limit the number of emails sent to a quantity over time? If there was a limit of say 1 email every 5 minutes for generic users (obvoiusly, some algorithym that would allow for small bursts would make sense), that would effectively stop someone from spamming from a regular account. If a person wants to setup a mass mailiing account they will have to pay for it.
If this was implemented then even the hijacking of a server would be pointless as the server would normally run in restricted mode. If spammers were to hack or create a false unrestricted account then they would be opening themselves up for far more than just spamming violations, creditcard fraud would make spamming a much larger risk.
Why do so many people keep looking for the next answer before they use the tools they already have?
...allow binary transfers. I can't tell you how much CPU time has been wasted by base64 encoding binaries, sending them over an inefficient protocol, and decoding them on the other end. yEnc does a good job but the whole encoding shenanigan is a major pain for anyone trying to send family photos or the latest AFI album. Please, IETF, make a better 8-bit clean push protocol, because SMTP is the only one we have.
"The lesson to be learned is not to take the comments on slashdot too literally." --Vinnie Falco, BearShare
I was the original poster. You got it basically right in your synopsis too.
To recount, my idea is for when someone sends a mail message, the message itself goes onto the local mail server and the header for the message would go to the recipient. The recipient's mail client would then download the message. However, it would be possible for the mail server to delete the message _before_ the mail client ever sees it in which case, the mail server tells the client this and the client would then throw away the header and the end user _would_never_even_know_ that the mail (spam) had been sent. This would be totally transparent. It would also allow of course, for sender of the mail to be able to tell if / when a mail message had been picked up. (Not read but simply picked up.)
One of the big advantages of a system such as this is that you know for 100% certainty where the spam (or other e-mail) is coming from. You don't have to spend time looking through forged headers etc. in order to send a complaint to the ISP.
ISPs on the other hand would be capable of canceling spam after it had been sent and before it was picked up by the end user. For example, someone send's 100,000 spams from an AOL account. Somebody notifies AOL that they received spam from the offending person, AOL looks and AOL then is capable of cancelling all unpicked up spams -- before they are ever delivered to the end user. Alternatively, AOL could also simply look on their servers and say: "hey, we have 100,000 messages that are waiting to be picked up, we had better look into this" and then make a determination from that point as to whether the mail should be canceled -- again before anybody (or very few) sees the spam.
Blacklists could easily be created too where the site is blacklisted for only a certain period of time. So after three days (for example) the blacklisting would go away automatically. This avoids the problem that many ISPs have where they get blacklisted due to a single user and then they can't figure out how to get off the blacklist. Using this approach, the blacklisting would only last for as long as the spambarrage continued.
Blacklists would easily be able to be created within certain organizations or groups of people who have similar "moderating" views rather than trying to make one (or very few) blacklist(s) meet everybody's needs as is now the case -- and often hurting people's ability to send and receive legitimate mail.
The protocol could not only specify what server the mail came from but also the user. So, for example, if someone were spamming from AOL, it might not be a great idea to blacklist AOL but only that user from AOL. This would work for mail systems where you know it is a legitimate business but with a few unruly users.
So, using this technique, it would be possible for a spammer to get a few spams out but it would be nearly impossible for them to spam very many people before it was caught by their ISP and canceled or the user/ISP was blacklisted for a period of time.
-Art
Pollarda@Lextek.com
Lextek: Suppliers of High Performance Text Retrieval Engines
... and neither does any of the other millions of friends and family members out there who don't give a rats ass about spam.
Where would the money go?
How would I pay for a "stamp?"
Which company/government body is going to manage my account for "stamps?"
I get just as much junk mail in my physical P.O. Box as I do in my e-mail box. Cost to send is NO deterrant.
This is not a solution.
Yes, using a tempfail for first time email is not a perfect solution, but it does help. In particular, making an open relay queue lots of spam will make the open relay's problem much more noticable to them.
Worse, the spammers who send their messages multiple times will still get through.
You shouldn't really tempfail the "first" time you see a sender-recipient pair, you should consider any pair that you have seen "recently" (weeks? months?) to be effectively a "first time". You should also tempfail any emails from a "new" pair for the first hour or two.
This will give you an hour or two warning about potential spam, giving spamtraps and blacklists a chance to kick in.
Remember, any email from a working mail server will still get through, it is just the first time there will be a slight delay.
SPF support for most open source mail servers can be found at libspf2.
> IP addresses can be forged fairly trivially
Why would such a wild claim get moderated upward? UDP can be spoofed, but TCP's three-way handshake is hard to spoof if the OS of the machine you're connecting to is smart when picking sequence numbers. We're talking about e-mail, which uses TCP. You could spoof TCP connections with some difficulty when many OS's used predictable sequence numbers, but that's a problem that has been fixed for years. Hello moderators?
And make the default text encoding UTF-8. The Linux Standards Base will soon make UTF-8 the default system encoding for Linux (well, the major ones that follow the LSB). Win & Mac are now Unicode-based. XML is UTF-8 by default.
Having the full character set in use by the OS (Unicode/ISO 10646) includable in a default email with no corruption or additional processing would certainly be a great feature.
This would allow even English speakers to have an enormously increased range of characters they can include in their text: math symbols, musical symbols, the full text of a Time Magazine article (professional publishers go way beyond Latin-1 in the characters they include in an article, and an email version these days usually corrupts the original to some extent), as well as mixtures of any other languages of interest.
XML in default form (UTF-8) could be included as simple text without corruption and without requiring that it be converted to an attachment.
If it's the default, UTF-8 will be quickly adopted by modern (non-abandoned) email clients and servers, meaning that they can all talk to each other in any language without reconfiguration.
I think this would be a major win.
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
You might get as much junk mail if the sender pays but you know what? You won't get the same quality of spam. You will get ads from your local grocery store instead of "MAKE MONEY FAST!!!!!!!" and "Add 12 inches to your penis!!"
Spam largely consists of cheap bullshit products, scams, porn, and crap that appeals to the insecurities of stupid people. Do you know why this is?
It's because the individuals who "run" the "businesses" in question are getting something for nothing. In other words, they're all just running scams. If you were to charge $0.05 for every e-mail sent, then it would become a cheap way to advertise, rather than a *free* way to advertise. Suddenly, sending out 5 million e-mails costs you $250,000 instead of $250 (if you include your own labour). For the paltry 0.01% (or less... I don't know what kind of numbers spammers get) response rate, suddenly it becomes a big losing game for everyone that's selling Viagra and porn and pyramid scams. That advertising would be replaced by people selling cars and groceries and books, and while they may still be capitalist scum, the products they're hawking are orders of magnitude less offensive than the crap that's currently being hawked by e-mail.
$0.05 per e-mail isn't going to break the bank for us regular joes - a couple hundred e-mails a month comes out to all of $10.00. Don't forget that it actually costs money to run an e-mail server, and it would be the people running that server that would be collecting the postage. It might even be a good idea to hand the whole system over to your friendly neighbourhood national post office.
Personally, I think this would be a small price to pay for vastly improving the quality (not to mention stemming the quantity) of the advertising in my mailbox. I'll also be happy with the fact that the people doing the advertising would be paying to support the e-mail system instead of working hard to bring about its collapse. As far as I can tell, this is a simple, elegant solution.
"No problem. I have the capacity to do infinite work so long as you don't mind that my quality approaches zero."-Dilbert
I've been using email on the net for 20+ years and have been as grateful as anyone that it hasn't cost.
However , of late, with other people making such gross abuse of the world's mail systems that I feel I am (and all of us are) paying anyway, it might be worth revisiting this question. I'm not 100% convinced that paying per piece of mail sent would be more expensive than, effectively, paying (in both time and in dollars spent on ISP infrastructure) per piece of mail received. I send a lot less mail than I receive, and I bet most people do.
Kent M Pitman
Philosopher, Technologist, Writer
Incase anyone's wondering, yeah, Art Pollard was the original poster, the name rings a bell now :-) Thanks Art!
The original post is still available to read.
mogorific carpentry experiments
... and just ban all HTML email. Bouncing it with a notice that says "Hey, I don't accept HTML email because of all the viruses and SPAM that spread that way... write me back again in Text." You could also put instructions for Outlook Express and other major email programs on how to switch it over.
It only leads to more problems. Email should never cost per email. I can only imagine the fraudulent behavior that would follow.
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
How many did I miss?
$x='S24;r)>63/* h@<5+oZ)32"5cz';$me='phroggy'x$];
$x=~y+ -xz+\0-Tx+;print$_^chop$me for split'',$x;
Hope this helps!
If you change your mind about what you sent, you can yank your mail before it is read. AOL already does this between members command "Unsend Mail"
So Bernstein isn't the most socially adept in the world, but his ideas are brilliant. Is anyone seriously considering implementing this?
:) )
It seems as though this could stop the spam problem overnight. (Or, at least over as many nights as it takes to get people to switch.
Hotmail.com or Yahoo.com in the headers of an email doesn't mean it came anywhere near those servers. I've gotten more than several spams that were apparently sent through open relays in China, Russia, or almost any other country, with hotmail.com or yahoo.com in the "From" field, but none of them actually touched either company's servers. Forging email is still surprisingly easy, if you can find a badly configured email server. How easy is it to do that? Just check some of the spam mail you get.
I finally went to popfile.sf.net, and though I still download the junk email, at least I can filter it out much easier. Other interesting spam filters include Mozilla(1.3beta+), and CRM114, though I haven't tried that.
None of this filtering is a real solution to the spam problem, and I think it will take something like the IETF or some standards org to get it right.
I've seen many spams that basically go to "every word in dictionary" at each ISP. I've seen a few that obviously tried "every 3-letter combination" as a login name (26 cubed=17576). It took spammers about a month to find a test 3-letter loginname on a medium-sized ISP, though I don't get many on it yet. There's no real need for any spammer to steal your email address from your own computer, unless your address is not found in the spammer's dictionary.
Currently, 73% of my email (by number of messages, not size) is spam. I did have my ISP turn off their "spam filter", because it tended to junk the wrong email (some relatives couldn't send me email), but still, 73% is a bit much.
Spammers use their own desktop mail programs, usually bypassing the ISP. These programs are specifically written to maximize output speed. Therefore, changing the ISP's SMTP server will have absolutely no effect on the spam transmitted thru the ISP.
Given that opensourced MTA's like Sendmail, Postfix and exim dominate the landscape, we would all get a *quick* relief from spam if the three MTA's chose a throttling/prioritisation mechanism based on hashcash and/or authenticated senders for relaying mail, enabled it as the default installation and backport it to be released with the next bug-fix.
Initially, this would not cause the users any ill-effects as all users would be equally disadvantaged. However, as the smarter users wisen up, they'd add hashcash headers to get priority mail sent, and this would eventually cause spammers to lose out.
The other name? Lotus Notes.
Unfortunately, not open source. However the server does run on Linux, and mail can be read with a web browser. Also, authentication AFAIK is only between Notes servers.
Sadly, this summary is still pretty complete.
As other people have pointed out, this is not an IETF working group - it's an IRTF research group, and as such its purpose is to try to understand the problem and the solution space rather than to propose a concrete solution for standardization. Many of the folks in the group don't seem to have figured this out yet either. As a result there have been a number of naive proposals for how to solve the problem, along with a huge number of arguments of the form "your proposal is broken in X way".
But the group just got started a few days ago, and is only barely starting to move in any particular direction. Trying to make any predictions about the group's eventual output from what has been discussed so far is probably not a useful exercise.
In the system you propose, I have potentially tons of mail on my system because someone else isn't checking their mail often enough. Does this mean that I just send a message to the sender saying that their email to "so-and-so" is too old and is being deleted?
Currently, I can send a message to someone, see it delivered (with a return receipt) and if they have room for it there, it will sit indefinitely until they check their mail. With your proposal a "busy" mail server may have very little room for the amount of mail storage necessary for sending due to the lag between sending the "header" and mail being checked. I suppose implementing a "send quota" might be an idea, but, again you are at the mercy of people who aren't checking their mail often enough - which may mean every few minutes if you have a small quota and send a lot of (or large) mail messages.
Also, what do you do about people who see the subject line, say "OH, I know what that's about" and delete it without ever retrieving the message? How long does the sending server sit on a dead message that will "never" be picked up?
Send quotas may be a way to stop spammers, but if I have, say a 10MB sent-mail limit and I have several emails with 2 or 3 MB attachments waiting to be picked up filling up my out-box, what can I do? Call them on the phone and say - "hey, pick up yer damn mail"? Kinda defeats the purpose of email.
I don't see any fool-proof way of avoiding spam. If I know your email address and want to send you something, I can do it. And as long as there are "free" email services where I can temporarily set up shop, spam 'till I drop and then move to another account without repercussion, you can't stop me short of running a manual whitelist-only system where only mail from people you're expecting mail from, and have listed as legit, will be accepted.
I administer a mail system that has several thousand users and processes a tremendous amount of mail daily. I have implemented spam control measures including blacklisting and heuristic scanning at the server level. dealing with spam takes an inordinate amount of my time, bandwidth and computing power. I'd love to blacklist every korean ip out there except that we have users who correspond with people in that part of the world.
I sure don't have any answers. I hate the thought of trying to legislate anything, because you can't. Korea won't give a crap if the USA says spam's illegal. What are we going to do?
Perhaps if there were a method of autogenerating server-wide mail rules where every message that arrived was tagged by the receiving server with a "Do you wish to block mail from this person?" link in it it might help. But then again spammers just use randomly generated addresses anyway. I don't have the answer, but I don't think this solution is it either. Interesting concept though.
"terrorism" and "pedophilia" are the root passwords to the Constitution
Anyone looked at the DJ Bernstein proposal for Internet Mail 2000?
If "we" (the internet community, for lack of a better phrase) cannot stop idiots from putting open relays on the net, there's no way we're going to persuade every mail server admin to actually implement rate limiting.
Build stuff. Stuff that walks, stuff that rolls, whatever.
Your message summarizes the ASRG mailing list quite nicely. Too nicely, actually. On ASRG, most responses aren't as polite as yours.
Build stuff. Stuff that walks, stuff that rolls, whatever.
I once argued that fighting spam is firstly a contract law problem, because we need to allow our computer systems to automatically enter into ad-hoc contracts in order to eliminiate spam. Once we can do that, the technically cards all fall into place.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
"Has anyone had problems with the computer accounts?" ..."
"Yes, I don't have one."
"Okay, you can send mail to one of the tutors
-- E. D'Azevedo, Computer Science 372
- this post brought to you by the Automated Last Post Generator...