By Latest Count, 95% of Email Is Spam

An anonymous reader writes "The European Network and Information Security Agency released its new spam report, which looks at spam budgets, the impact of spam and spam management. Less than 5% of all email traffic is delivered to mailboxes. This means the main bulk of mails, 95%, is spam. This is a very minor change, from 6%, in earlier ENISA reports. Over 25% of respondents had spam accounting for more than 10% of help desk calls. The survey targeted email service providers of different types and sizes, and received replies from 100 respondents from 30 different countries."

Comments

Logic?

[clang_jangle]

Less than 5% of all email traffic is delivered to mailboxes. This means the main bulk of mails, 95%, is spam.

I don't doubt that it's around 95%, but the logic of the above-quoted statement is certainly flawed.

Re:Logic?

[sdnoob]

your internet provider or mail server administrator is likely blocking more (a LOT more) spam than you see come through to your "spam folder".

95% spam is a reasonable estimate for a report coming out of the EU, i think; and is pretty close to what i see here in the US (about 9 of every 10 inbound messages to our domains is either blocked at time of delivery or filtered later on).

Re:Logic?

[Anonymous Coward]

Less than 5% of all email traffic is delivered to mailboxes. This means the main bulk of mails, 95%, is spam.

I don't doubt that it's around 95%, but the logic of the above-quoted statement is certainly flawed.

Link to full report:
http://www.enisa.europa.eu/act/res/other-areas/anti-spam-measures/studies/spam-survey/at_download/fullReport

There also appears to be selection bias in the sample. The confidence interval is also missing. Survey only took place in Europe and apparently one company in the US.

In short, this is a waste of someone's money.

Re:Logic?

[Anonymous Coward]

They'd block almost all of it if they'd just shut down internet access to infected Windows boxes.

Re:Logic?

[characterZer0]

Right. They are ignoring the huge volume of legitimate mail that hotmail/msn silently deletes in violation of the RFCs.

Re:Logic?

[Xeriar]

Right. They are ignoring the huge volume of legitimate mail that hotmail/msn silently deletes in violation of the RFCs.

Hotmail doesn't represent the majority of e-mail accounts, and usually it seems to be down solely to the incompetence of whoever is administering hotmail, rather than intentionally violating RFC. Same difference, I suppose, but it's certainly not a majority of the legitimate e-mail they get to them, anyway.

Re:Logic?

[Teun]

Survey only took place in Europe and apparently one company in the US.

In short, this is a waste of someone's money.

Only, huh?

27 nations and a population of >500 million forming the largest economic block in the world...

Re:Logic?

[drissel]

Daniel J Bernstein has a fix for spam - his Internet Mail 2000. You can see it at:

http://cr.yp.to/im2000.html

Regards,
    Bill Drissel

Re:Logic?

[stonedcat]

They'd block almost all of it if they'd just shut down internet access to Windows boxes.

Fixed

Re:Logic?

[Anonymous Coward]

This morning, over 6000 messages came through my domain of which 6 were legitimate. I spend more time waiting for the deletes to complete than reading my messages.
I think the 95% may be too low.
And if you account for the bits, the ratio is even worse, since my legitimate messages are usually a bit of text, whereas almost all the spam includes an attachment or image.

Re:Logic?

[fredklein]

... or if they use Email Certification.

Long story short, everyone who wants to send Certified mail has to be 'certified' by their ISP. (UN-certified mail would still be possible, if you wish.) Getting certified is nothing more than providing enough information to positively identify you, and costs a nominal fee.

In return, you create a public/private key pair, and give the public one to the certifier. The private key goes into your email server, which adds some headers to each outgoing email. One of these is encrypted with the private key. When someone with a certification-compliant email program receives a certified email, the program reads the headers, connects to the certifer's certification server, and downloads the public key. It then uses the public key to decrypt the encrypted header. If successful, it proves that email came from the specified server, and no one else.

If you get spam, your email client has a big 'report certified spam' button. Click it, and an email is auto-launched to the certifier of the sender. The certifier contacts the sender and demands an explanation. If sender was hacked, they fix the security hole and tell certifier they did so. If spam was not spam, or a misunderstanding, they explain.

If, OTOH, the sender does not reply, then the certifier revokes their certification, and from that moment on, all their (the 'sender's) emails are UN-certified.

What if a Certifier themselves is 'evil'? Well, it's certainly possible to have blacklists like they do now, but, instead of blacklisting IP addressed, which get re-assigned and cause trouble for their new owners, it would be evil Certifiers that get listed and blocked.

Eventually, it'll reach a point where any spam that is sent out will get the sender 'de-certified' almost immediately. That means everyone else probably never ends up seeing the spam at all (depending on how their clients handle un-certified emails. Most people will probably auto-trash them.)

However, white lists are still possible. If you like getting emails from a certain un-certified sources, just white-list them, and you'll continue to get them. You can also use challenge-response or keyword set-ups for people sending you un-certified email.

TL;DR:
By proving who send the email (or, more precisely, which server did), Email Certification can hold the server owner responsible. If they send spam, they get de-certified, which means in all likely hood, they lose the ability to email anyone at all. Spammers who can't get certified can't send emails anyone will see.

Re:Logic?

[Idaho]

Worse, a change from 6% to 5% "real mail", if that is indeed the case, isn't a "very minor change", it's a 20% difference!

Re:Logic?

[MROD]

If you check out the statistics I've been collecting at work then you'll see the figure is quite correct.

You should be able to see the stats here:

http://www.earth.ox.ac.uk/~steve/spamstats/

Re:Logic?

[MichaelSmith]

If you get spam, your email client has a big 'report certified spam' button. Click it, and an email is auto-launched to the certifier of the sender. The certifier contacts the sender and demands an explanation. If sender was hacked, they fix the security hole and tell certifier they did so. If spam was not spam, or a misunderstanding, they explain.

Why not just have the client reject mail from that sender and cut out all the dicking around?

Re:Logic?

[cenc]

yea, all those ISP's in China I am sure would be happy to issue certificates.

Re:Logic?

[fredklein]

That's certainly possible. But it only helps You, not everyone else. Reporting the spam to the certifier means the certificate will get pulled, meaning NO ONE gets that spam. And that's what will result in less/no spam for everyone.

Re:Logic?

[fredklein]

Cool. If any of the people they certify send spam, then people will complain to them. If they refuse to stop it, they get their certificates blacklisted, just like there are IP blacklists now.

Once all the Chinese ISP are blacklisted....

Re:Logic?

[MichaelSmith]

But since anybody can run the same client the outcome is the same, and you don't have to deal with issues of trust, and spamming in the "certificate pulling channel".

All my mail is signed with my certificate. My mail client can filter based on certificate. Anybody (you or me) can run a certification authority.

Re:Logic?

[fredklein]

::sigh::

(X) Users of email will not put up with it

There is nothing to 'put up with'.

(X) Requires immediate total cooperation from everybody at once

Wrong.

(X) Anyone could anonymously destroy anyone else's career or business

How?

(X) Lack of centrally controlling authority for email,

No need for one.

(X) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical

No one has tried it before, that's true.

(X) Countermeasures must work if phased in gradually
(X) Sending email should be free

Email Certification meets both these conditions: Users who have certification-complaint email programs would see the benefits, but non-certification-complaint clients would still work perfectly well.
Sending email would still be free. It's just the Certification (only needed for Servers, not users) that would cost a nominal amount.

(X) Sorry dude, but I don't think it would work.

I think it would. If you disagree, why not post why, instead of an annoying and inaccurate 'form letter'?

Re:Logic?

[Hurricane78]

Well, I use a greylisting system, with amavisd behind it, and the greylisting blocks 90%, before it even reaches the spam filter. (Which also keeps the resource usage down.)
Then spamd and the other spam systems linked into in amavisd throw out nearly all the rest of the mails.
I’d say 95% is a vast understatement. More like 99.5%.

Also, everything that is filtered by amavisd, still goes to the junk folder of my IMAP account, so I still can undo false positives.

Works pretty sweet for my own server.
I simply can’t say, that spam is a problem for me anymore.

Re:Logic?

[JSlope]

I think there should be used a technical solution to not allow somebody to use someone else's e-mail address.

might be a good thing

[symes]

If volume is increasing then this might mean returns are getting scarce for these parasites. and perhaps it will come to a point where no matter how much spam they deliver they still don't make enough. but then maybe i am dreaming.

Re:might be a good thing

[Joce640k]

Doesn't matter. There's no shortage of people who believe spamming will make them rich. Spam isn't going to go away just because it doesn't work.

Re:might be a good thing

[Rexdude]

No- it's the fact that there are people WILLING to pay for what's advertised through spam that it has proliferated so much.
If you can get even a handful of sales from sending out a million spams, you still make a profit.
Imagine if doofuses everywhere didn't send money to Nigerian princes promising them wealth, or ignored the viagra/cialis ads that keep appearing.

If there was a way to shutdown the payment gateways for spammers their means of making money would be forced to stop and discourage them from continuing.(maybe VISA/Mastercard shouldn't allow just anyone to act as a payment gateway for them)

More than 90% for me too

[Arancaytar]

I also get about 10 times as much spam as actual email. Fortunately, Google is pretty good at filtering that - the number of false negatives in my inbox has been less than ten this month, while I got over a thousand to my spam folder.

It's hard to comprehend how people deal without that level of spam filtering - I have relatives who regularly register new accounts in order to escape their spam.

Re:More than 90% for me too

[derGoldstein]

"better that ten guilty persons escape than that one innocent suffer" - William Blackstone

False positives are the bigger problem. The amount of spam drives the need to "profile" incoming content with greater scrutiny, leading more and more real E-mail to the spam folder. Some of that E-mail might be important. Whitelisting is only a partial solution -- you don't always know where an important E-mail is going to come from. I've had more than one occasion where I've missed out on an opportunity, and a couple of them were job offers. When you find something like that in the spam folder (which, like you said, is massive), it's very disheartening. I'll usually skim through the spam folder weekly, going over what ended up there over the past week -- it's usually a waste of time, but every now and again I'll see something that doesn't belong there.

Re:More than 90% for me too

[Blowit]

Want to reduce false positives, and your friends, colleagues and email partners to ask their provider to support either Domainkeys or SPF. Once they get on the bandwagon, their mail will no longer get false positive flagged.

Re:More than 90% for me too

[kandela]

Yeah, I know what you mean. Just last week I missed out on the opportunity to make a living just from surfing the web from my home computer! I can't tell you how disappointed I was that the email offering that 'chance of a lifetime' went to my spam folder.

Then there was the time I won a million dollars but because of my spam filter I never got to claim it in time. Or the time that the Prince of Nigeria sent a desperate email to me for help, but because of spam filtering I was never able to offer my assistance. I feel just terrible knowing that he was never able to access his fortune or reclaim his rightful seat on the throne.

Re:More than 90% for me too

[Antiocheian]

Have you ever heard of SpamAssassin, statistical probability or heuristics ?

You really think that you'd have to surrender your email to Google to fight spam ?

Re:More than 90% for me too

[allo]

or use a ham keyword.

Re:More than 90% for me too

[TheRaven64]

SPF doesn't say anything about whether mail is spam - most spam comes from domains with valid SPF records - it just says that the sender is not forged. Email from a domain with SPF is no less likely to be marked as spam than email from a domain without, however email pretending to be from a domain with SPF can be identified and bounced as forged. The main advantage of setting up SPF is that, when a spammer fakes your address as the sender, mail servers that aren't configured by monkeys (i.e. not gmail) will not then bounce thousands of spam messages at you.

Re:More than 90% for me too

[mcubed]

For the past few months, I've been volunteering at a transitional housing shelter, providing basic computer assistance to anyone who needs it. The guys at the shelter range in education level and in their experience with computers and the internet. Most have some basics down, many are perfectly competent or better, some have almost no experience. I have, just-in-time, stopped several people from giving out their social security numbers to spammers. I've had guys ask how come they can't get the free credit report the email said they could get without a credit card number. Just about all of them seem to understand, almost instinctively, that the sex-related spam is probably a scam. But I think you'd be surprised at how easily unsophisticated users can be taken in by what would strike many others as an obvious scam. And the more sophisticated the spam, the more people can get roped in.

When I'm helping someone set up an email account (sometimes, their first email account ever), I try to direct them to GMail because it seems to me to do the best filtering out-of-the-box. Many of them use Yahoo, and those are the ones I usually find trying to respond to a spam solicitation.

Re:More than 90% for me too

[BlortHorc]

Umm, you clearly need to read about SPF.

You can set an SPF record that says: mail claiming to have been sent from this domain will only ever come from this set of IP addresses or subnets.

So, for example, if the SPF record for northpole.com specifies that outgoing mail from that domain will only come from 227.2.43.8, when an SMTP connection is made from 19.2.55.87 to your mail server with mail from santa@northpole.com your mail server should at the very least increase the weighting it gives to the mail's spam likelyhood, and really should be quite happy to just simply reject the mail, no buddy, you don't have mail from santa.

Re:More than 90% for me too

[BikeHelmet]

Someone seems to have signed me up to every spam newsletter imaginable. Google filters about 3000 emails out per month. It misses one every couple weeks.

Re:More than 90% for me too

[ravenspear]

and bounced

No no no, bouncing forged mail is a horribly bad idea. That just clogs up legit people's inboxes with spam that it looks like they sent.

Forged email should simply be dropped by the server.

Re:More than 90% for me too

[growse]

3000 a month?

Try 3000 a *day* to my personal account. It is domain I guess, but I'm now discovering spamd/OpenBSD :)

http://www.growse.com/projects/spamwatch/

Re:More than 90% for me too

[BikeHelmet]

3000 a month is quite a lot for "legit" newsletters. Stuff like Sears.

Re:More than 90% for me too

[mqduck]

If there's one thing American have cared about since day one, it's returning royalty to their rightful seat on the throne.

Re:More than 90% for me too

[growse]

Ah, fair enough - I thought you meant all spam. No, 3000 a month is a hell of a lot for that!

Re:More than 90% for me too

[TheRaven64]

Tell that to the gmail admins. I got hundreds of bounces from gmail because the messages were spam, even though I have SPF records set up and, checking the headers, the address that they had been sent from was not one that my SPF records allowed.

Re:More than 90% for me too

[TheRaven64]

I have read about SPF, and I have it configured for my domains. An email having a valid SPF record does not mean that it is not spam. an email having no SPF record does not mean that it is not spam. An email having an invalid SPF record usually does mean that it's spam (or, at least, forged).

Most spammers set up SPF records these days. The post that I replied to (which you clearly need to read), said that you could reduce false positives by getting the sender to implement SPF. This is absolutely not the case. You mail is no less likely to be treated as spam by the receiver's spam filter if it has a valid SPF record than if it has no SPF record. The reverse, in fact, because most spammers now do use SPF so some spam filters will give a slight penalty to mails with SPF configured.

Setting up SPF will prevent spammers claiming to be from your domain. If your domain is whitelisted by the receiver, then that will prevent false negatives (mail not being marked as spam when it is) but it won't prevent false positives (your mail being marked as spam) if you are not whitelisted.

Re:More than 90% for me too

[jez9999]

Any newsletter that sends you regular e-mail without you first confirming your e-mail address should be treated as spam.

Re:More than 90% for me too

[BikeHelmet]

Totally agree. That's why I called them spam newsletters. :)

Accounting for help desk calls?!

[Anonymous Coward]

Now I am not a corporate email guru, but why would spam be the reason to call for help? In this day and age it boggles the mind. Even my grandmother can deal with spam without needing tech support.

Re:Accounting for help desk calls?!

[NatasRevol]

Your grandmother is smarter than most people in the office.

I too was the email guru once upon a time (last year). It boggled my mind that people simply could not understand that some email was spam, and that some valid mail got caught because their friends forwarded a forward or an ad company sent them an actual email. And I explained this to the same set of people over and over again.

Re:Accounting for help desk calls?!

[Antique Geekmeister]

_Reporting_ spam is often routed to the help desk. And the intricacies of reporting the entire, unedited message with all the headers intact is often beyond a casual email user. Particularly irritating email also climbs up the reporting priority list and wastes helpdesk time, such as email being forged from one domain to pretend that it is from another domain and getting other people's email being blocked or taking advantage of their whitelisted domains (known as "joe jobs").

Re:Accounting for help desk calls?!

[DarkOx]

This off topic but- don't you think OS X was born out of it being easier to make Unix friendly than fixing MacOS which they tried and failed to do internally? I really don't see how Windows has anything to do with it.

Re:Accounting for help desk calls?!

[clarkie.mg]

Are you sure ? I know a lot of inexperienced people who are overwhelmed by the number of messages in their mailbox. One 70 year old just told me she gave up on her mailbox because there were 750 messages in it. Another one, 50 years old, is drowning in advertisement messages - not even spam, she gave her email on legitimate shopping sites.

A third one, 50 years old, lost an email confirming her plane travel and ended up rebooking it ! When she called me, i found the email in 1 second by using the search function.

For an experienced user, it might seem easy to use some basic techniques like filtering, searching, sorting but most people just pile up the messages and only use "reply" and "forward" without editing.

I offer a one hour basic course for email management with an optional one hour for setting up filters and other tools but email is rarely considered a serious issue rather a tool to send jokes, porn or "how are you" messages, unfortunately.

Re:Accounting for help desk calls?!

[jonbryce]

I've had "my daugter is getting a load of inappropriate adverts that she finds disturbing, how do I stop it" several times.

Re:Accounting for help desk calls?!

[ckaminski]

I haven't deployed an Exchange server since Exchange 2000. Do newer versions mine the Junk-Mail folder for email to improve it's spam filtering?

I'm continutally amazed at MUA that don't have a mechanism for reporting spam. Especially Outlook.

Re:Accounting for help desk calls?!

[BagOBones]

The calls come down to the following three reasons:
- You have a great filter so when any spam gets through people complain because it is not the norm.
- You have an overly aggressive filter or the occasional false positive. People really hate it when they don't get important messages
- You have no filter or one that sucks, the users inbox is full of spam and they can't get through it all to find the real email

I'm surprised it's that low

[jimicus]

I was seeing more like 97% (once you excluded system generated internal emails - CVS and Bugzilla between them can generate a fair bit of mail).

The killer for running our own mail system in its entirety was when I did the arithmetic and our co-hosted secondary mail server was costing more than buying Google for Domains. That's before you even consider the document management Google for domains offers, which was just icing on the cake.

Re:I'm surprised it's that low

[hoboroadie]

According to the respondents in the report, less than 75% of the ISPs even forbid spamming in their Terms & Conditions. Dismaying news, IMO.

Re:I'm surprised it's that low

[Blowit]

Curious as to why you would need a secondary mail server for a small company when *IF* you purchase the right product, it can all be handled by one mail server. We use Surgemail and the load it can handle puts other commercial mail server software to shame.

Re:I'm surprised it's that low

[DarkOx]

Perhaps his small company does lots of business over E-mail as many do? E-mail is the primary interaction with customers for lots of smaller businesses and customers draw some pretty weird conclusions when they get even one NDR.

Also two or more mail servers is common the system is designed to work that way which is exactly why you can have multiple MX records for your domain. This way a sending server can try the other mail server if / when it can't contact the primary. Oh I suppose you could use some complex and expensive clustering solution so as to ensure your mail relay has 5 nines of uptime but that is likely much higher cost for an SMB which does not have dedicated IT staff and shared storage equipment.

No I think its probably a much easier cheaper solution to hire someone to simply setup two mail servers.

Re:I'm surprised it's that low

[jimicus]

Quite correct. The secondary MX wasn't for load balancing - Postfix can handle more mail than I could ever throw at it - the secondary MX was to minimise the risk of NDRs.

It actually caused as many problems as it solved because spammers seem to think "secondary MX == no spam filtering".

Re:I'm surprised it's that low

[TheRaven64]

Is there a list of IPs assigned to ISPs that permit spamming? If so, you can block them at the firewall; just redirect them to a tar pit that replies at one character per second that they have been blocked and should find a new ISP. If you really want to make a difference, do that for HTTP too; redirect people from those ISPs to a static page saying that they have been blocked from viewing the content on the site because their ISPs are encouraging network abuse.

Re:I'm surprised it's that low

[jonbryce]

If there is downtime on your main mailserver, or the telephone line connecting it to the outside world, you need a secondary server to pick up mail until it comes back online again.

Re:I'm surprised it's that low

[MrMr]

Slightly less dismaying if you consider that most countries have already adopted anti-spam legislation.

Re:I'm surprised it's that low

[BagOBones]

In real enterprise systems where email delivery has an SLA 24x7 you need redundancy so you can take parts of the system offline for maintenance or in the event of hardware failure.

Micropayments again

[Twinbee]

Micropayments. Yes I know it's been mentioned before, but one rarely hears of paying *each other* (rather than the host or government). It would be a good idea anyway even if spam didn't exist.

If we paid each other (say a penny or 1/10th of a penny), obviously the spam problem would be solved. (though some can charge nothing if they want) It also means that someone who gets a ton of email and hasn't got the time to read all of them will receive only the 'cream' of email. Only those who are willing to sacrifice say, a pound (or £10/£100 for super busy/famous people) would be able to email them.

As we know, Youtube has/is developing methods of payment to watch videos, and online papers are experimenting, so micropayments may be common sooner than we think.

Re:Micropayments again

[Nyxeh]

I had an idea to fix this:

http://piestar.net/2009/06/24/idea-fixing-the-email-system/

There are many better ways outside micropayments - which would add up on a large system (such as a forum or social networking site).

Re:Micropayments again

[Anonymous Coward]

but one rarely hears of paying *each other* (rather than the host or government)

Only if you don't read the discussions. A scheme like that is proposed every time the topic comes to "how I would end spam once and for all". Go ahead and try it. Oh, you want everybody to switch? See, that is a fundamental problem: If your scheme requires a critical mass of people to adopt the scheme at the same time, then it won't work. (There are more problems with pay-for-email and email-bond schemes, but that is the most obvious one.)

Re:Micropayments again

[russotto]

Micropayments. Yes I know it's been mentioned before, but one rarely hears of paying *each other* (rather than the host or government). It would be a good idea anyway even if spam didn't exist.

Because, as one of those irritating but often accurate form rejections points out, transaction costs make this impractical. You'd spend far more administering the payments than you would actually making them, so if you had a system where you paid someone $0.05 to receive your email, and they paid you $0.05 to receive theirs, you'd also each end up paying $0.50 in transaction costs to whoever handled the payments.

Re:Micropayments again

[Twinbee]

People can use both email types, until gradually the 'micropayment' emails become the norm, and the free, spam ones are ditched by more and more people. Not forgetting of course than even the 'micropayment' email can charge zero if that's what they really want. It 'subsumes' the old type in that sense.

If these kind of things didn't work, then we wouldn't have new connectors at the back of PCs. USB wouldn't exist for instance.

Re:Micropayments again

[Twinbee]

Well the way we handle transactions currently must be hopelessly inefficient. Computers are supposed to be good for this kind of thing.

Re:Micropayments again

[gilgongo]

If we paid each other (say a penny or 1/10th of a penny), obviously the spam problem would be solved. (though some can charge nothing if they want)

Your post advocates a

( ) technical ( ) legislative (x) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
(x) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
(x) It will stop spam for two weeks and then we'll be stuck with it
(x) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
(x) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
(x) Lack of centrally controlling authority for email
(x) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
(x) Jurisdictional problems
(x) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
(x) Huge existing software investment in SMTP
(x) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
(x) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
(x) Countermeasures must work if phased in gradually
(x) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!

Re:Micropayments again

[Antique Geekmeister]

We'll see micropayments work when we see fusion power. The overhead of authentication and actually processing money are so large that they are simply not practical for normal email, and the kind of idiot who does spamming now would simply steal funds from your mail servers.

Re:Micropayments again

[Blowit]

Surgemail already is micropayment ready... Just plug in your bank settings and you are ready to go.

Re:Micropayments again

[Twinbee]

What advantage will early adopters have from joining your scheme?

What advantage did early adopters of computers with USB ports have of getting their PC? As I said, these things will be added to server installations, and will be available as an *option* to the end user. People can have a 'normal' email and a 'micropayment' email.

I think the main problem is transaction cost. Once companies figure out how to send and receive micropayments for the cost of transmitting normal information over the internet (i.e. virtually nothing), then we have a winner.

I like the humour in the first part of your post :) Though, I did 'find' the Mandelbulb - that was *my* once in a lifetime I think... ;)

Re:Micropayments again

[Blowit]

This is why there would be a global payment system that mail servers would join... the user would purchase a minimum of 5-10$ of eStamps and would be distributed monthly to the providers. sender provider and payment provider would get 35% of the fee while 30% goes to the receiver provider. Problem solved.
Sender and receiver provider would make money while the actual sender is paying for it.

Re:Micropayments again

[ppc_digger]

4) The assumption that spammers would have a hard time spamming because they would have trouble keeping their servers reachable is false.

Sure, but then the servers couldn't be botnets, so you would know where the are and could take them down or block them.

Re:Micropayments again

[ookabooka]

Your post advocates a
( x) technical ( ) legislative ( ) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won’t work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
( x) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we’ll be stuck with it
(x ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(x ) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don’t care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else’s career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
( x) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( x) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( x) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( x) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( x) Extreme stupidity on the part of people who do business with spammers
( x) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

( x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( x) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( x) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don’t want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(x ) Sorry dude, but I don’t think it would work.
( ) This is a stupid idea, and you’re a stupid person for suggesting it.
( ) Nice try, assh0le! I’m going to find out where you live and burn your house down!

Re:Micropayments again

[mikael_j]

They get to pay extra to use email, of course! Clearly this is an "advantage" that everyone will want..

/Mikael

Re:Micropayments again

[Xeriar]

( ) Killing them that way is not slow and painful enough

Pray tell, what level of vigilante justice would you consider to be slow and painful enough?

Re:Micropayments again

[Twinbee]

Or get paid if they receive more email than they send. Of course it would all be negigible anyway if costs were $0.0001 per email sent anyway.

In the long term, it would balance out as they send and receive email.

Re:Micropayments again

[Twinbee]

What I meant was that probably very few devices supported USB in the beginning, so it didn't seem such a great feature at first. However, with the giant Apple backing it, there's the promise that it will eventually become standard.

Now you: What advantages will early adopters of your scheme have?

No spam, and ability to receive less and better quality email by charging arbitrary amounts. Some people who otherwise may not be reachable at all (being super busy), suddenly become available by being able to pay them to read your email.

Re:Micropayments again

[Twinbee]

Reply to this (my) post if you read it.

No need to be anon - I'm not going to get bitchy no matter how much I disagree.

Re:Micropayments again

[jeremyp]

People can use both email types, until gradually the 'micropayment' emails become the norm, and the free, spam ones are ditched by more and more people.

Won't happen.

So maybe 95% of the e-mail that is sent to me is spam. What is it worth to me to have to pay to send e-mails as I sit in front of my PC trying to decide whether to use the free system or the pay-for system to send my mate Bob an e-mail. The answer is apparently nothing. My e-mail is perfectly serviceable and spam is only a minor inconvenience. I don't see almost any of that 95% spam because virtually all of it gets filtered before it drops into my inbox.

You could argue that there is an infrastructure cost to all this spam and there is, but it is completely out of sight of most e-mail users and also probably negligible compared to the infrastructure cost of (say) providing the bandwidth for everybody to download TV programmes off the BBC.

Re:Micropayments again

[mikael_j]

I can't help but feel that a scheme like this would be loved by telcos all over the world, all they'd need would be a law that forbids email without this "feature" coupled with some sort of licensing scheme that required server operators to jump through a whole bunch of hoops plus pay a large yearly fee (perhaps labeled as a "downpayment" on that year's transfer fees so technically you'd only have to get the money once and then you could use the money for last year this year again but it would still be too expensive for startups and home users wanting to run their own server) and they'd have a de-facto monopoly on email and could start charging their users $0.05 per email or something silly like that, of course if you buy the "Email XL" plan you'd get 200 free emails per month and any further emails would be sent at the "discount" rate of $0.02....

/Mikael

Re:Micropayments again

[Hotawa Hawk-eye]

1) Create hundreds or thousands of throwaway email accounts.
2) Subscribe all those accounts to your target's mailing list.
3) Watch as your target has to spend a significant amount of money sending "Thank you for subscribing" emails and daily/weekly/monthly messages.
4) ???
5) Profit? ... perhaps not.

What I think could work would be an escrow system. When you sign up for your ISP account, you put some amount of money (say $20) in escrow with your ISP. After a certain period of time subscribed to your ISP with no black marks (say 3 months or so) that money is returned to you. However, if your account is terminated due to violation of the TOS, you lose that money. This addresses the problem at a local level (the ISP), ISPs can opt-in if they want (although other ISPs could then delay delivery of email from ISPs that don't use the system while they check it for spamminess), and it uses a capability the ISPs already have (their billing department.) To avoid abuse (ISP: "Oh, yeah, you um broke the rules. Sorry, we get your money.") we use the existing safeguards against bad business behavior, up to and including lawsuits if necessary.

It wouldn't _stop_ spammers, but it would make it more expensive and/or time consuming to spam, and that might be enough to cut out

Re:Micropayments again

[jonbryce]

People have been talking about micropayments for at least 10 years now. The problem isn't a technical one, see paypal.com for an example of how it is done. The problem is that as soon as people are asked for payment details, even for a very small amount that they wouldn't notice, they stop to think about whether or not they really need it, and generally decide they don't.

The problem is it takes a bit of effort to spend money. If you make it less of an effort, people don't like it because it makes them feel less in control.

Re:Micropayments again

[jonbryce]

If people have the option to send me an email for free, or to pay for sending me it, which one are they going to pick?

Re:Micropayments again

[jonbryce]

Most of the spam is sent through botnets. Who is going to end up paying this email levy? Mind you it might encourage them to clean up their machines, but more likely just mean some sensationalist articles in the Daily Wail and judges refusing to enforce payment.

Re:Micropayments again

[jfengel]

It seems to me that the key objection to this is the mailing lists.

But what if mailing lists were replaced by RSS? When you're delivering identical content to a lot of different users, letting them pull rather than push eliminates spam, and for zero cost beyond having the servers.

Having zombies' accounts drained is bad for them, but maybe it would make people more alert to them.

The dropped connections as you phase in the system is sad, but it's in pursuit of a more usable system in the end.

Re:Micropayments again

[amorsen]

Right now the vast majority of spam is sent by compromised computers. What would stop those computers from paying the micropayments? Sure, it'd be nice to collect a few dollars from the incoming spam, but I'd feel a bit evil taking money from random people around the world. Even if they should have been smarter.

You can say that fraudulent transactions are only a few percent of total transactions today, but it will difficult to build a micropayment system which has as many abuse checks as the current transaction systems -- and the current systems would kill your idea with transaction fees often in the range of $0.50.

Re:Micropayments again

[Tom]

While I find that list as funny (and often true) as the next guy, you failed in applying it.

Mailing lists and cooperation of everyone has long been solved in the micro-payment solutions. Central authorities are not needed, just a reasonably small set of payment handlers. And it's got nothing to do with blacklists, rather with dynamic whitelisting.

A little technical voodoo solves the rest.

Still, I agree with your result that it won't work, mostly because people have become way too used to the way e-mail works for any change that does not give them an immediate, tangible benefit.

Re:Micropayments again

[Changa_MC]

You do realize you just quote a line from a pre-existing answer form that ookabooka did not check, right?
So you need to find the guy who created that form, and ask him why he included it.

Re:Micropayments again

[jhdevos]

Also:
(x) It would make DDOS attacks abusing the system extremely easy.

(basically, the system reduces to a simple blacklist, which is automatically updated based on spam-reports)

Re:Micropayments again

[bhassel]

I don't see why mailing lists are an issue with a micro-payments system. There would be no need to attach a payment to mailing list emails (or to emails with friends/family, etc) since supposedly you would just add that address to your address book and they would be allowed through. After all *you* subscribed to the mailing list, so you are expecting emails from them.

It seems you would only need to attach a payment to an email if the recipient doesn't know you (and so wouldn't have you in their address book) or if it is urgent enough that you want to make *sure* it gets read.

Re:Micropayments again

[bhassel]

Unless you regularly send emails to people you don't know (and so wouldn't have you in their address book), I don't see why you'd need to have more than a few dollars in your account. So if your computer is owned, sure, you might lose those few dollars, but nothing more. It would be an incentive to clean up your system.

And as has been mentioned before, using an escrow-like service similar to Paypal would largely avoid a need for credit-card processing fees, etc, right? It seems all the payment between users of such a service could just be handled internally.

Re:Micropayments again

[amorsen]

Paypal is notorious for not getting the abuse checks right.

If we assume an email fee of one penny (highly unlikely though, because even a dedicated service without abuse checks would require that much just as a transaction fee), $10 would give the spammer a thousand emails to send. All of which would be delivered to the recipients, since that's the whole point of this system. I bet that's comparable to their current success rate, where 99.9% of their spams are caught by filters.

At the recipient end I'd be stuck with a thousand emails to read for a pay of $10, which is just not acceptable.

What do they mean by 'all'?

[Nyxeh]

Is that 5% sent is spam, or 5% that is delivered is spam? There are layers of spam blockers before any mail even gets remotely near anyones inbox.

Re:What do they mean by 'all'?

[WuphonsReach]

Yah, you can cut the rate of bad connections down by about 50%+ if you force the sending host to follow the RFCs.

- Incorrectly formatted HELO/EHLO greeting? 5xx Doesn't catch too many connections as the other end would have to massively screw up in order to trigger the invalid HELO rule.

- Giving a HELO/EHLO that is not a FQDN (fully qualified domain name)? 5xx Many botnets don't follow the FQDN rule and will give a randomly generated HELO name. I've never had a false-positive with checks like this.

- Giving a HELO/EHLO that does not resolve via DNS (see RFC 5321, section 2.3.5 where it talks about this issue in the 1st bullet point)? 5xx or 4xx if there was a DNSFAIL issue

- SPF record says "-all" for the MAIL FROM or HELO lookup and it fails to pass SPF? 5xx (At which point, you're simply following the instructions of the sender. If the record says "-all", they WANT you to reject non-conforming mail.)

- HELO/EHLO which purport to be from your own system? 5xx Know your servers, know who is allowed to put your domain into the HELO/EHLO and boot the pretenders. Easily done in Postfix with a few simple rules.

Most of those are standard checks in Postfix and will greatly reduce the amount of spam that you have to analyze in a more in-depth manner. Which results in a huge CPU/bandwidth savings if you can tell them to bugger off before the DATA command is issued.

I prefer to save block lists for the spam scoring system as there are too many false positives (and sometimes abuses of power) in the DNSBLs. Far safer to score rather then block - although Spamhaus' Zen list is extremely good.

Re:What do they mean by 'all'?

[DaveGillam]

Agreed on all counts, and don't forget adding honeypot MX records, as many bots will either target the highest-numbered MX, or only the lowest-numbered MX. Proper MTAs will follow RFC, and get around the honeypots, to the "real" MX hosts.

Re:What do they mean by 'all'?

[PNutts]

Yah, you can cut the rate of bad connections down by about 50%+ if you force the sending host to follow the RFCs.

Yes, I initially thought that's a good idea so I did. Unfortunately, I found out about it blocked about 10% of valid e-mails from misconfigured systems.

Re:What do they mean by 'all'?

[MROD]

Merely prescribe that your clients use port 587 (Submission) with a username and password. Even merely using the submission port would cut practically all the bots out of the equation (for a short while).

95% of slashdoters don't RTFA

[daveb1]

95% of slashdoters don't RTFA.

Re:95% of slashdoters don't RTFA

[Lennie]

Why should they, this is a discussionboard, not a bookclub. ;-)

Spam not equally distributed among message media

[Dilligent]

One thing to keep in mind is that even though it looks bad (and for email it certainly is..), most other mediums aren't quite as affected by it. I do get quite a bit of Spam on ICQ these days, but the ratio between spam messages and real messages is waaaaaaaay better than 20:1. I would expect the same to hold true for most other mediums as well, so that it might in fact be a good idea to use those as a separate alternative communication channel should your inbox become overwhelmed. Something i have noticed over the years is the reduction in Trojans and worms being sent (at least to my inbox). There was a time when i received around 50 trojan-emails a day, whereas now it has been quite a while that a spam mail did actually contain any attachment whatsoever. To summarize, yeah.. email looks bad, but there's a whole set of alternative or additional channels that can be used which aren't quite as saturated.

My spam count has gone down lately

[PetoskeyGuy]

I have no idea why, but my spam count has gone down. I have my own domain name and I used to receive about 100 spam per day. Lately that's gone down to 2 or 3.

I'm not doing anything different so I assume I fell off a list or someone upstream is fixing things.

Sometimes I run a filter that let's all plaintext through but whitelists mime and messages with http or www in the message. They get rejected at the server level.

I just turn it off when I register for new web sites.

Re:My spam count has gone down lately

[davmoo]

For what its worth, I've had the same thing happen on my own domain. As recently as 6 months ago I was averaging about 1500 spams a month, and now its down to maybe 200 a month. I'm certainly not complaining, but I'd love to know why it dropped.

Re:My spam count has gone down lately

[Blowit]

check to see if your provider is using SPF... If so, they end up blocking domains that does not have one or is spoofing a domain that does have SPF setup. This helps significantly to reduce the amount of junkmail.

Re:My spam count has gone down lately

[Anonymous Coward]

Got a letter from my ISP recently (the most important ISP in the country) telling something like: "All outgoing connection to ports other than 80 and 443 and xxx and yyy are now blocked by default, if you have a legitimate need to access these ports please log on to: .... and change your settings".

This is insta-death for a great many spam bot/relays.

Boom. Game over.

Re:My spam count has gone down lately

[amorsen]

I don't know where you get this from. All that SPF gets you is that you receive less "backscatter" spam, non-delivery-reports for mail you didn't send in the first place. For most people, that's a tiny fraction of the spam they receive. The majority of spam is sent from domains with valid SPF records and passes SPF checks.

Re:My spam count has gone down lately

[amorsen]

Insta-death for most of the Internet, too...

Re:My spam count has gone down lately

[Spad]

Not just backscatter, also spam "from" you - I used to get a lot of.

Re:My spam count has gone down lately

[FreeBSD evangelist]

I also have my own domain and run my own mail server. I seem to be running around 200 per day. Spamprobe does a pretty good job for me. It misses two or three per day, but more importantly almost never gets a false positive.

Re:My spam count has gone down lately

[amorsen]

SPF only protects the envelope from, not the "from" which is shown to the user.

Bill Gates

[Enderandrew]

http://www.cbsnews.com/stories/2004/01/24/tech/main595595.shtml

Bill Gates promised in 2004 that spam would be completely solved within 2 years.

Re:Bill Gates

[Anonymous Coward]

Well, that article is somewhat about captchas and I couldn't see any direct quotes from billy, considering that most email providers, live/gmail etc. use that tech. and my spam has virtually been reduced to zero in the last few years... No one can stop spammers from sending spam, but you can always filter it out.

Re:Bill Gates

[jimicus]

Well, that article is somewhat about captchas and I couldn't see any direct quotes from billy, considering that most email providers, live/gmail etc. use that tech. and my spam has virtually been reduced to zero in the last few years... No one can stop spammers from sending spam, but you can always filter it out.

The reason your inbox - or anyone's, for that matter - is not overflowing with so much spam that "just hit delete" is no longer an option is not because nobody's sending spam.

Neither is it because the magic email fairies are ensuring that you only receive legitimate email.

It's because some poor bastard is attempting to stop it. But for every counter-measure we take against spam, the spammers work on anti-counter-measures.

For those anti-counter-measures, we take contra-anti-counter-measures.

This doesn't work for very long, however. Before long the spammers have developed dis-contra-anti-counter-measures.

So the mail admins of this world devise un-dis-contra-anti-counter-measures.

I've run out of prefixes meaning "un", but you can see where this is going. It's a game you only win if you don't take part - which is pretty crap if you actually want to use email.

Re:Bill Gates

[Tom]

He's not lying, you know. He's just waiting for the perfect year to start his two-year-plan...

Re:Bill Gates

[StormReaver]

Bill Gates promised in 2004 that spam would be completely solved within 2 years.

And in 20 years, he'll claim to have no memory of having ever said that. And his apologists will claim that he's too smart to have said something so stupid. And history will have repeated itself yet again.

Re:Bill Gates

[Enderandrew]

In the second paragraph there is a direct quote.

“Two years from now, spam will be solved,” he told a select group of World Economic Forum participants at this Alpine ski resort. “And a lot of progress this year,” he added at the event late Friday, hosted by U.S. talk show host Charlie Rose.

Google it up, and you'll find several sources directly quoting him, not to mention follow-ups years later when Gates says it was a mistake to make such a claim.

Re:Bill Gates

[feepness]

640 days ought to be enough for anyone.

Re:Bill Gates

[jhdevos]

Yes. I conjecture that the only reason my current counter-measures (and that of GP) work reasonably well is because so many people use worse counter-measures -- which makes it less necessary for spammers to outsmart them (yet). It's not a solution, and by definition only works for a relatively small part of the population of email users.

Re:Bill Gates

[sootman]

And what kills me is that he COULD HAVE, the bastard. Or at least, made a very large dent in it. All he had to do was have MS release some patches for Windows and give them for free to EVERYONE, "pirates" included. According to a quick search, 80 percent of spam comes from zombies.

Re:Bill Gates

[Nightspirit]

I use outlook and I never see spam and I don't get false positives. That doesn't mean the spam problem is solved but at least my time isn't wasted (just bandwidth).

Re:Bill Gates

[jimicus]

Yes. I conjecture that the only reason my current counter-measures (and that of GP) work reasonably well is because so many people use worse counter-measures -- which makes it less necessary for spammers to outsmart them (yet). It's not a solution, and by definition only works for a relatively small part of the population of email users.

Mine WERE working reasonably well! Or so I thought - I was blocking well over 90% of incoming email to spam but I didn't have a mechanism for end users to check "may be, may not be" spam - and I didn't want to crank up the sensitivity without that.

But when the people worst affected are the executives who agree the pay every month, you have to ask yourself "how married am I to running my own email system in its entirety?". I can easily see a future where most sysadmins decide the answer is "not very" and wind up outsourcing at least the spam filtering.

Re:Bill Gates

[fatalwall]

I work for a company that ended up doing that!! We tried having a spam filter internal however after we looked at the cost of spam (appliance, internet usage, employees) we switched to an external group. All of our emails now go though there system removing the spam from whats sent to our email server.

Between that and blocking facebook and myspace we were were able to make due with the single T1.

Where they got there numbers?

[Blowit]

Look at my mail server's spam status.
the RBL has blocked 95% of the spam out there.
57.5% had no SPF records. Looks like SPF has gained a lot of ground now... almost half of the Internet is now using it.
Using Surgemail, I do not need to use 3rd party anti-spam systems as the anti-spam is handled by the mail server itself. It handled 4 million messages in a month and does not break a sweat. I love this mail server and no other system can persuade me to switch... Support is incredible, service top notch... can not praise it enough.

Spam status:
        RBL Denied 95.3% (1882484), Stamped 4.7% (93193), Checked 1975678
        Total score 3 or above 75.5% 123278/163348
        Aspam Score 1 or above 15.4%, ngood=987 nbad=2965 ncatcher=2521
        URL Database 13.6%, In database bad=12997 neutral=2168 fromnet=15138
        SPF hits (msgs) 68.8% 2753806/4002538, (no spf=2302652 57.5% pass=361145 of 4002493)
        SPF rcpts blocked 0.0% (0/698887) allow=0 dkf=5393
        Badfrom hits 0.0% bad=0 good=384559 mx=0
        Spam Bounce (0) 2.5%
        Helo failures 235981 5.7%
        SURBL 38.0% 94570/248869 0/0
        User spam actions Vanished:8 Bounced:21793 Stored:46
        Friends Allow:23059 Block:0 Confirmation:14944 (Bounced:2787 Replies:128 Spam-ratio:0.96)
        DomainKeys goodsigs=15730, badsigs=458, nosig=0, badformat=408
        SPFShare isspam=814 notspam=0 allow=0 web=2630 tell=0 (knowndb=270297)
        SpamC 104.09% (db 443774/34284) spam=108206 ok=36709 zero=103954
        From Blacklist 0 records, 0 hits
        False Pos 128/14944 0.86% (based on friend confirmations)
        False Pos 7732/41670 19% (based on msgs from friends)
        aspam_content.txt 7788 3.1%

Only 95%?

[Doc Ri]

I am surprised they conclude the fraction of good mails is as high as 5%.

From the CERN mail server report:

Incoming mails: 1992789
Rejected: 1952787 (98%)
Moved to Spam Folder: 14520 (1%)
Good mails: 25482 (1%)

Spam in Total 99%

And this is a good day. Often good mails are less than 1%.

Re:Only 95%?

[amorsen]

I'm guessing that they don't count mails to non-existent mail boxes as spam (it's dropped before the spam/no-spam determination). CERN probably counts it as spam.

What? But Bill Gates predicted end to spam by 2006

[jarocho]

Bill Gates called it, way back in 2004. And Bill Gates is never wrong about ANYTHING. So it's pretty obvious that whatever we've all been receiving in our inboxes since 2006 that looks like spam isn't. Probably, we're all just overwhelmed by all of the legitimate emails we're getting from our many, many friends nowadays, who really are just trying to tell us about some aweS0me dea1z on r0lexxes, and we just can't decide which of the incredible bargains to choose from. And it's actually Google and Yahoo's fault for not having deprecated their spam filters, even though spam now is a thing of the past (trying to make MS look bad, of course). So they keep catching your friends' emails as spam. But it can't be spam, because it's 2010 already. And Bill Gates said.

Stop the floodgates

[Bourdain]

I've introduced a number of people to a multitiered system which, for me, has almost completely solved my spam problem.

1) Do the unthinkable, actually pay for email service at a place, ideally, like www.fastmail.fm which uses spamassassin unlike the simpler less forgiving systems at yahoo/gmail/etc.

2) Use a handful of aliases (yielding unlimited email addresses) in order to sort mail to its relevant level of "attention"

e.g.
2a) john.smith@fastmail.fm would go to friends to use
2b) wellsFargo@level01.fastmail.fm would go to a site you trust like your bank and be filed in you level01 folder
2c) chineseCommerceSite@level05.fastmail.fm would go to your level05 folder and so on...

3) Beauty of the above systems is that when an address gets spammed (or the site sends too much garbage), you can easily disable it via a filter since each site should have its own email address

4) Further, you are less likely to receive obvious spam via setting a high spamassassin threshold and the fact that a site like fastmail subscribes to various RTBL's

Using this system, I've received barely anything more than 1 spam per month to any "un-aliased" address. The overwhelming majority of the time, said spams are properly flagged by spamassassin.

I hope this helps

Re:Stop the floodgates

[drinkypoo]

If any of your friends are using Windows, you need to give them a personalized address too, so you can see which of them have been owned, and issue them a new address only after they have formatted their system.

Re:Stop the floodgates

[Bourdain]

definitely not a bad idea, but in my experience, the number of times my personal address has been hit by things like that is extremely rare

I for one support my online freedoms

[Anonymous Coward]

ISPs don't need to specifically disallow something that is already against the law. So, you find it dismaying that over 25% of ISPs allow you to use the bandwith you pay for in any way you wish as long as you don't do anything illegal. I assume that the alternative is that ISPs begin regulating Internet traffic based on their arbitary interpretations of what you are ethically allowed to do. (See: Peer-to-Peer netlimiting)

The thing is, if I send many thousands of emails in one day, I might be sending e-mails to some online community I manage, it might be related to some service I offer, or any number of other legal and ethical uses of the bandwith I paid for. The ISP can't know what those are unless they actually read my e-mails or closely monitor them (something I really don't want my ISP to do!). Even if they call me "What are you doing?" they still have to take my word for it or violate my (and others') privacy. Even if they knew exactly what I was doing and personally thought it was unethical but knew it was legal, I would argue against it being their right to interfere.

Re:I for one support my online freedoms

[mikael_j]

By disallowing spamming an ISP has a specific line in the TOS that they can point to when a customer calls in screaming about their "intarwebs" being unreachable. "Yes sir, I understand that you are upset but it appears that we got several reports that large amounts of unsolicited email was being sent from your home, upon further monitoring by our technicians it was established that several thousand spam emails were being sent from your home and in accordance with paragraph 713 in the terms of service we disabled your internet connection, attempted to call you and also sent your a letter explaining the reason for us disabling your connection, if you want to have your connection re-enabled you will have to ensure that your equipment is no longer attempting to send out unsolicited email. You should also know that if this activity continues after we re-enabled your connection your connection will be permanently disconnected.".

Yes, I used to handle abuse cases for an ISP and got to explain things like this way too often, that was basically the opening explanation, most customers would bitch and moan for 10-20 minutes about how we had no right to cut off their precious internets and would claim that their computer was our responsibility (to which I would often reply with a car analogy along the lines of "If you let a stranger load your car full of explosives and walmart refuses to let you park your car in the parking lot, is it then walmart's fault that you can't be bothered keeping your car free of explosives?".

/Mikael

Re:I for one support my online freedoms

[hoboroadie]

If the ISPs don't prevent it then some one in government might decide that it should be illegal to litter people's inboxes with unwanted emails. Then where will your precious online freedoms be?
I support free speech, but draw the line at folks with megaphones following me into my front door.

Re:I for one support my online freedoms

[jonbryce]

In the EU, spamming is only illegal if it is to "personal subscribers" where you have had no prior business relationship. Viagra spam is illegal because of the content of the email, not the way it is sent.

Personal subscribers means non-corporate subscribers. You can spam companies with impunity. Depending on partnership law in the juristiction in question, you can spam some of them with impunity. For example, in Scotland, partnerships are corporate bodies, but in England they are not, so you can spam email boxes owned by Scottish partnerships but not ones owned by English partnerships. Sole trader business are "personal subscribers" in most juristictions so it is illegal to spam them.

It is illegal to advertise prescription drugs by any means in the EU, including by email, even if the recipient has consented to it.

Flawed logic

[KitsuneSoftware]

Unfortunately, 95% delivery failure does not mean 95% spam. Some spam gets delivered to my inbox, and I'm certain that some legitimate email gets blocked. Unfortunately, the businesmen who like to use "email marketing" have no idea how much of a problem it is, and the technical people doing the filtering refuse to bounce (instead of black hole) suspected spam as doing so would work as a DoS amplifier.

Re:Flawed logic

[Lennie]

I work at an ISP, we drop a lot on the floor, mostly SMTP-connections, but everything gets checked and scored very carefully first. We've been operating these mailservers for years, but never had a user complaint. Some may complain, but that's just because it's stuck at the sending provider. To give you an idea: 98% of what we get is not delivered to mailboxes.

Re:Flawed logic

[1s44c]

98% of what we get is not delivered to mailboxes.

If you block it at SMTP time do you still count that as mail you got?

Re:Flawed logic

[WuphonsReach]

"dropping to the floor" means that you accepted the message for local delivery (or further handling) and then silently discarded it.

If you're rejecting at SMTP connection time - you're not dropping mail on the floor - you're doing it properly by giving the origin server a 4xx or 5xx code.

(The worst thing you can do is to 2xx the message, and then decide later that it's undeliverable. At that point, you're either not obeying RFCs by dropping it on the floor or you're going to generate backscatter if you send out a bounce message.)

Re:Flawed logic

[Lennie]

OK, this was just a quick write up, what I meant to say was: 98% of delivery attempt including deliveries which get tags as high-scoring spam are deleted or kept for a few months or not accepted at the SMTP-level or TCP-level.

Re:Flawed logic

[Lennie]

OK, this was just a quick write up, what I meant to say was: 98% of delivery attempt including deliveries which get tags as high-scoring spam are deleted or kept for a few months or not accepted at the SMTP-level or TCP-level. Yes, we also drop connections at the TCP-level, if an IP-address has to many concurrent connections. And we obviously use 4xx and 5xx codes.

Accountability

[Kjella]

There's no single solution to spam, obviously at times I want people that have never sent me an email before to be able to reach me. Trying to derive whether it's spam from the content will always be an approximate process. But what is not so great is that currently, all the eggs are in one basket. If you get your hands on my email address, then it's valid for years and years, and I have no practical means of switching.

What would help a great deal, is if there was a standard way to generate and revoke an email address for a specific purpose, auto-alias any reply and in the reply include a forward to a different alias. Yes, occasionally I spell it out over the phone or someone has to type it in from paper and shortness and readability is important, but many times it is not. For example, I don't publish my email address here but if I could easily generate an alias ad453785cd76786da76b7678654aa@gmail.com and have it delivered to my real address with the possibility of nuking it I'd consider it.

The rest are really continuations of the same idea, because you'd get a lot of "harmless" mail saying like "Hello, I saw your post at [slashdot comment you made] and think your posts show just the kind of employees we are looking for. We at [bullshit company w/fake web page] would like to increase our technical staff and if you are interested, please send us your resume." which serve no other purpose than to reveal your unaliased address. For that reason, all mail sent to an alias should be replied to using the same alias.

The other issue is that for revocation to practically work, I can't have people who did get in contact with me over the slashdot alias that I'd like to stay in contact with keep using that alias. I have to either give them my real address or point them to a new alias. There's "Reply-To:" and just telling them in the email, but it's a bit weak. Finally a revoked address should optionally give a customer-chosen rejection reason, so that it could be things like "Switched alias, try [new address]" "This alias is expired, find my current address on slashdot.org", "If important, you can still reach me on phone 555-1234" instead of the default "No account with that name".

The best part of someone actually doing is, is that the whole system doesn't need to change. One web mail provider could create this exact setup with the controls to generate and revoke addresses, make sure your replies to an alias are aliased, update and control the Reply-To so you can't redirect it anywhere else and create the custom rejection messages. The only thing it can't do is make sure the recipient updates their address book or whatever, but if the ball gets rolling that will be fixed.

Make Only the spammers pay.

[aashenfe]

How about this take on e-mail postage. We know spammers/phishers send lots of e-mail, but receive very little or none. We use that to our advantage

Before sending e-mail, a sender buys postage, and it goes into their account. Maybe a penny a stamp give or take. When an e-mail is send, a stamp is taken out of the sender account and put into an escrow for each recipient. The e-mail is digitally signed for the escrow id, and sent like normal, but all spam filtering services then check the signature along the way.

When a recipient opens an e-mail, The escrow stamp set assigned for them is transfered to their account (e-mail client, or service provides this). Note: it can only be collected once for each person per e-mail, and it only goes to the account associated with the e-mail.

So after an initial stamp purchase, postage will transfer back and forth, and a normal user should never have to purchase postage again. A person, or company that sends lots of e-mail will have to keep buying postage to send. PHishing and spamming becomes economically difficult.

More reputable spammers/companies will have to buy postage to stay in business.

One last thing, users will be able to sell back stamps when there account starts to fill up, but at less of a price, to pay for the service and keep the validation servers running. So stamps are purchased at retail prices, and sold back at whole sale prices. Spammers/Hammers that stay in business end up paying for the service.

There is much more details, and ideas that can go along with this, but for the sake of brevity, I'll keep int at that.

Re: Make Only the spammers pay.

[WuphonsReach]

Your proposal fails for the same reason that countless other "pay" proposals fail.

The spammers already use hacked systems to send out their email. It's no big leap for them to also hijack the stamp/payment system to have the hacked system's owner pay for the spam run.

Basically, the bad guys will get away with paying nothing - and the burden will fall on legitimate users of e-mail.

The only possible upside to your proposal is that it would cost the hacked system's owner in a way that might encourage them to get their infected machine fixed. But the downsides far outweigh that negligible upside (and the ISPs can already identify and disconnect infected machines - a class action lawsuit against ISPs who don't would work better).

Re: Make Only the spammers pay.

[Leebert]

Oh, this is exciting, I've never done this before!

Your post advocates a

( ) technical ( ) legislative (x) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
(x) Mailing lists and other legitimate email uses would be affected
(x) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
(x) It will stop spam for two weeks and then we'll be stuck with it
(x) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it, anywhere other than Russia
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
(x) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
( ) Jurisdictional problems
(x) Unpopularity of weird new taxes
(x) Public reluctance to accept weird new forms of money
(x) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
(x) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(x) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
(x) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, asshole! I'm going to find out where you live and burn your
house down!

Re: Make Only the spammers pay.

[FreeBSD evangelist]

When an e-mail is send, a stamp is taken out of the sender account and put into an escrow for each recipient.

By whom?

Most spammers are running their own servers, and the mail sending program is custom (i.e. not sendmail). They buy bandwidth from an ISP, not email services.

Re: Make Only the spammers pay.

[aashenfe]

Maybe a little more description is in order
Any email that is not digitaly signed with postage would be blocked automaticaly for the users that use that choose to use the sevice. So the mail from spammers would never get through.
For a spammer to send send e-mail, they would have to use an API to contact the signing server, passing the credentials for an account to transfer postage from, as well as the sha sum of the email to be signed, plus a recipient list. A signiture would be returned from the signing sever that would be attached to the end of the mail before using standard methods to send.
Each send then costs the spammer ( or some poor sap who used week credentials on there account) or its blocked by the recipents client.

Win-win?

[oneandoneis2]

Maybe ISPs should stop fixating on P2P traffic and try harder to stop spam. Then they'd free up a ton of bandwidth AND make their customers happier.

Not for me.

[1s44c]

I use DNS blocklists, greylisting, and a bayes filter. I rarely see spam, maybe 1 or 2 stupid marketing mails from companies I have dealt with a week. My work has more or less the same setup and doesn't get much spam either.

This report must be counting mail blocked at the SMTP level as spam. That seems the only way to get upto 95%.

Re:What? But Bill Gates predicted end to spam by 2

[1s44c]

Bill Gates appears to know little about technology. Why else would be make such a stupid statement?

He reminds me of the support guy that just makes stuff up because most users will believe anything.

How many emails in your (gmail) spam folder ? 396

[clarkie.mg]

Please put the answer in the title of your response. Note that gmail deletes spam that is older than one month so if you answer for another spam system, count for the last 31 days or specify the length of time.

I have 396, much lower than the peak that has been around 900 for years then abruptly got to around 400 each month and remarkably stable.

Spam can be pretty useful occassionally

[cruachan]

I've had the same main email address since the mid 90s, so as you might expect it's on every spam list going, and on average I'm seeing 100 emails a day hitting my Outlook spam folder. However it's never an issue for me as I pay for the rather wonderful Cloudmark spamfilter which is near as dam it 100% accurate for my use.

So all I have is spam hitting my spam fillter at about one every 15 minutes. Which has on several occasions been a useful 'heartbeat' to diagnose when my there's something wrong with my connectivity - either the internet connection itself or the servers being hit by spam.

It's so reliable a diagnostic that I've even wondered if there's a viable commercial product in there based on the idea :-)

Sounds low

[nurb432]

If you take the % at home ( i host my own domain, and I'm the only real user... ) its more like 99.9% due to all the bounce backs and 'dictionary' emails which don't exist anyway..

At the office, its well over 98. ( external email only, not internal )

Re:Spam not equally distributed among message medi

[soliptic]

Yep. This is why I simply abandoned email. Sure, I have a gmail box so that I can "click on this link to activate my membership", but other than that it just slowly fills up with "newsletters" I never read. Any actual human being wanting to contact me online knows full well by now to send me a PM in some "walled garden" environment such as facebook or one of the music forum/communities I visit. Does this reversion to "walled garden" comms systems suck from a 'back in the day the internet was supposed to be all about...' techno-philosophical angle? Sure. Do I really give a shit? No. My friends can reliably message me over the internet, spam is a non issue, win win.

I know what you're going to say... what about people who aren't already my friends and don't already know I never read my email and they should hit me up on facebook or wherever? Well... yeah... what about them? I don't really feel my life suffers for all the unsolicitied but not spam email I would be getting in a parallel universe where spam doesn't exist. Anyway, the only instances of this I can envisage would be, for example, someone likes my music and wants to message me about it, in which case they'd have found it via a forum post or my myspace page, ergo they can message me via that forum or via myspace, so it's really a non issue.

I feel sympathetic for people whose circumstances are such where putting "My email address is blahblah@example.com" out in the wild is the only realistic choice they have, but personally I just thought, right, the whole email channel is saturated with shit, therefore I give up on that channel.

More like 99.9 %, 1 out of 1000 valid

[A nonymous Coward]

I have had my own domain since uunet! days. I drop connections when the envelope header is to a non-existent account. There are very few valid accounts on this domain. Here are the last three days stats on dropped and accepted connections (D dropped, N accepted):

D/837,780 N/941
D/935,298 N/884
D/901,749 N/832

This is 1 valid email out of 1000 attempts to a first approximation, 99.9% spam. Even with these, I still get several hundred validly addressed spams a day, most automatically junked altho I still scan the Subjects and Froms in case the odd transient correspondent was not white listed.

The bogus account names are complete nonsense, just tossing out names and seeing which ones stick.

This shows the true cost of spam...

[damn_registrars]

People can brag about their filters, white-lists, black-lists, etc, all they want. But if this count is accurate, it gives a good indication of the true cost of spam. If 95% of email is spam, then that means only 1 email of every 20 sent is legit. Which could be extrapolated to mean that what you "pay" - by ISP charges, etc ... for your email is also paying for 19 spam messages.

Because in the end, servers around the world are using bandwidth, storage, CPU time, etc, to relay spam. And those servers have to be paid for somehow.

So keep that in mind the next time you think of installing a fancy spam filter to solve your problem; you're really just displacing one cost for another by using some of your own resources to deal with spam once it reaches your inbox. If you want to actually help address the spam problem, look to the root cause of the spam instead of continuing to address only the effects of it.

Re:spam? like they used to have in the 90's?

[durdur]

If you hardly ever use your "real" email address, or only to very limited number of recipients, then yes, you are less likely to get spam. But if you use email a lot, even to people you otherwise trust, every time you hit send you are handing them your address - as well as transmitting it to any number of relays along the way. And any of your recipients can be malware infested.

Yet there are so few spammers

[Animats]

What's striking is how few different spams there are. When one of the major spammers is shut down, spam drops noticeably worldwide. Statistics like "the top N spammmers account for NN% of the spam" could be helpful. In terms of cost, the top few spammers probably have more impact than Al-Queda.

Maybe we could get major spammers classified as enemies of the United States, so the CIA could go after them.

Please learn to do math.

[seebs]

Imagine that the volume of non-spam email remains constant.

If spam was previously 94% of email, and is now just over 95% of email, that is not a change of 1% in the amount of spam.

Let's give concrete numbers. Imagine that there are one million non-spam emails per time unit. How much spam needs to be sent for spam to be 94% of email? The total amount of mail would be 16.6~ million emails, so 15.6~ million of them would be spam. Now imagine that the new amount of non-spam email is "less than 5%" -- let's say it's 4.9%. That would mean a total volume of email of about 20.4M, so about 19.4M of them are spam. So. That's a 23% increase in the volume of spam.

Now let's be realistic. Does anyone actually think the volume of non-spam email has *decreased*? I sure don't.

So this "minor" change is on the rough order of a TWENTY FIVE PERCENT INCREASE in the amount of spam.

Block China, and the likes

[cenc]

I managed to get a little over 50% spam reduction, and many other bad guy type connections by blocking most of Asia, Russia, and many other of the most notorious spam countries at the firewall level. If you have no good reason to every receive legit traffic from those countries, why let them connect?

You have to be careful about getting the ISP blocks right, but if most of your clients and customers come from a few known countries or especially inside just one country you can really crank down the number of potential targets you have to deal with in the logs.

Yea, people can attack through proxies and spam through proxies, but consider if you just eliminate half the computers in the World that do not need to connect to your servers.

Is this really that different than snail mail?

[brunes69]

If I add up the flyers, coupons, direct marketing, and other BS that hits my mailbox (including inside the statement envelopes), and compare it to the actual bills or statements I receive, by weight, I am pretty sure 95% of my snail mail is spam too.