Slashdot Mirror
Beat Spam Using Hashcash
Shell writes "If they want to send spam, make them pay a price. Built on the widely available SHA-1 algorithm, hashcash is a clever system that requires a parameterizable amount of work on the part of a requester while staying "cheap" for an evaluator to check. In other words, the sender has to do real work to put something into your inbox. You can certainly use hashcash in preventing spam, but it has other applications as well, including keeping spam off of Wikis and speeding the work of distributed parallel applications." If you're specifically interested in hashcash for your mail server, Camram has some interesting ideas -- their Frequently Raised Objections page may be illuminating.
Those damn police dogs can smell through plastic pretty well!
Protowall wouldn't let me read the article. Hm.
Aren't there plenty of available solutions today that make the sender "work for it?"
Funny this story should appear today.. I have been trying to find a mirror of hashcash.org for the last few days to read up on the whole idea. It's been down for a while now (or is there just some problem on my end?)
Please post mirrors.
ERROR 144 - REBOOT ?
Hashcash, Camram...... Jeez!
The previous stories weren't enough?
The spammers have heard of outsourcing too, I see a new job market emerging in this manual labour field
Business Voyeur
And remember, you can't spell "Budget" without "Get Bud".
Give me Classic Slashdot or give me death!
Your post advocates a
(*) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(*) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(*) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
(*) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(*) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(*) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!
Today, spammers buy and sell large lists of email addresses on CDs or other media. Each of these addressess took some mining to find it, and put it on the CD.
In the future (if this takes off), these lists will simply contain the hashes along with the addresses. This temporarily makes the spammers lives a bit difficult, but doesn't have a long term impact.
Spammers share information. The cost of all those hashes amortized over a few years to a large number of spammers is nothing.
How would mailing lists be affected by all of this?
I would think this would put a major strain on the server that distributes to the individual addresses.
The end effect of this is eventually bad, or utterly worthless.
Joe Sixpack wants to send a mail. If it takes him an hour to parse a key, he's not going to mail his mother anymore.
If a spammer has to spend an hour processing the key, he's just going to invest more of his time getting zombie PCs to get the work done for him.
Who wins here? Certainly no one.
Disclaimer: the hour was used as an example. I've no clue how long it takes, but the point should still hold.
The moral being, don't make the end users pay for the actions of spammers. We have laws for spammers now; it's time to start using them.
For example, Sourceforge sends site-wide update messages about once a month or so. They have tens, if not hundreds of thousands of users. If every one of those users used HashCash, Sourceforge would practically need a dedicated server farm computing hashes simply in order to send out its update notices.
This is a really, really stupid idea.
An awful lot of spam has been generated from machines infected by worms. If the spammer controls a thousand zombie machines, he'll have all the CPU power he needs...
You make me pay precious CPU time to e-mail my mother-in-law? you insensitive clods!
If you'd read the entire article before posting, you'd find your concern is addressed.
"Also, once minted, I don't want a stamp to be shared among every spammer who wants to send me mail. Therefore, hashcash takes two
extra steps (or at least recommends them as part of the protocol):
First, stamps carry a date. A user may decide to consider stamps older than a certain age invalid. Second, a hashcash client may, and probably should, implement a double spend database.
A double spend database is one in which each stamp may be used exactly once; if it is received a second time, it is considered invalid
(much as with a postage stamp after it is marked as processed)."
The author points out that a) a date is added to the string to be hashed and b) a database is kept for the day of hashes already used.
If you include the hash when you pass it out, step a) invalidates hashes of older days and step b) keeps the current days hashes from being reused.
So it doesn't matter if the spammers share. The hashes are one-times.
You are checking your backups, aren't you?
...because I was out of hash cash.
Funny, isn't there a Microsoft Research project that did this already?
Oh yeah, so there is, along with papers explaining how it works. So much for giving credit for prior work.
My other car is a cons.
I gotta read these headlines more carefully. I always thought SPAM was popular BECAUSE of people using Hashish. I would lean more towards doritos but thats just me.
If you want a virus built to generate stamps on zombies, just go over to Spamforum.biz and advertise for one. New ads over there this week include "PushMail Webmailer v1.0.2 ~ New, Fast WAP Webmailer for Sale (Gets by Filters)". There's even a banner ad for a firm that wants spammers: "3 different sites - Pharma - OEM - Cigarettes".
I think it would be great if /. implemented this. If posting took 200 sec of computation then only people with something interesting to say would bother. Much less spam and karma-whoring.
:)
Flamewars would be restricted to people at work posting from supercomputers
-J
Mailing lists are specificly dealt with in the list of frequent concerns.
We bought a vanilla smtp server for our gateway called Xwall. A few months ago they introduced greylisting.
Basically what it does is temporarily block suspicious emails. If it's a real SMPT server it will resend the message and the second time it will be allowed to go through. Spammers never use RFC compatible SMTP servers and simply send once in bulk and forget about it. This cut down our spam by over 90%.
Fur sail 2 u nou: 5 mil-leeun facter numberz
Yuz cun b-u-l-k f4ster wit dis CD uv all-ready calcoolated leest uf numbors. Fer onlee $99.95, u getz ohver fiv milyun numz ant wee tos in freeee a miliun fresh A-O-L addys. Vizut us @ hotprimefactors.biz to ordur.
now we need to go OSS in diesel cars
It seems to me that spam is just about unstoppable. As such, I find the best solution to the problem is to run a smart filter that learns as it goes. I run Mozilla Thunderbird and it's bayesian junk filter is damn good. I simply do not get spam in my inbox. I get a false positive once a month or so, but that is certainly acceptable. Yes yes.. That doesn't cut down on the congestion caused by bazillions of spam messages, but eh.. I still get 5 MBps to the house.
What is your penile percentile?
How about allowing a user to place a site on a list of sites that will be asked for the same hash (randomly chosen for each site) everytime? This means that sourceforge simply needs to save the hash and send it with subsequent emails.
Sort of like burning your harvest to keep grain prices high. Just send me a completed work unit of Seti-At-Home or Folding-At-Home in an email header. I am sure, given the incentive of every e-mail message advancing their goal, some of these projects can come up with work units that are difficult to calculate but easy to verify.
Maybe for once zombied Windows boxes will be more productive than they would be under their users' control.
the fact that not everyone is sending legitimate email with a powerful computing device. Something that could cause an inconvenience to a spammer with a boatload of cheap commodity 2Ghz desktop systems (other their own or a zombie army) will bring more modest systems to their knees. Handhelds, phones, old 486 systems recycled for use in the 3rd world, set top boxes, embedded systems, etc. will no longer be viable systems with which to send mail. And what about web mail providers?
These's simply no reason to resort to kludge solutions that depend on penalizing those who cannot afford top-of-the-line systems.
To me greylisting seems like the best thing to do. See:
g .html
http://slett.net/spam-filtering-for-mx/greylistin
and/or:
http://projects.puremagic.com/greylisting/
In a nutshell, it simply uses a standard 451 SMTP response that says "Hey, I'm busy now, can you call back in a minute or so?" To my knowledge, all standard SMTP servers respect this request, and little to none of the mass mailers do. And if they do, their bandwidth will triple.
Here's a log example:
Oct 15 15:18:17 example1.example.com sendmail[6955]: [ID 801593 mail.info] i9FJIGH06953: to=, ctladdr= (168/601), delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=121994, relay=example2.example.com. [123.390.141.456], dsn=4.3.0, stat=Deferred: 451 4.7.1 Greylisting in action, please come back in 00:01:00
If the mail never comes back, then the sender is now blacklisted. If the mail does come back, the sender is whitelisted.
Simplest and most standards compliant thing that I've heard of, and it seems to work.
It's easily solved. Just buy the CD of pre-calculated prime factors from the spammers.
now we need to go OSS in diesel cars
For these hashes, you cannot work on the complete hash space, otherwise it would take forever for someone to send a message because of how long it will take to find the hash. That means each message sent will have a subset of the hash space, or (more likely) large portions of the hash space will go unused.
If you're using the hash space uniformally, then armies of infected Windows PCs will take just a couple seconds per e-mail. What does the spammer care? Those CPUs are free/cheap. Just means it's time to find a way to compromise more machines.
If you're only using a subset of the hash space, store the results of each hash you try. Then, the next time around, finding the result is near instantaneous... Making the scheme innefective.
~D
This sig has been enciphered with a one-time pad. It could say almost anything.
Even easier than that. At 20 bit values, we're not talking very many different numbers here. These can be pre-calculated in a few hours, packaged on CDROMs, and sold to other spammers. Yet another way to make money on the net.
now we need to go OSS in diesel cars
Can't we come up with some workable system where the sender has to crank through a few iterations of seti or folding at home to pay for accepting the email? (I seem to recall a suggestion about cross-checking against another sender/server, but don't remember how they prevent cheaters.)
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
3.243F6A8885A308D313
I always liked Spam and Hash for breakfast when I was in Hawaii. :P
Only unsolicited mail needs a hashcash field.
Wikileaks, no DNS
This is extremely stupid.
If it works, users of email will put up with it.
Ideas similar to yours are easy to come up with, yet none have ever been shown practical
The only argument I see against the practicality of this solution is the point above, which is circular. An argument which depends upon a circular argument is also circular.
Armies of worm riddled broadband-connected Windows boxes
In my opinion, this is a strength of hashcash rather than a defect. Consider the ratios. I send maybe a few hundred e-mails a year to strangers. Each spammer wishes to send hundreds of millions. There aren't actually enough zombies to go around. The key is to allow the recipient side to adaptively adjust the amount of postage demanded based on multiple factors until there is no free lunch remaining for the spam kings.
Sorry dude, your spam response form has only served to add to the noise floor, not reduce it.
Furthermore, the whole premise of the spam form response is that a huge design mistake made by Unix hackers of old in the design of the e-mail infrastructure is beyond our collective willpower to fix.
I say: grab and backbone and deal with it. I'm embarrassed to be a member of such a pansy industry.
Spammers never use RFC compatible SMTP servers
And spammer tactics remain static, so the same techniques that worked five hours or five years ago will continue to work indefinitely. Not.
My next sig will be ready soon, but subscribers can beat the rush
I don't get this. It will just lead to a general slowing down of the services running on the Internet.
Where do people think that email is being sent from? A dedicated server that the user has somewhere dedicated solely to sending email?
Most people will be sending email through their ISP. And ISP that was coping with x,xxxx,xxxx pieces of email a day will suddenly now have to redo their email architecture to cope with the extra computational cost involved.
Other people send email using their webhost. The extra computational overhead will now mean that other users on the server will not have as much CPU to utilise and their sites will work more slowly.
What needs to happen is for more emails to be signed with people's digital identities. Then someone needs to create a network/service where people can 'vouch' for certain identities. Thus you can build up an associative trust network. And you spam filter can may a more informed judgement call on the validatily of the email it processes.
--
Fast Linux Virtual Private Servers
I dont know which spam is worse email email spam or spam the canned ham. Then to deal with eitther you have to get some hash
Dude I hear they invented this horseless carriage too! BIG NEWS! FILM AT 11!
I've read the article and what I have read doesn't impress me. So, you added cost to the spammers. Have you seen how much money these bozos make? Sorry, I think everyone is underestimating the cost.
IANAL, but I've seen actors play them on TV
(x) Microsoft will not put up with it
a ck/demo/lbdgn.pdf
Except that Microsoft are *ahead* of the hash cash scheme. They've developed a scheme that does the computation with something memory intensive.
Main memory is much much slower than the CPU and the difference in memory access speeds in a cell phone and a PC are much less than their CPU speed.
Memory based computations are harder to run in parrell. In principle you could have many computers working on signing a single message.
They've made is very difficult to run their algorithm in parrell. The Microsoft scheme is much better.
More information here: http://research.microsoft.com/research/sv/PennyBl
Simon.
Someone with a valid stamp is less likely to be a spammer. Simply include it as a factor when calculating probabilities!
Or ignore the X-Hashcash field completely. As you choose.
If you read the article, you'd see that this was precisely the way in which SpamAssassin uses hashcash : as one factor amoungst many in a general system of spam classification.
Wikileaks, no DNS
It's that in order for this to be useful, it has to be widely implemented. Anybody who sends a lot of legitimate email (e.g., hotmail) is going to need to buy a lot more CPU. So it's not going to get widely implemented. So it won't help. Sorry. :'(
Whitelisting is a simplifying concept, which one can understand more subtly as another factor to be accounted for in calculating probabilities, making your Baysian engine that much more efficient.
Wikileaks, no DNS
Why make spammers (and everyone else) pay in CPU cycles that accomplish nothing but waste time? Why not keep public keys in your address book, with a filter rule that runs a header's token against it, which could only be encrypted by the matching public key - authenticating the sender? Instead of separating the rich spammers (with distributed zombies spinning their cycles at someone else's expense) from the poor ones, we'd separate the authenticated sender from the spoofs, and put everyone not in our address book into the "bulk mail" folder.
;).
These CPU-wasting schemes are just a way to sell faster CPUs to nongamers who don't need them. While authentication uses the CPUs to better effect, improving the entire messaging system (and offering integrated encryption as a byproduct). Don't waste our time with counterproductive distractions (except Slashdot
--
make install -not war
I consider mailing lists a cute throwback to a much earlier time. Don't get me wrong, I subscribe to three or four myself. But every single one of them, I could just read on-line (and no, not all Yahoo lists, only one in fact).
To effectively eliminate spam, I would gladly visit a web page rather than have the same info appear in my mailbox.
Er... How does that differ from actual spam? I don't give two shakes of a rat's ass whether or not UCE comes from a "legitimate" source. I don't want it. Any of it. So, it really doesn't bother me that, for the benefit of no more "Free v1@6ra" email, I also lose out on "buy our totally legit ink cartridges" at the same time. I consider it a perk, not a problem.
The problem with technologies like this is that they need to gather widespread acceptance to become useful.
Quick grep on my mail archive (which is HUGE) failed to find single message with X-HashCash header. That means even if I would enable it now, it will be practically useless.
Of course wide acceptance could be achieved by the means of widespread grassroots campaign, but this is hard way. If somebody big like GMail, Yahoo Mail or MS Outlook or Apple Mail started to use it , that would have snowball effect.
Rather than yet more hackable encryption gimmicks, why not just impose standardized postage fees (estamps). It would be similar to paper mail: Sender pays; and receiver, ISP, and gov agengency all get a cut of the fee. This gives the ISP and gov financial incentive to hunt down abuse. If somebody sends without paying properly, then they don't get their cut.
Table-ized A.I.
Wow, they can see the future!
They've been telling people for YEARS that anything under the top-of-the-line computer won't be able to send email or brose the internet!
Job? I don't have time to get a job! Who will sit around and bitch about being broke and unemployed then?
reminds of what I used to call my student loans while in college...
100% Insightful
Although Bayesian method(s) are primarily used for fishing out spam, why not use them for general purpose mail-sorting? The databases can be trained and each message checked for user-specific categories: "Family", "BSD", "Support", "Online-Orders", ..., and -- of course -- "Spam". Messages, that don't fit into anything will continue to arrive into the main mailbox.
In Soviet Washington the swamp drains you.
Sorry, I replied to the wrong thread. you may all hurt me n... uh, ignore me. that's it. Ignore me.
The author mentions using this to prevent wiki defacement. This could already be done with some javascript and it could be browser independent. Just add some javascript that would hash the message, some part of the URL or page, or a salt and that would be a required part of sending.
It would require javascript enabled (or with a logged in user, you could remove the hash requirement), but it could also provide a seamless integration without any real changes required for the user.
This is already covered on the Frequently Raised Objections page.
Patrick Doyle
I mod down every jackass who puts his moderation policy in his sig. Oh, wait a sec....
Jesus, people, read the fucking article before commenting at least once in a while.
Why the hell does the embedded timestamp contain the year remainder mod 100 instead of just the year??
Sure, a zombie PC can compute a hash in just a couple seconds, but that's a few hundred emails per second fewer than it would be sending otherwise. Instead of sending 200 emails per second, it's sending only 1 every 2 seconds.
Since there aren't hundreds of compromisable non-zombie PCs for every zombie out there, this would necessarily reduce the spam by orders of magnitude. Of course, that would require adoption, which would never happen.
aQazaQa
He's not on broadband. It takes quite a while for his anti-virus to check outgoing mail. By the time the mail is actually being sent, he's often hung up.
So it costs him both time and money (extra phone call).
This would make the situation much worse.
The majority of home users with net access in the world are still on dial-up - yes, that is changing, but it's still the case.
If you'd take a moment to settle your knee down and read the very page you linked to, you'd see that PennyBlack shows HashCash as a prior work. Look. It's right there, at the bottom, reference B-02.
So much for giving credit for prior work, indeed.
Jeremy
Looking for a Python IRC bot?
We use Lotus Notes at work and I have no trouble E-mailing my greylisting server at home. Our mail relays happily delay the message for 6 hours and then resend it.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
No problem, calculate the hash offline, go online when it's complete. Yes, it's a pain, but so is putting a stamp on an envelope.
It stops people from sending emails to large numbers of people, hence it stops spam.
It stops people from sending emails to large numbers of people, hence it cripples mailing lists, solicited commercial emails and newsletters.
Lets go back to SPF + List of spamming domains to save our pain and ignore the chaff.
I don't get this. I'm getting spammed by jerkoffs who've gotten my email address from friends who use Windows computers. The emails are apparently from people I know and who I don't want to filter.
How does hashcash keep this spam out of my box? I _want_ the emails from the friends, but not the spoofed email.
Why not have it compute the stamp before you send the mail? You start a new mail window, that least intensive of applications. In the background it calculates the stamp while you type.
Under that system, you could make the stamps as much as a minute. Very few e-mails are written in less than twenty seconds, most take a few minutes. Really short messages go via IM. You still queue it to go after the stamp is ready to deal with the short e-mails, of course.
Now, with that said... I should point out that the real error in this system is that spammers will just build a database of known hashes.
My personal belief is that the only viable solution to spam is a whitelist augmented with a CAPTCHA challenge-response system.No, apparently they weren't.
I'm not a smorgasbord.
I believe the article writer underestimates the power of zombies, the number of zombies, the power of a roomfull of modded XBoxes bought used once XBox2 arrives, and the arrival of FastHashCash, in whatever form someone manages to create it.
I remember how the original Unix 'crypt' function was supposed to guard against brute force password attacks by taking a significant amount of a second to return each result. fastcrypt arrived later, and rather killed that defense mechnism.
While I applaud every effort to squash spam, this seems to be no silver bullet.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
From the hascash.org/faq website:
I tried to do this for a relative so he could send mail via my server regardless of which ISP he used (he travels a lot). Within hours a spammer had found my open relay, and within a few more hours my server had been blacklisted. I setup the ESMTP password extension to keep the spammers out, and deleted all the queued spam as soon as I saw what happened, but I think my server is still on the blacklist. I made efforts to get off the blacklist, but did not receive any reply.
Explain to me what would possess someone who seems to possess a good deal of knowledge about e-mail and computer to setup an open relay? There just is no reason and he DESERVES to be on that blacklist. What was so hard about the ESMTP password extensions that he didn't set them up first? Good grief.. this guy is insane.
it's still a win, simply because the number of spam mails sent out is still reduced -- even if there are lots of zombies computing hashes, they take longer to send out each individual message compared to zombies which aren't forced to compute hashes.
HAND.
If spam gets to your Bayesian filter (regardless of whether it's filtered or not), that means that some of your bandwidth is being consumed to transmit that spam. If spammers are forced to compute hashes (thus lowering their total output), less of your bandwidth is wasted on their crap. You still pay the same for your bandwidth, it's just that you get to use it, not you+spammers.
HAND.
Give me a break
Just to give some practical information:
I'm using hashcash in its basic form, not with Camram. I wasn't aware of Camram until just now, but will probably look into it.
All my emails are sent out with hashcash, and I have SpamAssassin lower the score of emails with hashcash.
The recommended hash length is at least 20 bits. I calculate hashes of 23 bits (per recepient), which takes about 2/3 sec on my Athlon 800. My SpamAssassin config requires at least 20 bits to lower the score, and lowers it more and more up to 26 bits (at which point it has -5).
I think that this is the most effective use of hashcash: once it becomes widely used, then spam rules can become tougher with less chance of false positives.
From reading the article, it looks like Camram is mostly a recipient-side addon to basic hashcash, which involves automated whitelisting and sending challenges to senders of "maybe-spam". Somebody sending hashcash like me will (from the look of things) get past Camram recipients without problems.
Camram seems a bit less cooperative than I'd like, such as using its own Bayesian filter instead of letting the user have an external one like SpamAssassin take a crack at the email. But these are implementational issues, not problems with the Camram concept.
If your filter is so good, instead of manually creating a whitelist, let it decide from the content of each message whether to challenge the remote server or not. You will get no more spam than before, and spend no more time than before, yet you'll make life a lot harder on a lot of spammers, and perhaps help stop the problem at its source. Duh.
Adoption will be slow. Many companies already have maxed out mail servers. Adding even an 1 second compute cycle to all outbound mail requires a fairly hefty increase in available resources, especially since most mail systems are chosen for bandwidth and IO not math processing power. What happens to a system during peak business hours when 100 people send mail with an average of 5 recipients each ... 500 seconds of computing ... ummm. Imagine a company that sends 5000 messages an hour, or 50000, or ...
If it's not at least a second on a reasonable machine than it's not going to cause ANY headaches for a spammer -- they are just text pumps they can send SO much more mail than a normal server because they don't care about logging, errors, bounces, rejects and retries.
The "use clients inside the company" idea is idiotic -- my mail server is going to punch through the DMZ directly to the desktop of my accounting staff and ask it to generate a key? I don't think so. There is a reason every company with any brains bans Seti/IM/etc. from their internal desktops.
Zombie writers will just interleave writing packets of the current message with SHA-1 calculation for the next message they are sending. Spammers have some really good programmers on their side. If you don't think of them as being at least as good as you are then you have already lost. They are already generating random text at the front and back of the payload, this isn't SHA-1 thing isn't a big deal.
Like SPF, spammers will be the FIRST people to generate proper keys. For the near future a valid key will be a STRONG indicator of spam not a "potential whitelist" feature.
Is that the case ?
The Internet's nature is peer to peer - 20050301_cs_profs.pdf
What needs to happen is for more emails to be signed with people's digital identities. Then someone needs to create a network/service where people can 'vouch' for certain identities. Thus you can build up an associative trust network.
:o)
Congratulations, you have just described a social solution to a social problem.
What these (hashcash/SPF/whatever) people fail to realize is that spam is a social problem, and you can't apply a technological solution to a social problem.
I'm glad at least ONE person gets it though.
I have a better idea for getting rid of spammers - beat them at their own game.
All we have to do is send out millions of legitimate e-mails to everyone, asking how their plants are doing, or what kind of turkey they're going to eat at Christmas. Eventually, people's e-mail will be so clogged up with messages from friends and colleagues that they won't be able to find the spam through it all. Misleading subject lines, such as 'Grow them six inches' or 'girls want more meat', to use examples for the above headlines, could be used to make recipients believe the message is spam, when it's actually well-intentioned correspondance.
Within a few years, people won't be able to locate actual spam from which to purchase products or services, and the spamhounds will shut down, defeated at last.
It's so simple, I don't know why no one's thought of it before.
--Dan
Judging by the +3 and higher comments, it seems that nobody is thinking outside the box. There is no mutual information between an e-mail not having a hashcash stamp on it and being spam. However, if an e-mail has a valid hashcash stamp, it's probably legitimate. Thus, while hashcash can't really help your spam filter reduce false negatives (spams that it lets through), it helps reduce false positives (legitimate e-mails that are blocked).
I personally stamp all of my outgoing e-mail with 20 bits of hashcash postage. It's easy to do and requires very little CPU time. Here's how I do it:
I have stunnel listening on port 465 which forwards connections to MEsmtpd. After authenticating the sender, MEsmtpd pipes the message to hashcash-sendmail which adds 20-bit stamps for each recipient to the e-mail and passes it on to sendmail. I don't have to do anything at all in my e-mail clients. There you have it, easy as pie.
Regarding that stupid "your spam solution won't work" checklist, Spam classification is a hard problem. It can't be solved by any one approach. Even though Hashcash won't stop any spam, it can still make your spam filter more effective.
P.S. SpamAssassin supports Hashcash. See Mail::SpamAssassin::Plugin::Hashcash.
Why not have it compute the stamp before you send the mail? You start a new mail window, that least intensive of applications. In the background it calculates the stamp while you type.
Under that system, you could make the stamps as much as a minute. Very few e-mails are written in less than twenty seconds, most take a few minutes. Really short messages go via IM. You still queue it to go after the stamp is ready to deal with the short e-mails, of course.
The reason this will not work is due to the way a typical SMTP connection actually works -
Steps:
1 - User writes email
2 - User sends email to their ISP's SMTP server
3 - The ISP SMTP tranfers message to destination SMTP server
4 - Destination SMTP server delivers mail to destination mailbox
5 - Profit (just kididng)
The checksum check will actually occur at step 3. Destination server will request the checksum from ISP's SMTP server - NOT FROM USERS MACHINE. Which means that the cost to large or even medium sized ISPs will be very significant. This means unless the end user machine will start sending email out directly to destination ISPs (bypassing step 2, a practice some broadband providers block to curb spam bots), this scheme will cost significant amount of money to ISPs in processing power. This also means that what you propose - calculation of the checksum on user machine during writing of the message is impossible.
True solution to the issue (or at least BEGININGS of a solution) should start with authentication of authorized SMTP servers for domains - like what Yahoo/Google/Microsoft & others were trying to do via DNS a few months back. (Whatever happened to that, BTW??)
-Em
RelevantElephants: A Somatic WebComic...
Welcome to the real world, where usually no solution is ever perfect. Instead, we try to reduce problems as much as possible
(*) Mailing lists and other legitimate email uses would be affected
No. You specify how many bits a particular source needs to spend, and therefore mailing lists that you know you are on would not be affected.
(*) Users of email will not put up with it
Users of email don't even have to notice.
(*) Armies of worm riddled broadband-connected Windows boxes
They are increasingly being filtered out by their providers anyway.
(*) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
While, of course, manually sifting through megabytes of spam or living with a significant number of dropped messages due to content-based spam filtering is completely "practical", right?
If the spammer sets up a mail server and finds a way to get external servers to send it mail, it can send spam via this server and then pass the hashcash requests from it's outgoing connections to it's incomming connections. In this way the spammer doesn't have to calculate a single hash - it distributes the computations among all of servers sending it mail.
How does it get lots of mail from external servers? There are lots of ways, but the easiest would be to sign up for mailing lists. There are millions of them.
This is not a solution. This is another way to tie me down to manual editing of whitelists. I already have to do that with current content scanners. Any spam solution that fails to improve upon current solutions by mitigating the need for a whitelist is not an improvement. They are all sufficiently accurate to identify virtually every spam, and what slips through must be so innocuous in order to evade the filters that it isn't much a bother. The problem is the false positives and having to maintain whitelists to deal with that. I don't need more true positives. I need fewer false positives. Hashcash does not solve this problem, so there's no reason to adopt it.
Edith Keeler Must Die
Ok, how about this one:
1. Users keep contacts list on a mail server. Mail server blindly accepts incoming mail if it's in your contacts list.
2. Non-contacts email get bounced with a turing test request. Sender has to effectively authenticate themselves as intelligent enough to pass.
3. Once authenticated (for this single email), the message is delivered.
4. Recipient has the option to add new contact to the list. Otherwise subsequent emails get bounced the same way.
So, this solves bulk-spam and allows new contacts to trivially send email to people, but at the same time completely blocks bulk spam.
Thoughts?
Although I think SPF is also a good idea (and would complement hashcash nicely), SPF tends to create problems for folks running small-time domains. For instance, my webhost doesn't give me the ability to set the txt record in my DNS.
Find free books.
Most of those devices (actually, most desktops period) send their mail through an MTA. Have the MTA add the hashcash, optionally requiring the device to do SMTP AUTH.
Brent J. Nordquist N0BJN
http://shit.slashdot.org/article.pl?sid=04/11/10/1 811251
And the idea here is to make them pay that price to someone else (maybe even themselves)? Great...
Any "payment" solution should preferrably result in the recipient getting paid. If I'm to tolerate the junk being sent my way, I'm not satisfied with seeing proof that some hardware vendor has earned money on the affair; I want that money myself as compensation.
In order to enforce a processing delay on each e-mail message, I find it much easier to have my own e-mail server keep the SMTP session on hold for a suitable number of seconds, than to design a computationally expensive algorithm for the sender to execute. Then the spammers can invest in as much hardware they like; they will still have to stand in line along with everybody else in order to talk to my server. By enforcing the delay uniformly against all clients (except perhaps a few whitelisted friends), a zombie network will not offer much of an advantage over a single host to the spammer.
Instead of having the sender pay for more hardware, the recipient should be allowed to pay for less.
It's an intriguing idea, but as others have pointed out, not that many computational tasks lend themselves to quick and inexpensive verification. Another problem is that you don't have any easy way to verify that the Seti@home work unit is only being used to send you this particular e-mail, rather than being used on a million different spams.
But if you think a little more broadly, the implication is that this would create the first open market for CPU cycles. Spam would turn into a legitimate business, just like paper junk mail, i.e., you would only get a few spams a day, and they would all be from people who had actual products to sell (not Nigerian scammers, fake pills, etc.). The new legitimate spammers would be willing to pay people for their spare CPU cycles.
When someone buys a computer, they've invested, say, $1000 to buy a few years worth of CPU time. That investment is then almost all being wasted. When I'm not using my computer, but it's turned on, 100% of its CPU cycles are being wasted. When I am using my computer, 99% of its CPU cycles are still being wasted.
This could create a whole new cottage industry of people farming out their machines to compute hash cash. The good news is that individuals could make back enough money from their investment that they could get the computer essentially for free. (Assuming an efficient market, spammers should be willing to pay you very close to the cost of your machine in return for 99% of its CPU cycles until it becomes obsolete, because that's essentially what it would cost them to buy such a machine themselves.)
You'd pretty quickly get a lot of inflation. Once the market was really functioning efficiently, the cost of CPU time would go way down, because there would no longer be 99% waste. This would give people an incentive to raise the bar on the hashcash computations they'd demand in order to send them an e-mail, and it would also give people an incentive to buy faster computers. (The hardware manufacturers would love it!) But now an interesting thing is happening: you have a worldwide distributed supercomputer that is getting bigger and bigger, driven by economics.
Then the real question is what all those CPU cycles would be used for -- worthless computations like hashcash, or interesting ones, like seti@home. Personally, I can think of one really good application for all that CPU time: real artificial intelligence, like HAL in 2001. With, say, 10^8 computers worldwide available for parallel computation, I think it should actually become feasible to do something like a direct simulation of a human brain, in real time.
Find free books.
- The result of the trusted unit is right.
- The result of the new unit differs from all results to different workunits.
- He answered the challenge in time.
the mailer is allowed to send mail else he will be blocked for some time.
Option: If b units have been replaced, exchange the set of "trusted" workunits completely with a trusted other mailserver.
Some ideas about attacks on this:
First, the attacker needs to ban some of his zombies by trying to find out which units are "trusted" and which are "new".
Poisoning the trusted units would be pretty hard, because it needs at least c zombies and no "honest" person trying to send mail in d hours.
Its possible though to keep the server from getting new trusted workunits by sending false responses for the new units. The server could notify the admin in this case, because it it a pretty save sign of an attack. In this case the attacker still need to calculate b workunits and 1 workunit every f minutes - pretty expensive for spammers.
(The server is under load too, while being attacked, but not more than the attacker. Without the attack there is not much additional load.)
What did I miss here?
Are there any other contenders for the most obnoxious recurring duplicate story? This one has come up so many times in the past couple years that it's not even funny. There are others in that category. Which one is the worst?
Are we really so hard-up for news that we're posting yesterday's failed spam solutions today? Why not post a story about breaking the color barrier in baseball - it may not be relevant to the site (although that's even questionable lately), but at least that one worked.
One thing that caught my eye:
X-Hashcash: 1:20:040927:mertz@gnosis.cx::odVZhQMP:7ca28
^^^^^^
Why are they doing that mistake over and over again? They are talking about a method that's computationally expensive, yet they "save" 2 bytes by using 040927 instead of 20040927.
Of course it runs NetBSD. BTC: 1NT7QvbetmANwaMzhpVL6
What's your point? Using humans to compute hashes for give e-mail addresses? Letting the spam victims input their data themselves?
Either approach is silly on a large scale. TFA says that computing a hash on a modern CPU takes less than a second. If you provide the infrastructure to serve an input form that deals with the needed amount of hashes, you can as well compute them yourself. Let alone getting enough users to actually visit your form.
I don't get it.
Of course it runs NetBSD. BTC: 1NT7QvbetmANwaMzhpVL6
Thie whitelist here is for a different idea.
Whitelist merely means that the sender could send message WITHOUT having to calculate the Hash value.
Basically, senders falls under two category.
Whitelisted - Receives without having to require a calculated hash.
Non-whitelisted - Need to have a hash on header to be accepted.
In US, you can easily buy enough major firearms to wipe out your neighbourhood but a few little fireworks are banned.
You could do your own version of that without the SPF - if foo@foo.com gives you a hashcash from IP address 1.2.3.4, you could whitelist any mail from foo@foo.com at that IP address (or probably even that /24) even without an SPF header, but SPF gives you additional support.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Somebody who replied below said that network administrators spend lots of time responding to abuse complaints - but that's not true for users like him - he doesn't complain about spam that he never sees, and the people who should be responding to the abuse complaints are the senders of spam, not the recipients.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
SpamAssassin gives bonus points for hashcash hashes. (http://spamassassin.apache.org/tests_3_0_x.html) It'll be too expensive to calculate the larger hash values for spammers, so a 25 bit hash will really nock down your spam score.
User agents can start calculating hashes in emails without really any negative affects. Email software will ignore it.
As time goes on, more and more MUA will include hashes in the email they send and spam filters will give bonus points for correct hashes. It is not a silver bullet, but it makes it easier to differientiate spam from nonspam.
A different idea, as in different from what? The very purpose of whitelisting is to exempt certain senders (or their messages) from the restrictions placed on everybody else.
When Hashcash is presented as a solution, objections are raised against Hashcash for penalizing the wrong parties, and whitelisting is offered as a response to those objections, it really boils down to manually maintaining whitelists to avoid using Hashcash at all against anybody you want mail from. That's no solution, that's like inventing a car to obtain faster transport but have a man walk in front of it with a red flag to warn pedestrians and others about the approaching vehicle... Kind of refutes the argument for building it in the first place.
I have a simple solution to the spam problem: Just stop accepting any mail at all! One drawback is of course that it may affect legit senders, but you can whitelist those. It's not the ultimate solution, but it's a small step in the right direction! ;-)
Besides, you presumably need to maintain a whitelist anyway, so friends and other frequent correspondents don't get caught by your spam filters. (That's even more true if you're on a mailing list that discusses spam or spam tools, because such discussions commonly include spammy words and phrases even if they don't originate at bad IP addresses.)
if you're an email user trying to SEND mail to a Hashcash user, that's of course much more annoying, because some stupid robot is telling you you've got to do something hassly to get your mail delivered. Depending on how well the user interactions are designed, it may be more or less annoying (if some robot tells me that I have to install and run some piece of software on my computer to be able to send you mail, forget that! But if all I have to do is download some web page and wait until the Java applet gives me a number to type, that's only mildly annoying.)
Non-hashcash systems like TMDA typically make you either type in a captcha text or sometimes just reply to the email, which demonstrates that the real email you sent wasn't from a forged address, and they usually do auto-whitelisting. And people complain about them too, but unlike hahscash, they usually understand it.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Hashes are mathematical functions that make it easy to turn a specific input value into an output value, but make it very hard to find an input value that will produce a specific output value. So hashcash and similar functions specify an output and make the sender apply lots of brute force to find an input value that produces that output - but it takes almost no time to run the calculation once to see if the input works.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Sorry guys, but this is not going to work for a much simpler reason than anyone else here is listing... Hardware crypto.
While it WOULD be expensive for a spammer to do computations on their CPU, they won't do it... They will go out and spend $100 on a PCI crypto card, and other than using up a few more watts of power, they won't know the difference. Crypto is just one of the things that hardware is great at, and general-purpose CPUs aren't very good at, and that's not going to change any time soon.
Perhaps VIA (in addition to their hardware AES) will include a hardware SHA-1 accelerator, and make tons of sales to spammers. That's not going to drive up spammer costs significantly.
Also, this will only make a tiny difference in the speed of zombie PCs sending spam. After all, those 2GHz Windows machines have been network-bound up to now, so this will just mean the CPU meter going up is something else that will get ignored by the owner.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Finally! An answer to the actual question I posed, instead of an out-of-context answer.
You sir (or Ma'am) have made my day.
You are checking your backups, aren't you?
Someone has to mod this post through the fucking roof. This is an important aspect of the battle against spam, and very few people "get it".
In Soviet America the banks rob you!
You're being silly. I dare wager that I've expended considerably more effort in administering email systems than you have. But just to be clear : I *want* to solve the problem of Unsolicited Bulk Email. *Solve*, that is, not mitigate. And re-read my post. Would you conclude from it that I don't use such tactics on my own mail servers? Or indeed a range of other measures that sure, work quite effectively today, but likely won't work tomorrow?
Another example : some spamware chokes on multi-line 220 greetings - that's a handy tip that those in the know can take advantage of, but it's not a solution to the problem of Unsolicited Bulk Email. Ditto for secondary MXs that always respond with a 451. Indeed, the irony is that the more widespread such idiosyncracies become known, the less effective the tactics become. That's the nature of the current arms race, and half-baked solutions that don't actually solve the problem just lead us in circles. Hash cash is a half-baked idea. TMDA and challenge response are half-baked ideas. SPF is partially baked at best. SenderID inhabits an alternative reality. DomainKeys shows a glimmer of potential. Internet Mail 2000 is an example of something that I think actually attempts to *solve* the problem, but I won't deny that it's anything other than radical.
So please, for everyone's sake, don't stop showering.
My next sig will be ready soon, but subscribers can beat the rush
If the postage is 20 bits, then that's only a search space of 1 million. Just precompute them all (would take less than 12 CPU-days) and you can answer any question in O(1).
The way I read it, the postage is the number of leading zero bits in an SHA1 hash, which is 160 bits long. In addition, the recipients email address is part of the string to be hashed, so I don't quite see how they can precompute this into a lookup-table.
Greylisting is when you configure your mail server to reply, the first time a message is sent to it from a particular originator, with a SMTP try-again-later message. This requires the upstream server to hold the message in their spool for a certain amount of time. The next time they try, it'll be accepted.
I use it. It works, brilliantly; it's reduced the flood of incoming spam from several hundred messages a day to about 5. And it means that the spam gets blocked before it gets transferred, which means it doesn't use up my (or anyone else's) bandwidth.
Get more information here. My favourite greylisting SMTP proxy is Spey, but then, it would be: I wrote it...
No. What it means is that you haven't bothered to understand hashcash before you complained about it on /.
What you call a "checksum step" is really a hash collision generation step. It can occur either before step 2 or before step 3. If you run a small outgoing mail server, you can have it add hashcash stamps to all mail that doesn't already have them. If you run a large mail server, you obviously have to pass the mail on without hashcash, and it's up to the sender to generate the hashcash stamp.
All my outgoing emails contain hashcash, and it works equally well whether it's sent to the recipient's server directly or through my ISP's mail server (for several domains that refuse direct smtp from my netblock).