Slashdot Mirror

http://www.hashcash.org/
http://pthree.org/2011/03/03/the-sad-state-of-hashcash/

This type of proof-of-work algorithm can ONLY ever catch on if someone seriously tackles the chicken-and-egg problem of ensuring it's supported in all mail agents AND clients.

The world needs:

-client hashcash plugins for: gmail, yahoo, thunderbird, hotmail
-mail agent plugins for: sendmail, postfix, exim etc.

and they all need to be plug-and-play.

This is known as hashcash. One big reason that it doesn't work on the web is that, currently, users will be stuck with some slow JavaScript version of the algorithm, while a sufficiently determined spammer can use a fast C version, and end up with much less work required to post. So it's nearly impossible to set a cost that is cheap enough for valid visitors, that will be a sufficient deterrent against spammers.

Make 'em work for it!

A cleaner version of the link of thing you suggest is hashcash. The idea is that you force anyone using your service to invest certain resources, with the idea being the investment would be acceptable for a single user, but unacceptable for a massive attack.

The problem with hashcash, though, is that computing power is dirt cheap, especially in this day of botnets. The Storm botnet, taken as a whole, peaked last year as one of the world' most powerful computers.

I think we'll be able to come up with a captcha system that works reasonable well for reasonable periods, making using word problems, cultural questions, or some kind of clever pattern recognition problem. (Of course, any captcha is going to discriminate against somebody: the blind, the deaf, the dumb, the ignorant, etc. Unfortunately, that's a fact of life.)

I think we're better off in the long run destroying the economics of spam than continuing this arms race. Unfortunately, destroying the economics of spam requires regulation and legislation.

The porn hole is still a big problem though, and there's really no way around that. You can think of various cryptographic schemes, sure, but fundamentally, a capcha still relies on something transmitted to our sense organs. And what we can transmit, we can easily record and replicate elsewhere.

HashCash http://www.hashcash.org/

It penalizes the big guys instead of the small guys, that's why it hasn't taken off. Also, no one seems to want to promote any solution that doesn't put somebody in control of something.

I think you're referring to something like HashCash. Sounds interesting, but I'm not convinced it would work.

I'm not really surprised. Would you trust someone who plonks an open relay on the net (1c) and then wonders why he ends up in an ORBL? I can understand if it was a mistake but, to further incriminate himself, he goes on to say he then set up SMTP_AUTH (which should have been done in the first place and also proves he could have done it correctly) and moans about "blacklist operators and ISPs... intentionally sabotag[ing]" his poor mail server. I would laugh if it wasn't so tragically typical.

Rearrange: Himself. Blame. Got. Only. To. And shit happens.

The goal of CAPTCHAs in most situations is to make the business of using a bot not cost effective.

You can do this by slowing the bots down, and not stopping them entirely.

Humans will wait 30s to enter a site they should be going to, this is death to bot operators (even with large botnets). Like what hashcash does for anti-spam.

An example, is here to protect email addresses.
Yes, you could write a cleaver tool to do the math in compiled C and not JavaScript, but the cost is still there.

One word: Hashcash. Basically you prove that you wasted a couple seconds worth of CPU to send your message. I believe SpamAssassin already recognizes Hashcash headers, not sure about other filters. But if you're really ready to start dropping email en masse in favor of a whitelist-style approach, this is the simple and elegant solution.

What about hashcash support?

Hashcash is a denial-of-service counter measure tool. Its main current use is to help hashcash users avoid losing email due to content based and blacklist based anti-spam systems.

A hashcash stamp constitutes a proof-of-work which takes a parameterizable amount of work to compute for the sender. The recipient can verify received hashcash stamps efficiently.

From the hashcash.org site:

"Hashcash is a denial-of-service counter measure tool. Its main current use is to help hashcash users avoid losing email due to content based and blacklist based anti-spam systems. A hashcash stamp constitutes a proof-of-work which takes a parameterizable amount of work to compute for the sender. The recipient can verify received hashcash stamps efficiently."

It's not going to affect normal users that much (except maybe list services) but by marginally increasing the cost of sending out thousands of messages at a time, it alters the economics of spam and makes it a non-viable way to do business.

Once I set this up on my Wordpress blog, the level of comment spam dropped to zero *immediately* and I haven't had a single incident since.

Laws are a dumb idea because they don't actually prevent spam, just deter it, and only if the spammer cares. I don't think we need to rely on possible spammers to have good common sense.

What we need to do is make it more expensive for spammers. Hashcash has been mentioned several times before and it's probably one of the best ideas. It simply makes sending massive amounts of email very expensive while costing the average person little to nothing.

iRate and TiVo are really the same idea, so let me suggest Penny Black. Sorry. Hashcash, Instead.

Note that Hashcash was first, but that Penny Black 'looks' more like innovation because of the marketing presentation. Hashcash, due to its straightforward presentation, appears as a mere technical fix. And this is how most Open Source invention and innovation is hidden, IMO.

And that is why I wish every emailer would support hashcash. Make the sender's computer perform an expensive computation to send a valid email. If it took 30 seconds to a minute for the fastest consumer PC to create a valid stamp, mass spamming would be nearly eliminated. If spammers got faster machines, just increase the strength of the hashcash stamp required to accept their mail. This, in combination with whitelists, would nearly solve spam.

Try http://www.hashcash.org/ It's a way of making a postage stamp for e-mail, allowing safe passage through spam filters. You could ask your business partners to use this to get their mail through quicker.

Spammers won't sent you encrypted mail.

It is way too computationally expensive.

Spam programs are designed to work extremely fast, using very little CPU to send a message.

That is why things like hashcash would work, they'd make it economically unfeasible for spammers.

Encryption takes quite a bit of work (just less than unauthorized decryption :)

The same principle behind the anti-spam uses of hashcash could be used here. Google throws some javascript into the page, that calculates a hashcash stamp and add it to the requested URL whenever an ad link is clicked. URLs without a stamp are considered fraudulent and ignored. Javascript runs on the client side, so you take a few milliseconds or a second of their processing time. Barely noticeable to a person actually clicking the link, but it will put a dent in someone who's trying to perform thousands or millions of clicks per hour.

I'm amazed that I haven't seen more about Proof of work tokens for spam-fighting.

Proof of work tokens are hashes (like md5's) that take a relatively long time to compute and are very quick to validate. For most purposes, adding a few seconds to the delivery of email is unnoticable. For spammers, however, it greatly decreases the number of emails that can be sent out within a period of time.

Even though this does not completely eliminate the problem, it can significantly reduce the amount of time spent sifting through spam. Used in combination with public-key cryptography, it could even allow for mass-mailings from known users. (For instance, the Red Hat mailing list.)

The current problem with spam is a result of the fact that it takes almost no money to send spam. Increasing the amount of time spammers need to use in order to send out email is the only way to make a dent.

Links:

HashCash.org

Reusable Proofs Of Work
Currently down, but look at the google cache

(I'm new here so I don't know if this has been posted on every spam thread)

It seems to me that the only decent technical solution to this is something like Hash Cash, which has the end result of restricting the amount of mail a computer can send per unit of time . . . at least, it would be a good addition to any existing measures. How practical is this? Would it scale properly? Etc.

And although the process of exchanging "toke'ns" was highly "cryptographic", ultimately not a lot of work got done...

Anyway, I got confused there for a minute, but I am better now. This might help others:

From http://www.hashcash.org/

Hashcash is a denial-of-service counter measure tool. Its main current use is to help hashcash users avoid losing email due to content based and blacklist based anti-spam systems. A hashcash stamp constitutes a proof-of-work which takes a parameterizable amount of work to compute for the sender. The recipient can verify received hashcash stamps efficiently.

You might have a point if this scheme involved using money. In this case, however, the "payment" is a proof-of-work. The user is paying in CPU cycles "spent" to send the message.

They didn't develop the "payment" system, they use Hashcash.

-Peter

X-Hashcash: 0:040503:adam@cypherspace.org:271cc51dc3355f5a1b8f 092f

* Mailing lists. [...] if there is a way for legitimate mailing lists to bypass the challenge, then spammers can equally bypass the challenge.

Hashcash is generated for the mailing-list address. The recipient would add the mailing-list to their list of addresses they accept mail as, and a spammer can not send to the list without including hashcash. So the limitation for mailing-lists is that the spammer can send mail to many people (the list subscribers) for the cost of one stamp; if he sends directly he has to send one stamp for each recipient.

* Robot armies [of 0wned machines].

Clearly someone wit lots of owned systems can send lots of mail; but still less mail than they could without hashcash.

* Legal robot armies. [...] Large spam groups can afford purchasing hundreds of systems for distributing an computational cost.

They can do this (and doesn't matter with it's legal or not btw, they'll do it anyway), but it will cost them more per mail which will cost them, so they will send less mails and be economically incentivized to target their mails by buying demographic data etc. (eg. so you would be less likely to receive spams in languages you can't read, or on topics you are not interested in).

Another aspect is that legitimate users do not send mails to lots of new recipients; most email exchanges are conversations over a period of time with sends and receives. Some of the hashcash based systems use hashcash only for introductions, and exempt recipients from hashcash after that based on crypto tokens (or just whitelists) (eg CAMRAM, TMDA do this).

The argument here is that hashcash can be set to higher cost as it is only borne once per new recipient for normal users.

It would be a shame is Microsoft would go for a proprietary system. Especially since an Open Alternative already exists.

I heard some guy from Microsoft talking about some of MS's spam plans, after billg committed the company to stopping spam by 2006. They seem to really like the idea of hash cash, which certainly seems like the most reasonable bolt-on solution.

I think the best bet for Microsoft's anti-spam campaign would be to be as open as possible with the process. If they could come up with a standard for hash cash, enable it on every Exchange server, as well as provide it for every Sendmail, Qmail and Postfix server, they would have a huge PR victory. Everyone would be focusing on how Microsoft cured spam and they could start to shake their buggy image.

They've got two temptations they'll have to avoid if they want to win this battle though. The first is their culture: they're notorious for only using standards when it suits their needs. They need to be political about getting the standard accepted everywhere, which means playing nice with the Internet as a whole. The second is to try and use this to throw their monopoly weight around. If they say "only Exchange servers can user our powerful anti-spam techniques" people will turn off the spam protection so that they can get mail from Linux mail servers. I'm pretty sure they're too smart for the second one.

Basically, this is intuitive to most Slashdot readers. Open networks are bigger than closed networks and a network's value is exponential of its size. If MS can make an open spam solution they'll have helped build a very valuable network.

The camram project is very close to releasing 0.2 which will make available a hybrid sender pays system which will work for systems handling a single user through a few hundred users. With this release will also come the information of how to convert any content filtering antispam defense into a hybrid sender-pays system like camram.

As of today, 3 systems support sender-pays using hashcash: gnus, spamassassin, and camram. it's important for more systems to support an open standard for sender-pays. So if you are deeply involved in an antispam content filter, please consider adding hashcash as part of the system.

check out http://www.camram.org http://www.hashcash.org

I think the Forbes article is reasonably balanced. It is right in one thing: turning junk fax prevention or spam prevention into a big lawyer-fest is not the best way.

What we need is a system where the recipient decides whether to accept a message, rather than showing all messages that the sender chooses to send. This decision by the recipient can be made by filtering for particular keywords, but that can be worked around and generates an arms race between spammers and spam-blockers. Better would be a system where a small payment is required, *set by the recipient*. So if you really dislike getting faxes, set the charge at one dollar and then businesses will only send to you if they have a good chance of recouping that cost (which means: a product you might actually want to buy).

It's not necessary that the recipient get the $1 payment; it would be enough for spam-blocking to require the sender to take a dollar bill and burn it. Or you can use computing time as a substitute for money, as in Hash Cash or Penny Black.

The great thing about Bayesian filtering is that it's adaptive. So they would have to dramatically increase the rate at which they discover and use filter-killing tricks for this to work.

But I worry that eventually, some companies that advertise via spam will learn to speak in a human voice. Surely this is possible for some products or scams. Advertisements don't have to look like advertisements, especially if they are only trying to pique your interest in a product that you will then go buy (or vote for) offline.

Even you will have to read the message carefully to realize that it's unsolicited bulk email. In such cases, we can't expect good accuracy from Bayesian filters, and the message will take more of your time.

Basically, advertisers adapt. A parallel example: If we get too good at zapping TV commercials with our TiVOs, they'll switch to more insidious product placement in the shows, so that the commercials are indistinguishable from the content.

Collaborative spam-filtering methods like Vipul's Razor might hold more promise. But the character of spam could shift to evade these filters, too. Spam might eventually come to resemble a bigger form of junk snailmail, or telemarketing -- where there are lots more advertisers but each one does a better job of targeting to a smaller list of customers (thanks to database companies like Experian). By flying under the radar with smaller lists, an advertiser might be able to stay out of the database of known spams. (With a small list, few recipients may bother to report the spam, so you can't distinguish it from solicited bulk mail that has been accidentally or maliciously reported as spam by several people.)

In the long run, I think we have to solve spam in the email architecture. I've always thought hashcash was the most promising idea, and it is now being pursued at Microsoft Research. There are also more radical proposals like Tripoli.

You can be open-minded and only discard mail from tags that get abused, or paranoid and only accept mail from tags you've specifically whitelisted. You can be obvious about the tags - betty@veronica.archie.com, or subtle about them - orggl@veronica.archie.com is "betty" in rot13, or cryptographic (use tags with the correct hash, so you can robo-check them, or longer tags with elliptic-curve signatures), or creative (Annalee Newitz uses a different username at techsploitation.com on each of her newspaper columns). And of course you can seed your web pages with spammer bait, so any person or machine that sends mail to stupidharvester@username.domain.com gets blacklisted.

My comment about crypto being overkill comes from a perspective of ten years of hanging out with the Cypherpunks, and doing crypto for years before that. There are other ways crypto can be useful - Adam Back's Hashcash work (and Microsoft's recent Penny Black stuff), Digital Signatures on email to reduce forgery, or simply requiring all email to you to be digitally signed or encrypted or both because that's too much work for most spammers. You could use it to build traceability, but that's not always good, and making it mandatory, centralized, and universal is very very bad from a civil liberties perspective as well as probably unworkable.

The technology is fairly old, it's known as Hash Cash.

It has known shortcomings, but it is one of the best solutions out there.

Its main problem, however, was not yet known when it was invented: That spammers would control huge zombie networks, as they do today.
With 100k zombies (which is not uncommon), the spammers can still send out 10k mails per second, or those 25 mio. spams the topic speaks about in under one hour.

You can use Hash Cash where the 'payment' is in a small amount of CPU time burned by the sender. So no central authority is needed.

If everyone (where everyone is the three single entities of Microsoft, AOL and Sendmail) would agree to implement a compatible HashCash solution (spec publicly available, of course), then SPAM would be prohibitively expensive (too slow since the sender is paying in CPU cycles).

As soon as you cross the threshold from profitability to loss, SPAM all but disappears from the internet. And unlike your "If everyone just..." scenarios, the everyone I'm describing is actually realistic.

Look, if we could convince every sysadmin on the planet to upgrade their MTA's, we could just implement HashCash and be done with it. And this guy wants us to concurrently update all our DNS maps?

Some objections to this have been (1) how do you process the payments without giving control over the internet to some evil corporation? (2) it's impractical to redesign the e-mail protocols and infrastructure, (3) mailing list operators can't pay to send every e-mail. Well, #1 is obviated by schemes like hashcash, where there's no real money involved. Re #2, this RSS example shows that the e-mail infrastrucure can and will be replaced, and there are ways to do it without having to make everybody change over to a new system overnight -- it can be done piecemeal. The RSS system may also show that #3 is not such a big deal, because maybe newsletters shouldn't go through the same channels as e-mail. (Note that the US postal service doesn't deliver newspapers.) Also, #3 was kind of silly anyway, because people can have a whitelist, and exempt people on their whitelist from paying to send them e-mail.