Lessig Wagers His Job On Anti-Spam Theory

kien writes "Lawrence Lessig is betting his position at Stanford on his anti-spam legislative recommendations. From his blog:'First the analysis: Philip Jacob has a great piece about spam and RBLs. The essay not only identifies the many problems with RBLs, but it nicely maps a mix of strategies that could be considered in their place. But, alas, missing from the list is one I've pushed: A law requiring simple labeling, and a bounty for anyone who tracks down spammers violating the law. Here goes: So (a) if a law like the one I propose is passed on a national level, and (b) it does not substantially reduce the level of spam, then (c) I will resign my job. I get to decide whether (a) is true; Declan can decide whether (b) is true. If (a) and (b) are both true, then I'll do (c) at the end of the following academic year.' The Declan referred to in point (b) is Declan McCullagh." Update: 01/07 02:45 GMT by T : Speaking of whom, here is Declan's acceptance of Larry's bet.

Comments

First problem with this solution:

[swordboy]

Lawrence Lessig is betting his position at Stanford on his anti-spam legislative recommendations.

Umm...

You *don't* need LEGISLATION to fix this problem (isn't that what technology is for?). Fix the technology (or lack thereof), and you've fixed the problem. There are several very good ideas floating around out there that don't require an office of homeland spam in the whitehouse.

Stupid lawyers...

Re:First problem with this solution:

[Guppy06]

"There are several very good ideas floating around out there that don't require an office of homeland spam in the whitehouse."

What amazing reflexes you have in your knee-jerk reactions. You could have a future in television news. Just because there is a federal law passed on something doesn't mean there will have to be federal enforcement of that law.

Consider federal anti-junk-fax laws. If you get an unsolicited advertisement on your fax machine, the sender owes you $500, collectable through your local small claims court/justice of the peace/etc (if need be). Essentially, all this law does is explicitly spell out the rights of the owner of the receiving equipment and make it easier for the recipient to claim damages without having to carefully explain how junk faxing is essentially trespassing each and every time.

The FCC doesn't enforce this law. The FBI doesn't enforce this law. You enforce this law.

I personally think the idea of expanding the existing junk fax law to include spam would be easier to enact (add three or four words to existing law) and easier to enforce (track down spammers for a guranteed $500 instead of just a chance at $10,000), but I'm obviously biased.

Now calm down before you shatter your kneecap.

Re:First problem with this solution:

[sfe_software]

Name one technological measure which has a zero false-positive rate

Bayessian Classification

a low false-negative rate

Bayessian Classification

and a snowball's chance in hell of being adopted.

Mozilla has (very preliminary) Bayessian classification. So far, that part works great - not a single false-positive in weeks of use (I've been using it since 1.3a was released), and once they add the ability to auto-mark-as-read and move/delete SPAM, I'm all set.

The problem should address spam at the server side, since it's already wasting space by the time it's allowed onto a client machine.

I'm not sure if you are referring to the origin server, or the receiving server (in which case it has already wasted space/bandwidth), but the receiving server could easily implement Bayessian filtering as well. It would take some work on the part of the clients to make it work (or perhaps simply forward junk mail to a local address that classifies it as SPAM?)...

I personally am okay with doing this in the client, as long as the Mozilla team continues to improve this feature. Currently I'm still interrupted and must mark the messages as "read", but eventually I won't have to ever see SPAM.

I'm normally not all that fanatic about software or software-ideas, but Bayessian filtering just plain works. If some implementation were to add common word-groups instead of just word occurrances, it might even be more rock-solid, but even as it stands in Mozilla's implementation, it has serious promise.

Implemented as a Perl script on the server-side, one could easily eliminate the problem all together for each user (since everyone has a different idea of what constitutes SPAM).

A classic example of this: Yahoo mail uses a more global approach to SPAM classification (BrightMail I believe). Unfortunately the RedHat Eratta mails fall into the Junk folder, since apparently many Yahoo users consider it SPAM. Similarly, I still get "notification@mailsweeps.com" SPAM in my inbox, no matter how many times I report it as SPAM.

This is where Bayessian filtering, which works on individual users, solves the problem.

Anyway, if it isn't obvious, I'm all for using technology to solve the problem, especially now that a very promising technology is currently available. Legislation won't help, unless it's globally enforced, and even then it still won't help much. Bayessian lets the user define what he or she considers SPAM, which will vary from user to user, making it the most logical approach IMO.

Re:First problem with this solution:

[Sheetrock]

A decent idea I've seen along these lines (barring your third criterion -- but I remind you we're still waiting for things as important as IPv6 to be deployed) has to do with requiring the sender of an e-mail to generate a computationally-expensive hash collision, dubbed 'hashcash', of the message that is computationally-inexpensive to verify by the systems forwarding the message to its destination. In a nutshell, a computer sending e-mail can be required to spend an arbitrary amount of time to generate this data, as the alternative would be to have the mail discarded by any mail server/relay implementing a check for the data.

There are more details here. Obviously, there's more to creating a workable system than this, because such an atmosphere would make it impossible to run a large-distribution mailing list, but it should be possible to get around such problems with a little ingeniuity, such as allowing the recipient of such mail to exempt certain IP addresses at the mail server from having to generate hashcash. My favorite part of this scheme is that, implemented properly, it could stop spam before it leaves the originating ISP.

Re:First problem with this solution:

[ergo98]

Mozilla has (very preliminary) Bayessian classification.

Just as an aside it's "Bayesian". I'm not launching into pedantry but noticed that when I tried doing a search on it (good old Google and its suggestions).

In any case, the success of Bayesian Filtering is because it is rare: Do you think that spammers couldn't dedicate some time and create a "norm" email if these filters were widespread? The only reason that they haven't is because users utilizing it as an anti-spam technique are rare, though if it took off it would be rendered impotent quite quickly. In other words if you like it so much, don't go around advertising it.

Re:First problem with this solution:

[sfe_software]

...what happens when your filter is attuned to emails between you and your buddies, and suddenly a proposal comes in from an employer, or a partner, or a customer? This single lost email could be incredibly damaging...

Personally, I will always at least review the subjects of the 'junk mail' periodically. Currently the Mozilla implementation doesn't treat Junk any differently, other than setting the flag. When it's able to "Move to Junk Folder" I'll still double-check.

The difference is, I can do that once per day. A quick scan of subject lines will rule out the vast majority at a glance (mentions of Viagra, Toner Cartridges, etc) and the questionable ones will get opened for further examination. I estimate about two minutes out of my day, on my own time, to make sure no false positives are in the Junk bin.

Beyond that, Junk mail coming in won't interrupt me while I'm working. I do occasionally receive important mail that needs immediate attention, and the absolute worst case scenerio (in my plan) is that an important message will be marked as SPAM, and only be seen by me at the end of the day.

In practice this would be very rare. So far, I've only had a couple false positives in the early beginning, and these were mailing lists I'd agreed to receive messages from that otherwise may have sounded like SPAM. Now these are getting through no problem.

So, end result -- I don't think Bayesian filtering is the end-all solution, but to be able to classify email upon arrival, and later double-check its work, is the best solution I've seen yet. Sure, some users will end up "trusting" it and might get burned, but that's their fault for putting too much trust into Bayesian (or any) filtering.

Re:First problem with this solution:

[Anonymous Coward]

And you don't mind indirectly paying for the spam that you don't see??

Fine, but I do mind, even if I don't ever see it.

Re:First problem with this solution:

[casio282]

Hmm, would you mind sending me this post in an email? Never mind, don't bother. You mention "mortgages", "penis enlargement", and "Nigeria". It will never get past my Bayesian spam filter.

Thank goodness /. isn't a mailing list!

Re:First problem with this solution:

[mesocyclone]

I suspect Bayesian filter will only work for a while. Spammers have a lot of money, and they can use it to hire a lot of creativity. If Bayesian filters get very popular, spammers will engage in counter-counter measures, just like a lot of them have already done to other techniques (return address filtering, IP filtering, and these days, simple keyword filtering).

I would bet that it is relatively easy to make a bit of spam that would pass most peoples' Bayesian filters... since most people are fairly alike in their email - or at least there are large subsets worth going after.

Ultimately it will be sort of a Turing challenge - my spam filter vs. your spam trying to emulate any person (not every person) that I might ever want to get email from. Doesn't sound too hard to me!

As many have pointed out, the real problem with Spam is that it as is an economic activity where people other than the spammer pay most of the costs (externalitites). The cost of mass email is only going to get cheaper unless some pretty stern measures are taken. I like Lessig's approach. Make the bastards pay. They are stealing resources!

Of course, we should do the same thing to telephone spammers, but so far nothing has been done there. It is a lot easier to propose legislation than to get it passed.

BTW... I got one of those TeleZapper things (disclaimer: I have not finanical interest in the product). It really does reduce telephone spam. Unfortunately, like most technological solutions, it also has false positives. When the library computer calls up to let me know a book is available, TeleZapper freaks it out before it delivers a message (I know this from watching caller ID). [another aside, the Phoenix, Arizona public library is so primitive that it still can't send you *email* notifications!]

Re:First problem with this solution:

[CableModemSniper]

Wow, that has to be the most interesting thing on spam I've ever read. It's almost like using Bayesian classification to stop spam is as bad as unesscessarily using anti-biotics. The spammers (or bacteria) evolve and get around it, and become even more annoying then they were before.

Re:First problem with this solution:

[poot_rootbeer]

No spammer has ever made any money by spamming me yet, so do you think they will make less money if I filter their emails and never look at them?

No, but they WILL send out more emails so they can continue making the same profit with the reduced response rates due to increased filtering.

Filtering is a band-aid, no matter how accurate or how transparent it is. The only real solution is to stop spam at the source. And while legislation is itself another band-aid (at least until a better mail system, one that's not susceptible to spammers' tricks, is developed and universally implemented), it'll at least reduce the bleeding.

Please resign now

[argoff]

Phew, I might get moded out of exsistence on this, but IMHO he should resign now. His views on copyright monopolies are simply wrong. He reminds me of the people who thought that the free states could peacefully get along with the slave states, but in the information age. He simply refuses to understand that we are quickly entering into an age where either all information will be controlled or all information will be free. Information is so easy to copy, modify, and manipulate - there can be no middle ground.

There is an old saying, give me my tea hot or iced, but if it is lukewarm I will spit it out of my mouth. His position that intellectual property still has a place in the information age while decrying all it's problems is just that.

Re:Please resign now

[peacefinder]

I don't agree with you, but I certainly hope you're not modded out of existence. Yours is an interesting point of view; I'm going to have to think about your "slave-vs-free state" analogy.

However, I think Lessig's immediate resignation, as you suggest, would be a serious setback to the "freedom" of information. (And it's obvious you don't mean "as in beer". :)

If he is right that the middle way is viable in the long term, and he acieves it, then life will be pretty good. Information will be less free than in your ideal, but it will be much more free than it is now.

If he is wrong as you suggest, and the middle way is not viable in the long term, then his work does not harm your cause. In this case, it will be chiefly relevant for having moved people away from the belief that complete control is viable. Perhaps he will win a non-viable middle way, perhaps he won't... but either result improves the cause of freedom of information. (Keep in mind that this contest will take decades to win; the only close end is defeat.)

Information freedom doesn't have enough prestigious voices, speaking in places that matter, for any of them to be lightly cast aside. Whether you agree with him or not, Lessig is, at the moment, the most viable opponent to the idea of total information control*... and that idea must be defeated before we'll have the chance to quibble over the system that takes its place.

You may have valid reasons for spurning the middle way and its supporters. You should have a care, though, that in spurning the middle you don't end up on the side that you like least, for lack of allies.

*: This is a matter of opinion, of course... there are other candidates. But I haven't heard of anyone else arguing this before the US Supreme Court or other institution of similar importance. And no, /. doesn't count. :)

Why I'd take the bet

[stand]

What I don't understand about Lessig's proposal is how would he enforce the bounty part of the law against off-shore spammers. Suppose I get an unlabelled spam from someone and I manage to track down the spammer as originating in Mauritania. How do I get my $10,000 from this guy. Is the US going to invade Mauritania to get it?

Sting the bastards into oblivion

[Black Copter Control]

Some time ago I found that spammers had managed to hijack the Windows proxy set up by one company that I worked for. When I found it, they were essentially using the full 1.5Megabit pipe to pump spam into the universe. Given that they were hijacking the computers for financial benefit, this was clearly illegal -- both in Canada (where I live) and in the US (where they were doing most of their business).

This leaves me thinking: shouldn't it be possible to use the ham-fisted anti-hacking laws against these bastares??? Not for spamming, but for hijacking peoples' computers to do the spamming with. I'd love to treat these bastards to 6-10 behind bars. Far better than a $100K fine that would be little more than a locense fee.

I tried to get an agreement with the company for the right to sue on their behalf in return for me helping to lock down their systems... They didn't go for it. My alternative approach is that I'd like to set up a similar system, wait for them to hack into it, and then do a hunt for the bastards running the scam. Any holes in this plan? (other than the probable difficulty in properly trackingg these people down?)

Re:He's no fool... international?

[smallpaul]

The spammers who are U.S.-based would merely move offshore.

It isn't the person pulling the trigger on the spam that matters. It is the business sponsoring it. For most of these marginally profitable businesses, (penis extenders?) it would be easier to do something else rather than move offshore. Plus, the money has to get from US consumers to the people offshore. There may be legislative ways to make this difficult.

What about Alan Ralsky? (aka: pond scum)

[Anonymous Coward]

Ok, this is *very* offtopic, but does anyone know about what Alan Ralsky is up to at the moment - the physical mail should have started to get to him by now at least...

Sounds like this guy is going to be out of a job.

[theLOUDroom]

that is, even if the law was ever passed.

How can this guy forget that the internet is not contained entirely within the jurusduction of the US?

It's nor like the spammers need to move elsewhere anyways, all they need is some non-logging proxy outside US borders and they can post with impunity.

Let's not forget the number of spammers already located outside of the US, either.

The internet just does not work the way this guy thinks it does: there is never going to be a day when everyone just follows the rules and plays fair

The way to handle spam is not with laws, it's with technology. Legislative bodies move too slowly and don't understand the technology, nor the scope of the internet.

What needs to be used is a combination of many different technologies: filtering, blacklists, whitelist, etc.

The internet is a huge shared network. So big, that prentending that you can trust every node on it is moronic. Software needs to be designed to recognize when a node is misbehaving and deal with it as well as possible. This goes for not just spam but other types of internet abuse, such as DOS attacks, trying 100 passwords in a row, etc. If a computer is going to be connected to an untrusted network it needs to be able to properly handle all kinds of unwanted data. To me that's just common sense.

Fraud laws don't stop me from getting Nigerian scam emails, do they?

The best way to fight spam is to develop software that isn't vulnerable to it, just like we fix other vulnerabilities. The reason we have spam is because our software isn't good enough.

Think of an unfiltered email systen as accepting input from a web form without doing any checking on the data it's recieving. It leaves you open to tons of really easy attacks. (If someone puts a meg of text in a field and submits it, your cgi scripts are probably going to go apeshit.) It's just bad design and it's about time we fixed it.

Re:I'm surprised!

[ergo98]

The US contains a large quantity of pc's and internet connections (if not most internet connections anymore). A law in the US alone will reduce the flow of spam massively, as these 300 million people use the internet disproportionately.

While US citizens may "use the internet" disproportionately, overwhelmingly my spam is sourced from Asia as of late. In the days of old people like Spamford Wallace could take credit for the majority of spam, but today I would imagine far more prevalent is distributed spammers in far away lands.

Having said that I'm certainly for laws: Often these spammers ARE profiting off of Americans so it seems fair that seizure of their credit card/paypal/etc funds would be just.

Perhaps the best law of all would be one banning people from responding to or buying stuff from spams...

Re:Rubbish

[Guppy06]

"Listen to him complain about collateral damage - collateral damage is the point of blackhole lists!"

And this is a good thing?

Let me modify a few of the nouns in your rant and see if you still agree with it.

Killing US citizens is the solution, not the problem. If we didn't punish these ignorant civilians they would continue supporting Israel. Every citizen of an Israel-friendly country is voting with their silence - for persecution. The US government has proven that they will not act against Israel until they are threatened, and the only way to do that is to kill civillians to the point that they start losing votes. Collateral damage IS the point of terrorism - otherwise its useless.

The ends do not justify the means. Innocent until proven guilty unless spam is involved? No thanks.

(Do I think RBLs are a form of terrorism? No. But I do not accept the idea that collateral damage is OK.)

Then make more than one Bayesian filter

[yerricde]

what happens when your filter is attuned to emails between you and your buddies, and suddenly a proposal comes in from an employer, or a partner, or a customer?

Bayesian filters don't have to classify a message only as "Spam" or "Not Spam". You can train them to recognize several categories such as "Work"/"Not Work", "Buddies"/"Not Buddies", etc.

Long monopoly terms are the problem

[yerricde]

Having cheap knock-offs of your designs or technology made by China or whoever is fine for consumers, but who put up the money to create the technology in the first place?

Does the inventor of a novel information technology product (not a drug) really need a 20 year monopoly to pay for the product's research and development?

Does the author of an operating system really need a life + 70 year monopoly to pay for the product's research and development?

Some monopolies do benefit society. But like all things, monopolies should exist in moderation, and this is why Dr. Lessig has gone to court to argue against monopoly term extensions.

Betting..

[steveheath]

I've some experience of betting (I am currently working in the industry). The first thing I was told when I started here was "What constitutes a bet". Among other things a bet must be time-bounded. This means there must be an end-date at which point the bet dissappears. Mr Lessing probably doesn't mean that he will resign (whatever job he's doing at the time) if the law is passed EVER as part (a) suggests. Presumably there should also be a time-bound on part (b) too..?

Bad bet

[T.E.D.]

A US law can't have much effect, for the simple reason that most of my spam these days comes from outside the US. If you could wave a magic wand and stop all US-based spam, you'd hardly make a dent in it.

In fact, the majority of my spam these days comes in using one of the various eastern pictographic fonts. Not only can't I read it, I can't even make out the symbols. I might as well be getting 50 emails a day of line noise.

How about an email watch dog?

[jhamm]

I've tossed around the idea of doing this many times throughout the past couple of years, but more so now that spam is at a point of ridiculous proportions. How about any new incoming mail being held by a watchdog first without appearing in your inbox, which sends a reply to the sender with a message like: "This is the first time you've sent me email. If you wish for your email to be seen, then reply with the word PLEASE in the subject line." From then on, the watch dog would allow emails from that sender to pass through. Otherwise, the spam emails are never seen. I suppose that there may be some spammers that do reply, but it's a manual act, especially since you could set your own pass subject ("PLEASE"), or even have it change to a random word each time. I bet this would cut down on spam significantly. What do you all think?