Using Email Networks as P2P Spam Filters

Oscar Boykin writes "New Scientist is running a story on using the social network in email as a P2P network. The idea is that email networks have structure that is conducive to a type of search called percolation search . This means email clients could query the social network of email users to filter spam. This story is based on a preprint available."

Comments

Nice...but not necessary

[PenguinBoyDave]

Since switching to Thunderbird, I get nearly no spam...maybe one or two per day. I like fancy stuff, but when simple works, go with it!

Re:Nice...but not necessary

[Dukael_Mikakis]

I use gmail, which does an excellent job at filtering spam.

But I think this could even be a step back. Like the parent says, I think most informed people have solved the issue of filtering spam pretty effectively (Thunderbird, Yahoo, Gmail, Bayesian filters, etc.) and so we don't generally *see* much spam.

The *REAL* problem with spam is traffic and network pollution. Spam wastes a ridiculous amount of bandwidth and (through spyware) hijacks our systems' cycles to do something that is (with filters) ultimately to no end. This seemingly won't solve the bandwidth consumption issue and might worsen the problem by polling all your friends over the network and then using your personal cycles to scan said email against all the known spam on your friends' computers.

People forget that the true detriment of spam these days is the traffic it causes, not cluttering your inbox (if you're smart).

Re:Nice...but not necessary

[Anonymous Coward]

I agree with the main point of your post, but wanted to address something you said.

I use gmail, which does an excellent job at filtering spam.

I see this stated here on /. a lot. While gmail seem to be improving in this regard, I haven't had the experience of excellence.

A couple months ago the majority of my spam was actually legitimate email from my mailing lists. As of this moment, I don't see any legitimate mail in my Spam folder.

However, about 20% of the actual spam I get ends up in my Inbox.

I agree this is better than getting 0% spam in my Inbox and a ton of false positives, but my idea of excellent is for me not to even really think about spam at all. In other words, if a spam shows up in my Inbox, I want it to be a rare occurance. I want to be genuinely surprised.

Potential for harm

[davidwr]

Imagine the potential for harm if I infiltrated a social network and then identified my enemies as spammers, either deliberately or because I or the software agent I use was somehow tricked into doing so.

Social network-based spam-detection is a part of, not a total, solution, and its limits need to be recognized.

Isn't this basically how Razor works?

[forevermore]

Granted, I just skimmed the article, but isn't this exactly how Razor works? (simplified) Communities of people flag messages, senders, etc. as spam, and the mail server (or in my case, spamassassin) compares the messages to the community spam archive for matches before delivery.

Reduces to a standard spam filter

[tdvaughan]

According to the article the method works by asking its network of email users if they've seen the spam before:

Similar software on each computer that receives the query would then check the message against its own spam database, and so on, until a match is found, or the message is deemed original.

So it can't deal with spam that includes a unique random ID and would tag emails from a mailing list as spam. Once more: nice try, but it won't work in the real world.

would that really be good?

[overbom]

If I were a spammer:

I'd change an email client to respond with any message from certain folks I don't like to report all of their messages as spam to poison the social network. a couple of clients out there saying "yup, I've already got a message like that here, and my user marked it as spam".

think globally, act locally, right?

Not a particularly new idea...

[Otto]

This isn't a new idea... except that they propose to integrate it into the mail client and have everybody you've ever sent mail to or received mail from be a potential contact, weighted by frequency that you email them. That's a bit new, but not as effective as it seems.

For one thing, it would block mailing list messages, which are messages that you probably do share with your contacts.

For another, it does not consider that most spam has random keywords seeding into every copy sent, so those would have to be ignored somehow, which introduces a fuzzy match algorithim, which means the possibility of false matches exists, and since you're asking others (probably all using the same algorithim against their databases) you have increased the chances of a false match being found.

In any case, collaborative networks already exist in a better form. Users mark messages as spam when they get them, a flag is created and sent to some central place that all users check against for matches. The algorithim for fuzzy matching resides in one place and is only used as an indicator in spam assassin in any case, not as the sole indicator..

Large scale systems like Google's GMail can use people flagging messages as spam to filter similar enough messages from other users, sort of thing. I'm pretty sure they do something like this, in fact, as my GMail account has *never* made a mistake in it's spam detection.

And so forth. There's better ways than relying on a random query of your contacts to see what they think.

spam filters should reduce network load

[sPaKr]

Skipping past the security issues. One of the goals of spam filters should be reducing network load not increasing it. If we have to send our spam to several differnt peers to be scored this would compound the network load problems. Mostly this is a bad idea(tm) from the get go. I think the only thing that will really stop spam is to force something like pgp(gpg) signatures on all mail. Here's hoping the new national ID cards will have public certs encoded on them. It would be cool if someone would step in and get PKI working for the rest of us. Also we should drag the boddies of spammers through major cities behind a horse, while allowing victums to beat the spammer with large sticks like golf clubs.

Bigger problem...

[Not_Wiggins]

What is one person's spam is another person's desired mail. I'm not talking about advertising, either. For example, I know for a fact that there are a lot of people out there that "knee-jerk" react to service messages from their bank, credit card, whatever... stuff they even signed up for that they mark as spam. Since I want to get my "your payment has posted" email, do I want to rely on the network of people around me that signed up for the same thing with the same company and report it as spam because they're too lazy to just unsubscribe?

Re:Standard Form Letter

[Linux_ho]

I'll add my own here:

(X) Similar to DCC and Razor, but far less bandwidth efficient than either

You should also have checked:

(X) Users of email will not put up with it
(X) Requires immediate total cooperation from everybody at once