Slashdot Mirror

Some of it is UCE, partially due to a couple dingbats with similar names who apparently think my gmail account belongs to them.

This isn't spam; at worst, it's bacn with a case of mistaken identity.

As someone whose full-time job is preventing spam (I work on Akismet, which checks about 380MM Web comments per day for spam), my general response to these kinds of questions is this: Fighting spam is hard because what's spam for you is not always spam for someone else, and spammers are continually changing tactics -- what worked to prevent spam yesterday may not work as well tomorrow, so it's a constantly moving target.

In my experience, GMail's filter is just ok. I see about 50 spam per day end up in my spam folder, 3 or 4 that make it to my inbox, and maybe one false positive per month (when I bother checking). That's a 94% success rate with a 0.3% FP rate (based on my ham email activity), assuming that they're not instantly discarding blatant spam that wouldn't even merit ending up in the spam folder (which they very well might be doing). If Akismet had this same success rate filtering comments on my blog, I'd have to manually mark 230 comments as spam each day instead of Akismet's missed spam average of about one per day. I don't complain about it though, since fighting spam is hard (see above).

http://www.stopforumspam.com/ http://akismet.com/ Depending on your forum software, someone will most likely have done the hard work and integrated these services to do what need. I use these on vBulletin to moderate spam posts.

tried "show random page" on your site: looks like there are much more spam-ads than inventions, unless I'm very unlucky... You should consider a spam filter like Akismet for instance http://akismet.com/

Akismet is the best thing for blog spam prevention ever. I can't believe you've never stumbled across it before. It uses statistical analysis to identify spam, and the more people use it, the better it gets. If everyone used it, the blog spammers would just disappear because their attacks would be completely ineffective.

Akismet is useful but it has its limitations.
* works only for public or non-sensitive stuff, because it involves sending entire messages (including sender's IP and email address) to a central server
* it doesn't pick up on new spammers fast enough. I've had a wave of my.mashable.com comment spam recently, they all looked very much the same but Akismet didn't complain. Yes, I submitted them as spam. After two days I was tired of waiting for Akismet and added an extra filter.

Basically, it's a neural net

I don't think so. Just some Bayesian filtering. Actually it's as closed as can be. They won't tell how it works except that they use "hundreds of tests", "dozens of factors", "weighing and clustering" and some "secret sauce". The boss seems to have some ego issues that make him use that sauce for personal vendetta as well.

Akismet is great for comments and such. Basically, it's a neural net using user submissions to determine whether or not a submission (sent automatically from your site for checking) is spam or not.

Just use http://akismet.com/ to reject the junk, open the content up to the world, and we'll all be better off.
Well, at least all of us except for the journals.

This is why you need a queryable, updateable public spam database like Akismet where, with a little effort in telling it the odd time it gets it wrong, you can eliminate 99% of spam. This might not help for a registration script, but you could use it on the content ultimately used by the registered user to determine whether the signup was likely a bot or a human.

http://akismet.com/

Your best bet for forum spam would probably be a bayes filter - much the way you'd deal with email. if it's small scale and non-commercial, you could use akismet. This is generally not a viable solution if you're running a high traffic commercial forum (we looked in to it, it was going to cost us between $15 - $20k per month). In the end, it was more viable to develop our own solutions in house. This won't stop them from making bogus accounts, but it can help to cut down on the amount of garbage that litters your forum.

Link spam is very easy to filter-out, because spammers have to use links and unobfuscated keywords, otherwise spam won't benefit them. And there are additional methods which can be used like observing odd "browsing" patterns, poor quality of HTTP/HTML implementations. Good'ol blacklists work too.

There are services that successfully implement content-filtering, like Akismet and (mine!) Sblam, which has accuracy over 99.85%.
If you take into account "false positives" CAPTCHA causes by blocking disabled users or just discouraging posting, content-based filtering may be more effective than any "bulletproof" CAPTCHA.

Akismet is what a lot of Wordpress users (and many other bloggers) use to prevent comment spam. They've got a pretty neat stats page that shows the volume of spam they have blocked from their creation. They are relatively new, so the fact that the graph trends upwards so quickly also has to do with the fact that their userbase is still growing. But it's unquestionable how large a spike I saw in the end of November and December. Particularly over the Thanksgiving/Christmas holiday weekends. I have a personal server in my house that was MELTED by the amount of hits to my dinky little blog. It would go up and then 30 seconds later would be unresponsive and have to be forcefully rebooted. It even killed my D-Link router.

I'm posting AC so slashdot doesn't melt my server again...

Akismet is what a lot of Wordpress users (and many other bloggers) use to prevent comment spam. They've got a pretty neat stats page that shows the volume of spam they have blocked from their creation. They are relatively new, so the fact that the graph trends upwards so quickly also has to do with the fact that their userbase is still growing. But it's unquestionable how large a spike I saw in the end of November and December. Particularly over the Thanksgiving/Christmas holiday weekends. I have a personal server in my house that was MELTED by the amount of hits to my dinky little blog. It would go up and then 30 seconds later would be unresponsive and have to be forcefully rebooted. It even killed my D-Link router.

I'm posting AC so slashdot doesn't melt my server again...

and install Akismet

My own weblog was recently hit by comment spam. I was extremely irritated, and initially considered captchas as a potential solution. But several problems with captchas ultimately lead to me seeking alternate solutions.

The first problem with captchas is the barrier it puts up, however small, between you and the users of your site. Apologies for the corney analogy, but captchas are a speedbump on the information superhighway. People hate running into them.

The impediment to visually disabled users is also a big one to consider. It's not just fully blind people. People can be shortsighted, colour blind, dyslexic or perhaps simply shortsighted users relying on specialist software to read your website. You're letting these people down by adopting this practice and that's something I would really feel bad about doing.

But the biggest reason not to use captchas is spammers increasing abilities to interpret them. At even a five percent success rate in interpreting captchas, a spammer can bombard your site with requests and still get something through. They're just using the same model as they did with email, and it will work.

Instead I chose some other plugins available for Wordpress to help with the spam. Akismet sounds like it could work as a kind of distributed spam check/blacklist of sorts, though I am wary of the fact that a private company is running the service. I also installed Bad Behaviour, though it's clear that eventually some spammers will adapt their behaviours to this.

Ideally what I'd like is a true bayesian comment spam filter plugin for wordpress, but so far I haven't been able to find one. Such filters have done wonders for me in Thunderbird for my email spam, with something like a 99.99% sucess rate and no false positives. Clearly the situation is quite different with comment spam, but all the same it would be nice to have one.

I envisage that the comment spam situation will get a lot worse as time goes by, regardless of any pagerank type algorithm changes. Comment spam will no doubt become as ubiquitous as regualar spam and I can forsee dozens of "splog" post per day in the not too distant futre. My opinion is that Blog software should come with robust, adaptable and self updating anti-spam software on by default before this problem escalates out of control.