Blog Comment Spam Removal

mattwarden writes "The back-and-forth between spammers and mortals continues. Anyone with a MovableType blog that is even remotely on the map has no doubt been hammered lately with comment spam, comments made on entries by a script or program in an attempt to increase search engine page rankings. Prior to today, one had to manually delete each of these comments. No more! Jay Allen has developed a plugin for MovableType that removes these spam comments based on a blacklist (of both hostnames and regular expressions) and intercepts new spam comments before they are made. There's even a one-click link included in the comment notification email that makes it easy to de-spam your blog."

Comments

Oh well.

[The Fink]

I was going to write one of these for my couple of sites, but instead, I came to the conclusion that I'd be better off disabling comments.

The more I think about it, the less I think comments are actually, you know, useful in a personal web-space of any kind. Few of the comments I get at least are of any real value, other than to indicate that either (a) I'm being spammed again or (b) someone human is actually reading my site (for which I'm always grateful, although I have other ways to find that out anyway).

Heh

[Henry V .009]

I am just going to say: Neat.

/. needs TrackBack

[GeorgeH]

Heh, I just posted something about comment spam and a possible solution to my website...

So what else can be done about it? I'm surprised no one has mentioned Bayesian filtering of comments. Like most people who've heard of it, I first found out about Bayesian filtering from A Plan for Spam, and how it can identify spam. Since then virtually every spam blocking system has started using Bayesian techniques for at least some part of identification.

Read the rest...

Any connection to Slashdots recent problems ?

[Crashmarik]

I have noticed slashdot is getting alot of 500 errors and it seems that there are more than a few sites that are organized to produce negative comments for slashdot. Are the 2 phenomena at all related ?

yay!

[croddy]

it's about time someone did to the weblogs what the bloggers are doing to the rest of the internet.

I, for one, welcome our new blog-spamming overlords.

Re:yay!

[Anonymous Coward]

say, how is this offtopic, eh?

Re:yay!

[Anonymous Coward]

For the same reason this is the same thing was what spammers did to the rest of the web community, asswipe.

Well, that about says it all.

[The Fink]

A front page slashdot article with a grand total of eight comments.

Shows what the average slashdotter thinks of "blogging"... can't say I'm surprised (and not in a negative way).

I wonder, if the term "blog" and derivatives (which I personally detest, but that's another matter for another forum...), put people off - if it had been omitted, I wonder if more people would have read and commented.

Re:Well, that about says it all.

[Kalak]

/. seems to forget that *Slashdot is a blog.* Look for the definition of a blog, and you'll find /. fits. It's one of the most successful, is on steroids, and has gone commercial, but this is really Taco's blog and we are all just guests. /.ers not caring about blogs is kind of like a human that doesn't care about oxygen. It shows how pathetic many /.ers are, since they don't know the definition of what they're slamming (standard procedure around here).

I'm surprised the spammers don't hang out here...Oh wait, they just spam the articles instead of the comments.

Re:Well, that about says it all.

[JoseMonkey]

I think the definition you linked to is a bit too broad in its definition; by that definition, cnn.com is a blog. (Granted, it's commercial, but that didn't stop you from calling /. a blog, did it?)

I don't understand this strange trend to classify all online publishing (colloborative or otherwise) as a "blog." I think when people typically think of a blog, they think of a online journal of sorts. It's a subtle difference, but I think it's important. Otherwise, everything is a blog. And that's just silly.

Incidentally, it reminds me of the whole hubbub in the corporate world over enterprise portals during the last few years. No one can quite decide what one really is, but I'll be damned if I don't want one for my company!

send code to badgrammar123@hotmaill.com

[Anonymous Coward]

I was just thinking of such a filter for LiveJournal code-beggars this morning: anything that detected a mail adress containing digits should catch the majority of slackjawed monkeys.

Bayesian filtering and more

[fooljay]

Lots of people have mentioned Bayesian filtering. My plan with MT-Blacklist has always been to hook into Movable Type and then create what would essentially be a sub-plugin API. In that way MT-Blacklist becomes the engine that other people can write filters for.

If you use Movable Type, you are probably familiar with Text formatting filters. This would work in much the same way. Someone could hook in a Bayesian filter, or an IP filter, or a Swedish Chef filter. Whatever. It's coming...

-j-
http://www.jayallen.org/

Suspicions confirmed.

[Kyrthira]

I knew there was a reason I liked LiveJournal. Lots of the fun, not so much of the hassle. (Just some drama-llama stuff now'n then.) I say it's worth the $25/year for a paid account. I gave up on blogs/guestbooks on my websites long ago, because all I'd get was spam; either for the purposes listed in the article, or for some bloody pr0n site or another.

Re:Suspicions confirmed.

[Anonymous Coward]

Well you will end up with plenty of "traditional" spam if you get a paid account, and enable the @livejournal.com e-mail.
Bots only have to find for LiveJournal usernames (which is of course very easy), and send mail to username@livejournal.com. Most of the accounts are not paid, and so the e-mails will bounce. But about 10% (paid users) will go through, many more than by doing a dictionary attack on an ISP mail server. This proves a very effective way of harvesting emails, especially given the recent trend to avoid or hide mailto links.

Re:Suspicions confirmed.

[Joe Decker]

While I have a LJ with such an email addy enabled, I so far (it's been two years) don't get spam at that address. There's an option, which I've always had turned on, to insert spaces or some such in the display of said emails to hinder botharvesting, perhaps it's been effective. *shrug*

I do get tons of spam at other addresses, sadly.

"Mortals"?

[John Hasler]

> The back-and-forth between spammers and mortals
> continues.

I'm fairly sure spammers are mortal. If I ever catch one I'll find out for certain.

Scriptygoddess talks about this

[ChipMonk]

She has an entire entry about comment spam, and what to do about it. For myself, I installed her Comment Queue Script/MT Hack. Works like a charm.

hope no one took offensive countermeasures ...

[HealYourChurchWebSit]

I sure hope no one took the offensive and used tools such as wget, lynx and/or curl for evil instead of for good ... as some suggested in a SlashDot Thread last month.

As tempting as such technologies are ... that would be to the level of the spammers.

Which is why I'm so glad I can ignore such countermeasures and simply use Jay's excellent plug-in!

rats

[ChipMonk]

The Comment Queue died two days after I installed it. So, I'm trying out Jay Allen's MT-Blacklist.

Image based password

[too_bad]

Like many websites (including the whois query page at netsol) have the image
based passwords. Basically, they are images with some text with a lot of wavy lines
and the assumption is that it is hard for programs to do an OCR on them, but easy for
humans to read and understand the text.

Just make the bloggers read and re-enter the text in the slightly-obfuscated images before they
can enter their comments. If they spent atleast a few minutes composing their article
it should not be to hard to type in a few more letters to be allowed to post.

It's happening to me 4-5 times a day

[cosmosis]

Yes, I was beginning to think it was only my blog that was getting this. I'm now getting 4 or 5 spams on my comments every day.

My Solution

[cheesyfru]

I have a blog that I built around my own content management system. In order to thwart comment spammers, I take away their incentive. Most of them spam because they want GoogleJuice(tm). When they post their URLs, my software rewrites the address bounces them through a filter. The filter page takes their URL, and uses Javascript to redirect the user to the address. The net result: browsers click the link and get to the destination, while search engines will never know the difference.

Combine this with a well-placed notice that there's no reason to spam, and now my comment spam is next to nothing.

Legislation

[dekashizl]

I want to open up a discussion on legislation. Obviously some sort of law making unsolicited advertisements illegal would be a big help in thwarting the SPAM problem in many arenas (email, blogs, phones, faxes, etc.). But legilation also feels like we, as technologists, are giving up and admitting that we can't solve this problem. What do people think about this? Should we be true Americans and let the legal system sort this out?

Re:Legislation

[Anonymous Coward]

Legislation is tossing the baby out with the bathwater in most cases. If an anti-"spam" law gets passed that covers a broad collection of electronic media, e-mail, web forums, blogs, then its not hard for some enterprising attorney to get someone jailed for a beefy term because their opinion differs, or was an unpopular person.

Lets try a technological solution first, rather than trying to fatten more attorneys' wallets.

Oh, it won't work even if passed -- comment spammers would just web proxy off another country's computers just like e-mail spammers do.

Great site guys!

[Flakeloaf]

This website is a good place for discussion of your topic. My website is a good place to buy discount Viagra. Increase your manliness and be sure to satisfy your partner. Order now and receive a carrying bag for the stacks of condoms you'll no doubt be needing.