Slashdot Mirror
What Ways Can Sites Handle Spambot Attacks?
Amazing Quantum Man asks: "I'm a member of a site devoted to nitpicking TV shows and movies. It has always had an open posting policy — no registration required, and you could use any name you wanted. This policy was instituted way back in 1998, and led to some quite fun, freewheeling threads on various boards. Recently, we have come under spambot attack, with spambots posting links to gambling and porn sites on every single discussion board on the site. The admins have been trying to block IPs, but it's useless against a botnet. As a defense, it looks like the site is going to require registration, and disable anonymous posting. Many regulars, while they understand the need, are concerned that the freewheeling character of the site will be lost. Let me continue by saying that I'm not a site admin, merely a member there. Also, if it helps, the site in question is running Discus. Has anyone here been in a similar situation? How did you handle it, and what did it do to the 'culture' of your site?"
What do you think about need for registration and still keeping old open way of posting?
Just log in and later post with whatever nick you want. Just don't trace it or anything. You can even prepare some kind of statistics for users (how many post they posted). And of course implement some captcha.
"an experienced, industrious, ambitious, and often, quite often, picturesque liar" - Mark Twain
See if you can set up a CAPTCHA that must be completed before the post can be put up. Multiple missed attempts could even ban an IP. Just be sure you have some alternate means for people that have issues with their vision.
Even people that believe in pre-destiny look both ways before crossing the street.
It has always had an open posting policy -- no registration required, and you could use any name you wanted.
There's no reason why that should change. Just add CAPTCHAs of some sort or another to the posting system. No more bots posting crap (although the CAPTCHA system might need to be changed every now and then depending on the strength of those chosen).
Amavisd-new has had p0f support for detecting the OS of the sending mail server for quite some time. It detects the OS type of the incoming mail connection and adds a header indicating the results. You can then use SpamAssassin to detect the OS an add an appropriate point total. Since few "real" organizations use desktop OSs for mail relays, you can usually assume a high probability of spamminess from such.
-- Minds are like parachutes... they work best when open.
Akismet is a very good antispam. It blocks 99% of spam on my forum.
CAPTCHA doesn't work, many spambots can solve CAPTCHAs.
I would suggest maybe putting in Captchas for every spot you might submit a post, etc. This way, bots cannot or have more difficulty making posts. Here are more links I had on these, but I haven't looked at them in a while...
"The past was erased, the erasure was forgotten, the lie became truth." ~1984 George Orwell
Don't let anonymous users post links to other websites.
I loathe CAPTCHA, although I may end up implementing it on my system. I could also be convinced that a "which of these N pictures are kittens?" test might work.
I run a small old-school weblog on my own content management system. Middling PageRank (6 or so), a couple of hundred readers. I just had the spambots discover my Wiki, but in the process of cleaning up that mess I was shocked and amazed by the emergent behavior I'm seeing in spambots. Every form on my site that could get random info plugged into it, including search fields and new user account information; I'm going to have to make new user accounts far less easy to get. All of a sudden I'd ballooned from under a thousand registered users to forty-five hundred.
I don't like verified email accounts, and I think those are going to get attacked fairly soon too, but some sort of way to more strongly tie an identifier to an actual human has to fit into the mix.
One of the things I'm excited about is the notion of cross-site identifiers, like OpenID, and distributed reputation. Something that lets my site collaborate with other sites and say "I trust this URL". Users will still have to jump through the "are you a human" test, but will only have to do so once within the confines of a trust network.
On my guestbook, I just say "posts must begin with the word Banana, which will be automatically stripped." It works. Some spambots are actually human so it doesn't stop them, but it's super-simple.
I used to get a ton of spam on my guestbook. I tried doing lots of little things in the code and it turns out the spam was being submitted without them filling out the HTML form. To force this to happen, I found a neat idea on some German website (it's down, so I mirrored it). The code will not accept a post if there is no number/checksum pair. ...' "
That cut out a lot of the spam. The rest has been gone since I added another, required field "What is my first name?" It is like a captcha but much easier. No one will complain that they get it wrong. For your site, maybe something like "Finish the name of this show 'I Love
This post climbed Mt. Washington.
Use Bad-Behaviour and mod-security.
These two work perfect for me.
Grundgesetz * 23. Mai 1949 - 30. November 2007 - http://www.vorratsdatenspeicherung.de/
I've been battling this for years now. Ironically, the best way to stop spambot attacks is to homebrew your own CGI stuff. If you can't do that, rename all the scripts to non-standard names so that the common URLs are not found.
I've been using keyword blacklists. They have proven to be very effective. If you don't allow people to input names of common drugs or strings like ".php?" or ".asp?" you can knock out a lot of the affiliate/redirect spam.
The biggest problems have been with the popular messageboard apps. We've simply stopped putting up messageboards, or set them to require registration and manual approval to post. It's really disgusting how if you leave a forum unlocked, it'll take about a week before it's full of ads for online drugs and sex sites.
I've had to deal with spam attacks on both my personal site and a forum I use. In both cases, we tried to ban IP addresses, then tried invisible methods of stopping spam (eg hidden required fields populated by javascript), and nothing worked.
In the end in both cases, we've just had to use a CAPTCHA system. Spammers tend to use multiple IP addresses (and I do mean in the hundreds, a lot of them proxies or botnet-controlled boxes) so banning simply doesn't work.
I've tried doing things like only requiring a CAPTCHA if the comment includes "http" or similar techniques. It doesn't work, I've had spam that simply consists of "Hi, great site" posted 30 times.
I don't know why, but spammers don't seem to care whether their spam even has the potential to turn into revenue for them or not..
CAPTCHA is the only viable method, IMHO.
For those worried about accessibility; offer a non-CAPTCHA'ed form and manually review it; most users will be able to post perfectly well and for the few that can't enter the CAPTCHA, they can still post to the site, but with a delay as you check it for spam.
On Wordpress you have the option of requiring moderation only the first time an individual posts. Once you have approved one post by them they no longer are moderated.
Sure it still involves trolling though moderated spam to find the genuine posts, but if you don't have massive traffic it works fine.
Three Squirrels
There's no need to make people register accounts, just have them enter an email address whenever they post. The first time any one email address is entered, a message is sent to that address and the post is delayed until they click a link in that email. You need never display the email address anywhere and don't even have to save it at all (a hash of it works just as well).
It also has the added benefit of being more friendly to visually-impaired people than CAPTCHAs would be.
After seeing this presentation on OpenBSD's spamd, which profiles and greylists SMTP connections coming from botnets, I'm convinced of the need for HTTP POST greylisting.
Point is twofold: slow the bots down (or stop the dumb ones altogether) and block obvious botnets completely.
SMTP has the handy retry message. For HTTP, we would need to store the original POST request, and return a response with a 10-20 second meta-refresh to a confirmation url. Anonymous posters won't mind the wait, and the time window gives us time to watch for additional POSTs from the same ip, and blacklist them outright if they match a spammy profile.
registration required to post a URL or email address?
every day http://en.wikipedia.org/wiki/Special:Random
Check your users against DNSBLs. Originally intended to block out malicious mailservers via their IP addresses, they are applicable on webservers as well. Via sorbs you can check for open HTTP and SOCKS proxies (interresting for you), open SMTP servers (not very interresting for you), webservers with unpached vulnerabilities, hijacked IP netblocks and malicious (in bed with spammers) network service providers. Other lists include the here recently mentioned Spamhaus list, and various DULs (dial up user lists). See the Wikipedia article for some of them.
I used DNSBLs at my former employer to block users coming through open proxys from registering domains. We saw that every phisher who bought a domain name came through an open HTTP proxy and used a stolen credit card. So using DNSBLs was the only viable option then.
Meme of the day: I browse "Disable Sigs: Checked". So should you.
I have successfully blocked comment spam by rejecting messages with http:/// in them. Most of the spam contains links, so this can be extremely effective. Maybe on the site in question, reject anonymous posts that have http links in them, and if you have a site you need to post, you have to get an account.
Sean
Comment removed based on user account deletion
I've had good success with grep(1), using a file filled with various words culled from spam.
I also recycle known spam through the search software, so it automagically updates itself. Seems to work well, and the
best part is that as your anti-spam technology improves, the people behind the spam robots tend to give up on your site.
Not a web designer.
Comment removed based on user account deletion
I dealt with this same issue on a message board. For years it did not require registration to post and with a small cadre of level-headed moderators we had a lot of fun. It was good for everybody, from regulars to one time guests who just wanted to ask a question.
Then, about two years ago (I think), the message board spammers began to get exponentially worse. Poker spammers were most of it, but I also saw a number of porn site spammers and some guerilla marketing campaigns that were awful. The evening that the one "documentary" on M. Night Shyamalan played on SciFi a huge number of posts and threads from "people who watched the film and wanted to talk about it" appeared. Obviously a bot network, because there were easily a hundred posts and the IP addresses were checking out as valid.
I tried everything to avoid registration. Banning IPs was useless, because they were bot networks. I made rules to discard posts that matched known spams - new, different ones came in. I discarded multiple posts or duplicate posts - the bots made posts that were different. I made rules to discard posts with certain URLs - no good, way too many URLs were rolling in. I changed the name of post function files in the Phorum message board - the bots adapted or were adapted. I made rules to prevent multiple posts within a certain period by the same host - the bots slowed down their posting. They posted with http code, they posted with bbs code, they posted plain text. In the end, after about two months of too much effort, I enforced registration. The problem has been solved ever since.
As a result of the registration I am certain my message board is not as robust as it once was. The simple fact is that registration drives away people who could become good members of the community. Another simple fact is that I have seen a number of boards turned into useless crap by spambots.
I dislike CAPTCHA, so registration was the lesser of two evils. However, if there is a mod so that Phorum can enforce CAPTCHA for guests, thus allowing them to post without registration, maybe I should check it out.
Andrew Borntreger
Champion of cinematic disasters
I've been active for quite some time on a site dedicated to DIY tube guitar amps (ax84.com). We have a lounge area where anything goes, but the posting policy is quite loose, with all sorts of fun stuff occuring within [otherwise] on-topic threads as well.
After getting hit with several posts by auto-spammers, the maintainer instituted new rules.
You can register, which requires nothing more than a valid email address, handle and password (AFAIK, I registered when he was first testing logins). But we also have people who don't want to register for a variety of reasons-- from wanting to stay off the grid to just not caring. These people get a temporary login if they answer a question that is easy for humans, less easy for a bot. It could just as easily be a "pick the number from the image" thing or whatever.
At any rate this has been in place for a month or so, and I don't see any difference at all in the community. It's still a free-wheeling, fun place, but no spam. A win-win from where I sit. It's possible the non-registerers are unhappy, but since Chris included them in the discussions of how to handle things, and they are still there, I have to assume they're "happy enough". I am.
``- Some sites use CAPTCHA... but I don't like it. I'll bet you I make a mistake in the CAPTCHA at least 30% of the time, which is just frustrating.
- Verified email accounts - this is what I tend to use. User signs up, email with password gets sent. Some people don't like giving out their email for fear of SPAM and such.''
You could give your users choice. Either enter your email address (which will be used to send you a verification code), or solve this riddle. The riddle could be a captcha, but I prefer all-text things. Anything that's easy for humans, but difficult (as yet) for programs will work. "In the next field, enter every first letter of this sentence."
Please correct me if I got my facts wrong.
I had a ton of these on my message board.
My solution:
1) Generate a random number with "todays" date interleaved into it and put this as a parm on the input form.
2) When they post the message, check the referrer string for a valid "datecode" from the referrer URL.
3) I still was getting some slipping through but only for 1 day since they will not go once the date has changed. So then I check if the message has 3 or more links, then don't allow it.
Now I only get 3 or 4 a week, probably entered by hand.
If it became a real big problem, what I would do is change the code above to instead of encoding the date into the random number, encode the actual server time within the random number and only allow a post that is within 5 minutes. That way if they entered a message manually, copied the URLs and then tried doing it over and over again it would only work for 5 minutes.
Buy V1AGR/\ Now!
Naked sluts waiting to chomp your butt for Free!
CEEAlLLiS, only $19.95 a pack. Act now!!
Use HTTP BasicAuth, and give a simple username/password (e.g. "forum"/"forum") in the instruction message that BasicAuth allows you to send to the user in the dialog box. It's nice and quick, well supported, and Spambots don't seem to be able to cope with this just yet; but it won't be long :-/