Slashdot Mirror
jQuery Dev Bemoans Overwhelming Spam On Google Groups
angryrice tips a blog post by John Resig, lead developer for jQuery, about the failure of Google Groups to manage spam, declaring attempts to use it as a public discussion system "completely futile." Quoting:
"The final straw was placed upon my patience with the Google Groups system a few weeks ago. Spammers are now spoofing the email addresses of existing group participants to sneak their messages through. Previously you would've seen a delightful 'FREE MOVIE DOWNLOADS' spam from 'freemovies123@gmail.com' — but now you'll see it coming from existing group users — or even the group moderators themselves. This cheat completely bypasses the moderation system since the spammers are pretending to be pre-moderated users. The Google Groups system is completely fooled. The spam message comes in claiming to be from an existing group participant — and according to the Google Groups interface there is no difference. If you click the user's name you'll be taken to a full listing of that user's posts (with the spam messages delightfully interspersed)."
Looks like a good time to learn to admin a mailing list.
Caveat Utilitor
You get what you pay for.
Maybe if we created a mail header with the pgp signature of the message in it we could train our spam filters to filter on that. Google could silently inject the header into its mail clients... no one would need training. Email would look the same. Clients unaware what to do with the header could ignore it. Inside systems like Groups you could see "verified" or not on the email.
[signature]
Cue Spam Comments in 3...2...1
I used to be an avid newsgroup participant way back in the day. The flamewars were legendary, and the amount of technical information exchanged on some of those groups was beyond description.
If there were a way to use spammers for fuel, I'd have no qualms solving our energy woes that way ...
If libertarians are so opposed to effective government, why don't they all move to Somalia?
Isn't GPG / PGP email signing perfectly suited to handle this?
All you need is a way to build a tree or chain of trusted signatures. The root of the tree could be the person who created the group.
Yahoo chat as well seems to be overtaken by this spamfest. They have tried to address it with captchas, but the spammers simply go ahead and entire the captcha code and keep spamming. They could require credit card verification to make it harder to open massive numbers of accounts, i suppose. Maybe they could have some sort of scanner that would look for sequences that could identify common patterns in spam messages and flag these messages for moderation. Even moderation itself is ripe for abuse with moderators who abuse that power that they have. Perhaps another solution is a voting system on particular messages like that on slashdot, in this case, simply as to whether the message is spam or not, the messages which are voted to be spam are basically collapsed but could be opened with a click, or can be shown with a show "spam marked messages" feature. Could be useful both on chat and also on message boards.
The spammers Behavior are really destructive in many ways, this is just one of them. It really should be seen as sabourtage against infrastructure and a bigger efford should be made to follow the trail of money and take down those people who makes the money.
Google's really dropped the ball on spam blocking with Blogger too. I host a couple of random blogs on there, and they've all been hit with a ridiculous amount of spam in the last year. Blogger doesn't even give you something like Akismet... :(
If this is a Usenet group that Google Groups is just providing an interface to, I guess it's time to bring back the cancelbots. UDP against Google. It's come close before.
If this is one of the Google Groups that's a web forum, then they need to require that you actually log in before posting.
... and you can expect to get rained on.
The problems described in the article: Having it so it's not completely obvious a group is moderated, having a choice of either moderation of every post or no ability to control spammers, flamers, and trolls, and no protection against forged moderation sound like issues caused because Google groups tries too much to be like Usenet.
Usenet was a very good idea in the 1980s and early 1990s, before the internet became anonymous and spammers started moving in. My favorite thing about Usenet is that it's easy to read it offline (Google "Leafnode") for people who do not have a continuous connection to the internet--this was the norm in the UUCP-dominated 1980s, when just about nobody had a direct internet connection.
I recently posted a blog about the death of Usenet:
http://maradns.blogspot.com/2009/07/memories-of-usenet.html
MaraDNS is an open-source DNS server.
Clever, too.
Why the hell haven't they put the same spam filters that they use for Gmail on the discussion lists?
Is there anything better than clicking through Microsoft ads on Slashdot?
Time to move away from the antiquated system of mailing lists. Web based forums are much easier to control and a far, far better way of sharing information with users. I hate coming across an otherwise useful site and then having to go to a mailing list to see what other users are talking about.
Google has some of the weakest around. And whats more is becaue Google uses domain keys it is a desired domain because that stuff gets through the spam filters better.
I wish Google had an automated honey pot system where you could drop a google address, and any google account would instantly get shut off for sending mail to it. The idea is you plant the email address in a place where automated spambots will harvest it and poof! no more spammer.
Of course it could be used for abuse and if passed off as a legit account, so there needs to be some registration and tying of spam honey pot accounts to their owners for accountability.
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
Google Mail has a feature in Labs whereby they identify social groups within your email contact so that if you exchange a lot of emails between a certain group of people and suddenly add a new recipient it will flag a possible problem. Surely it would be possible to apply a similar methodology to Google Groups only with the IP addresses messages originate from - send from a new IP assignment and the message gets moderated, no matter how many successful posts you've made from elsewhere.
UNIX? They're not even circumcised! Savages!
But maybe this will kill Google Groups and thus eliminate 99% of the spam on Usenet. We can hope, anyway.
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
I've been wondering if/when Google would make some sort of effort to deal with the problem. You'd think that a company that's gone out of their way to hire brainiacs could come up with *some* sort of solution. I'm a little surprised they've let it spin this far off into the weeds.
I see a lot of Google's products needing the oh so familiar Beta label again.
Seriously, Google's offering is not without it's serious drawbacks, and I suspect that the good stuff is to be had from actual paid services. However, this kind of letting crap slip where people can spoof the name of a valid member is a serious Alpha quality flaw. What's the point of identifying anyone, if everyone can pretend to be everyone else? I mean that is the actually concept of identity, to uniquely label something as different as other things.
I think Google is trying to take on more than it can handle and it is beginning to really show now that they've removed the excuse of "Beta".
Maybe it's because they want to encourage you to use Gmail, which they control and can extract some income from, instead of Usenet, which they have only a passing acquaintance with and can't squeeze a penny out of.
Google Groups was a good idea with a bad implementation. Last time I checked, there was no fast way to report a spammer, you have to click 3 or 4 times and be redirected to different pages before having just one message successfully reported.
I created and admin a Google group for my son's high school team. We have coaches about 120 parents in the group.
Even though it's a pain in the ass, I chose to moderate messages for new members. Still, spam gets through. As the group's admin, it's embarrassing to see graphic messages and know that all the parent's on my kid's team are seeing it. Also, moderation means that some messages may not get through in a timely manner.
I'm looking to migrate the group to an alternative now.
Welcome to my nightmare
I think you're gonna like it
I think you're gonna feel... you belong
A nocturnal vacation
Unnecessary sedation
You want to feel at home 'cause you belong
Welcome to my nightmare
Welcome to my breakdown
I hope I didn't scare you
That's just the way we are when we come down
We sweat and laugh and scream here
'cuz life is just a dream here
You know inside you feel right at home here
Welcome to my breakdown
Whoa
You're welcome to my nightmare
Yeah
Welcome to my nightmare
I think you're gonna like it
I think you're gonna feel... you belong
We sweat laugh and scream here
'cuz life is just a dream here
You know inside you feel right at home here
Welcome to my nightmare
Welcome to my breakdown
Yeah
As a concerned legitimate user of /. I must offer these words...
FREE V1AGRA!!
FREE MOVIE DOWNLOADS!
UNLIMITED P0RN FREE!!
(pdxp is no longer with you. the spambots are now infesting his brain, and you are all next)
Usenet continues to be a wonderful source of binaries and technical info.
It is still the source for most of the releases I see on p2p networks.
It is no worse than the political pap and S/N ratio say on a site like Slashdot.
Usenet simply requires a thick skin and the willingness to self-manage your experience. Those unwilling to do so have been complaining of "tragedy" since the 2nd week of Usenet's existence.
And sadly they are bringing their puckered asses and regulatory sensibilities to the rest of the Internet, turning it into a suburban picket-fenced nowhere.
I manage a moderated google group and I have received spam "from the group" from someone who is not a member. This makes me think that they sent it directly to me and just spoofed the headers to make it appear to come from google to get past my local spam filter. I wonder if this is what is really happening?
"Computer Scientists can count to 1024 on their fingers" (non-mutant, non-mutilatated, human computer scientists)
Maybe they pool their resources in Google Wave and ditch Google Groups as soon as Wave is ready.
Google Groups serves as a face to Usenet, yes, but it also advertises itself as a place to create new groups which are hosted by Google, as an alternative to setting up your own mailing list. I suspect the jQuery folks are using a Google hosted group. The spam situation is indeed ridiculous, and Google could indeed do something about it. They even have "report spam" buttons on all the messages, but so far as I can tell clicking on those buttons has no effect. At the very least it should hide the messages from me that I mark as spam. But no, it doesn't even remember which messages I've marked as spam from login to login. They've just dropped the ball for some reason.
We were having some problems with this on the wimax hacking google group.
About a month ago I set all posting options to members only (read is still public, the group is listed in the directory, and there is no moderation). I then set it so people need to request an invite to join. The signup page says "Sorry, about the inconvenience, but spam was starting to ramp up, so now users have to request membership manually. Anyone who is human is welcome, and encouraged to join."
There has been zero spam since the change.
It would be nice if there was an option to just let people solve a captcha to join the group, but until then this solution is working fine.
Maybe the answer is to block posts to USENET that come in via Google. That seems to be the source of the trouble.
Looking at the newsgroup "comp.lang.python", all the spam seems to be coming in via "posting.google.com" with GMail return addresses. Bulk-created phony gmail accounts are such a source of spam that they should be blocked until Google gets their act together. At this point, we have to view GMail like Hotmail, another free email account system made useless by spammers.
Hotmail is widely blocked. Next, Gmail?
Why don't they set up an option for the admin to require all posts to be validated by a digital signature?
I use google groups, and maybe i'm lucky, but I don't get spam hardly ever in my groups.
I get more spam in my mail then I do in groups, and I probably get less emails then I do new group posts a day.
(i get maybe 10 new emails a day, and 100+ new messages in my fav groups)
Be seeing you...
...how exactly do the spammers know which users are pre-moderated on which groups ?
Just blasting all addresses, regardless of validity may be a good tactic for standard mailboxen, but it seems to me that the ratio of pre-moderated to not-even-subscribed on any given group would be pretty prohibitive. Coupled with the presumably already reasonably low positive feedback on spam (which is not to say that the roi is bad, mind you), and you *should* get only fragments of percents of successfully inserted mails - UNLESS you have prior knowledge of which addresses will work on which groups.
What a depressingly stupid machine.
So, let me get this straight. The purpose of this article is to complain that there's a popular public service for communication, and ohnoes it gets spam!!1!1!!! HALP HALP HALP And what's more, *gasp* a developer of at least a semi-popular library of sorts doesn't like spam! This changes EVERYTHING! My entire perception of the internet itself has changed FOREVAR!
The reason, at least to me, seems abundantly clear: Google has the attention span of a three year old. They fixate heavily on something for a while... then their attention drifts and they are off to the next shiny thing. They've got a lot of products, but no clear vision or effective management.
then they need to require that you actually log in before posting.
That is really up to the group administrator.
To control who can post in your group, please follow these steps: 1. Click on the "Group settings" link on your group's homepage. 2. Select the "Access" tab. 3. Choose an option under "Who can post messages?" and click "Save Changes." If you choose "Managers only," owners and managers will be the only ones able to post messages to the whole group. If you choose "Members only," your members will be able to post, but non-members will not. The "Anyone can post" option allows all Google Groups users to post in your group.
Also, you may want to consider the option "All posts are held for moderation" if you wish to review messages before they're posted to your group. This option is located toward the bottom of the page, in the "Message moderation" section.
For the safety of your group, only members have the options of uploading files, and creating and editing pages. It's up to you whether you'd like to also restrict these actions to moderators only.
Not sure if that helps for the spoof aspect of the declared problem. May be. Maybe not. I've never ran a Google group.
Reply to That ||
When they spent money to buy Dejanews, there must have been some plan to make money. Oh wait, this was during the dotcom bubble, when having a plan to make money was seen as crusty old world thinking.
I beg to differ, after having borne the brunt of sustained malicious trolling.
In the long run, you can fight spam but not trolling. Trolling is a deeper problem, which obviously cannot be fixed by technical means. Flamewars are a close second.
Spam levels were above 96% in some groups I accessed. And more than 90% of the spam came from Google Groups. I guess they put it on autopilot without any spam checks and walked away. So I just blocked all of Google Groups in my killfile. At least for now, any legitimate posts from there I will see if someone from outside Google Groups posts a followup and includes it. But some of the groups are just dead, now. In a couple cases it's definitely due to the spam.
now we need to go OSS in diesel cars
This is more to do with Yahoo Groups than Google Groups but they seem similar. Recently I've joined several Yahoo Groups about specialized ham radio topics. Nearly all of them keep their archives private. I have apply to join (basically push a button and say who I am) and then wait for approval from the admin. Once approved I can read the archives and also post. Posting from members is usually unmoderated. It's painless enough but still very frustrating when I'm just searching around for information and a quick look at the archives is probably all I want.
I don't mind having to join if I want to post but do they achieve anything by keeping the archives private? Yahoo obscure the email addresses so spammers' 'bots are not going to get much from them. I've asked several admins "why do you keep the archives private?" and have not received a convincing answer. It usually goes something like "I understand your frustration but we have a lot of trouble with spam" and sometimes goes on to imply what a silly question I asked. Well ... I still don't see how keeping the archives private helps to reduce spam. I haven't been a group admin so maybe I'm missing something.
I can understand keeping archives private or non-existent for a group on a personal or private subject but that doesn't apply to these groups.
My guess is that this is Yahoo's default setting when a group is created and few admins really think about it. Of course Yahoo want as many people as possible to join.
Bingo. They need a moratorium on new products for 3 years while they chain the engineers to big, burly product managers and get all of their offerings on the same page.
Of course, that's (more or less) what happened at Yahoo!, and Google took the opportunity to fly right past them.
You're a group of technologically literate people. Why don't you just sign your messages and verify based on signature, rather than something completely meaningless like email-address?
And once again: Why the hell does google not sign all messages which pass through gmail as "really did come from this address"?
-- 'The' Lord and Master Bitman On High, Master Of All
Google's web apps are notorious for poorly handling email headers.
Case in point:
sparr@domain.com is subscribed to the group@googlegroups.com
sparr@gmail.com checks sparr@domain.com via POP, and can send as sparr@domain.com
using the gmail interface to send email From:sparr@domain.com To:group@googlegroups.com (Sender:sparr@gmail.com) fails, resulting in a bounceback message *TO SPARR@GMAIL.COM* stating that sparr@gmail.com is not a member of the group.
I could understand some concern over spoofing and authentication if this was coming from a third party, but I am sending from a google application to a google application. Gmail has already verified that I have permission to send From:sparr@domain.com, why doesn't ggroups trust that?
Google flew right past them in what sense? Google is bigger by some measures, and rakes in a ton of revenue from serving advertising across the web, but... Other than search, where Google and Yahoo! compete head to head - Google's offering is almost always second best.
are you getting all this, google? priceless business advice from slashdot comments. it's not too late to change now and save your business.
Why do people want to move away from that which is "antiquated"? Many technologies are antiquated but they're still the best way to do things for most people. In vitro fertilization may be easier to control but I kinda like the old way of doing things. The CZ75 is a fine pistol but the 1911 is still at least as good. And while web forums have their uses, mailing lists and usenet groups are still the best way to simply move information without visual decoration. They also have many wonderful advanced features noted by other respondents.
That was the obvious question to me too. A buck a user a month for Postini has been without a doubt the biggest bang for a buck that I've spent at my current job. And you get the MX servers in the bargain.
If this is essentially the same thing as Usenet, it's no wonder. NNTP was designed in the days when we were generally able to trust people not to be malign - it's a very trusting, open protocol, and when people or servers broke the rules, sensible people would stop peering with them. Sophisticated, malign groups of people are a problem in any system, but particularly for systems where there's a lot of built-in trust.
For every problem, there is at least one solution that is simple, neat, and wrong.
In general, Google wants to solve every problem with an algorithm - and if it can't be solved with an algorithm, then by definition, it can't be a problem.
Spam can't be solved with an algorithm.
This isn't just Google Groups, Blogger is collapsing under spam too.
I myself just wrote about this the other day.
You can be an insane coder too, read: Insane Coding
Google doesn't work backwards. If you've been on Wave yet, you can tell that this is the way they want to go with group discussions.
Back in the day when Dejanews was a "cool web 2.0" like thing for Usenet and Usenet was still popular, they could manage the actual, pro spammer attacks with handful of people. Those were the days when CNET had "help.com" which allowed complete newbies to post questions to Usenet.
Now Google, with impossible to imagine computing resources lets the core Usenet _and_ their own private groups gets polluted by trivial spam. Yes, trivial since even my stupid mail filters can sort that kind of spam without even touching bayesian etc. filters.
It is almost like pyramid scheme. Spammer uses Google groups infrasacture to post pirate software download forums which are solely gathering income from Google adwords. That happens on a big5 one, not some alt.conspiracy low traffic thing.
In first days, I thought Google didn't care on purpose of promoting their own, closed, moderated fake groups but it was a total tinfoil hat theory. They simply didn't/doesn't have competency to carry that kind of job which 2-3 experienced admins did while Usenet was 10x-20x more popular.
Why on earth such an advanced developer, especially in that area would require "Google Groups"? Why not run a private NNTP/web Hybrid which is also perfectly available to index/use?
I bet there are solutions using JQuery itself, not my area so I just shut up at this point.
Ask any experienced Admin, Google News would be "delinked" long time ago if it wasn't carrying "Google" name.
I have seen amazing things and they weren't some "123movies" spam, they were coming from real World criminals, UN wide accepted terrorist organizations and some real huge pyramid scheme running guys.
When you confuse it with a real "managed" server and you spend your time reporting it with headers taking your own time to Google, they send back a freaking template saying they are using your report for statistics etc. I wonder if any law organizations spare their time to watch public messages instead of wondering after teenagers downloading some movies.
I've not seen the spoofing from existing members, but I've enabled "moderate new members" and spam to my group has dropped to zero. I've moderated a few legit new members but I've also caught 100% of the spammers (who need to be manually identified and deleted). Albeit this has risen to almost 1-2/wk fairly consistently.
That 90% of the problem is that alternative jobs don't exist? (On that note - only have to wait, First World lifestyle is inexorably converging on the average Nigerian's.)
you had me at #!
Perhaps you haven't visited lately. Although many groups are harmed by spam, there are thousands of active (non-spam) newsgroups.
you had me at #!
Maybe they could use a decent spam filter, such as mollom (http://mollom.com), which is pretty adept at classifying content.
I am the Shield Anvil. And I am not yet done.
They should block the entire IP address block of the village / town it came from. Then let vigilante justice take its natural course.