Buried By The Brigade At Digg

Slashdot regular Bennett Haselton writes in with an essay on a subject we've dealt with internally at Slashdot for years: user abuses of social news... this time at Digg. He starts "Alternet uncovers evidence of a 'bury brigade' coordinating efforts to 'bury' left-leaning stories on Digg. Digg had previously announced that the 'bury' button will be removed from the next version of their site, to prevent these types of abuses, but that won't fix the real underlying issue — you can show mathematically that artificially promoting stories is just as harmful in the long run. Here's a simple fix that would address the real problem."

Even if you just arrived from Mars and have never heard of Digg, that description of the service should make it obvious how easy it is to game the system, by rounding up groups of friends to vote on stories that you want to promote, or to bury stories that you want to kill. The former type of abuse (and it is abuse, under Digg's Terms of Use; search for "organized effort") is far more common, since people usually have more incentive (commercial or otherwise) to promote their own work than to bury someone else's. And in fact, Digg has announced that the next version of the service will remove the "bury" button, replacing it with a "Report" button for reporting bona fide cases of abuse, not just to bury boring stories.

The thinking seems to be that abusive "digging" to promote a story, is less harmful than abusive "burying", and this has the ring of plausibility — that a creative effort is better than a destructive one. After all, Alternet had previously highlighted several artificial right-wing "digg brigades" mentioned in their story (Diggs And Buries, theliberalheretic, etc.), but they didn't blow the lid off of the situation until their report on the Digg Patriots bury brigade, as if to say, "Now we've found something really scandalous!" Annalee Newitz cheekily reported on how she bought votes to boost a story to the front page of Digg, but probably would have felt guilty if she'd hired a service to bury someone else's story. And when a Digg user organized an effort to bury Ron Paul stories that he thought were "spamming" the system, Ron Paul supporters protested that they were merely organizing to vote up stories they agreed with — the clear implication being that this was more honorable than organizing to vote stories down.

But this, I think, is a fallacy. If a story's ranking is artificially inflated, then the extra eyeballs for that story have to come from somewhere, and they come from users paying less attention to the other stories that the phony up-and-comer pushed out of the way. Artificially bumping a story up is just as harmful as artificially burying a story, but the harm is distributed among many innocent victims, not just one. (By the same reasoning, in fact, you could argue that burying a story does no net harm to other users of the Digg site, because the harm done to one story is cancelled out by the benefit to all the other stories that rise in prominence when the victimized story is pushed out of the way. So by strict economic logic, recruiting friends to boost your own story at the expense of everyone else's, is actually more harmful than organizing a bury brigade!)

So I don't think that Digg's replacing the "bury" button with a "report" button will fix the problem. For one thing, obviously groups could abuse the "report" button in the same way — issuing calls to action to report a story for violating the TOU. Since a flurry of bona fide abuse reports is presumably what Digg uses to identify and remove truly abusive stories like MLM spam, how are they going to tell the difference between these cases and cases of abusive "reporting"? (My suggestion: See if there is a sudden change in the percentage of users who view a story and make an abuse report. For stories that are genuine TOU violations, the percentage of users who "report" it should remain steady; for stories that are victimized by a "report brigade," you'll see a sudden spike in viewers and in the percentage of those viewers who report the story for abuse. This might have worked for detecting and stopping the bury brigades as well, although we'll never know now.)

But more fundamentally, even if this change does stop the "bury/report brigades" from killing stories at will, that only fixes the most obvious symptom of the underlying problem, which is that the system can be gamed by recruiting your friends to vote either way. It won't stop "brigades" from artificially promoting shallow stories that agree with their opinions, which does the same net harm overall.

Indeed, the most long-term harm that the DiggPatriots Yahoo Group might have done is that their cheating was so egregious that it makes other examples of cheating look benign by comparison, and might prevent people from realizing that "benign cheating" is just as harmful. As detailed in the Alternet report, the DiggPatriots group talked openly about cycling through different Digg accounts and circumventing bans on their IP addresses. The welcome message to the Yahoo Group told new users that the group was operating "under the radar." The group leader, a woman with the handle "bettverboten," talked about how to prevent Digg from monitoring their actions. And of course the vast majority of posts were calls to bury stories. But what if all of that had been inverted? If the group had operated in the open, while still focusing on recruiting conservative members? If each user limited to themselves to only one Digg account like they were supposed to? And if they focused not on burying stories, but on digging stories that promoted their viewpoints? Just as bad. It just doesn't sound as bad.

I still think the only way to make Digg a true meritocracy, would be to use some version of an algorithm I outlined in an earlier article, inauspiciously titled "How to Stop Digg-cheating, Forever." The gist of it is that in addition to collecting votes from friends, stories should be shown to a random subset of users on the site (perhaps in a box that occasionally appears at the top of the screen when they're logged in), who are asked to vote it up or down. The votes of a random sampling of users would be more representative of how much value the story would have to the Digg community as a whole. Even if most users who are asked to vote on a "random story" simply ignore the request, all you need is to show the story to a large enough sample that you can measure the difference in responses to a truly good story vs. one that has been promoted by digg-cheaters. You don't necessarily have to run this procedure for every story, only the ones that are about to gain some benefit from a large number of diggs (such as being pushed to the front page), and you need to decide whether the story really deserves that big boost. The only way to game that system would be to organize a group of dedicated Digg users so enormous that they constituted a significant percentage of all users on the system — something pretty hard to do without getting caught.

Still, the only site that I know of, that uses a version of this "random sampling" algorithm is HotOrNot.com, which lets you recruit your friends to vote on the "hotness" of your picture on a scale of 1 to 10 (by sending them a link to that specific picture), but also shows a stream of random pictures to visitors, so that your picture can collect votes from strangers. If the votes from the users who visit your picture via the link are significantly different from the votes from users who see your picture via the random stream, then HotOrNot discounts the votes from users who view your page via the link. This prevents digg-style gaming from people who want all their friends to give them a 10. (Note that if you think about it, this is essentially the same as always throwing out the votes from people who visit your picture via the link. If you collect votes from group A and B, but you only count the votes from group A if they agree with the votes from group B, then you're really only counting votes from group B! All the extra votes really give you is the ability to brag that X many people voted on your picture.)

This seems like the simplest way to prevent Digg-cheating, although there may be others. Still unresolved is how to solve the general problem of "gaming" in traditional media and the blogosphere. For the foreseeable future, it's going to be the simple truth that if a major media outlet wants to run a story, it will be heard, and if no media outlet wants to run it, it won't be heard, regardless of how many viewers or readers would have voted in some hypothetical poll that, yes, they want to read that story, and yes, they liked it afterward. That's true for Internet articles as well, except to the extent that a deserving article might be rescued from obscurity by Digg, but the more that system can be gamed, the less it will reward articles that really deserve it. Digg is gameable because power users can recruit votes from their friends; the media and the blogosphere are so obviously "gameable" that we don't even call it "gameable," because "power users" — media outlets and A-list bloggers — can run whatever they want. Right now, the only way I can think of to change this situation that is even logically possible, would be for a site like Digg to adopt some version of the random-sampling algorithm, and to continue growing in power until a significant percentage of the public (not just Internet users, but everybody) relied on it for information. Then, if you had something important to say, people would hear it, but you wouldn't be able to cheat your way to the top.

The ultimate irony is that Alternet's story may never have seen the light of day, if it hadn't been the beneficiary of the same gameable, non-meritocratic inefficiencies that exist in the media-blogo-outrage-o-sphere, just as they exist on Digg. Yes, the Alternet story deserved to be heard, but you don't get the publicity you deserve, you get the publicity that you organize, and Alternet had the organizational publicity structure in place to get their voice heard. If a kid blogging from his bedroom had infiltrated the Digg Patriots group and made essentially the same discovery, would anybody ever have heard about it? (Well, maybe, because of the political hot-button factor — but even then, only after the story had been picked up by a major site like Alternet.) A truly meritocratic Digg algorithm could make it possible to get a good story out without a lot of organizational support behind it — and to ensure that an organized effort can't kill a good story either.

Comments

tl;dr

[biryokumaru]

Yes... a simple fix...

Re:tl;dr

[Vintermann]

It IS simple. And it's very traditional, too: the first true democracy, Athens, relied heavily upon random sampling to achieve popular representation. It's called sortition.

It's one thing slashdot got right, too, or at least more right than reddit and digg. Mod points are awarded randomly here, if you've been member for a while (a year, isn't it?). Since we don't get to mod all the time, we do it more conscientiously when we do. A fair evaluation by a representative sample gives far better results than what you get elsewhere, which is usually empathetic votes from people who very strongly disagree or agree with you.

(I admit, I could be a little biased here. I get much, much lower mods on reddit compared to slashdot, where it sometimes feels everything I write gets to +5!)

Rob Malda, fact is you were right in a way digg and reddit simply weren't. Their approach was appealing in the start, but didn't scale well as their readership soared. Why don't you capitalize on this more? Take the next step?

Allotted mod privileges is great, but it should also be random which comments and stories were eligible for moderation. Maybe just a tenth of the comments on each story, with these sorted on top (treewise), so that you avoid the Matthew effect, that already highly modded comments/stories get all the attention.

The firehose could really shine, if you took your old ideas (which are the same as the old ideas of the Athenians) to their logical extension.

And what about yelp?

[Anonymous Coward]

I have been yelping for about a year and I see the owners of places abusing the crap out of that system. I now actually have yelp staffers emailing me asking me to change my reviews at the bequest of an owner of a restaurant or it will be removed...

Re:And what about yelp?

[aquila.solo]

That's not from the owners of places as much as extortion from Yelp itself. If a company doesn't pay to become a "preferred member" or some such BS, then Yelp shows the unfavorable reviews. If the company is a "good citizen," then Yelp shows more of the favorable reviews; even to the point of pressuring users like you to modify their posts.

In short, Yelp is pretty much useless as a source of unbiased information.

How is this new?

[Lumpy]

Slashdot had this problem long before Digg even existed or was even an idea.

CmdrTaco tried several ways of dealing with it, but it still exists today. Shill accounts designed to moderate down a disliked opinion. Mod down mobs. I have seen this stuff in action on lots of people's posts.

Typically the shill actions and mob actions get undone by the general populace but you can see the effects by looking at the moderation of a hot topic post. 30+ moderations with a crapload of overrated,troll, etc.. when the post was 100% op topic are a prime example of this.

Re:How is this new?

[SashaMan]

I totally disagree. While of course this happens to some extent, and while in general Slashdot has some pretty common points-of-view (FOSS supporters, generally libertarian leaning, etc.), the level of groupthink and mob rule is many orders of magnitude less on slashdot than on digg. While part of this may be due to the audience, I think the biggest factor is the moderation system. There is no "agree" or "disagree" moderation on slashdot. There are certainly many times I've moderated stuff as interesting or insightful even if I didn't necessarily agree with the sentiment of the poster.

On digg, it's all up or down. You'll frequently see comments like "**** Republicans!" rated very highly. Whether or not you agree with Republican political views, putting four asterisks before their party name adds nothing to the discussion. You rarely, if ever, see a comment like that rated highly on slashdot, unless there's something sarcastic behind it. Of course, now you'll probably see lots of comments like that as responses rated highly :-P

Re:How is this new?

[glavenoid]

Yes. I'll let my .sig explain my views on this matter. Actually, on second thought the 120 character limit is, well, limiting.

Metamoderation used to actually *do* something (I think). It allowed the metamoderator to determine whether or not the moderation was fair. It did this by posting the comment and one of its moderations, but not the handle of the person who posted the comment. In this way, the metamoderator was not instantly swayed by *who* was making the comment, and the entire point was to determine if the "+5: Funny", "-1: Troll" or "+2: Insightful" were warranted moderations.

As it is now, I don't even know what the fuck metamoderation is supposed to do, or what it is for. I have no idea. All I know is that there is no decent explanation of what metamoderation is supposed to do, how it is going to accomplish it, or what. To those of us who used to be regular metamods, the new system is entirely broken.

Slashdot is never wrong!

[aronzak]

The fix is just to go to Slashdot for all your news, ever.

Slashdot is never wrong, right?

Re:Haha

[Abreu]

I am becoming increasingly jaded at the USian right wing and their Obamapanic

Really guys, your president is Center-Right from the perspective of the rest of the world, and it is just sad to see him try to meet the right wing halfway in all of his policies, only to be branded a "dirty commie" over and over again...

The USA needs Democrats with balls to propose truly liberal policy, not watered down compromises, imho

He's wrong

[recoiledsnake]

But this, I think, is a fallacy. If a story's ranking is artificially inflated, then the extra eyeballs for that story have to come from somewhere, and they come from users paying less attention to the other stories that the phony up-and-comer pushed out of the way. Artificially bumping a story up is just as harmful as artificially burying a story, but the harm is distributed among many innocent victims, not just one.

Nah, burying skews votes by not allowing corrections. Lets imagine that there are 50 people who are gaming the system by being an organized collective and that Digg needs 50 buries to kill a story. If it was Reddit, the 50 downvotes could be balanced by, say 100 upvotes. But on Digg, not even 1000 'diggs' can counter the 50 buries. This allows a small group to have a significant chilling effect and effectively a veto on the content. Artificially bumping up is much less harmful.

Why bury is worse

[fermion]

Skimming the essay the basic assertion seems to be that having too much to read is worse than the occasional killing of a story. This is common thinking of those that wish to protect us from unfavorable information. That there is someone who knows better than you what you need to know. Of course a selection process is neecessary, there is no way to print all the minutia that goes on in the world, but that selection should be based more on interest rather than the facts presented in the story. For instance, if one is interested in Miley Cyrus, then one wants everything on the subject, not just the Disney edited factoids.

Which is why burying is worse. Burying is act of preventing people from hearing differing opinions. While it is true that artificially inflating the importance of information also has negative effect, many different viewpoints can be overinflated, so we still end up with a variety of opinions. A comment system allows all to reflect on those opinions.

It is true that groups can game the system to inflate the ranking of stories, but look at it this way. On has a finite amount of time. It is relatively trivial to use the time to bury selective stories, but becomes more complex if one wants to do the same thing by inflation. One has to inflate a larger number of stories, and at the same time others are doing the same with stories they agree with. All sides are probably going to inflate the stories that reflect best on them, as inflating politically correct but embarrassing stories would not be beneficial.

At the end of the day, and inflation policy is more likely to result is a selection of the best stories from a variety of opinions, while a bury policy will likely cause the best stories to be buries simply because a few people disagree with the viewpoint. The question is one interested in presenting information that people can choose from, or if presenting an opinion in hopes that everyone will agree.

Re:Haha

[bsDaemon]

The last time Democrats in this country had balls, they seceded from the union to keep their slaves.

Catch 22

[hellfire]

We also need a populace which will support Obama if he leans left and shows jerks like Palin just how big his balls really are.

But that's the true problem. We have an uneducated, jaded populace that doesn't vote their heart (if they vote at all), we have two parties who don't want to lose any control on government they have to allow a major third party, and we have a bunch of fat lazy rich people who also control much of the media who want to maintain their control on government as well.

Obama was a good choice, IMHO, but he's basically been given crap to start with, and anything less than diamonds from that crap is spun as failure by the political machine. No he's not perfect, but the entire country has been positioned as center right, and our system of checks and balances, while good, has been pushed to the right hard over the past few decades and we don't have enough force to push it back. Even if we did it will take time as our system of government was built to create "stability", and major changes are sometimes harder for no other reason than it's hard to change the status quo.

Re:Catch 22

[inKubus]

Bush had 9/11 and Iraq, arguably both were capitalized on to avoid checks and balances. It's funny when Obama passes health care the crazies fear socialism but when Bush increases the size of the government by 50% after 9/11 and they didn't even look twice, flying their flags on their SUVs.

Obama needs another four years so he can actually do some real socialism. This is really the modern way to provide basic human needs and it is possible to be fair with the increased real-time statistics and data reporting (social intelligence) being built as we speak. I think the U.S. is tending to be more socialist as we realize that some of this is possible to do right with government guidance (if not total service). There are lots of examples where a free market with pure competition is just not possible, and health care is one of them.

The crazies are supporting a borderline fascist policy put forth by the republican mainstream for 8 years, where corporations can do no wrong. At the same time they were increasing the size of the goverment, except only the part that serves the corporations and their interests (security, defense, empire-building, etc)! But thanks to the evangelical base, whose leadership is probably not even Christian, they have a decent sized force of 2-3 million that will do just about anything "for the lord" so they can make it look a lot bigger than it is (this story being a good example)

Between the two you have the vast majority of americans who don't have feelings on the matter and vote with their pocketbooks. Obama has not raised taxes on the middle class. He's really sending a ton of money back to the people (yes, they are borrowing it from the future, but Reagan invented this), and he's helping America to modernize and provide streamlined services for all it's citizens. He's the first techie president. However, you gotta be careful what you do for labor unions. We are probably better off with regulated labor unions and more jobs just to keep people off the streets. Of course, the real reason those jobs are being lost is that the crazies don't feel the need to obtain advanced education, which means they aren't real useful in a modern economy about moving stuff at highly lean and efficient paces, with lean manufacturing hopefully done by robots so you can "hire and fire" just by flipping a switch.

So, crazies: get an education, enjoy your free health care and stop being racist--it's so 60 years ago.

Re:Haha

[operagost]

Funny enough, the voting patterns switched after the Dem party passed the Civil Right bill back in '65... the formerly "I'll never vote Republican" voters switched at the "betrayal".

Strange, because a greater percentage of Republicans voted for the bill.

The Democrats had a HUGE majority back then, and the Presidency... so we know for sure who was opposing REAL progress.

Re:**** Republicans

[corbettw]

Middle-aged Idiot Leading Fascists?

Re:Haha

[dgatwood]

Actually, a significant group of Democrats were strong supporters of Lincoln's position in the war. No surprise since even back at that time, both political parties shared some significant common roots, having both been formed by pulling together members from the same basic set of defunct parties that existed previously. The party divisions were (and to an extent, still are) largely arbitrary.

As for the "pro-slavery" Democrats, one could reasonably argue that this was the last time in the history of our country when politicians actually represented the views of their constituents.... Not that their constituents weren't wrong, but it's hardly fair to blame the politicians for actually doing their jobs (for once)....

And to be fair, it was never about keeping their slaves, but rather to protect states' rights to decide whether or not to allow slavery. You know, the same sort of states' rights agenda that Republicans are pushing at the moment. Humorously, even in the Civil War era, the Republicans' view on states' rights depended solely on which party held the most power in the Federal Government. When Federalists were in power, they screamed "States' Rights!" at the top of their lungs, claimed to be for a smaller (federal) government, and generally tried to impede the Federalists' progress. The moment Republicans came into power, they took as much power as they could get and no longer cared about anyone's rights. Sound familiar? It should. It still happens in both the Democratic and Republican parties today, with just as much vigor.

And like most governmental issues today, there was a lot of money involved in the slavery debate. No surprise, again, that at least initially, the wealthy slave owners won, keeping their power, up to the point of splitting off into a separate country. It would have remained that way, were it not for somebody standing up, saying "No, this is wrong", and being willing to take the country to war to make the point.

But in the end, they shot him for it. Who is standing up now? Certainly not the Republicans, and certainly not the Democrats. Today, the people with the money win, because everyone is looking for the next big handout and no one wants to take a bullet.

These days, neither party cares in the slightest about states' rights except when they can use it to their political advantage. It's all just a charade to ensure that neither party every truly has to answer to the public as a whole. Don't blame me. I Voted for Kodos. At least a cartoon character is a real change from what we have now.

Re:Haha

[PopeRatzo]

Ah yes, the Dixiecrats

The racist past of the Democratic Party is an interesting story.

Clearly, there were lots of racists who called themselves Democrats, but over the past several decades, with the ascent of people of color in the Democratic Party, those racists would have become more and more uncomfortable as Democrats. Today, a black man is the head of the Democratic Party. By definition, any serious racist would obviously not remain a member of a party that is led by a black man.

So where do you think those racist Democrats went? Maybe they just stopped voting, maybe they joined some third party (though the numbers don't really bear this out). There's really only one party to which the racist "Dixiecrats" could have gone.

There have been 98 black members of congress. Since 1900, only 5 of them have been African-American. There are currently zero African-Americans among the 178 Republicans in the House of Representatives.

Re:What?

[M. Baranczak]

the near epileptic fit the left had over Bush

Yes, the blind, irrational hatred of George W Bush was a sight to behold. What did he do to earn such enmity? Besides the two wars, the secret prisons, the torture, the illegal wiretaps, Katrina, and the collapse of the economy, what exactly did he do that was so bad?

Re:Haha

[PopeRatzo]

Please, Media Matters is a left wing front organization. Everyone has a bias, but theirs is most obvious.

This is an example of another way to silence debate. Any media organization that is not Fox or some right-wing blog is not to be taken seriously because they are liberal. Even when all they do is show verbatim clips and transcripts of the things that are being said in the right-wing media. No clever editing like Breitbart, just letting the Right say what they want to say. I've heard that same line about "such and such newspaper/network/blog/magazine is a left wing front organization" used to describe every network but Fox, every newspaper except Sun Myung Moon's Washington Times, every radio station except those owned by Clear Channel of SRN, every publisher except the Eagle/Regnery group. Every university is a "left-wing front". Every union is a "left-wing front". Basically, for the Right-Wing in America, at least half of the country consists of left-wing fronts or far left lackeys. Half the nation is made up of traitors, the enemy. In 2008, according to the Right Wing, more than half the American voters were on the "far left". And why is every Democratic member of congress on the "Far Left" according to the Right? Wouldn't there have to be some people on the plain old Left? No, they're all "Far Left" but if you ask them, everyone on the Right is "Center Right".

Freedom of speech, except for liberals. Freedom of religion, except for muslims. Freedom to pursue happiness and equal protection under the law, except for homosexuals.

The current Right-Wing in the US has a very interesting set of beliefs, one that will be studied by historians for generations to come