Slashdot Mirror
Greatest Task of Web 2.x: Meta-Validation
CexpTretical writes "This Technology Review article about Web 2.x problems fails to mention the 800 pound gorilla in the room when it comes to fulfilling the dreams of the Semantic Web — i.e., assumptions about the validity of metadata or tagging schemes. We can add all of the metadata and/or tags we want to web resources but that does not mean that the 'data about the data' honestly or accurately describe the resource or are 'about the data' at all. This is why Google does not place much importance on the metadata already contained in HTML document headers for search ranking, because it cannot be trusted. And to validate it would require more effort than to search and index that data from scratch. Ensuring or verifying the validity of metadata would be a task equal to that of initially creating it, but would have to be repeated on an ongoing basis. Hence all of the talk about 'trusted networks,' which then require trusting the gatekeepers of those networks. Talk about 'semantics.'" Slashdot's moderation and meta-moderation offer one example of getting useful metadata in a non-trusted environment.
The tagging system might be a better example, or at least an example of mostly useless meta information.
You are definitely correct, but I wonder if this would be the same in a search environment like Google. First, you have a much broader selection of people that can mark meta-data as being accurate or not. Second, people will not see the meta-data without specifically searching for it. This means that the people searching for "swingers in Milwaukee" will most likely be people that don't frown upon such behavior. There are still obvious issues, like people searching for more general controversial terms like creationism/evolution or people that disagree with a certain behavior organizing against certain sites by "moderating" them poorly. I could easily see this happening in politics and religion.
You have enemies? Good. That means you've stood up for something, sometime in your life. --Winston Churchill
Basically, every group and forum is going to be self-selecting to an extent. Even the stories are self selecting, I think. Stories about copyright law enforcement from the entertainment industry get the opposite vibe than stories about companies violating the copyright of GPL'd software. I really hope it's from two different groups of people.
"Linux sucks" I think it actually ok by this point. There have been enough articles about how the Linux GUI stinks, how DVDs don't play, etc that I don't think people are as offended by "Linux sucks" as they once were. Or maybe there are just fewer Slashdot Linux zealots around for whatever reason. Stories that appear on the front page seem to have decently good moderation, IMO. Even political stories usually tend to have differing viewpoints modded up.
But if you want a real challenge, go to the Games section and opine that any Nintendo product sucks. I think the Wii controller is a stupid idea, but if I posted that I'd be modded to -1 in an eye-blink. And don't dare say anything about the Xbox 360 being a decent product! Now there's some serious moderation abuse.
Comment of the year
Moderators are chosen from a large pool according to rules described in moderation guidelines. It stands to reason that if these moderators come to consensus about a post, then that consensus would be descriptive of the post.
No, it just means that their behavior, and the meta-mods of their mods continue to enable their periodic role as mods. In other words, groupthink also impacts who gets to mod.
Don't disappoint your bird dog. Go to the range.
You're completely correct. Most people don't moderate based on how a post or a story meets a certain set of criteria. They only ever get to the level where they "agree" with a story, or "disagree" with it.
When it was first becoming popular, I used Digg for a few weeks. Various people would post comments in stories I had submitted, saying how they had just "buried" the story as "OK, This is Lame" because they disagreed with it. Of course, that's now how Digg is intended to work. It's about a story's merit as a story, not about how it may conflict or agree with one's opinion.
I've seen other people suffer a similar fate. John C. Randolph, who many of us know for his past work at Apple, is often the victim of that sort of stupidity. Unlike 99.99999% of the Digg users, he has some clue as to how Apple works, and what sort of projects they're working on. Yet time and time again he's the victim of morons who outright claim his stories are "inaccurate". Unfortunately, there are so many morons that they completely outnumber those of us who know who jcr is, who know of his great work, and who know how perfectly accurate his information is.
At least here at Slashdot, there's some limit as to who gets moderating privileges. It tends to be only the most intelligent individuals. Contrast this to Digg, where any 12-year-old cocktard is given the ability to moderate stories that tower above their intellect and understanding.
I don't think that meta-validatino can *ever* work.
It's a lazy shortcut to somebody with a brain doing the editing/moderating themselves. The masses are NOT always right and are often wrong, in fact (Wikipedia). Meta-validation is a way to let "the users" do the work, even though those users are generally not qualified to do so. The whole value in say, a web site, is offering useful, accurate information to other people who don't already know that information. Meta-validation is essentially mod rule, with no order or methodology. Meta-validation is a shortcut to profit, and as a result, it will never result in good, long-term information.
Some of what I see, as one of those computer scientists/software engineers, as the benefits of Semantic web related developments, like making some AI type web apps more plausible, possible, and/or effective, others might not appreciate. An analogy from city/road planning might be the choice between the local route versus the highway. Many travelers want the highways with the associated signs that help them find their way quickly from point A to point B, while most of the businesses along the way want all of the traffic to continue to flow through their streets and local routes because they need the business. Similarly, on today's web, what site wants much of its traffic diverted before it even stops by because some metadata has diverted it. Always winners and losers.
When I moderate, I view all comments, even the ones with negative scores. That's the responsibilty of moderators, yes? The moderators have to wade through the sewerage so that you don't have to.
With that in mind, I have no idea why your message is rated as insightful.....
Here on Slashdot, there is a selection process and a reputation system that determines who has the ability to moderate. How does this "Web 2.0" address the fact that anyone can attach and moderate tags?
We all know what to do, but we don't know how to get re-elected once we have done it
We do get to the point where the moderators come to a consensus (or at least a majority overrides a minority) but that doesn't address the issue of whether or not the moderators are correct in moderating a comment up or down. In most situations there is no one correct answer, so how can you get everybody to agree? I am not going to moderate a post insightful if I disagree with the content, and I wouldn't expect anyone else to do so either. In a situation that it is essentially voting for whether you think the post makes a valid point, it is pretty obvious that groupthink will prevail most of the time.
An example of unaccountable, gameable metadata that generates untrustworthy info that is almost as useless, through abuse, as it is useful.
Slashdot's moderation could:
Those few improvements could introduce some accountability and feedback into the now mostly abused meta/moderation system. Until then, Slashdot has little to teach the world about the right way to accumulate useful metadata in an untrustworthy environment.
--
make install -not war
I only recently started posting on Slashdot, but I find your claim that the moderation system is mostly abused pretty inaccurate. While your suggestions for improving the system seem like they would be useful, moderation, which is certainly not perfect, successfully enables a large amount of people to share ideas and thoughts. Usually, at least in my experience so far, the truly thoughtful and thought-provoking posts get modded up, not (only) the ones that most readers agree with. I haven't seen anything that could rightly be called abuse, although I'm sure it exists on a small scale.
To bring this back to the point of the article, the type of metadata associated with moderation is of one of the simplest types. It results in a category and a number. The process of generating that data, though, is complicated yet functional, and I would say it's as good a start as most other systems currently out there to validating metadata in a community (I'd like to see some examples to the contrary).
It's only about .5% of the way to generating truly descriptive metadata, but it does what it needs to do. Now the tagging idea...that has a long way to go.
For my part, I have no inclination to agree with your assertion, because in the 2 years I've been meta-moderating daily, I haven't seen more about 1% of posts[*] that show such symptoms.
:) ), altogether. I had other reasons for no longer wanting to post under a pseudoanonymous moniker, but it's fair to say that the reason I stopped at that particular time was that it was a pretty soul sucking experience to know that nobody would read anything I wrote. He was always there, using his several accounts to drag most of what I posted down below reading threshold before anybody got the chance to read it. Basically, posting to Slashdot turned into posting to the mod stalker. That didn't appeal to me, M2 didn't work, and Slashdot editors were uncooperative. I saw no alternative to give it all up.
I won't address the groupthink claim, but I can address the more general claim that M2 is decent at catching unfair mods. I used to have a mod stalker once, who modded every single post I wrote either overrated or redundant. As you might figure, those appellations are impossible, or at least very demanding, to mod in M2. On the rare occasions that my posts got highly rated before he was able to drag them down below the reading threshold for most posters he also waited with moderation until the thread was dying, so that nobody would catch on to the unfairness of a clearly non-redundant post being marked as such.
Eventually I posted a comment in some thread directly addressing him, because I know he read everything I wrote. He responded to it, bragging about what he was doing (as an AC, of course). He admitted that he didn't really have a good reason for it. Mainly he just thought it was fun to harrass me. He said he had been doing it to many other posters here, and that nobody ever caught on to it. According to him, M2 was a joke. I see no reason to disbelieve him.
What happened? Well, I emailed Taco directly, asking him to look into it. Of course, I never received a reply. After that I stopped using the account, which I had had since the early days of Slashdot (long before young punks like you arrived, user 781340
That's just my story, of course. But now at least you know that things like these happen, and that M2 is severely broken when it comes to preventing them.