Slashdot Mirror
User-Generated Content Vs. Experts
Jay points out a Newsweek piece which suggests that the era of user-generated content is going to change in favor of fact-checking and more rigorous standards. The author points to Google's Knol and the "people-powered" search engine Mahalo as examples of the demand for more accurate information sharing. Quoting:
"User-generated sites like Wikipedia, for all the stuff they get right, still find themselves in frequent dust-ups over inaccuracies, while community-posting boards like Craigslist have never been able to keep out scammers and frauds. Beyond performance, a series of miniscandals has called the whole "bring your own content" ethic into question. Last summer researchers in Palo Alto, Calif., uncovered secret elitism at Wikipedia when they found that 1 percent of the reference site's users make more than 50 percent of its edits. Perhaps more notoriously, four years ago a computer glitch revealed that Amazon.com's customer-written book reviews are often written by the book's author or a shill for the publisher. 'The wisdom of the crowds has peaked,' says Calacanis. 'Web 3.0 is taking what we've built in Web 2.0--the wisdom of the crowds--and putting an editorial layer on it of truly talented, compensated people to make the product more trusted and refined.'"
> Last summer researchers in Palo Alto, Calif., uncovered secret elitism at Wikipedia when they found that 1 percent of the reference site's users make more than 50 percent of its edits
Wtf. Why is this 'secret elitism' ? IIRC, the story was something along the lines that what happened typicall was that a large 'plain text' commit tended to be submitted by an actual expert, and then hundreds of small commits were made by this 1% that was to wikify the text, format it nicely, add references etc.
To me, that sounds more like a 'secret janitorial staff' than a secret elitism.
Big media companies are finally starting to "get" the Internet and join the information age by finally making meaningful contributions online.
Wisdom of crowds is far from dead though... and may I say let's not get in the habit of referencing "Web 3.0" PLEASE.
If you can read this... 01110101 01110010 00100000 01100001 00100000 01100111 01100101 01100101 01101011
The Yellow Pages?
>Because experts are never wrong. Infact, did you know experts
>always completly agree
That's not much of an argument. Of course experts disagree - but either side knows far more about a given topic than the average Joe 12-pack on the internet. Unfortunately, experts don't have nearly the free time of Joe 12-pack. Meaning in many cases the well-meaning but uniformed will engage in editing wars with the true experts and there's no way to prevent someone from reposting the same crap over and over.
Brett
The article says, basically, that the experts are advertising supported. It will all fall apart as web advertising collapses again. Thank you, Adblock Plus!
Besides, with all but the newest and pre-release products, I get much better information reading a spec-sheet and browsing user opinions than I do from an expert review.
This is going to be a case of "Your betters know better", which simply will not fly. I will take wikipedia with its inaccuracy any day, over a closed publication model with a range of possible slants to its subtle editorializing and crafty omissions, all created by funding requirements.
No, thank you. I'll pass.
enefesdi bhootparamdi
if a thing is worth doing at all, it's worth doing right. -- H.S. Thompson
Damn, if these "Periodicals" come out daily, we'll call them newspapers; and weekly ones we'll call magazines. Heck even some of the highly technical ones we'll call Journals.
Shit, then people can go to school to become writers/authors or even Journalists. I bet a whole industry can sprout up from this. If the content is good enough, I'll even pay for it. I wonder if they can deliver it to my doorstep every morning by 7am, so I can read it with my morning coffee.
And this is different from the current situation in what particular way?
Faster! Faster! Faster would be better!
Probably. The biggest 'problem' with wikipedia and it's ilk is that it takes readership away from the monetised publishers who previously held sway on the provision of information.
Yes, sometimes it sucks. But sometimes books do too, and the edition on your shelf won't magically correct its errors and ommisions if you wait a few days. For that you need to buy a new edition, and hope the problems are gone.
...but I'm sort of getting tired of user generated content and user powered free-for-alls. Everyone likes to hail Web 2.0 as a revolution in democracy, but it really isn't. It's who screams loudest, and who can afford the opportunity cost of sitting around all day reverting edits, creating their own Digg army, or spamming links all over the place. And everyone is doing it now, so the time it takes to find something worth reading, watching or listening too isn't worth it even when the price is free. It's one of the reasons I find myself coming to Slashdot to actually find articles, instead of Digg/Reddit/etc. And on a larger scale, I actually find myself going back to "old media." Picking up a newpaper (or at least reading something with an editor online, like the NYT), listening to NPR, getting a subscription to Wired, buying CDs and box sets of old shows, and so on and so forth.
-- Aaron
mob justice maybe, but certainly not wisdom. this nonsense has turned the internet into one big soap box with very little meaningful content.
If you mod me down, I will become more powerful than you can imagine....
'Web 3.0 is taking what we've built in Web 2.0--the wisdom of the crowds--and putting an editorial layer on it of truly talented, compensated people to make the product more trusted and refined.'
Truly talented, *compensated* people. Thank god that capitalism is finally in charge. They had take the elections but I just new it had to be media too.
Buzzwords discovered during a quick reading:
Netizen
choice fatigue
Web 3.0
wisdom of the crowds
What the hell is "choice fatigue" anyway? Are users overwhelmed by the sheer amount of data aggregated by Google and the like? Is the author implying that we're too lazy/tired/inept to handle more than one or two obvious sources of information? A combination of both? These trend stories only hold weight when constructed with ambiguous phrases, hurried research, and lack of in-depth explanations. He damns Amazon, Wikipedia, and craigslist in a matter of four sentences using flimsy support at best. Dark days for the internet heavyweights indeed.
Also: when will the "Web x.0" label finally die? This is a serious question. At the current buzzword usage rate, we'll arrive at Web 10.0 by 2015. So the tech trend story authors will either have to qualify the phrase using several paragraphs, assume readers understand all 10 evolutions of the web, or stop using it altogether. If it's the latter -- oh god please let it be the latter -- then at what number will it stop: 4.0, 5.0, 6.0? Anyone want to take bets?
It's too expensive for one thing. Take for example the medical field. How much of the body of knowledge is "Level 1 Evidence"? You'll find that only a small proportion of what is in a medical textbook meets this standard, because it requires a formal review panel of experts systematically analyzing properly undertaken studies with blinding and so forth, and even then you have to take it with a grain of salt most of the time. It's just so terribly labour intensive that the job has to be restricted to narrowly defined, very important areas of interest.
Ordinary people are much better than experts at offering real, useful knowledge with everyday applications. People are after coalface experience and are getting it for nothing. Hard to beat that.
Also, when it comes to current affairs / intelligence, nobody trusts a commercial or government agency on these things anyway. At least they won't for long, because you get to see how useless their information is.
Do it yourself, because no one else will do it yourself. [beta blockade 10-17 Feb]
This story and others like it are part of a move by publishers and the traditional media to undermine a phenomena that they are terrified of because it makes them less relevant to many people. While Wikipedia is imperfect it is still no worse that the traditional media, which has always been vulnerable to corrupt editorial manipulation, marketing scams and shoddy journalism.
Myself, I think I'll stick to letting everyone contribute. That way I can see all the expert views as well as all the interesting notes and crackpot additions that non-experts add. Since Wikipdia started limiting contributions I've found it a lot less useful and less enjoyable to use. If I wanted to read a smaller, more limited, more expert opinionated, source I would grab the encyclopedia off my shelf. What made Wikipedia great was it's huge amount of information with stuff you wouldn't find in the encyclopedia. It gave you one heck of a place to start with and then through your own research you could sort through the information provided to see what was from experts, what was interesting side material experts wouldn't tell you about, and what was just crap. Rather than censoring non-expert material it's better to highlight expert material while leaving everything available.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
If that's really what Web 3.0 is going to be, it will fail. Adding an editorial layer DOES NOT SCALE.
It's hard to imagine that we'd give up all the truly valuable contributions from the wisdom of the crowds, of which there is actually quite a bit, in order to filter it through an editorial layer which by it's very definition would result in a much, much smaller pool of knowledge. That idea is essentially nonsense.
Filmo The Klown
While there is some truth in that, the problem here is more subtle. Stated simply it is:
How do we tell that this person is an expert? What actually distinguishes them from another user?
This is a serious problem because there are a whole lot more people who claim to be experts than there are who have anything useful to contribute. The "wisdom of the crowds" never really existed - crowds are quite stupid - but Wikipedia 'solves' the problem of finding the experts by building a system where you don't need to bother, and (here's the important bit) nobody has ever come up with anything that works better. Nothing will change until/unless somebody does come up with a better solution.
The only thing 'better' is peer-review, but scince WP is an encylopedia and not a journal, I don't see the point.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Sure someone came up with something better: his name was Aldus Manutius, and his invention is nowadays called a "publisher" - one guy whose reputation depends upon his ability to pick good editors who can themselves vet books for quality. (Technically, a lot of people contributed to the invention of the publisher, but Aldus is a good stand in for the group.) A publisher's reputation for quality directly affects the prices he's able to charge for his works, and thus his livelihood. The problem with Wikipedia and projects of its ilk is that, unlike open source software, there's no boundary of usability that instantly tells you whether or not it's crap (bad code doesn't work, and that's obvious to users, not just experts). You can't apply a "survival of the fittest" model to written works that are intended to represent reality accurately, because the only check on their growth is the reading and use of such works by often inexpert readers who judge the works on criteria other than accuracy (because most of them don't have the requisite knowledge to judge their accuracy). One possible check is reputation - nobody wants to have their name attached to a gross error, so folks are more likely to take care with their research and arguments if they are forced to take responsibility for what they write (see Areopagitica), lest someone who is an expert demonstrate their error for all to see. The problem with wikipedia isn't the fact that it's open to any contributor; it's the fact that there's no assignment of responsibility for entries to authors who have to maintain a consistent identity (if someone demonstrates your failure to check your facts, you can just switch to another sock puppet and nobody's the wiser) and the failure to vet editors (if say Alex Ross were to write an article on John Adams the composer, I could go in there and change it to say that he was an early Baroque composer for the harpsichord, and there'd be nothing to prevent me from camping on the entry to revert any corrections, because in the eyes of Wikipedia what I have to say about John Adams, as a musical novice, is as valid as what was written about him by a professional music critic. But if I say that John Adams died three hundred years ago and wrote harpsichord music, people who come across it will shrug; if Alex Ross does so, he'll lose his audience and Adams will call him and say "are you on crack?" The editors need to be people who have something at stake; and the authors need to be held responsible for their work...and ultimately, the publisher needs to be held accountable for the quality of the whole, or else you end with good articles only in the hard sciences where inaccuracies are obvious to all relevant readers and in narrow specialties where no one would bother writing entries except for those who are already heavily invested in them, and worthless articles on any subject in which a measurable fraction of the general public has an emotional investment, but no particular expertise.
I thought Web 3.0 was supposed to be the Semantic Web. So now Web 3.0 is going to be like Web 2.0 but with more gatekeeping? Does that make the Semantic Web Web 4.0?
I hate buzzwords.
It's "peer-review" but we are all the peers.
After all, what were "experts" before they became experts, except "users"?
Or, to put it another way, an expert is a user who other users call "expert".
You are welcome on my lawn.
The summary seems to suggest that merely paying people will automatically elevate their content to a realm of professionalism and accuracy. That's silly. If anything paying people introduces potential conflicts of interest etc.
This looks like the common mistake where people assume that something is worth more if they paid more for it. Or it may be a flimsy attempt to commercialise sites like Wikipedia, I'm sure many business types salivate at the prospects of "monetizing" such a huge site - well, it's huge *because it's good* - Wikipedia never even had to advertise, users flocked to it because it was useful, and that's testament to the fact that the system "works".
Personally I don't think there is even a problem that needs to be solved. GP is exaggerating badly, as is the summary (I didn't RTA). On the whole, Wikipedia works incredibly well - really, it's 'nothing to see here, move along', focusing on the 0.0001% of problem areas and blowing it out of proportion to suggest an epidemic of problems suggests sensationalism or an ulterior motive to me.
I don't see anything wrong with 1% making 50% of the edits at all, that is a natural distribution for projects of that nature, you see the same pattern in open source development, and it's not a bad thing at all. Actually I would've been surprised if the pattern had been anything else. We don't judge the content based on stats about the nature of the editing process, we judge the content on the content, and it's good, very good. Not perfect, but nothing is.
...editors pick the most accurate ones for posting... Heh. If we could only get these "editors" of which you speak, Slashdot might become just such a site!If a job's not worth doing, it's not worth doing right.
You're kidding, right? Apparently you've missed out on Margaret Jones, or James Frey, or the entire bogus memoir industry that produces crap like this with the help of a ghost writer. I work for a publisher, and simply put, they rarely fact-check. Instead, what they do is send prerelease books to reviewers. The hope is that the reviewers will be smart enough to catch glaring errors. How knowledgeable the reviewers are depends somewhat on the audience of the book. College textbooks typically go to professors and grad students. Trade paperbacks can go to pretty much anybody, but usually quotable people or professional book critics.
In any case, this is exactly the same mechanism that Wikipedia uses: throw it out there and see if anyone catches something. As a practical matter, publishers cannot fact-check. They do not have the resources. The only books I would depend on fact-checking for are the ones that claim to do so as a principle of their cognitive authority: dictionaries and encyclopedias. The imprint I work for publishes several hundred textbooks a year, and reprints darn near a thousand. We have a little over 200 employees. See what I'm getting at?
Even scientific articles are "fact-checked" this way: throw it out there. Typically the reviewers are peers, and quite knowledgeable. This works better than with trade publishers because the reviewers have specific knowledge about that particular field. But does the publisher fact-check themselves? No! I should add that the pay scale for reviewers goes up depending on the relative reliability of the reviewers. Reviewers for scientific reviewers are often paid in the several hundreds range. Reviewers for college textbooks in the low hundreds (sometimes in trade for other goodies), and trade paperback reviewers, not much, if anything. Often it's for the privilege of seeing pre-release stuff.
There's only one kind of publishing where fact-checking (aside from dictionaries, etc.) is done as a rule: journalism. But there have been many scandals there as well. There was a study mentioned in the book Trust Us, We're Experts that said that nearly half of the Wall Street Journal's article's were simply slightly modified press releases. And the Wall Street Journal is regarded as one of the more reliable papers! I think I only need to mention cable TV journalism for you to see where I'm going with this.
The publishing industry is not reliable. They're in it for the money. Books like Frey's sell just as well, if not better, than the real ones. Just look at the demand for O.J. Simpson's book-- a book that never even claimed to tell the truth! People want something juicy, and the publishing industry is happy to give it to them. Sorry, ptrourke, your premise is false.
Could you tell me the difference between peer review and the moderation system that we have in, say, /.? How is "this is accurate and scientific" in many cases different from "yeah, I agree with this theory"?
Of course, 99.999% of all those theories that get discredited in the scientific circles are the usual esotheric FTL drives powered by some mystic cold-fusion-in-your-basement that's fueled by the next perpetual motion machine, that's a given. But basically that's what peer review comes down to: I agree with you.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
See? That's the way you do it!
I can moderate slashdot. I cannot participate in any peer-reviewed scientific journal that I am aware of, because I have no scientific credentials that would be accepted by any credible scientific journal.