Web of Trust For Scientific Publications

An anonymous reader writes "PGP and GnuPG have been utilizing webs of trust to establish authenticity without a centralized certificate authority for a while. Now, a new tool seeks to extend the concept to include scientific publications. The idea is that researchers can review and sign each others' works with varying levels of endorsement, and display the signed reviews with their vitas. This creates a decentralized social network linking researchers, papers, and reviews that, in theory, represents the scientific community. It meshes seamlessly with traditional publication venues. One can publish a paper with an established journal, and still try to get more out of the paper by asking colleagues to review the work. The hope is that this will eventually provide an alternative method for researchers to establish credibility."

Comments

Wikipedia

[jgtg32a]

This is exactly what Wikipedia needs to implement.

This will allow it to overcome the credibility problems that it has.

Still needs a root

[orclevegam]

The problem of course is that at some level you still need to have a known good reference for the whole "web" to work. It doesn't help your credibility at all if you've got a paper signed by 100 of your closest crackpot buddies. What this does provide is the ability for someone in addition to established authorities to vet a work, such that a well respected member of the scientific community can easily and in a verifiable fashion signify his approval of a paper.

Re:Still needs a root

[DancesWithBlowTorch]

Actually, it doesn't need a root. Quite on the contrary, the developing graph could give amazing insight into the structure of research communities. It would be possible to identify researchers forming links between otherwise almost disconnected areas of research, and to find the great minds at the centre of such blocks. There is no "root" to the web of scientists. Even people like Erdös were only ever local subroots.

I think this project is a great idea. Unfortunately, it currently seems to consist of only a command line tool to sign reviews with GPG. That's nowhere near enough if it is to thrive beyond the CS world. It needs a simple, rock-solid GUI, and most importantly, lots of eye-candy for the graph. It will need to look cool and work well to build up the momentum for this to work at all.

Weird objection

[nine-times]

I'm sometimes bothered by the stress on studies being "verified" by something like a peer-review process. Not that I don't understand why it makes sense. It's a pretty reasonable attempt to sort valid work from crap, but...

There's still a certain way in which it's just an appeal to authority. It's people saying, "We should accept what this scientist says because other scientists say that he's right." I guess what I'm saying is that I worry that, as a process like this becomes more technical, people will be more likely to confuse a statement like, "This study has been reviewed by other scientists and seems to have merit," with something more like, "This study is correct, infallible, and indisputable."

And I guess part of the reason I worry about this is that there may be cases where what "everyone thinks" (i.e. the common conception even among experts) is wrong, and some random nutcase is right. It almost never happens, but it happens sometimes. It seems to me that a technical method of assigning trustworthiness of ideas in a web of trust might possibly lead to having all the groundbreaking ideas go into a spam filter somewhere, never to be seen again.

Re:Weird objection

[evanbd]

As a third party, there is no way I have the time to follow the chain of logic that results in a modern scientific paper from first principles. At some point, I have to accept some of the preconditions of the paper without verifying them, because doing otherwise implies that I am an expert in the particular field the paper is relevant to. And there are plenty of cases where I want to make use of a result from a field that is related to my work but in which I am not an expert.

Appeal to authority is the fundamental reasoning technique I apply in such cases. A respected expert says it is so, and so I will trust them until I have reason to believe otherwise. That trust should not be blind -- if I am presented with reason to, I will happily re-evaluate that trust. Perhaps the expert is mistaken. But, in the interest of actually getting something done myself, I will accept as a default position that the experts know what they're talking about.

Re:Weird objection

[ralphbecket]

Peer review is actually pretty weak. It's mainly effective at spotting obvious howling errors. Peer review is not the same as replication and, indeed, many reviewers don't bother to check the equations or data presented in a paper unless they are genuinely suspicious of the conclusion. Replication, not peer review, is the gold standard of science.

Very poor idea

[littleghoti]

What is important is *anonymous* peer review. There needs to be a mechanism for new scientists to question established researchers without lasting detriment to their careers. On another note, what I thought this article might be about was CiteULike, which is great. Any academics should check it out

Re:Very poor idea

[Anonymous Coward]

They are anonymous--the same way student reviews of a professor in a class with 5 students are anonymous. By the time you're doing anything original, innovative, and remotely interesting in a field, you're going to have 20 peers tops--very likely only 5 or so. Some of your reviewers will be unrelated and able to check basic mathematics for correctness but not much more.

Honestly, I think I haven't yet seen someone receive comments back where they couldn't take a good guess at who the originator was...

Already got one

[burris]

Scientific publications already have a web of trust in the list of cites at the bottom. Publications don't get cited unless they are notable in some way.