PubMed Commons Opens Up Scientific Articles To User Comments

New submitter smegfault writes "In a new trial, PubMed Commons has been released. Until now, post-peer-publication results were restricted to letters to the editor of scientific journals; and even then some journals don't accept letters to the editor. With PubMed Commons, scientific peers can comment on PubMed-indexed articles without the interference of journal editors and peer reviewers. At the moment, eligible for participating are: 'Recipients of NIH (US) or Wellcome Trust (UK) grants can go to the NCBI website and register. You need a MyNCBI account, but they are available to the general public. If you are not a NIH or Wellcome Trust grant recipient, you are still eligible to participate if you are listed as an author on any publication listed in PubMed, even a letter to the editor. But you will need to be invited by somebody already signed up for participation in PubMed Commons. So, if you have a qualifying publication, you can simply get a colleague with the grant to sign up and then invite you.' However, reports are in that anyone with a PubMed / NCBI account can sign up on the PubMed home page."

Comments

Scientists arrive at web 2.0 ten years later

[Anonymous Coward]

Oh look, 2 out of 10 people found this comment helpful !

Will definitely need slashdot's karma system

[Anonymous Coward]

Will goatse redirects get a "+5, Informative" on the proctology articles?

In theory...

[Anonymous Coward]

... more eyeballs are a better thing, but with the rise of corporate PR firms and successful propaganda campaigns against science. I have serious doubts without some kind of moderation system and real identity system. In science I believe open sane debate is only possible with realnames.

Re:In theory...

[HiThere]

realnames aren't necessary, and can be a real hinderance. Handles, however, do seem to be necessary, so that one can evaluate their trustworthiness. And slashdot id#s are useful, as they let one easily gague how long a particular handle has been in use (in case you don't recognize it), but dates might be better.

P.S.: What's your handle? OK, your realname? Don't want to say?

It'll end up like this

[Nimey]

you can make up to 88 dollars an hour working from your house. Click here for one weird tip on how to lose your bellyfat. U R homo. Ron Paul 2012! bit.do\bNB9

Re:It'll end up like this

[Anonymous Coward]

No, it'll more likely end up not used at all. It sounds like they throw up so many hurdles, if I were one of the eligible people to comment, I wouldn't.

Re:It'll end up like this

[stranger_to_himself]

No, it'll more likely end up not used at all. It sounds like they throw up so many hurdles, if I were one of the eligible people to comment, I wouldn't.

The British Medical Journal (and I'm sure other journals) have done this for a while, and it is used and is useful. People tend to use it as publishing a 'letter to the editor', but its a lot less selective.

This will all end in Tears

[WillAffleckUW]

I predict this will all end in tears.

Tears in the fabric of space and thyme.

cool study bro

[Anonymous Coward]

After you read Dr.Johnson's titillating study on the asexual behavior of female ants among all male colonies, why on earth would I want to read koolguydouch3's comment "cool study bro"?

Correct me if I am wrong, but PubMed is a place for publishing papers, studies, etc.; why would you want to read nobody's comments?

Re:cool study bro

[stranger_to_himself]

After you read Dr.Johnson's titillating study on the asexual behavior of female ants among all male colonies, why on earth would I want to read koolguydouch3's comment "cool study bro"?

Correct me if I am wrong, but PubMed is a place for publishing papers, studies, etc.; why would you want to read nobody's comments?

The comments are often very useful when posted against articles on specific journal websites. Like all academic publishing, you'll have to use some critical faculty to separate the wheat from the chaff.

Popular Science just decided to do the reverse

[JoshuaZ]

This is sort of amusing since PopSci decided to stop having comments. They did so because of evidence that comments really are a net negative. See http://www.sciencemag.org/content/339/6115/40.summary?sid=9b37fd35-5bb4-4bbe-89e7-b1054f5ecdd1 and http://news.yahoo.com/blogs/sideshow/popular-science-ends-reader-comments--says-practice-is-bad-for-science-002245622.html.

Re:Popular Science just decided to do the reverse

[geekoid]

This is a very different group.

I still wish PopSci used it as an opportunity to try other methods, like a pay to post comment section

Re:Popular Science just decided to do the reverse

[Anonymous Coward]

like a pay to post comment section

Those are called ads.

Interesting contrast

[cold fjord]

It is interesting to see the contrast between a hard science publication opening itself to comments while Popular Science has stopped accepting comments. Given the controls they are putting in place, and the importance of maintaining a profession reputation, I expect their experience with comments will be better than relatively uncontrolled access.

what this will look like:

[Vesvvi]

I'm going to go out on a limb and predict where this will go first: improved metadata and citation networking. I'm an eligible author with pretty good experience with the system.

The initial comments will not be excessively negative. As I've mentioned before on Slashdot, publications are a summary of findings and never the full story: the authors are always holding back. On average, if it looks like they've overlooked something (from the standpoint of the reader), it's more likely to be an error or oversight on the reader's part than the authors. I think people generally appreciate this point, so they'll be conservative in their criticism to avoid looking foolish.

Getting cited is a really big deal, and not being cited (when your work is highly relevant to the topic) is considered a serious slight. I've seen nasty phone and email messages bounced around because of this. So in the context of comments, you're going to see a lot of things along the lines of "They should have considered author X, work Y from 2003 because it is highly relevant." This is a safe comment to make, but it can also be used to make a subtle point, drawing attention to competing work the authors chose to ignore, etc.

There won't be a lot of novel observations/data/interpretations being presented. Online comment pages will not be considered a place to stake your claim on an idea. Hence, people won't want to be "scooped", and they will reserve key insights for themselves.

There will be a lot of referencing preprint sources as they become more popular. This will be a new form of citation: retroactive citation of "future" (current) works, and it will greatly improve the citation network. This is important because that network is critical (besides in-person networking) to follow the development of a research field.

Re:what this will look like:

[JanneM]

PLoS has had comments for a long time, and the result is - mostly no comments at all. I suspect the problem is that you're posting in your professional capacity, under your real name. That means you can really hurt your reputation with an off-hand comment; on the other hand, they count for nothing as far as your CV and employment prospects are concerned.

Posting a comment is really a losing proposition, with no upside (you can always contact the first author directly with questions) and potential downsides. So people, rationally, don't comment. Unless PubMed has figured out a way around it, I suspect this will be the end result this time around as well.

Re:what this will look like:

[Anonymous Coward]

I can think of two specific instances where I would have loved to have a system like this:
1) a paper claimed to offer evidence against a model that I proposed, but their experiments failed to test my model
2) a paper reached absurd conclusions because they failed to consider the well-understood dynamics underlying my model.

There is a lot to argue about even with staking a claim to an idea because ideas are often ignored/misunderstood long after they have been presented, and even after they have been accepted by the small community of experts.

Re:what this will look like:

[umafuckit]

PLoS has had comments for a long time, and the result is - mostly no comments at all. I suspect the problem is that you're posting in your professional capacity, under your real name. That means you can really hurt your reputation with an off-hand comment; on the other hand, they count for nothing as far as your CV and employment prospects are concerned.

Posting a comment is really a losing proposition, with no upside (you can always contact the first author directly with questions) and potential downsides. So people, rationally, don't comment. Unless PubMed has figured out a way around it, I suspect this will be the end result this time around as well.

Quite. When you look at the way scientists behave in public you see that they're usually very cautious about criticising another's work. For example, questions following a talk at a meeting are usually very tame, even if the talk has glaring holes in it. It's karma in action: it's in everyone's interest to be nice in public because everyone gets up on the podium eventually and they want an easy ride when they're there. Also, the purpose of a talk at a meeting isn't to get grilled but to share data. Talks within an institute can get more rough if there's an ass in the audience whose feathers get ruffled. Similarly, people might be more forthright at a poster at a conference, where the interaction is more direct and fewer ears are listening in. Interactions by e-mail have, in my experience, been very diplomatic indeed.

There are plenty of shitty papers in my field that end up in the very top journals because the lead author won a Nobel and knows the editors, etc. Whilst it would feel great to see the opinions of the rest of the field nailed to the wall, nobody will benefit and it will only hurt the PhD student or postdoc who's first author.

Wait for the kooks and quacks...

[grub]

Wait for the kooks and quacks to start polluting vaccine and autism related articles with their woo.

Better idea

[StripedCow]

This is great news, and about time.

But I have a better idea: what if Google scholar implemented a user-forum for each of the papers they list?
That would mean that instantly, this service would become available for any paper from any journal.

Godwin's Law

[linear a]

Let me be the first to say that they are all worse than Hitler.

Perhaps something ArXiv should consider?

[amaurea]

At times I've missed this from ArXiv too. Cosmocoffee is an arxiv overlay that sort of allows this, but such an approach is only effective is a significant fraction of arxiv users were too use it, which is not the case.

A few years ago (2010, iirc), a paper making a controversial claim was published on arxiv, and within one month three papers had appeared disputing that claim. During the next month, a counter-rebuttal was published, followed by counter-counter-rebuttals. In effect, a discussion was going on on arxiv in the form of articles. But scientific articles come with a pretty lage overhead, so perhaps it would be good to have some quicker way of commenting on an article than writing a full paper oneself.

Such a commenting system could be used to implement open, distributed peer review. Coupled with some sort of reputation system and meta-moderation, that would make make scientific journals obsolete (in my field, nobody reads journals (that happens on arxiv), and we only submit articles to them for peer review). Another positive consequence of this is that the discussion going into peer review, which is usually hidden, would be out in the open. Such discussions are usually quite informative, and it would also let people see if the review was fair or not.

While one may fear trolling etc., I think the answers and comments on stackoverflow is a good example of how a powerful reputation system can practically eliminate trolling and vacuous comments. So I think something like this would be doable. I hope PubMed's experiment works out.

Summary speculation incorrect

[daenris]

The bit at the end of the summary that suggests anyone with a PubMed/NCBI account can sign up are either wrong or they fixed it. I have an account and a number of publications, but no grants from the sources they're searching, and it doesn't allow me to request an invite or register with my email.

Re:Summary speculation incorrect

[Anonymous Coward]

Are you the corresponding author?

Re:Summary speculation incorrect

[smegfault]

I have an account and a number of publications, but no funding. I was invited in and got accepted. The report that anyone with an account could sign up was taken from a comment on the original blog posted linked in the article; but apparently it's now fixed or it never worked.

Comments? How trite.

[FilmedInNoir]

I feel I'm best expressed through the modern medium of memes. Opinion