Slashdot Mirror


Science Magazine "Sting Operation" Catches Predatory Journals In the Act

sciencehabit writes "A sting operation orchestrated by Science's contributing news correspondent John Bohannon exposes the dark side of open-access publishing. Bohannon created a spoof scientific report, authored by made-up researchers from institutions that don't actually exist, and submitted it to 304 peer-reviewed, open-access journals around the world. His hoax paper claimed that a particular molecule slowed the growth of cancer cells, and it was riddled with obvious errors and contradictions. Unfortunately, despite the paper's flaws, more open-access journals accepted it for publication (157) than rejected it (98). In fact, only 36 of the journals solicited responded with substantive comments that recognized the report's scientific problems. The article reveals a 'Wild West' landscape that's emerging in academic publishing, where journals and their editorial staffs aren't necessarily who or what they claim to be."

194 comments

  1. Click by mynamestolen · · Score: 5, Insightful

    How many of the open access journals rely on click through advertising? Follow the money, I say.

    --
    work in progress
    1. Re:Click by moteyalpha · · Score: 3, Insightful

      Science Magazine did a bad experiment about submitting a spoof scientific report so that you would click on them! How can you trust a science magazine that uses bad scientific methods to make a point. Real scientists create experiments that can be reproduced and independently verified and they did not. Q.E.D.

    2. Re:Click by jklovanc · · Score: 1

      What is stopping someone from "independently" creating a bogus paper and submitting it to numerous peer-reviewed, open-access journals and analyzing the results? It seems reproducible and independently verifiable to me.

    3. Re:Click by tlhIngan · · Score: 3, Interesting

      How many of the open access journals rely on click through advertising? Follow the money, I say.

      I think they're all trying to figure out their business models.

      I know some respected organizations have created open access journals, though they rely on member fees to pay for the costs. Others rely on the author to pony up some cash (some up to $1500) which pays for it.

      I think the author-pays is an interesting one - and quite possibly might be a way to cut down the number of bad articles - after all, if you're not willing to pony up, you probably don't have enough belief In your research.

    4. Re:Click by ffflala · · Score: 2

      Few if any, I'd guess. Academic journals are usually not ad-based publications. The "open access" model described here means that either the author pays for the publication, the author's institution, or the publication has an institutional grant to pay for it.

      When you follow the money, it leads you to two groups. At the bottom of the pole will be the individual scammers who've set up these "journals" -- the article mentions a few professors who were at best slipping and at worst cynically & intentionally running this simply for a profit. For the individually-published papers, it stops here.

      For the rest, the money continues to a publishing company that cynically generates profits using a catalog that includes one, several, or only fake journals. These publishers include some big names in the traditional closed-access academic journal model (where subscriptions, often incredibly expensive ones, are required to read journal articles), such as LexisNexis owner Elsevier, Wolters Kluwer, and Sage.

    5. Re:Click by alexander_686 · · Score: 3, Informative

      It depends. What impact will publishing a paper have on your career? If âoepublishingâ 10 papers is the difference between a associate professorship and a full professorship, $15,000 is cheap.

      One might ask how valuable fake papers are â" and it turns out they can be worth quite a bit.

      http://www.economist.com/news/china/21586845-flawed-system-judging-research-leading-academic-fraud-looks-good-paper

    6. Re:Click by moteyalpha · · Score: 3, Insightful

      It would be a de novo experiment as the first was not independent. It seems much like the Microsoft "Scroogle" ads. It made me think they must be desperate to employ such methods. "Coke says Pepsi sucks, Coke confirms it" It was not intended to be a real serious poke at them as I really like their magazine and they do have good articles in my opinion. It sounds like marketing department logic at work here.

    7. Re:Click by Anonymous Coward · · Score: 0

      Doctors lie to patients ad give placebos. Does that make the placebo effect invalid?

    8. Re:Click by ATMAvatar · · Score: 5, Informative

      In this "experiment", what was the control group?

      --
      "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
    9. Re:Click by Anonymous Coward · · Score: 0

      There are plenty of nature and science papers that no one ever gave more than a cursory read just because PI is known.

    10. Re:Click by MacDork · · Score: 1

      How many conspiracies, like Global Warming, are built on such unshakable foundations as this. Consensus! Peer reviewed! :)

    11. Re:Click by Anonymous Coward · · Score: 0

      Wow now I read the article. Basic science fail of this entire project by not submitting to any closed-access journals. Given the premise, I have to assume they did and ignored the results.

    12. Re:Click by girlintraining · · Score: 4, Insightful

      Real scientists create experiments that can be reproduced and independently verified and they did not. Q.E.D.

      This is less about a failing of science and open publishing journals than the fact that on the internet, reputations can be shed like a snake sheds its skin -- you're just a few clicks away from a new account and a new identity. This has been a long-studied problem in cryptography -- how to create trust networks in public key crypto with key signing parties, etc. That the lessons learned there apply to social networking sites and open publication journals as well requires only the smallest amount of creativity to see.

      If you want honesty, you need to have some way of punishing people who are dishonest. It really is that simple; You need a way to saddle them with a cost that can't be shed by simply switching identities. And the best way to do that, for better or for worse, is a central authority in the real world that matches online identities to real-world ones. Everything else is varying degrees of broken.

      Create a blacklist of people who have lied and although you may be able to overwhelm the system for awhile, it is self-correcting... eventually it will run out of people willing or able to get blacklisted, and the quality will then start to rise as people are forced to be responsible for what they say and do.

      --
      #fuckbeta #iamslashdot #dicemustdie
    13. Re:Click by Captain+Hook · · Score: 1

      I think the author-pays is an interesting one - and quite possibly might be a way to cut down the number of bad articles - after all, if you're not willing to pony up, you probably don't have enough belief In your research.

      That fails to address an issue like the Heartland Institute financially backing anyone with a paper which supports one of their own policies.

      --
      These comments are my personal opinions and do not necessarily reflect the opinions of the other voices in my head.
    14. Re:Click by Kijori · · Score: 4, Informative

      What would you like to control for?

      The null hypothesis is that the journals have sufficiently good review processes to avoid publishing papers with obvious and fatal flaws. If you submit a paper with obvious and fatal flaws and it is published, that hypothesis is not true. It's proof by counter-example, and no control is required for it to be valid.

    15. Re:Click by K.+S.+Kyosuke · · Score: 1

      And you think that trying to reject the null hypothesis in case of traditional journals is not worth the time? Given the fact that they apparently checked for the quality of the rejection letters, I think it would be worth their while to try.

      Also, there's at least one high-quality open access journal that will reliably reject all your bad papers. ;-)

      --
      Ezekiel 23:20
    16. Re: Click by Kijori · · Score: 1

      No, but that's a completely different point and has nothing to do with a lack of a control. There would still be no control group (or need for one).

    17. Re:Click by moteyalpha · · Score: 1

      You make some valid points. The problem is not in motive, but in the implementation. This is a subject that winds its way through all science and in signal processing it is SNR ( signal to noise ratio ). The ability to assign identity or origin to a signal is only one aspect of dealing with chaos.
      Strict regulation of information assumes that the mod is a perfect gatekeeper.
      The black swan is something that I often look for in all the tripe. My interests vary from the norm and so any communication channel centered about an accepted norm would never contain all the information that I seek. In Asimov's book The Foundation it is the mutant Mule that represents that black swan event. It is my position that raw data is better for my purposes and though it may be convenient to have others sift through the garbage it does sometimes throw out things that might interest only me.

    18. Re:Click by Rhipf · · Score: 1

      I have to admit that I didn't read the article but...
      It would be nice to see how closed access publishers fair by comparison.

    19. Re:Click by aestrivex · · Score: 1

      JoUR isn't an open-access journal, it is only available via subscription.

    20. Re: Click by Anonymous Coward · · Score: 0

      The null hypothesis is that the journals have sufficiently good review processes to avoid publishing papers with obvious and fatal flaws. If you submit a paper with obvious and fatal flaws and it is published, that hypothesis is not true. It's proof by counter-example, and no control is required for it to be valid.

      If the point of this study was to test that null hypothesis (# of acceptances =0) then this study was pointless to perform. We know scam journals exist beforehand so at least one would accept one of the papers. Extrapolating from that one journal to the "population" of open access journals would clearly make no sense so it would be a failed study design.

      In reality there was no null hypothesis, this is just a report on the data collected (which is fine). The data can be used in the future to build models of what kind of journal are likely to have shoddy peer review processes.

      The problem is that the authors *chose* to report only on a subpopulation of all journals (open access) and then imply that this property is related somehow to the quality of the journals they sent the fake papers to. There is never any link made between being open access and quality of peer review.

      The subtitle of this article is:
      "A spoof paper concocted by Science reveals little or no scrutiny at many open-access journals"

      This is a true description but it is misleading because the property of open-access was selected by the researchers themselves. This is no different from saying "15% of smokers get lung cancer". It is purely descriptive (which is fine) but obviously provides no evidence regarding the relationship between smoking and cancer.

      So, why was the choice made to only submit to open access journals?

    21. Re:Click by Anonymous Coward · · Score: 0

      How many of the open access journals rely on click through advertising? Follow the money, I say.

      Exactly. That is the same way unauthorized download sites (kimdotcom) make their millions..ad click-throughs, and placements. They couldn't care less whether it was real science, as long is it 'looked' sciencey.

  2. Awesome, now let's test schools the same way by intermodal · · Score: 3, Insightful

    Seems like degree-mills are more common than actual universities by the same token.

    --
    In SOVIET RUSSIA... erm...NSA AMERICA, the Internet logs onto YOU!
    1. Re:Awesome, now let's test schools the same way by VortexCortex · · Score: 1

      Seems like degree-mills are more common than actual universities by the same token.

      To solve the problem simply eliminate final exams and meaningless accreditation and implement entrance exams for jobs. Instead of the traditional degree mill one of the others stories shows how Makerspaces could be the answer: "And here we have a 3D printed degree, and over here a Degree Lathe, and this one's a 3 axis CNC degree mill..."

  3. The total number of these journals is irrelevant by Russ1642 · · Score: 1

    What matters is the results from the top journals only, or maybe expand that to only the ones that people are currently trusting. The same argument is made about the number of crappy apps in a specific app store. Go ahead, add another ten million crappy apps to the library. It's irrelevant. Show the number of crappy apps that actually get downloaded or show up high in search results. Nobody cares how many fake journals are out there as the majority are painfully obvious. What matters is if the top ones have poor quality.

  4. Controls? by Anonymous Coward · · Score: 5, Interesting

    Did he send the bogus articles to closed publishers too? How did the rates compare? I tried to RTFA, but didn't see anything about controls.

    1. Re:Controls? by icebike · · Score: 2

      Isn't the point of open access journals to let the Science world decide if the papers are any good rather than some gatekeeper?

      --
      Sig Battery depleted. Reverting to safe mode.
    2. Re:Controls? by godrik · · Score: 4, Insightful

      No that is not the point. The point of OpenAccess papers is to allow a larger communicatino of the papers by removing the barrier of ridiculously high access fees. Accessing a single paper can cost $50 for a researcher that do not have the proper subscription. OpenAccess journals are mainly designed to take the editors and publishers which ask for a ridiculously high publication fee. or cost of access.

      Open Access does not mean that anything get published in there. Though as a reviewer for many computer science journal, I can guarantee you that everybody can publish in there... assuming the level of contribution and style are up to standard of scientific method and writing. That is a very difficult thing to achieve for a non academic because of the time comitment in "learning" how to write these papers.

    3. Re:Controls? by Anonymous Coward · · Score: 0

      Isn't the point of open access journals to let the Science world decide if the papers are any good rather than some gatekeeper?

      Who is the "Science world"?

    4. Re:Controls? by uslurper · · Score: 1, Insightful

      Maybe the cost is not so rediculously high if you consider that serious journals pay for the staff and resources to actuially read what is being submitted.

      --
      oldhack: "Security is a waste of money until shit hits the fan. 5 minutes later, it becomes waste of money again. "
    5. Re:Controls? by Anonymous Coward · · Score: 3, Informative

      Peer reviewing is done on a voluntary basis by other researchers. A journal doesn't pay anithing for that..

    6. Re:Controls? by TheGavster · · Score: 4, Informative

      There is a certain amount of irony in someone attempting to prove that open access journals publish bad science through the use of bad science. I read the article, and his only mention of testing closed publications is in his conclusion, quoting a colleague who suggested just such a step. He discounts this by restating his thesis (that open access journals are more numerous and publish more papers than closed ones) before shifting topics.

      --
      "Because Science" is one step from "Because old book". Try "Because of my experiment testing my falsifiable assertion".
    7. Re:Controls? by Anonymous Coward · · Score: 0

      What on earth are you talking about? Referees are almost always unpaid. Most editors are also unpaid.

    8. Re:Controls? by Anonymous Coward · · Score: 0

      You are aware that paid academic journals have been caught publishing total crap too?

    9. Re:Controls? by Anonymous Coward · · Score: 1

      First of all "rediculously" is not a word, I think you meant "ridiculously", that out of the way, it doesn't matter to informally educated people like myself that the "serious journals" pay for the staff and resources (they don't btw), what matters is that we can't afford $50/article to read these journals, and we can't afford the hundreds of dollars ISO and the like charge for standards. Our counterparts in formal education don't have to purchase journal articles as their institutions are subscribed to the major journals at greatly subsidised rates versus what we would have to pay as independent students/researchers/educators if we were consuming journal articles at an equivalent volume.

      Also there are many journals, such as Royal Institute Transactions, that, as individuals, we can't subscribe to, as the only pricing available from the publisher (in this case RI Publishing) is for institutional subscriptions, well in excess of what any independent researcher could afford.

      The bottom line is independent researchers (the general public) are financially restrained from access to scientific results, which in most cases were funded in the first place by public grants. We are paying for science that we don't get the results of. At least some grant bodies, most notably the NIH require public publication* in a timely manner.

      * The irony here is "public"ation, that we have to make the distinction now because so much supposed publication isn't public, in spite of the original meaning of the word.

    10. Re:Controls? by Anonymous Coward · · Score: 0

      "That is a very difficult thing to achieve for a non academic because of the time comitment in "learning" how to write these papers."

      Are you serious? How hard can it be to learn to write a 'technical paper' up to some arbitrary standard? Give me a break.

      "assuming the level of contribution and style are up to standard of scientific method and writing."

      Almost sounds arbitrary.

      Then again given the sad state of America there are people that can barley formulate a sentence.

    11. Re:Controls? by Anonymous Coward · · Score: 0

      Sadly this is probably incompetence rather than fraud. It is widespread amongst those receiving PhDs these days. There is no way any competent scientist could publish this without sending the fake paper to closed access journals. This should never have been published, indicating everyone in the supply chain is incompetent. The only alternative is to assume massive degree of predetermined fraud.

  5. Bias by goodmanj · · Score: 1

    Wow, clearly I should avoid publishing in those no-name open journals, and stick to big-name proprietary journals like Science!

    Science is just a liiitttle bit biased here. I don't doubt the result, but I'd like to hear it from a neutral source.

    1. Re:Bias by Anonymous Coward · · Score: 5, Insightful

      Actually, it's really really badly done.

      To actually make any of the conclusions (or inferences) about the quality or rigor of open-access journals REQUIRES a control group of traditional journals to be operated on in a similar manner. In other words, there needs to be a sting on both open-access and traditional journals simultaneously.

      Without that, no claims can be made. None. Not even one. Because we DO NOT KNOW how many traditional journals, like Science, would also have accepted their falsified paper(s). It's possible the traditional journals could have lower standards of quality and rigor than the open-access group.

      Science and AAAS (of which I'm presently ashamed to be a member) should be blasted for publishing this tripe. It needs to be retracted, immediately. If they want to have the slightest shred of credibility here, they should at least conduct scientifically rigorous stings.

      Disgusting.

    2. Re:Bias by goodmanj · · Score: 5, Insightful

      Yes, but. This isn't entirely a binary scientific question. If the question were "are open-access journals worse than traditional journals?", you'd obviously need a control. But "Is the peer review process at open-access journals acceptable?" is not a scientific question, but one of values and personal preference. Most people would decide that a 50% failure rate is not acceptable, control or no control.

      Now, we're all *very* curious to know whether traditional journals fare better than open ones, and Science is showing bias and intellectual dishonesty by avoiding that question, BUT that doesn't mean that this study has no value.

    3. Re:Bias by Anonymous Coward · · Score: 0

      Did you also notice that nearly half the OA journals were taken from a list of "unprofessional" journals? Seems a bit too much like stacking the deck for my taste.

      And we already know that Science's record is not perfect ( http://en.wikipedia.org/wiki/Sch%C3%B6n_scandal )!

    4. Re:Bias by ColdWetDog · · Score: 1

      No, it's not disgusting. It's biased and would be better with a control arm (which the author admits to). It also points out a significant issue on it's own - that there are a lot of scams in open access journals.

      The more interesting question certainly is "are traditional, purportedly higher quality journals any better?" The author, or someone else, could certainly do that and I suspect someone will. But his methodology stands alone. He was not trying to find who was better or worse, just if there was a problem in the first place. It is a biased, somewhat arbitrary view of the scientific publishing word but he does bring some clarity to this rather murky field.

      --
      Faster! Faster! Faster would be better!
    5. Re:Bias by Anonymous Coward · · Score: 0

      Maybe the published paper *is* the control group. Hmmmm??

    6. Re:Bias by stenvar · · Score: 3, Insightful

      In fact, over the years, Science has published numerous scientifically fraudulent papers, some of which were pretty blatant. So, in a sense, we already have a control. In addition to control experiments, it needs three more things experiments usually need: a statistically representative data set, a justification, and a clearly defined hypothesis. It lacks all of those.

      Peer review isn't meant to eliminate all errors from scientific papers, it's simply intended to make life a little easier for readers by weeding out papers they are probably not interested in. So, if the hypothesis is that "lower cost journals have less stringent peer review", that doesn't require any testing: it's almost certainly true, but it doesn't matter to anybody. Publishing a bad paper in a peer reviewed journal doesn't hurt anybody, except maybe the reputation of the journal.

    7. Re:Bias by Anonymous Coward · · Score: 0

      IT doesn't point out a 'scam,' it could as easily be explained by reviewers who aren't on the lookout for scam papers because (A) they aren't expecting them, and (B) they aren't paid enough to worry about it.

      There are plenty of possible explanations, you can't draw any conclusions from the results except that one paper was accepted by several journals when it shouldn't have been.

    8. Re:Bias by Janek+Kozicki · · Score: 1
      you didn't rtfa, did you?

      From the start of this sting, I have conferred with a small group of scientists who care deeply about open access. Some say that the open-access model itself is not to blame for the poor quality control revealed by Science's investigation. If I had targeted traditional, subscription-based journals, Roos told me, "I strongly suspect you would get the same result."* But open access has multiplied that underclass of journals, and the number of papers they publish. "Everyone agrees that open-access is a good thing," Roos says. "The question is how to achieve it.

      so he didn't miss it, maybe he is doing this right now, but isn't telling

      --
      #
      #\ @ ? Colonize Mars
      #
    9. Re:Bias by carpefishus · · Score: 1

      " But "Is the peer review process at open-access journals acceptable?" is not a scientific question, but one of values and personal preference. Most people would decide that a 50% failure rate is not acceptable, control or no control." The only inference is that the ones that accepted the paper are crap. This says little about the ones who rejected. Your implication is that since 50% failed the group failed.

      --
      Facts take all of the premium out of arm waving - T. Reynolds
    10. Re:Bias by Obfuscant · · Score: 1

      so he didn't miss it, maybe he is doing this right now, but isn't telling

      Perhaps this paper itself is the test, and Science failed while the others passed?

    11. Re:Bias by Anonymous Coward · · Score: 0

      If there is evidence of dishonesty why would you trust the data they report?

    12. Re:Bias by Anonymous Coward · · Score: 0

      But they don't tell us who to trust? More likely this is a cry for help and they all failed. It only got published in Science because of the anti-open source angle.

    13. Re:Bias by Anonymous Coward · · Score: 0

      Exactly, it's not a scientific question, ergo, it's wrong for them to claim to be answering it scientifically.

      This criticism applies to most social science.

    14. Re:Bias by hweimer · · Score: 2

      Science and AAAS (of which I'm presently ashamed to be a member) should be blasted for publishing this tripe. It needs to be retracted, immediately. If they want to have the slightest shred of credibility here, they should at least conduct scientifically rigorous stings.

      I also doubt that they adhered to their own guidelines for Human research studies:

      Informed consent must be obtained for studies on humans after the nature and possible consequences of the studies were explained. All research on humans must have IRB approval.

      --
      OS Reviews: Free and Open Source Software
    15. Re:Bias by Phillip2 · · Score: 1

      "Is the peer review process at open-access journals acceptable?"

      It's a perfectly scientific question; you can address most questions, including those about values and personal preference, with a scientific methodology.

      But it's a poor question, because open access has no bearing on the question -- Is the peer-review process acceptable.

      All he has done is sub-select a biased, non-randomised group in the first place. To justify this, he should be comparing open access
      to something else, otherwise, it has no role in the experiment.

      Basically, he's taken a journalistic approach -- performed a test which gave him the answer he needed for the story -- and not a scientific
      approach. It does raise a lot of questions about peer-review. In particular, it raises the question of what sort of peer review did this paper go
      through?

    16. Re:Bias by Anonymous Coward · · Score: 0

      It may have value if you like a ring in your nose and someone leading you about with an attached rope. Even the sketchiest of corporations have the sense to contract an "independent" organization to perform tasks such as this "sting". Were these submissions made to websites that no reasonable person woujld take seriously at a glance? My guess is yes.

  6. Democratization by SecurityTheatre · · Score: 5, Insightful

    A lot of people cite the democratizing power of "open access" and "crowd sourcing". I feel this is an example of the same principle at work.

    On one hand, it is easier for those that are not entrenched within the bastions of power to be heard, but on the other hand, all data received from these sources must be treated much more cautiously.

    In the past "being published" was a big deal, as it required a fairly high bar of factual accuracy, and that is still the case of many prestigious journals, but in the rush to Twitter-ize research and accept as many publishable details as rapidly as possible in the name of profit and prestige, the barriers to entry have eroded.

    In much the same way that hard investigative journalism with strong ethical guidelines, verifiable sources and solid editing will always have a place in my heart, these reputable journals can serve to establish a foundation of trust in the scientific arena. And now, in much the same way that one should treat any writing within the "blogosphere" as suspect until verified, many open access journals must now be treated with the same level of suspicion until it is proven otherwise that they hold themselves to a higher standard.

    TLDR: Democratization is not always a good thing.

    1. Re:Democratization by icebike · · Score: 1

      On the other hand, publishing nonsense can quickly be modded troll, if the journals have such a mechanize in place.

      Moderating by scientists in the field seems better than letting some gatekeeper decide which new ideas get to see the light of day, and which get deep sixed simply because they are unpopular points of view at the moment.

      How much actual damage can be done by publishing rubbish? (Its a serious question, because I don't pretend to know the answer).
      Aren't all results subject to verification by peers anyway?

      --
      Sig Battery depleted. Reverting to safe mode.
    2. Re:Democratization by phantomfive · · Score: 1

      You might think so, but being published is not always necessarily hard. That is, the story is a repeat of an experiment that was already performed, except with a print journal instead of online journals.

      Most likely what will happen over time, is that some open access journals will gain more prestige than others, and some will be more reliable than others; just like with print journals.

      --
      "First they came for the slanderers and i said nothing."
    3. Re:Democratization by 93+Escort+Wagon · · Score: 2

      Moderating by scientists in the field seems better than letting some gatekeeper decide which new ideas get to see the light of day, and which get deep sixed simply because they are unpopular points of view at the moment.

      I take it you're unfamiliar with how journals such as Science decide whether or not a paper should be published?

      --
      #DeleteChrome
    4. Re:Democratization by fermion · · Score: 5, Insightful
      First the disclaimer. I do believe that professionally peer reviewed journals and reporting still has a place. I pay significant sums of money to subscribe to a newspaper, a few top magazines, as well as Science and Nature. They serve a purpose and, to me, are worth the costs.

      That said Science is not beyond reproach on accuracy. Both journals has had a very scandalous path over the past few years with their accepting clearly fraudulent papers. In July, evidently, Alirio Melendez had a paper retracted. This researcher fooled many major journals with at least 13 papers. Science also published the paper on bacteria living on arsenic, which is generally seen as having major issues. I recall reading a paper related to dancing and sexual attraction, maybe in Nature, being retracted due to fabricated data.

      That said, there is little wrong with a single suspect paper being published. This is how scientists communicate. There is little protection against fraud such as occurred in this case because it is so patently silly. Building a system to protect against such silliness would mean that we would no longer be focused on science. The real problem here is that the popular media does not understand the difference between a single piece of research and the process of research. Places like /. should know better, but they don't. The process of science is to reproduce and extend results. When a bad paper corrupts the process, as has happened when Science and Nature has published suspect paper, that is a problem. These journals, having high impact factors, have a responsibility to proctor what they publish. A backwater online journal does no necessarily have such responsibility, rather relying on the ethics of the researcher and a faith in the process of science to ferret out unethical and silly people like these.

      What is truly alarming is the simple bad science present in this research project. This experiment has no control group and does not try to match the target journals to an equivalent paper journals.

      If the research was done properly the open access journals would be matched with closed journals on the basis of several relevant criteria, like impact factor, cost to publish, region predominately served, or the like. This is the way research is done. One can't just go out onto the street, ask 10 people who you don't like if they ever thought of killing someone, then claim that everyone in this group are murderers if 7 say yes.

      The paper would then be submitted to all the journals, the results generated using well known statistical methods, and then, if there is some degree of confidence, the results published.

      My prediction is that if you were paying a closed source low ranking journal to publish a paper asserting that the moon was composed of coagulated casein in a mesh of lipids they would not blink at printing it.

      At the end of the day, in this case Science is no better than your average corrupt advertising agent.

      --
      "She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
    5. Re:Democratization by petermgreen · · Score: 2

      AIUI what happens is they send the paper to a small group of reviewers who they regard as experts in the field. The reviewers aren't supposed to know whose paper they are reviewing but they can often figure it out anyway just from prior knowlage of who is doing what.

      Some of those reviewers will do their job as honestly as they can (though I bet they will still be more faourable to stuff that confirms their beliefs), others will deliberately try to discredit any paper they see as being from a rival so they can later publish their own paper in the area with less competition.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    6. Re:Democratization by stenvar · · Score: 1

      On one hand, it is easier for those that are not entrenched within the bastions of power to be heard, but on the other hand, all data received from these sources must be treated much more cautiously.

      More cautiously than what? A Science paper? Science has published numerous scientific frauds over the years. In fact, if you're looking for a high profile scientific fraud, you're more likely to find it in Science than in an open access journal, because that's where the rewards are highest.

    7. Re:Democratization by mpe · · Score: 1

      Moderating by scientists in the field seems better than letting some gatekeeper decide which new ideas get to see the light of day, and which get deep sixed simply because they are unpopular points of view at the moment.

      Even for more trivial reasons like disliking the author or where they are from.
      Science isn't ment to work by "argumentum ad populum" or "argumentum ad auctoritatem" in the first place.

      How much actual damage can be done by publishing rubbish? (Its a serious question, because I don't pretend to know the answer). Aren't all results subject to verification by peers anyway?

      Sometimes a good way to test a theory can be obvious to an "outsider", but completly overlooked by "experts in the field". Even more potential "loss of face" if it's someone pointing out a basic flaw in the reasoning behind a popular theory.

    8. Re:Democratization by Anonymous Coward · · Score: 2, Insightful

      "And now, in much the same way that one should treat any writing within the "blogosphere" as suspect until verified, ..."

      Oddly enough, the entire point of science is that all claims are suspect until verified. And I applaud anyone that approaches any testimonials in that fashion. Whether it's in the blogosphere of the sciencesphere.

    9. Re:Democratization by danudwary · · Score: 2

      Close. The reviewers are actually the anonymous party, and they see the author list. So there's much more potential for abuse.

      A quality editor can usually see through bias in a reviewer, and I've seen them override a reviewers decision if they think there's a problem. There have certainly been unfair, biased reviews from people with an agenda (arguably it's more common in grant reviews, in my experience), but this is not usually perceived as an endemic problem. In many journals, a submitter can recommend reviewers, and can recommend against reviewers whom they don't want to see the paper. With some exceptions, scientists are generally a pretty ethical lot since we all have to live with the peer review system, and thus everyone knows that rampant abuse would just hurt everybody.

    10. Re:Democratization by danudwary · · Score: 1

      However, there is a clear difference between a fraudulent paper, and a shoddy paper in which the experimental results are clearly an error.

      Catching fraud can be very tough for a reviewer, since they pretty much have to rely on the author's word that the primary evidence exists. They don't get to go look at the students' lab notebooks, or whatever. If someone wants to fabricate a graph, or photoshop a gel, that's going to be hard to figure out. It's only going to be caught when someone with the interest, knowledge and proper resources attempts to replicate the experiment.

      In this case, the fake paper's conclusions are clearly a mistake. They state a dose dependency, and then show a graph with a flat line. Kindergarten mistakes. Even an undergraduate with a basic understanding of what a biological assay is should be able to spot the wrong conclusions right away. You'd literally have to not read the paper to not miss it. If that gets past a reviewer, that's a much bigger problem.

    11. Re:Democratization by stenvar · · Score: 1

      However, there is a clear difference between a fraudulent paper, and a shoddy paper in which the experimental results are clearly an error.

      Yes: fraud actually causes harm, whereas a shoddy paper doesn't.

      Even an undergraduate with a basic understanding of what a biological assay is should be able to spot the wrong conclusions right away

      And that's precisely why it doesn't matter.

    12. Re:Democratization by Anonymous Coward · · Score: 0

      remember the french scientist who diluted antibody to less then one protein per tube, and still got immuno rxn ? Benviniste ?
      in nature
      and the arsenic replaces phosphate paper in science
      both clearly for the shock value (and it worked, they gots lots of PR)

    13. Re:Democratization by mvdwege · · Score: 1

      Science has published numerous scientific frauds over the years.

      That's the second time you have asserted that. How about, given the subject under discussion, you provide some hard numbers?

      --
      "I know I will be modded down for this": where's the option '-1, Asking for it'?
    14. Re:Democratization by Registered+Coward+v2 · · Score: 1

      First the disclaimer. I do believe that professionally peer reviewed journals and reporting still has a place. I pay significant sums of money to subscribe to a newspaper, a few top magazines, as well as Science and Nature. They serve a purpose and, to me, are worth the costs.

      That said Science is not beyond reproach on accuracy. Both journals has had a very scandalous path over the past few years with their accepting clearly fraudulent papers. In July, evidently, Alirio Melendez had a paper retracted. This researcher fooled many major journals with at least 13 papers. Science also published the paper on bacteria living on arsenic, which is generally seen as having major issues. I recall reading a paper related to dancing and sexual attraction, maybe in Nature, being retracted due to fabricated data.

      True, but fabricating data carefully to make to support your conclusions and fabricating data and the accompanying experimental manner to clearly be bad science is two very different things. In this case, the author was careful to be sure any decent peer review would reveal the flaws in order to gauge how well journals conduct reviews.

      That said, there is little wrong with a single suspect paper being published. This is how scientists communicate. There is little protection against fraud such as occurred in this case because it is so patently silly. Building a system to protect against such silliness would mean that we would no longer be focused on science. The real problem here is that the popular media does not understand the difference between a single piece of research and the process of research. Places like /. should know better, but they don't. The process of science is to reproduce and extend results.

      Scientists communicate by publishing results of experiments that are designed to be scientifically valid. They may disagree about conclusions or methods and check the underlying experimental approach. A scientists may reach erroneous conclusions, parts of the experiment may be found to be problematic, etc.; but at least the paper is subject to a rigorous peer review to identify any obvious flaws before publication. What the author did was not fraudbut a simple test to see how well open access journals conform to accepted standards of scientific publication.

      When a bad paper corrupts the process, as has happened when Science and Nature has published suspect paper, that is a problem. These journals, having high impact factors, have a responsibility to proctor what they publish. A backwater online journal does no necessarily have such responsibility, rather relying on the ethics of the researcher and a faith in the process of science to ferret out unethical and silly people like these.

      I think some of those "backwater online journal(s)" rely more on willingness to pay to be published.

      What is truly alarming is the simple bad science present in this research project. This experiment has no control group and does not try to match the target journals to an equivalent paper journals.

      This wasn't science but investigative journalism.

      --
      I'm a consultant - I convert gibberish into cash-flow.
    15. Re:Democratization by Anonymous Coward · · Score: 0

      >> and then, if there is some degree of confidence, the results published

      There is value in publishing negative results, too.

    16. Re:Democratization by Anonymous Coward · · Score: 0

      That's the second time you have asserted that. How about, given the subject under discussion, you provide some hard numbers?

      Here is eight retractions from just a single scandal:

      J. H. Schön, S. Berg, Ch. Kloc, B. Batlogg (2000). "Ambipolar Pentacene Field-Effect Transistors and Inverters". Science 287 (5455): 1022–3. Bibcode:2000Sci...287.1022S. doi:10.1126/science.287.5455.1022. PMID 10669410. (Retracted)
      J. H. Schön, Ch. Kloc, R. C. Haddon, B. Batlogg (2000). "A Superconducting Field-Effect Switch". Science 288 (5466): 656–8. doi:10.1126/science.288.5466.656. PMID 10784445. (Retracted)
      J. H. Schön, Ch. Kloc, B. Batlogg (2000). "Fractional Quantum Hall Effect in Organic Molecular Semiconductors". Science 288 (5475): 2338–40. doi:10.1126/science.288.5475.2338. PMID 17769842. (Retracted)
      J. H. Schön, Ch. Kloc, A. Dodabalapur, B. Batlog (2000). "An Organic Solid State Injection Laser". Science 289 (5479): 599–601. Bibcode:2000Sci...289..599S. doi:10.1126/science.289.5479.599. PMID 10915617. (Retracted)
      J. H. Schön, Ch. Kloc, B. Batlogg (2000). "A Light-Emitting Field-Effect Transistor". Science 290 (5493): 963–6. Bibcode:2000Sci...290..963S. doi:10.1126/science.290.5493.963. PMID 11062124. (Retracted)
      J. H. Schön, Ch. Kloc, H. Y. Hwang, B. Batlogg (2001). "Josephson Junctions with Tunable Weak Links". Science 292 (5515): 252–4. doi:10.1126/science.1058812. PMID 11303093. (Retracted)
      J. H. Schön, A. Dodabalapur, Ch. Kloc, B. Batlogg (2001). "High-Temperature Superconductivity in Lattice-Expanded C60". Science 293 (5539): 2432–4. Bibcode:2001Sci...293.2432S. doi:10.1126/science.1064773. PMID 11533443. (Retracted)
      J. H. Schön, Ch. Kloc, A. Dodabalapur, B. Batlogg (2001). "Field-Effect Modulation of the Conductance of Single Molecules". Science 294 (5549): 2138–40. doi:10.1126/science.1066171. PMID 11701891. (Retracted)

      And, by the way, you must be scientifically illiterate if you need other people to provide "hard numbers" for this sort of thing.

    17. Re:Democratization by mvdwege · · Score: 1

      So that's a single instance. With retractions. Hardly numerous. And caught by further peer review.

      Seems like the process is working just fine, and Science is acting responsibly by retracting the papers.

      And as for your snark, it is usually the duty of the one stating an assertion to support it, so fuck off.

      --
      "I know I will be modded down for this": where's the option '-1, Asking for it'?
  7. Who? by Anonymous Coward · · Score: 0

    I've never heard of this person John Bohannon.

    1. Re:Who? by Beardydog · · Score: 2

      I think he's building a railroad...?

  8. Invitations to publish by Anonymous Coward · · Score: 1

    I get regular solicitations via email to submit papers to these open-access journals - the big selling pitch is usually speedy review and acceptance. It's very disquieting.

  9. Science is the new religion by nine-times · · Score: 2, Interesting

    I've made comments before comparing science and religion, and too often people think that I'm a religious person trying to belittle a genuine quest for knowledge. On the contrary, I think the genuine quest for knowledge is an amazingly worthwhile thing. However, science has become a method for the "practitioners" and "priests" to exert social, economic, and institutional influence by swaying the beliefs of those who are not educated enough or informed enough to differentiate between genuine knowledge and blind dogma.

    It's less that I'm a backwoods book-hating theist. It's that I've "lost the faith" and don't believe in what we call "science". We've gotten into muddy waters, studying soft sciences in ways that will never reach definitive answers, and allowed politics and media to have too much sway. We've gotten better at engineering, and worse at knowledge.

    Go ahead. Mod me as flamebait.

    1. Re:Science is the new religion by Anonymous Coward · · Score: 0

      studying soft sciences in ways that will never reach definitive answers

      Psychology, for one.

    2. Re:Science is the new religion by HornWumpus · · Score: 3, Insightful

      The soft sciences were never rigorous. Nothing has been lost or gained.

      Very few are fooled. Sociologists/Psychologists/Economists can say they've 'proved' something till they're blue in the face. Nobody will take them seriously.

      --
      John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
    3. Re:Science is the new religion by phantomfive · · Score: 2

      Science is the process of getting closer to truth by experiment. Here's Richard Feynman explaining it.

      The institutions and scientists at the heads of those institutions have become corrupted (and purged) multiple times throughout history (Lord Kelvin the traditional example), but the principles of science seem sound and correct over time.

      --
      "First they came for the slanderers and i said nothing."
    4. Re:Science is the new religion by Anonymous Coward · · Score: 2, Informative

      On the contrary, I think the genuine quest for knowledge is an amazingly worthwhile thing. However, science has become a method for the "practitioners" and "priests" to exert social, economic, and institutional influence by swaying the beliefs of those who are not educated enough or informed enough to differentiate between genuine knowledge and blind dogma.

      You have no idea what you are talking about. A paper published anywhere is just correspondence. It is intended for scientific community. That's all.

      If you can't tell a boson from a photon, or you don't know what HDL actually is beyond the talking points you see on TV, then journals are NOT INTENDED FOR YOUR CONSUMPTION. It is like reading latest materials research while you don't know how to join two 2x4s together without using fasteners or glues. And journalists are just as bad or worse than general public.

      If you want to listen to real knowledge, to real conclusions, then ONLY deal with scientific consensus. Organizations like IPCC are there to present a consensus and that's what they do. If you start reading individual papers, you will not know what they are talking about and you may not even know what research is simply wrong because you are not in that field.

      If you really really really want to read journals, then only stick to reviews or reviews of reviews that present a consensus, not original research. Original research is useless unless it is repeated, studied and understood. And most of papers are just that - research that may or may not be valid that may or may not have any immediate application.

      We've gotten better at engineering, and worse at knowledge. Go ahead. Mod me as flamebait.

      They go hand-in-hand. You can't get better at engineering without getting better at science. And if you try to diminish science, then the end result will be the same as Roman Empire.

      Finally, if you try to understand science based on some blog entries by random people about some papers, then you are completely lost. What you are reading is religion, and not science.

    5. Re:Science is the new religion by Anonymous Coward · · Score: 0

      Oh, really? There seem to be all sorts of pseudo-intellectuals who spam links to useless study after study (which haven't even been replicated or anything of the sort). A lot of people are fooled by the soft sciences.

    6. Re:Science is the new religion by Anonymous Coward · · Score: 0

      that was so last century. Science is a religion now.

    7. Re:Science is the new religion by ColdWetDog · · Score: 1

      Oh, I think you have a point. Lots of people deify 'science'. Even people who are supposed to know better (ie, this crowd). It's hard, we're stupid humans, not Vulcans. Science is a weird, counter intuitive thing to most people. Science knowledge is also enormous. No one human being can understand but a tiny fraction of what goes on and thus be in position to truly debate the merits of something. This crops up all of the time in the Climate Change debate. Yes, to really understand it you can go back to the research, tackle the requisite skills to understand the data and pour through thousands of articles in dozens of fields. Nobody can do it, so there is going to be a leap of faith about something for everyone.

      The good news is that Science usually gets it right. Eventually. Fits and starts, forwards and backwards, but there is no substitute for being (more or less) correct.

      Can look like a real mess from down in the trenches, though.

      --
      Faster! Faster! Faster would be better!
    8. Re:Science is the new religion by nine-times · · Score: 2

      Science is the process of getting closer to truth by experiment.

      That's insufficient to explain what science is, when it is what it should be. And then science today is not what it should be. "The process of getting closer to truth by experiment" is not what most people are talking about when they talk about 'science'.

    9. Re:Science is the new religion by Kwyj1b0 · · Score: 1

      However, science has become a method for the "practitioners" and "priests" to exert social, economic, and institutional influence by swaying the beliefs of those who are not educated enough or informed enough to differentiate between genuine knowledge and blind dogma.

      While I agree with you on some aspects, you seem to miss out in that (in theory) other people can call upon those priests to verify their miracles. True, most people never do it. The scientific method requires that there are people to verify claims and catch mistakes. For most important problems, however, there are enough of people to catch mistakes sooner rather than later.

      Losing faith in science would be like losing faith in open source because most people aren't educated enough to view the source and verify the code. What matters is that you trust that enough smart people have viewed it and validated the results. My obscure open source project might have a backdoor, but Firefox is unlikely to have one. The problem isn't with open source or the priests, it is that the method assumes enough of resources, which isn't likely for unimportant projects.

      In some fields where computers are being used for huge simulations, the problem gets worse. I've known lots of papers that get accepted based on models that were flawed (in Engineering) because there was a sign change in a 20,000 line code. The problem is very hard to catch. However, if it is important enough (or enough eyeballs are looking at it), it will get caught.

      I'm not sure what you mean by better at engineering but worse at knowledge.

    10. Re: Science is the new religion by Eugriped3z · · Score: 1

      He didn't mean knowledge, he meant worldliness (experince that challenges belief when God doesn't smite you), verily.

      Scientific methodologies used to develop knowledge don't require belief, but they demand a rational mind and offer discrete explanations. Religious sytems don't require a rational mind, but they do demand belief, offers no explanation but dogma.

      The difference is in the intended purpose of each. Science seeks to explain the external world and allow people to manipulate it. Morality is an after thought. Religion seeks define the internal world, control the definition of morality and manipulate the people outside it.

    11. Re:Science is the new religion by Anonymous Coward · · Score: 1

      Science is so much richer than a thing only for "practitioners". It's specialized, but I think people underestimate their ability to understand the basics of a scientific topic *if* they are willing to devote the effort to understand it. Problem is, most people aren't. They want the quick answer in a couple of newspaper articles, press releases, or maybe a wikipedia page. Nope. You'll have to dig further. That's not a fault of science, but the challenge of learning subjects that are genuinely complicated.

      I'm not interested in swaying the public like some kind of religious leader, I'm interested in informing them and motivating them to ask questions and look into things more deeply themselves. Don't merely "trust" me. I want them to be skeptical and cautious.

      If you want to believe that science is like religion, about the only way I would agree is that both of them are human constructs and subject to the same kind of social faults as any other human project. But that's where the similarities end. In criticisms of science there's way too much mileage given to the idea that scientists are somehow "dogmatic". Sure, some are, but on the whole science is not. It's quite willing to consider new ideas and surprising results, IF there is good evidence for it. When I see people faulting science as being too dogmatic, usually I find that's just an excuse for offering a really weak scientific argument themselves, and then the advocates of that idea try to blame science and scientists for the problem. Obviously it's never the person proposing the idea that has a problem, it's just those dogmatic scientists!

      Or is it that you're one of those "science purity" types that thinks the only legitimate science is physics?

      Don't get me wrong. I agree with you that politics and media have muddied things considerably, and sometimes scientists are as much a part of that problem as the politicians and media types. Scientists can have agendas too. But overall if it's good science it gets out there, if scientists make a genuine effort to communicate it accurately, and if people are interested enough in it, they can figure it out even if they aren't scientists. They just have to care enough to put in the effort. You make it sound like such an effort is futile. It isn't.

      I wouldn't mod you flamebait for your view. I just think you're wrong. Nothing wrong with that.

    12. Re:Science is the new religion by Anonymous Coward · · Score: 0

      You're right on the money IMO. I'm a fan of science, and I'm a Bible believing Christian. I don't think in the truest sense the two have ever been incompatible. It wasn't until they were both hijacked by people who had interest in neither that both became so poluted.

    13. Re:Science is the new religion by Anonymous Coward · · Score: 0

      What? You mean like all those zombie fucks out there who think Tesla "invented" alternating current because there is some infomeme out there saying he did? Yeah, I love those fuckers.

    14. Re:Science is the new religion by VortexCortex · · Score: 3, Funny

      Very few are fooled. Sociologists/Psychologists/Economists can say they've 'proved' something till they're blue in the face. Nobody will take them seriously.

      I agree. You know which branch really bugs me though? Entomology.

    15. Re:Science is the new religion by hexadecimate · · Score: 1

      C'mon. Won't some of you out there with mod points rate this funny? Just to up the ante.

    16. Re:Science is the new religion by Iskender · · Score: 1

      That's insufficient to explain what science is, when it is what it should be. And then science today is not what it should be. "The process of getting closer to truth by experiment" is not what most people are talking about when they talk about 'science'.

      What does public perception have to do with what science is? They're two completely different things. How do you know what "most people" think about this?

      You say science today is not what it should be. Do you realise it never has been? The founders of natural science were known to work based on strange religious ideas, not to mention the whole alchemy background thing. Christianity continued to shape science significantly right up to the 20th century. By that point, the social sciences you so seem to loathe had already been founded. Add to this the fact that the very definitions of science we use today are from the 20th century.

      I do not know when you think this time of better science took place. But, rest assured, science was far from any ideal then, too. Just like any other human endeavour it's just an approximation of the right way. And a wonderful approximation it is.

    17. Re:Science is the new religion by Trogre · · Score: 1

      Well that's probably because, as a name, "Tesla" sounds much cooler than "Pixii".

      It's also fashionable to favour American folks over French, even if Tesla was originally Serbian.

      --
      "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
    18. Re:Science is the new religion by phantomfive · · Score: 1

      "The process of getting closer to truth by experiment" is not what most people are talking about when they talk about 'science'.

      Then 'most people' are wrong. It isn't the first time.

      When you find someone who doesn't understand what science is, merely educate the poor fool by showing him the movie. Let Richard Feynman explain.

      --
      "First they came for the slanderers and i said nothing."
    19. Re:Science is the new religion by AthanasiusKircher · · Score: 2

      Very few are fooled. Sociologists/Psychologists/Economists can say they've 'proved' something till they're blue in the face. Nobody will take them seriously.

      Umm, what are you talking about? Too many people take them seriously. Loads of people pop pills all the time because of what psychologists have decided is "normal." Heck, the livelihood of most people in most developed countries is highly dependent on the people in control of the money supply following various economic theories -- and when those theories fail, the economy tanks.

      Maybe "hard scientists" won't take these things seriously. But the vast majority of people in the world seem to -- often with detrimental results due to misplaced faith in shoddy theorizing.

      That said, many experiments in "soft sciences" could produce better results. In my experience reading a lot of these studies (not just in things like psychology and cognitive science, but even medical studies and health issues), one recurring major issue is misuse and misunderstanding of basic statistics. A lot of studies use a 95% confidence threshold, and exploratory studies often try out dozens and dozens of potential correlations. So they're bound to find some something that fits their "significance" threshold, even if they collected random data.

      These minor (and possibly meaningless) correlations are then generally spun into some significant finding in the discussion sections and press releases (a finding that often depends on the broadest possible interpretation of some minor data blip), and pretty soon this new idea becomes established within the field when a few other studies with some minor statistical blip also seem to provide "confirmation."

      Five or ten years later, somebody runs another better-designed study specifically on the topic, and it turns out there's no correlation at all. It was just a statistical ghost, or a badly designed set of data manipulations or collection... or just somebody trying to turn their crap data into something significant.

      But true randomized trials with human subjects that can accurately target something specific are often difficult to fund and sometimes even impossible (or unethical) to do. Nevertheless, even with limited options for good data, we can still be more realistic and cautious about what we claim out of it. Instead, there's always a rush to ascribe great meaning to every statistical blip. To me, that's the biggest problem in the so-called "soft sciences."

    20. Re:Science is the new religion by AthanasiusKircher · · Score: 1

      but the principles of science seem sound and correct over time.

      How long is "over time"? From my perspective on history, scientific "principles" and methodology are continuously changing. The standards of what constituted valid "scientific argument" were vastly different over time -- Copernicus, Kepler, Newton, Lavoisier, Darwin, etc. all had incredibly different views on what constituted acceptable scientific methodology. Yes, they all collected data, and "experiments" have been performed in various ways over the centuries, but the foundational axioms of how theories related to the world, what philosophical assumptions we make about scientific arguments, etc. have changed radically, even since the dawn of the "Scientific Revolution."

      And lest one claim that "Well, those guys all lived very long ago, and at least since the middle or late 1800s, we've basically been doing the same thing," I beg to differ. We've only come to understand a lot of details of statistical paradoxes and misleading aspects of data in the past century, for example. And given how central data manipulation is to most of experimental science nowadays, I have to say that the way we approach science is radically different from even a century ago. Hypotheses that would have seen perfectly legimitate in the early 1900s would be immediately thrown out today -- and not just because of changing knowledge... because of methodological issues. Scientific methods and principles continue to evolve as older methodologies are phased out, and (supposedly) better ones are discovered.

      Frankly, unless you define "the principles of science" as something so vague and meaningless as "collecting data from experiments to better understand the world," I don't see how there's that much similarity to what historical figures were doing. It's a continuously evolving system, and the "principles" change to fit the current methods.

    21. Re:Science is the new religion by Belial6 · · Score: 1

      Say that when a child Psychologist helps kidnap your child and then tells the judge that your child is suffering from "Oppositional Defiant Disorder" because they keep saying they want to go home. Judges and juries take them serious, and men with guns enforce the will of these judges and juries.

    22. Re:Science is the new religion by phantomfive · · Score: 1

      It may seem that the idea of doing an experiment and believing the results seems obvious to you, but it wasn't obvious for millenia. That's basically it, science looks at the evidence, as opposed to what any authority might say.

      Once again, to quote Feynman, "The principle of science, the definition, almost, is the following: The test of all knowledge is experiment. "

      --
      "First they came for the slanderers and i said nothing."
    23. Re:Science is the new religion by mjwx · · Score: 2

      I've made comments before comparing science and religion, and too often people think that I'm a religious person trying to belittle a genuine quest for knowledge. On the contrary, I think the genuine quest for knowledge is an amazingly worthwhile thing. However, science has become a method for the "practitioners" and "priests" to exert social, economic, and institutional influence by swaying the beliefs of those who are not educated enough or informed enough to differentiate between genuine knowledge and blind dogma.

      A lot of people who dont understand how the scientific method works and only get their science information from tabloid news papers think this way.

      It doesn't make it true.

      The difference between science and religion is that science actually questions itself, it is designed to be questioned and if proven wrong, science has to change. Religion has no such requirement and even when proven wrong beyond all doubt, has no impetus to change.

      You seem to think what you read from tabloids is real science, it isn't. This is why you think it's a grand conspiracy to "exert social, economic and institutional influence". Science does no such thing, the scientific method is the search for truth from experimentation and itself is not above question. However it appears that you dont understand the basics of it, you think a scientist starts with a result and works backwards where in reality, they start with a hypothesis and experiment, then gets a result. If the result does not match the hypothesis, the hypothesis was wrong and needs to be changed.

      Any "social, economic or institutional" change that occurs from science is due to the result changing a hypothesis held by that "society, economy or institution" that has been proven incorrect. The problem is, people who dont understand the science tend to think of this as a conspiracy to undermine them.

      --
      Calling someone a "hater" only means you can not rationally rebut their argument.
    24. Re:Science is the new religion by Anonymous Coward · · Score: 1

      Five or ten years later, somebody runs another better-designed study specifically on the topic, and it turns out there's no correlation at all. It was just a statistical ghost, or a badly designed set of data manipulations or collection... or just somebody trying to turn their crap data into something significant.

      It's worse than that because people are discouraged from publishing "negative" results. They are "not interesting".

    25. Re:Science is the new religion by Anonymous Coward · · Score: 1

      To clarify:
      Based on my experience with the literature, (lack of) proper training in experimental design and statistics, and general culture of "selling your idea" I believe that for medical research the false positive rate of publications is greater than 80%, I would not be surprised if it was as high as 98%. I suspect that tens of billions of dollars a year are being wasted due to these reasons just in the united states for that sub-field of biology.

      I am sure the social sciences are worse.

    26. Re:Science is the new religion by vigour · · Score: 1

      Very few are fooled. Sociologists/Psychologists/Economists can say they've 'proved' something till they're blue in the face. Nobody will take them seriously.

      I agree. You know which branch really bugs me though? Entomology.

      You sir deserve mod points.

      Relevant

    27. Re:Science is the new religion by Anonymous Coward · · Score: 0

      It's less that I'm a backwoods book-hating theist.

      But of course you are. You're dressing it up a bit, but in the end it's all the same garbage vomited out of your mouth:

      However, science has become a method for the "practitioners" and "priests" to exert social, economic, and institutional influence by swaying the beliefs of those who are not educated enough or informed enough to differentiate between genuine knowledge and blind dogma.

      Science is testable. That's why it's science, and will never be faith, and why you're being willfully obtuse by equating scientists with priests. I suppose you're also one of those morons who keeps insisting that Atheism is a religion years after "it's a religion like bald is a hair colour" debunked that entire line of reasoning.

      Go ahead. Mod me as flamebait.

      Between the obvious trolling and the obvious attempt to inoculate yourself, you deserve to be.

    28. Re:Science is the new religion by nine-times · · Score: 1

      And all *true* Scotsmen would agree with you.

    29. Re:Science is the new religion by nine-times · · Score: 1

      What does public perception have to do with what science is? They're two completely different things. How do you know what "most people" think about this?

      Because we're people, talking about 'science', so what we're talking about when we talk about 'science' matters. You may have a idealized concept that you intend when you say 'science', but if that's not what most people mean by the word, then suddenly conversations about 'science' are not about your idealized concept. They're about the other thing. The thing most people have in mind.

      And your idealized concept probably doesn't live up to the hype in your own mind, either.

  10. Mediocrity in Academics by EMG+at+MU · · Score: 4, Informative

    For years we have known that there is a glut of graduates in the system. I remember my freshman year at university, the attitude of a lot of students was "the Masters is the new Bachelors, you have to have one to get an entry level job" or when I got closer to graduating it was "well I don't want to be done with school and my parents are helping me out so I'm going to go for my Masters". While education is awesome, the fact is that you don't have to be all that smart anymore to get a Masters or PhD.

    Even as an undergrad I was pressured to publish. I didn't have the time nor the resources to do anything meaningful but my professors all said that I had to publish to even consider going to graduate school. They said that pretty much no matter what I do, even if its not novel or valuable to the academic community there will be a journal that will publish it. That's the current state of academics now.

    Lets be clear: I'm not talking about MIT or Berkley. I'm talking about the thousands of research institutions across the country that while also doing amazing research, churn out Masters and PhDs like a printing mill. When you dilute the pool of researchers there is going to be subpar research. When there is a glut of subpar research there will be journals that see the business opportunity and publish anything you pay them to publish. This is not new.

    1. Re:Mediocrity in Academics by Anonymous Coward · · Score: 0

      While education is awesome, the fact is that you don't have to be all that smart anymore to get a Masters or PhD.

      Agreed. The only problem being... you never did.

    2. Re: Mediocrity in Academics by Eugriped3z · · Score: 1
      Exactly what and where were you studying that you felt pressured to publish as an undergrad?

      It seems odd that a prof would push so hard unless you had a very specific and competitive grad study program in mind, something important to offer or you prof didn't.

    3. Re:Mediocrity in Academics by Talennor · · Score: 1

      And while the current reality isn't exactly new, we're creating new ways to understand and handle it. Can subpar research and publication be harnessed to advance us or is it just a drag? Is more education in our current model helpful to the new masses of PhDs? What does that do for science? Society? Not everyone uses their high school education in their jobs or really needs to even be literate, but it's made our society better in many ways to educate as many people as we do today. There's so many cool questions about what this is today that we're seeing and what it means and what we can do with it!

      --

      //TODO: signature
  11. Where are the names? by Anonymous Coward · · Score: 0

    I didn't see the names of the creationists who accepted the articles listed. Since it's known that this group is the only one practicing dubious science, I'd have preferred if their names were given. You know, to keep track of them.

  12. Not submitted to proprietary journals? by mspohr · · Score: 4, Insightful

    Science has an axe to grind here, obviously, and this "experiment" is seriously biased.
    It does not appear that it was submitted to any closed, for-profit journals (like Science). It would have been much more interesting to see how many of them would have accepted the paper.

    --
    I don't read your sig. Why are you reading mine?
    1. Re:Not submitted to proprietary journals? by Anonymous Coward · · Score: 0

      http://www.michaeleisen.org/blog/?p=1439

    2. Re:Not submitted to proprietary journals? by Anonymous Coward · · Score: 0

      The experiment was clearly biased towards low-quality open access journals, not just open access in general. There is more profiteering in low-quality open access journals simply because the publication costs are so much higher (typically 2-3x closed journals).

      Anyone with experience with submitting papers to peer review in higher-quality journals (open or closed) knows that it is difficult enough to get a good paper published.

    3. Re:Not submitted to proprietary journals? by DerekLyons · · Score: 1

      Science has an axe to grind here, obviously, and this "experiment" is seriously biased.

      Please, feel free to explain the bias - because you signally fail to do so.
       

      It does not appear that it was submitted to any closed, for-profit journals (like Science). It would have been much more interesting to see how many of them would have accepted the paper.

      Indeed, it would have been very interesting. But not doing so is not an indication of bias on Science's part - anymore than taste testing chewing gum but not bubble gum is a bias against chewing gum.

    4. Re:Not submitted to proprietary journals? by Anonymous Coward · · Score: 0

      Link shows a bad paper was submitted to Science and accepted.

    5. Re:Not submitted to proprietary journals? by mspohr · · Score: 1

      Basic reading and comprehension.
      Try reading it slowly (without moving your lips).
      Have you ever heard of a "control" group? (Hint: It's a basic part of the scientific method)

      --
      I don't read your sig. Why are you reading mine?
    6. Re:Not submitted to proprietary journals? by JanneM · · Score: 1

      Yes, they should have submitted it to a similar number of similarly ranked closed-access journals and seen if there's any difference due to the open access policies specifically. As it stands it's sort of interesting, but doesn't tell us squat about open access.

      --
      Trust the Computer. The Computer is your friend.
    7. Re:Not submitted to proprietary journals? by Anonymous Coward · · Score: 0

      Bingo.

    8. Re:Not submitted to proprietary journals? by Bill+Dimm · · Score: 2

      It does not appear that it was submitted to any closed, for-profit journals (like Science).

      But they did submit a bogus paper to Science. It was titled "Who's Afraid of Peer Review?" The paper lacked a control group, but Science published it anyway in spite of its obvious failure to measure up to scientific standards.

    9. Re: Not submitted to proprietary journals? by Eugriped3z · · Score: 1
      It's self-promotion, pure and simple. It's obviously not a scientific experiment. It's a "hand wavy" sort of thing, an estimate, a facsimile, a wild-ass guess. It's like reality TV. There should be a term of art to describe and distinguish it from a formal experiment or legitimate research. Maybe Science should create a new category, like "reality typing."

      After all, they are asking us to look askance at anything outside their rarified realm of the rigorous standards they claim to uphold. Why do their editors believe they should be able to get away without meeting the same level of scrutiny? Perhaps all is not what it seems at the top?

    10. Re:Not submitted to proprietary journals? by Anonymous Coward · · Score: 0

      Yeah, it is biased for journals run with strict review processes. Also, Science is published by the non-profit American Association for the Advancement of Science. When you get this wrong, how the fuck can I take anything you say seriously?

    11. Re:Not submitted to proprietary journals? by DerekLyons · · Score: 1

      Basic reading and comprehension.

      Is something you very badly need, because you utterly failed to either comprehend or answer my question.
       

      Have you ever heard of a "control" group? (Hint: It's a basic part of the scientific method)

      Yes, I've heard of a control group. No, it's not applicable here. If you're testing open access journals, you compare one to another (like for like). You don't do experiments on apples and use oranges as the control group. Or, to put it another way, you're a moron with no more clue about the scientific method than the pencil on the desk in front of me.

    12. Re:Not submitted to proprietary journals? by lubaciousd · · Score: 1

      So, you're saying the point of this study was to establish that open access journals aren't 100% fraud-proof? I don't think anybody would have doubted that result. I don't doubt that paywall journals aren't 100% fraud proof either, but I have trouble believing they're more reliable just because some amount of fraud has been demonstrated using this method on only open access journals. Maybe other journals have used different methods to evaluate paywall journal acceptance rates, but isn't the point of a control to do essentially the same thing in case the methodology introduces a systematic aberration in an expected result?

      This is apples and oranges - finding out that x out of y apples have parasites tells me something about whether or not an apple will be parasite free, but if I'm hungry and I need a parasite-free fruit, it would be a lot more useful to have the same test applied to the oranges as well.

      This is absolutely a control case that would greatly increase the informative value of the results, and no amount of insulting other people will change the fact that you are the one who is wrong. Surely you're trolling, Mr. Lyons.

    13. Re:Not submitted to proprietary journals? by Belial6 · · Score: 1

      How can you say that? They didn't divide the paper up between those that published the fake paper, and those that did not. They divided the groups between open access journals and... well... we won't talk about the other group. They then painted the open access journals that rejected Science's fake paper with the same brush that they painted the ones who accepted it.

  13. The question is: by Anonymous Coward · · Score: 0

    What are the numbers if you try the same with pay for read journals?

    I'd guess if you try the same with several hundred non-open journals the number would not be that different...

  14. really? by stenvar · · Score: 2

    His hoax paper claimed that a particular molecule slowed the growth of cancer cells, and it was riddled with obvious errors and contradictions.

    And this is different from the average Science paper... how?

    1. Re:really? by Anonymous Coward · · Score: 0

      His hoax paper claimed that a particular molecule slowed the growth of cancer cells, and it was riddled with obvious errors and contradictions.

      And this is different from the average Science paper... how?

      They admitted it?

  15. Is that any worse then traditional journals. by MouseTheLuckyDog · · Score: 1

    Ask any academic and they will tell you how to get articles published no matter what.
    Did they try this with traditional journals and get better results?

    BTW, the readers are supposed to be trained in the subject. They should be able to spot a paper riddled with obvious errors and contradictions.

    BTW didn't Pons and Fleischmann, publish a paper in Science?

  16. Re:The total number of these journals is irrelevan by goodmanj · · Score: 4, Insightful

    The problem is that serious decisions are made by people who have no idea which journals are top quality. Bad tenure decisions, bad engineering choices, and god forbid bad medical decisions are being made daily on the basis of nothing more than "hey, the European Journal of Chemistry sounds legit."

  17. Peer Review by Anonymous Coward · · Score: 0

    I am not a scientist, but isn't the point of publishing a paper in a journal, so it can be reviewed and tested by other scientists around the world who will either confirm or reject (parts of) it.
    If this is the case, then what's the difference between this and any other erroneous paper, other than the fact that in this case it was done intentionally?

  18. Arsenic in DNA by Anonymous Coward · · Score: 0

    Science published the "Arsenic in DNA" paper. They decided to publish it because the results were unusual, therefore "interesting". They didn't seem to care the results were false.

    1. Re:Arsenic in DNA by Anonymous Coward · · Score: 0

      They didn't know the researcher had falsified the data. The purpose of peer review is to find papers where the results don't mesh with the data. The purpose of publishing is so that *other* researchers can replicate the experiment and validate the data.

      Publishing a paper where the data has been carefully crafted to appear legitimate isn't a failure of the publisher, it is a failure of the researcher who submitted the paper. That failure will be exposed when others attempt to replicate the experiment, and cannot. Strangely enough, that's exactly what happened.

      The paper being discussed here (in relation to the article), is one where the data and the conclusions were obviously at odds. Failure to reject such a paper for publishing is indicative of a failure of peer review. Thus, it is of note that fewer than 50% of the journals 'peer review' processes caught the errors.

  19. some pretty valid pushbacks on this article by Anonymous Coward · · Score: 1

    This article is being widely panned as lacking controls, published without any critical review, and driven by self-interest from a traditional publisher with the most to lose from Open Access taking off (as it is). Some have gone so far to assert it's an over-reach for how badly it was done, and will make Science as a journal look partisan.

    For example, quick scan brought up these three scathing responses:

    Mike Eisen (HHMI Berkeley Professor)
    http://www.michaeleisen.org/blog/?p=1439

    Peter Suber (Author of the book "Open Access", Director of the Harvard Open Access Project, Faculty Fellow at the Berkman Center)
    https://plus.google.com/u/0/109377556796183035206/posts/CRHeCAtQqGq

    Mike Taylor (programmer with Index Data and a research associate at the department of earth sciences, University of Bristol)
    http://svpow.com/2013/10/03/john-bohannons-peer-review-sting-against-science/

    I'm sure this will heat up some much needed debate about poor quality journals and the failings of peer review, but with the lack of any controls at all, it says basically nothing about open access as a model for publishing.

  20. John Bohannon by Anonymous Coward · · Score: 0

    He's published on Facebook and LinlkedIn, he must be legit.

  21. several valid pushbacks from this article by drDugan · · Score: 4, Informative

    This article is being widely panned as lacking controls, published without any critical review, and driven by self-interest from a traditional publisher with the most to lose from Open Access taking off (as it is). Some have gone so far to assert it's an over-reach for how badly it was done, and will make Science as a journal look partisan.

    For example, quick scan brought up these three scathing responses:

    Mike Eisen (HHMI Berkeley Professor)
    http://www.michaeleisen.org/blog/?p=1439

    Peter Suber (Author of the book "Open Access", Director of the Harvard Open Access Project, Faculty Fellow at the Berkman Center)
    https://plus.google.com/u/0/109377556796183035206/posts/CRHeCAtQqGq

    Mike Taylor (programmer with Index Data and a research associate at the department of earth sciences, University of Bristol)
    http://svpow.com/2013/10/03/john-bohannons-peer-review-sting-against-science/

    I'm sure this will heat up some much needed debate about poor quality journals and the failings of peer review, but with the lack of any controls at all, it says basically nothing about open access as a model for publishing.

    1. Re:several valid pushbacks from this article by drDugan · · Score: 1

      and two more

      "Bohannon’s article is hostile. He submitted articles only to OA journals and by omission thereby erroneously links the failure of peer review to a single business model"
      https://www.martineve.com/2013/10/03/whats-open-got-to-do-with-it/

      "worthless travesty"
      http://gavialib.com/2013/10/which-is-it/

  22. SCIENCE is just such a predatory journal by Anonymous Coward · · Score: 0

    This paper is a tour-de-force of ironic brilliance.

    This is a paper describing an experiment with NO CONTROLs, and SCIENCE snapped it up because it met their editorial biases. Bohannon sent the other buggy paper to no closed-access journals, and this one to no open-access journals.

    http://svpow.com/2013/10/03/john-bohannons-peer-review-sting-against-science/

    Such subtlety must be admired.

  23. What do we want in a paper? by Okian+Warrior · · Score: 1

    I've been studying this (publishing) for some time, in the context of learning, verifying assumptions, and the scientific method.

    It turns out that there is really no bar in scientific publishing. It doesn't have to be understandable, nor innovative, nor even correct. You only need to be ethical (ie - don't lie about the data), cite anything that you got from other sources, and show that there is less than a 1-in-20 chance that you are wrong (p > 0.5).

    What exactly do we want in a published paper, anyway?

    Many cancer studies can't be reproduced. Many studies are statistically significant but valueless (the IQ of people in NYC is higher than Chicago by 1 point: this can be statistically certain but have no practical significance). There are lots and lots of ways to frame the conclusion the wrong way such as confusing correlation with causation, reversed conditionals (if the defendant is innocent, there is a 1 in 1 billion chance that this evidence is wrong), and other logical errors.

    Then there's the enormous economic incentive of needing to publish to keep your job, that reviewers will oppose maverick thought and agree with community beliefs, and that no one examines their assumptions.

    Would you like to publish a paper? MathGen will write one for you. Pass it around and chances are it will be accepted.

    So when I talk to people about my research, the inevitable comment is "you should publish". And my inevitable answer is: why?

    What do we want in scientific papers? What are they even for?

    1. Re:What do we want in a paper? by stenvar · · Score: 1

      What do we want in scientific papers? What are they even for?

      They are for communicating useful ideas. And papers do that and do it well.

      But you don't need peer review or journals for that. With the Internet, people increasingly just put papers out as tech reports, and they get cited and used by others.

  24. Re:The total number of these journals is irrelevan by semi-extrinsic · · Score: 5, Interesting

    In Norway, we have a "level" system that is used in academia throughout the country. It is used for evaluating researchers and research groups when it comes to employment, tenure, funding etc. Your "point score" is summed up, 2 points for publication in a "level 2" journal, 1 point for "level 1".

    A journal is either "level 2", "level 1" or "level 0". "level 2" is a selection of top journals from each field in science, 2000 in total (for all of science, from computational physics to the sociology of music). "level 1" means the remaining serious peer-reviewed journals. "level 0" either means "bullshit journal" or "journal that was founded just last year".

    Researchers may nominate journals for a change in status, e.g. 2->1, 0->, etc. The decisions are made by a government-appointet body on a yearly basis. It's nowhere near perfect, but it's a lot better than nothing.

    --
    for i in `facebook friends "=bday" 2>/dev/null | cut -d " " -f 3-`; do facebook wallpost $i "Happy birthday!"; done
  25. Entrapment! by OakDragon · · Score: 1

    That science journal had been flaunting its low standards all over town!

  26. The problem is more that of peer review by WillAffleckUW · · Score: 3, Insightful

    It's more who is reviewing the material, and to what level, that matters.

    A lot of high-grade peer reviewed journals, like Science and Nature, have been hoodwinked by researchers from South Korea and China, where cheating is more endemic than here.

    Or, as most of us say, Wait Until The Second Journal Article.

    --
    -- Tigger warning: This post may contain tiggers! --
  27. Re:The total number of these journals is irrelevan by Obfuscant · · Score: 4, Insightful
    The total number of these journals is perhaps the more relevant part of this article. There are 304 journals that are potential relevant places for that one submission? How can anyone keep up with the current science in any field when there are 304 places to look? Never mind that many of those aren't sufficiently vetting the product.

    And if you are just writing them off and basing your reading on the "top ones", of what value are these?

    While science journals are often used by researchers to find out what their colleagues are doing and can thus be vetted by the reader, they are quite often the bases for undergraduate and graduate educations, and putting deliberate crap in front of them is a Bad Thing.

  28. Karma! by almechist · · Score: 1

    This is the same problem the internet has faced right from the beginning, and is not confined to academia: who do you trust online, and how can you be sure they're on the level? Someone or something is needed to weed out the bad apples... In other words, moderation. And yes, the same basic principles apply equally to discussion forums like Slashdot as they do to online scientific research journals. Ultimately it comes down to reputation, and some form of karma system. Slashdot's system uses temporary moderators selected by an algorithm, but for scientific journals there is currently just the one site mentioned in the article, run by one guy who is single handedly attempting to keep track of who the legitimate journals are. I don't see why this function couldn't eventually become more automated, perhaps even incorporating random moderation and meta-moderation overseen by an algorithm, just like Slashdot. There's been plenty of research into reputation management systems over the years, surely there must be something that could apply to the chaotic research journal situation described in the article, perhaps even an already existing software package. The phenomenon of open online academic journals is relatively new, these things usually work themselves out over time, as with any new technology. The idea of open journals is a good one, sooner or later some system of useful self-regulation will emerge, and the useless and/or predatory journals will eventually fall by the wayside.

  29. He would have submitted it to 304 closed journals by Deliveranc3 · · Score: 1

    But it would have cost him millions of dollars. And there was no chance that the open journals would pay him to discredit their competitors.

    Is there a list of the open journals that caught the fraud?

  30. The problem isn't open access journals... by blahplusplus · · Score: 2

    ... the real probem is that as problem size increases, the human brain just can't deal with all the stress and energy one must expend to fact check everything. This is why we need automation in checking papers for errors and contradictions, i.e. the number of facts you need to know grows exponentially as things get more complex. What we're really seeing is that the human brain is the biggest bottleneck since human beings have limited time and energy. So no one should be surprised it's easy to 'dupe' or game a system because the resources you need to stop untrustworthy people is unrealistically expensive. Any area of human endeavor is only as good as the people themselves.

    1. Re:The problem isn't open access journals... by jimbodude · · Score: 1

      I'd mod you up if I had points, so instead I'll just say that I think there is truth to what you have said. Researchers, professors in particular, are very busy people and can only spend so much time on a thorough review of a submitted article. Finding some way to help automate this would be a great research project and a great help to the whole process.

  31. subjects are extremely vital by Anonymous Coward · · Score: 1

    Shady journals and conferences might be a problem when it comes to someone applying to a job at some small company that doesn't have any PhDs in that field, but for anyone who's spent any time in legitimate academia, this isn't an issue. If someone comes claiming to be an expert in networks with a whole bunch of great publications, and none of those publications are in SIGCOMM, NSDI, CoNEXT, etc. I'm going to assume they're a fraud.

    I like the comments above about this being perhaps somewhat of a shrill reactionary takedown attempt on open access venues. Paywalled conferences/journals are absolutely disgusting, given the setup today - a bunch of professors (and/or their grad students) reviewing papers for free, because it's what's expected of them. There's really no excuse, especially with USENIX as a model (good peer review, they run nicely put together conferences, and once the conferences are done, the papers are free for the world to read).

  32. Re:The total number of these journals is irrelevan by VortexCortex · · Score: 1

    Mod Journal modder up!

  33. Re:The total number of these journals is irrelevan by ShanghaiBill · · Score: 1

    How can anyone keep up with the current science in any field when there are 304 places to look?

    1. By reading specific articles that colleagues recommend.
    2. By using a search engine.

  34. The Wikipedia Bio In Full by westlake · · Score: 3, Informative

    I've never heard of this person John Bohannon.

    John Bohannon is a biologist, science journalist, and dancer based at Harvard University. He writes for Science Magazine, Discover Magazine, and Wired Magazine, and frequently reports on the intersections of science and war. After embedding in southern Afghanistan in 2010, he was the first journalist to convince the US military to voluntarily release civilian casualty data. He received a Reuters environmental journalism award in 2006 for his reporting on collaborative research in Gaza. He was also involved in some controversy over an article he wrote critiquing the Lancet survey of Iraq War Casualties.

    At Science Magazine, Bohannon also adopts the ''Gonzo Scientist'' persona, where he ''takes a look at the intersections among science, culture, and art -- and, in true gonzo style, doesn't shrink from making himself a part of the story. The stories include original art and accompanying multimedia features.'' As the Gonzo Scientist, Bohannon's research on whether humans can tell the difference between pate and dog food led to Stephen Colbert eating cat food on the Colbert Report.

    Bohannon is probably best known for creating the Dance Your PhD competition, in which scientists from all around the world interpret their doctoral dissertations in dance form. Slate Magazine ran a profile on Bohannon and the competition in 2011. He performed with the Black Label Movement dance troupe at TEDx Brussels in November 2011, where he satirized Jonathan Swift's A Modest Proposal by modestly proposing that Powerpoint software be replaced by live dancers. Bohannon then went on to perform with Black Label Movement at TED 2012 in Long Beach.

    Advisory Board - John Bohannon

    While visiting the Harvard University Program in Ethics and Health, he is working on two areas of research: 1) torture --- in particular the complicity of scientific and medical workers in torture, and 2) ethical problems involved with obtaining global health data, stemming from his journalistic coverage of the controversial attempts to estimate the health and mortality of the Iraqi population since the US-led invasion.

    After completing a Ph.D. in molecular biology at the University of Oxford in 2002, John focused on bioethics as a Fulbright fellow (2003 --- 2004) in Berlin.

    1. Re:The Wikipedia Bio In Full by Anonymous Coward · · Score: 0

      (Posting AC for obvious reasons.)

      I worked with Bohannon a bit when he wrote up our research project a couple of years ago in a major popular magazine. The gonzo style was very much in evidence; and we were amused by it. We needed the publicity and he ended up giving us a decent writeup, though it wasn't at all focused on the things we thought were important. Instead of a more neutral article just presenting our work it ended up being like a human interest story. It was a little frustrating working with him because of what he focused on, and a little amusing to read the finished writeup, but we were happy with it.

  35. Well ... by PPH · · Score: 0

    ... this appears to be a Global problem in scientific publishing. I'm glad that some journalists are finally Warming up to the problem.

    --
    Have gnu, will travel.
  36. DARPA Inference Cheking Kludge Scanning by utkonos · · Score: 1

    This article sounds like the DICKS plugin for nmap that was described in this issue of hakin9. This was a beautiful trojan horse.

  37. Re:The total number of these journals is irrelevan by dgatwood · · Score: 4, Interesting

    A journal is either "level 2", "level 1" or "level 0". "level 2" is a selection of top journals from each field in science, 2000 in total (for all of science, from computational physics to the sociology of music). "level 1" means the remaining serious peer-reviewed journals. "level 0" either means "bullshit journal" or "journal that was founded just last year".

    Here's the problem with doing that so systemically: it is fundamentally anticompetitive, and leads to stagnation. Nobody would bother submitting to a "level 0" journal because it won't earn them any props at all, which means that the journal can never become anything more than a "level 0" journal. This means that you don't get fresh blood with new ideas on the review boards, so progress moves at a snail's pace. There's something to be said for disruptive innovation, even in academic publishing circles.

    Also, the entire notion of judging the value of your scientific contribution based on what journal agreed to publish it is as absurd as judging the value of a car based on what dealer sold it. A paper should stand or fall on its own merits. A good article that pushes science forward, even if published in a minor journal, should weigh significantly in your favor for tenure, and a lousy article, even if published in a major journal, should not. A system that does the opposite is abject stupidity, pure and simple.

    --

    Check out my sci-fi/humor trilogy at PatriotsBooks.

  38. Who funded this study? by Anonymous Coward · · Score: 0

    Who handed over the cash for this study? I'm guessing Elseiver, Springer or some other big publisher who use academic subscriptions as their cashcows had a hand in it.

  39. Which Journals? by Anonymous Coward · · Score: 0

    Which Journals accepted the paper?

  40. Re:The total number of these journals is irrelevan by Obfuscant · · Score: 1

    1. By reading specific articles that colleagues recommend.

    Hmmm. So you know only what they know. Could work ok.

    2. By using a search engine.

    Search engines find words and phrases. They don't vet the material they return, nor do they usually return only what you are actually looking for. You've replaced the problem of finding relevant articles in the tables of contents of 304 journals with the problem of finding relevant articles in the 345,289 hits returned by Google.

    And if you are going to just Google for the articles, why have 304 journals at all? Just put all the articles on the web and use Google to find them.

  41. Oh my! by WeeBit · · Score: 0

    One man alone can be pretty dumb sometimes, but for real bona fide stupidity, there ain't nothin' can beat teamwork.

  42. Re:The total number of these journals is irrelevan by mysidia · · Score: 1

    How can anyone keep up with the current science in any field when there are 304 places to look?

    You don't need to look in 304 places; only one or two.

    Read Slashdot news for nerds, stuff that matters

    I hear everything that really matters winds up on there :)

  43. Re:The total number of these journals is irrelevan by Anonymous Coward · · Score: 0

    Also, the entire notion of judging the value of your scientific contribution based on what journal agreed to publish it is as absurd as judging the value of a car based on what dealer sold it.

    That's right - because no journal ever published an article without checking it over.

  44. Sum Ting Wong by Grand+Facade · · Score: 0

    Wi Tu Lo
    Bang Ding Ow

    --
    Rick B.
  45. Re:The total number of these journals is irrelevan by Anonymous Coward · · Score: 0

    I think that open-access is a great idea, however like most open services and products they lack any tight reviews by experienced people in a particular field. This gives these anti-science people more reason to discredit science.

    I've said it before, science isn't a guarantee, and is not written in stone, research and new findings will and should always be taking lightly with an open mind. But with the media/press, internet taking everything literally, even tho I do have hope a majority of the populous do not just jump in head first believing the media/press.

  46. Re:The total number of these journals is irrelevan by rtb61 · · Score: 1

    The whole journal publishing idea seems like one great big obfuscating scam. It seems like publishing has nothing to do with it at all, it is all about the reality of peer review or it's absence. What seems to be the most important issue here is not publishing but the article peer review process. How many people reviewed it, who are they and what are their qualifications with regard to suitability to be involved in the review process. When left to private for profit enterprise this seems basically to be a major screw up and as lives appear to be at stake with regard to what is done with this information a much more government regulated and controlled environment needs to be set up to monitor and control the whole peer review and publishing process.

    --
    Chaos - everything, everywhere, everywhen
  47. what did they expect??? by l3v1 · · Score: 4, Interesting

    In all seriousness, I - as a researcher myself - understand the need of easy access to publications. However, I never supported the open access models that came into existence and are being built and pursued today. Why? Because it's all about the money and a lot of such journals absolutely do not care about quality, or about having big name editors who'd perform very thorough revision of reviews and make proper decisions about paper acceptances. Big journals have good editorial and review staff, and they simply can't allow them to be bad and irresponsible, because they actually care about their reputation and credibility. New breed open access journals on the other hand only care about revenue.

    The instititue I work at has mandated open access publication as well as others did, however, they did not provide funding for us to actually publish open access versions at big name journals, so we try to play the system whenever we can, and publish in traditional journals with traditional publication schemes. I do not care about some politician-flavored scientists' (most of them not even publishing) dreams about some utopistic open access world. I care about publications appearing in credible journals, reviewed by credible people, producing quality publications - even if they are only attainable for money.

    --
    I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
  48. Self-serving stunt by quax · · Score: 1

    Reputable journals are only marginally better. Just witnessed a back and forth where some research was attacked by a prominent scientist. The assumptions the latter made weren't quite on target, so the attacked researchers submitted a paper pointing this out. This passed anonymous peer review but then the paper solicited the opinion of this star scientists. He dismisses the paper with the most bizarre arguments that give the impression that he didn't even read it. Then the prestigious journal turns around and endorses this position and rejects the paper.

    Absolute astounding and sobering.

    Disclaimer: I am not a party or author in this but covered the controversy on my blog, and hence have been shown the rejection email.

  49. Re:The total number of these journals is irrelevan by steelfood · · Score: 1

    Yes, but that requires management to actually read the papers in order to make an informed decision and *gasp* know what they're doing.

    It's so much easier to just count the number of published papers. It is college after all, where they rate wine by the ABV.

    --
    "If a nation expects to be ignorant and free in a state of civilization, it expects what never was and never will be."
  50. Re:The total number of these journals is irrelevan by Anonymous Coward · · Score: 0

    If you still think this way and are a scientist you are part of the problem. You people are stupidly setting yourselves up as fall guys.

  51. Re:The total number of these journals is irrelevan by NeutronCowboy · · Score: 1

    I need to move to Norway. For some reason, it seems to be the only sensible country on earth. Now if we could just tilt the rotational axis of the earth a little farther up, and we'd solve it's only problem: eternal days and eternal nights.

    --
    Those who can, do. Those who can't, sue.
  52. Re:The total number of these journals is irrelevan by Anonymous Coward · · Score: 0

    Read journals from before "the government" got involved in funding. I mean pre-WW2 and then those by people trained from that era (decreasingly so up to the 90s). The average report was 10x as credible even with far less tech and no charts. They carefully describe what they did along with describing individual results and why they differ from each other. Now it is just average plus some error bars.

    More importantly, experiments were designed to gain evidence for or against a theory rather than to say something was unlikely due to chance.

  53. a little secret us scientists have by Anonymous Coward · · Score: 0

    we all know that most - way more then 50, more like Sturgeon's law level - of "peer reviewed" journals are crap
    we don't say it, cause we might have trouble with funding.
    The Journal of Molecular Biology is still pretty good, and used to be one of the most prestigous journals in the field
    iirc, the back page had condensed instructions for authors which included
    a significant finding is often followed by replicate findings in other organisms. in general, the JMB won't publish such follow on reports....

    lets have some truth here: "science" magazine is the top 1%, or even 0-.1% of science papers
    most of these other journals are just crap; 3rd rate scientists publish in them and cite each other, but nothing of importance happens, as you can verify by looking at citation rates

  54. Re:The total number of these journals is irrelevan by TapeCutter · · Score: 2

    A paper should stand or fall on its own merits.

    Yep, and quality papers is what they should be competing for. The journal ranking systems used by univisities (not just in Norway) are designed to give more weight to journals that have a long track record of doing that. This is why the Nature and Science journals at at the top of the list, their long publishing history and track record of quality papers speaks for itself. A low ranked journal will stay a low ranked until it's track record is such that it can be deemed a reliable source. If it does nothing to improve it's record then it follows it will never be respected.

    A good article that pushes science forward, even if published in a minor journal, should weigh significantly in your favor for tenure, and a lousy article, even if published in a major journal, should not.

    The "impact factor" of individual papers is generally weighed by the number of citations, not the name of the journal.

    --
    And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
  55. Or by Anonymous Coward · · Score: 0

    Or just tell us who caught the issues so we know what open journals are worth reading. This headline could have been really uplifting. What about "List of the 36 most reliable Open Access Journals."

  56. social science is to science as social justice is by Anonymous Coward · · Score: 0

    "...all men are Created equal and endowed by their Creator with certain inalienable rights..."

    This is a religious statement. But religion gets you laughed at these days, while science put a (heterosexual married White cis)man on the moon and split the atom, so science is infallible.

    So the first part has to be reinterpreted as "all identifiable demographic groups have equal average intelligence". Which is a scientific statement, is laughably false, and is boldly asserted and backed up by threats as much as any religious dogma ever was.

    The second part needs to be reinterpreted as "governments which respect these rights are better than governments which don't". This is _not_ a scientific statement, and can never be transformed into one, until you define what makes a good government, which is a value judgement.

    There are people who will assert that both parts are settled science. They call themselves "progressives", and what they mean by "science" is not the same as what we mean by "science".

    For every class of decision a modern government makes, from diplomacy to economics to issuing fishing licences, there exists a caste of scholars in the social sciences, carefully selected for their race, gender identity, sexual orientation, intelligence and/or political reliability, who "use the methods of science" to divine the correct public policy. None of these professors is in any way, shape or form responsible for the success or failure of these policies.

    Decisions are not personal, but procedural. A procedure is a better procedure if it cuts more stakeholders into the loop - if it is a more open process. Here we see clearly what the State is doing: it is building a support base from its own employee roster, and it is purchasing support by exchanging it for power. The feeling of being in the decision loop produces a remarkable effect of emotional loyalty, no matter how trivial the actual authority may be. There is just a slight downside to this: when socialism fails, no one is responsible. No system of ideas, even, can be responsible - for a system of ideas would be an ideology, and public policy is not determined by ideology. Thus many will tell you that economics failed in the crisis of 2008, but no one can possibly do anything about it.

  57. Seems more like an excuse to censor by s.petry · · Score: 1

    While you raise an interesting point, that open journals should be very suspicious and scrutinize probably better than closed systems, the point is regarding a study which was not at all scientific. If an experiment is done to show people accepting bad papers and only one group is tested, how is this "science". More importantly since this is an article, how is it "fair" journalism?

    Since I see garbage on closed proprietary sites as well, why would they not also submit the same bogus papers to closed journals? If you want examples read just about every corporate sponsored report on GMO foods and Global Warming. Perhaps they did sample closed systems, and they did not publish the result. Did the result not favor their implication that "open is bad"? This becomes a very important question.

    Hell, maybe it's a spoof report to see how many suckers fall for the gag. I doubt it, since censorship has been a hot topic to the string pullers for quite some time.

    Analogy time for the people not seeing it. This "study" and article is like having people inspect Super 8 motels and finding roaches in more than half the rooms, then claiming "More than half of all Hotel rooms are infected with roaches.".

    That is obviously sensationalizing a study which is not at all scientific.

    --

    -The wise argue that there are few absolutes, the fool argues that there are no probabilities.

    1. Re:Seems more like an excuse to censor by Anonymous Coward · · Score: 0

      Oh, dear. An appeal to 'fair' journalism. Journalism isn't about being 'fair'. It's about reporting on the *facts*, and providing the necessary factual background so that people can *understand* those facts.

      The study in question wasn't a survey of *all* scientific journals, it was a survey of *open access* journals. For that reason it is completely valid to *only* send the bogus paper to open access journals, just as you would only measure the temperature of people with the flu if you were looking to study the temperature of people with the flu.

      Unlike your analogy, the study didn't make the conclusion about all scientific journals. In fact, it did just the opposite. It gave conclusions about *only* open access journals. Strangely enough, that's the set the study was restricted to.

      s.petry is yet another sad indictment of an education system. Basic reading comprehension failure.

  58. Re:The total number of these journals is irrelevan by Anonymous Coward · · Score: 0

    Here's the problem with doing that so systemically: it is fundamentally anticompetitive, and leads to stagnation. Nobody would bother submitting to a "level 0" journal because it won't earn them any props at all, which means that the journal can never become anything more than a "level 0" journal.

    Thiswould only be true if you wanted to start a journal which only accepted submittions within the country. Meanwhile, there are nearly 200 countries in the world, each one with their academia. If a journal is started and in spite of having a pool of nearly 200 countries-worth of academics it doesn't manage to attract a respectable amount of relevant papers then Norway isn't the one to blame here.

  59. Zeljko Svedrzic Ph.D.who published in PLoS by Anonymous Coward · · Score: 0

    Science Magazine "Sting Operation" Catches Predatory Journals In the Act is a trash act, since the author of then sting operation did not test together open-access and the traditional journals.
    In 2001 Nature published papers that NSAIDs can be inhibitors of gamma-secretase, and thus potential cure for Alzheimer's disease. The authors observed inhibition when they used NSAIDs at several hundred microM!?!?!? At that concentration any aromatic compound from a compound library would be a hit. No surprise that up to now we had several hundred drug candidates for Alzheimer's disease that failed.

    Furthermore J Biol. Chem, i Nucleic Acid Research published papers that mammalian Dnmt1 is a highly processive enzyme, even though the enzyme activity was expressed in raw cpms, and the enzyme can not make more than two turnovers in 5 hours!!?!?! The processivity studies got more than 300 citations!?!?!

    Traditional journals are as trashy or good as PLoS One. We all know all of that, and we know how to read papers to spot trash. We just do not want to pay extreme prices to read research supported by the taxpayers money.

  60. And now, repeat the same experiment with... by ArsenneLupin · · Score: 1
    ... Elsevier Journals.

    I'm convinced that if the right buzzwords and writing style is used, a sizeable percentage of so called "reputable" journals will fall into the same trap!

    1. Re:And now, repeat the same experiment with... by ArsenneLupin · · Score: 1
      Actually, as somebody else here has pointed out, they did submit one single dodgy paper to a reputable journal as well, and it got accepted! (meaning 100% acceptance rate in "reputable" journal, versus only 51% in open-access...) So the study's conclusion should not really be what it looks like at first glance... Ok, admitted, the sample size of control group is way too small, but that's needed to make the paper dodgy.

      ... but in any case, it's an interesting twist of the liar's paradox... I say, a twist, rather than an example, because no control group at all is even worse than one with a sample size of 1.

  61. Umm no by Weezul · · Score: 4, Insightful

    Clicks are not the problem. Journals don't get any money from advertisement clicks. Real problem is :

    At present, "Open Access Publishing" mostly means "Author Pays". If the author is your customer, then obviously you publish whatever they want. We must abandon the extortionate academic publishers like Elsevier all together by building an arXiv overlay filters that take over the journal's role of reviewing and declaring papers important. And these must be paid for by tax money because the customer should be society.

    Just like with universities, Britain has rampant grade inflation because the students all pay 15k USD per year (9k GBP). St Andrews has a 98% graduation rate. A 98% graduation rate tells me the university did basically no "selection" on their admitted students, all selection occurred when an admissions person read their test scores from high school. In other words, the student is the customer and the product is a little piece of paper. This is why Britain sucks so bad at engineering and must create that blatantly bullshit ranking system by THES to make themselves look good.

    In continental europe, almost everyone who finishes high school can attend university without paying, but the universities select students by failing out the shitty ones, well society is the customer and the students are the product. It's infinitely more fare because gaming the system in high school does nothing and people who never really hit their stride until the find challenging material do well.

    --
    The Christian religion has been and still is the principal enemy of moral progress in the world. -- Bertrand Russell
    1. Re:Umm no by Compaqt · · Score: 1

      >It's infinitely more fare

      Um, you must mean more fair, less fare.

      --
      I'm not a lawyer, but I play one on the Internet. Blog
  62. Re:The total number of these journals is irrelevan by Anonymous Coward · · Score: 0

    I need to move to Norway. For some reason, it seems to be the only sensible country on earth. Now if we could just tilt the rotational axis of the earth a little farther up, and we'd solve it's only problem: eternal days and eternal nights.

    Well, Norway also produce this.. Btw. the 24-hour dark in winter/24-hour sun in summer is just northern Norway.

  63. Re:The total number of these journals is irrelevan by dave420 · · Score: 0

    Use scholar.google.com or another specialised search engine, and quit your whining. Or did you honestly think you'd found a massive flaw in the system all by your little self, and then thought it awesome to bash keys in a cloud of pride in order to spread the word? Muppet.

  64. Thank the underwriters by ThatsNotPudding · · Score: 1

    "In closing, we would like to thank the underwriters of our story: Elsevier."

  65. Global Warming Citations by Anonymous Coward · · Score: 0

    As part of my PhD work on credibility in the AGW debate landscape, I studied 9,332 papers published in support of AGW. Those 9,332 papers had 726,712 citations, of which 631,989 were from "open access" journals. Of those 726,712 cited works, 711,349 of them had at least one author who was connected to the citing author either by being at the same institution or by sharing authorship on another paper.

    I also studied 1,454 papers published "against" AGW. Those 1,454 papers had 154,365 citations, of which only 16,403 were from "open access" journals. Of those 154,365 cited works, 14,589 of them had at least one author who was connected to the citing author either by being at the same institution or by sharing authorship on prior paper with similar findings to the citing paper.

    "Anti" papers had far more citations per paper, and from more "closed access" journals, than the "pro" papers did, and far, far more of the "pro" papers can be connected professionally to the citing author, which creates credibility problems, especially with citing someone who co-authored a prior work with you.

    Oddly I am having to fight my own institution to include this data in my dissertation. I wonder why. Researchers at my institution has published 9,423 papers in support of AGW and just 492 against, and I suspect there is significant administrative coercion that is the root cause of that.

    1. Re:Global Warming Citations by Anonymous Coward · · Score: 0

      Well, I suspect (based on your post here) that the reason you're having issues including this data is that you seem to be attempting to draw conclusions about those numbers which aren't actually supported by the numbers themselves. A raw citation count tells you nothing of the quality of the work being cited. A raw relationship graph between cited authors tells nothing about the motives behind the citation, and tells nothing about the quality of the work being cited.

  66. Formatting. by leuk_he · · Score: 1

    management could review on formatting, since that is what a lot of these open access journals do.

  67. Re:The total number of these journals is irrelevan by Slashdot+Parent · · Score: 1

    The problem is that serious decisions are made by people who have no idea which journals are top quality.

    I'm not an academic, but I am fairly certain that the publishing portion of tenure decisions means publishing many papers in high impact factor journals ("impact factor", by the way, is the google search term that you're going to want to use if you want to look more into journal rankings).

    There are established and readily-available metrics to measure journal quality. Department chairs, researchers, engineers, and doctors are not stupid people. They know that a paper in an open access journal whose results have never been replicated might as well have been "written" by an academic bullshit generator.

    --
    They don't grade fathers, but if your daughter's a stripper, you fucked up. --Chris Rock
  68. Re:The total number of these journals is irrelevan by Anonymous Coward · · Score: 0

    Yep. You 1) only randomly find out about things that your various colleagues and friends have found, and find interesting or insightful. Or 2) Only find out about things that you're specifically looking for. And 3) find out about anything any of those sources point you toward.

    Welcome to how the rest of life works, just without the advertisement-driven drivel.

  69. Peer review??? by Kodack · · Score: 1

    The last time I checked, the editorial staff at a journal wasn't expected to vouch for or ascertain the scientific merit of a paper. Journals are the realm of proof reading for spelling errors etc etc, not doing peer review.

    The way it's supposed to work is

    1. A paper is written
    2. A paper is submitted to a journal
    3. The journal ascertains the subject of the paper
    4. Journal sends copies of that paper to other experts on the subject of the paper
    5. Those peers then vouch for or criticize the paper
    6. Journal either rejects or accepts the paper for publication.

  70. Re:The total number of these journals is irrelevan by Obfuscant · · Score: 1

    Wow. Glad to see the level of civil discourse in /. has not dropped considerably.