Slashdot Mirror

← Back to Stories (view on slashdot.org)

Scientific Study Finds There Are Too Many Scientific Studies

Posted by Soulskill on from the my-study-can-beat-up-your-study dept.

HughPickens.com writes: Chris Matyszczyk reports at Cnet that a new scientific study concludes there are too many scientific studies — scientists simply can't keep track of all the studies in their field. The paper, titled "Attention Decay in Science," looked at all publications (articles and reviews) written in English till the end of 2010 within the database of the Thomson Reuters (TR) Web of Science. For each publication they extracted its year of publication, the subject category of the journal in which it is published and the corresponding citations to that publication. The 'decay' the researchers investigated is how quickly a piece of research is discarded measured by establishing the initial publication, the peak in its popularity and, ultimately, its disappearance from citations in subsequent publications.

"Nowadays papers are forgotten more quickly. Attention, measured by the number and lifetime of citations, is the main currency of the scientific community, and along with other forms of recognition forms the basis for promotions and the reputation of scientists," says the study. "Typically, the citation rate of a paper increases up to a few years after its publication, reaches a peak and then decreases rapidly. This decay can be described by an exponential or a power law behavior, as in ultradiffusive processes, with exponential fitting better than power law for the majority of cases (PDF). The decay is also becoming faster over the years, signaling that nowadays papers are forgotten more quickly." Matyszczyk says,"If publication has become too easy, there will be more and more of it."

112 comments

  1. "Publish or die" killed the science star by Anonymous Coward · · Score: 5, Insightful

    Predictable, but sad outcome of the popularity contest that our lives have been converted to. Now mandatory for nearly all lifestyles and incomes.

    1. Re:"Publish or die" killed the science star by peragrin · · Score: 3, Insightful

      Or maybe the fact that there are more scientists publishing today than there were 20 years ago.

      The article not the study, completely ignores one fact. The world population has tripled in 70 years. More papers being published is a result of more people existing. While not the sole reason, it is something to remember.

      --
      i thought once I was found, but it was only a dream.
    2. Re:"Publish or die" killed the science star by riverat1 · · Score: 1

      Or maybe the fact that there are more scientists publishing today than there were 20 years ago.

      Add to that the expansion of science. The more we know the more there is to study which would naturally produce more studies.

    3. Re:"Publish or die" killed the science star by riverat1 · · Score: 2

      And I'll add to that science is progressing faster than in the past. Computers help speed up analysis. More researchers help speed the process.

      Regarding cites, as science progresses the people getting the cites now are the ones who cited the earlier paper before. It would be interesting to see a cite tree to see how people who cited you are getting cited and so on. That might be the real measure of the strength of a study.

      (Sorry to reply to myself but I had to add that.)

    4. Re:"Publish or die" killed the science star by gweihir · · Score: 5, Insightful

      And even worse: Publish positive results or die. As a consequence, failed experiments get repeated all the time, because nobody else knew they failed and there is a high level of incentives to lie or at least overstate success. The root-cause, IMO, is the bean-counters that allocate funding. They do not understand that Science is exploration, that mostly it will fail and that well-documented failure is just as important as success and does not in any way reflect negatively on the scientists involved. But the bean-counters only want to see "success", and by that they make it much, much harder to obtain.

      Scientific culture needs to re-invent itself. As it is, it is mostly a problem and does not benefit society much anymore.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    5. Re:"Publish or die" killed the science star by jellomizer · · Score: 2

      19th century system to a 21st century world.
      Science today is far more complex then it was a hundred years ago. Back then it was easy to get a superstar scientist. Experiment with a few hundred dollars of equipment you can find a new principal. Publish it and you are big news.
      Most of the easy stuff had been found we get some rare finds such as the discovery of graphine, but most of today's work is with expensive equipment needing a larger teams of scientist. That publish or parish methodology is antiquated. The better approach would be open and accessable sharing of data and results in real time where more can work on you work of progress, and less trying to be Mr. Know it all scientist, who will get the Nobel prize for stumbling on the best answer.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
    6. Re:"Publish or die" killed the science star by sg_oneill · · Score: 1

      Or maybe the fact that there are more scientists publishing today than there were 20 years ago.

      The article not the study, completely ignores one fact. The world population has tripled in 70 years. More papers being published is a result of more people existing. While not the sole reason, it is something to remember.

      It does make me wonder if we're hitting some sort of plateau where its just too hard to be a generalist in any field anymore. Like, is physics too specialized now that we won't see another Einstein who mastered multiple fields of physics?

      Keep in mind, the early thinkers where often philosophers who where expert in ALL the fields of science, mathematics AND humanities, and would write wide ranging books covering all the fields and innovating in all the fields. This, I suggest is now completely impossible for one person to achieve.

      --
      Excuse the Unicode crap in my posts. That's an apostrophe, and slashdot is busted.
    7. Re:"Publish or die" killed the science star by Anonymous Coward · · Score: 0

      Predictable, but sad outcome of the popularity contest that our lives have been converted to. Now mandatory for nearly all lifestyles and incomes.

      It's predictable but it's not sad. The state of the art moves on. Original papers get superceded by further study and they are referenced instead. How many people refer back to the original seminal work that pioneered a field? And how often would it be useful to do so? Once something is taken on-board and taught at university you don't expect to have to refer to it each time. In other words papers can't explain things from scratch or they'd end up university text books. They instead assume that the state of the discipline is known by researchers in the field and build on only the latest work. That older work in turn refers to the older papers and so on. We stand upon the shoulders of giants.

    8. Re:"Publish or die" killed the science star by Anonymous Coward · · Score: 0

      We stand upon the shoulders of giants.

      Nope, you stand on the shoulders of your dissertation mentor, and he's not a giant, but a small nut in the for-profit degree mill.

    9. Re:"Publish or die" killed the science star by Anonymous Coward · · Score: 0

      Most of the easy stuff had been found w

      If you sincerely believe it was 'easy' to find the 'principals' of science three centuries ago, you lack education and perspective. It was at least as hard and expensive to procure experiment equipment (for the lack of options to buy, scientists had to be DIY experts), and jumping over the prejudice and inertia of the field, and the society at large wasn't much easier than convincing today the average Russian 'patriot' or American 'republican' that global temperatures are, indeed, rising because humans burn fossil fuels.

      This is why the rate of major discoveries hasn't changed all that much -- we get one or two each century if that, even with the huge expansion of possibilities that modern society has provided. Science is HARD, has been, will be.

    10. Re:"Publish or die" killed the science star by jellomizer · · Score: 1

      Let's take politics out of the issue, because there will always be people who will fight the science on both sides of the Spectrum. Liberals have a good history of not believing science that points out that something is actually save, because their nature is to try to change things. Conservatives have a history of no believing in science that points out that something is dangerous, because it is their nature to keep things the same.

      But to the point when we think scientist we think of a person who is master at all science diciplines. This was far more common back in the 18th and 19th century. Today's science is far more specialized, because if you try to be just a scientist, then you are spread too thin, perhaps the equlivlant of multable AS degrees.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
    11. Re:"Publish or die" killed the science star by dj245 · · Score: 2

      19th century system to a 21st century world. Science today is far more complex then it was a hundred years ago. Back then it was easy to get a superstar scientist. Experiment with a few hundred dollars of equipment you can find a new principal. Publish it and you are big news. Most of the easy stuff had been found we get some rare finds such as the discovery of graphine, but most of today's work is with expensive equipment needing a larger teams of scientist. That publish or parish methodology is antiquated. The better approach would be open and accessable sharing of data and results in real time where more can work on you work of progress, and less trying to be Mr. Know it all scientist, who will get the Nobel prize for stumbling on the best answer.

      There is plenty of easy science still yet to be done in taboo subjects. The possibilities for illegal drugs alone are huge. Can't get funding? Crowdfund it. There are plenty of people who will contribute to good science in these areas, like this one which essentially is just putting people on LSD in a fMRI machine and looking at the results. I donated some money and it looks like 1279 other people did too. They are currently at 177% of their funding goal with 34 days remaining.

      Right now, there are obviously a lot of donors and too few studies using crowd funding. That will surely change in the near future but I still think that is a far easier task to find 1280 people willing to give you $50 instead of finding 1 person willing to write a check for $66,000 (44,500 British Pounds). There are plenty of people like me who want to see research into these areas and are willing to pay for it.

      --
      Even those who arrange and design shrubberies are under considerable economic stress at this period in history.
  2. Too many studies to keep track of? by ericloewe · · Score: 1

    The solution is simple. Throw out studies that sound "too meta".

    1. Re:Too many studies to keep track of? by ShanghaiBill · · Score: 5, Insightful

      The solution is simple. Throw out studies that sound "too meta".

      Another solution would be to shoot idiotic journalists that misrepresent what studies say. The actual study does not say there are "too many" studies. What it says is that, since there are more studies, individual studies are cited less frequently, and may be read by fewer people. But nobody expects every scientist to read every paper published in their field. I probably read less than 1% of the papers published in my field, but if there is a specific topic I need to research, I often can't find enough papers that focus on what I need. So, from my point of view, there aren't enough studies.

      Also, there are too many books published. Proof: Amazon lists over a million titles, and there is no way that one person can read them all.

    2. Re:Too many studies to keep track of? by PopeRatzo · · Score: 1

      The actual study does not say there are "too many" studies. What it says is that, since there are more studies, individual studies are cited less frequently, and may be read by fewer people. But nobody expects every scientist to read every paper published in their field.

      If only we had the technology to be able to search the available research for specific items of interest, so we wouldn't have to rely upon poorly-written study titles and could narrow down the available research to the items that apply to our own narrow subject of interest.

      There may not be too many scientific studies, but I'm starting to get the feeling that there are just too goddamn many people calling themselves "journalists" without actually performing recognizable acts of journalism.

      --
      You are welcome on my lawn.
    3. Re:Too many studies to keep track of? by Anonymous Coward · · Score: 2, Insightful

      If only we had the technology to be able to search the available research for specific items of interest [...]

      Actualy, the technology is here, the problem is in the paywalls. Probably No scientific institution in the world has access to all the journals that cover the relevant fields of the institution.

      Big problem.

    4. Re:Too many studies to keep track of? by rmdingler · · Score: 1
      Information overload.

      I feel like this is indicative of the internet age, rather than isolated to academia, and the solution has evolved to become a penchant for sorting through the noise.

      Perhaps true insight is to be realized by a more talented eye toward the sifting through the chaff

      --
      Happiness in intelligent people is the rarest thing I know.

      Ernest Hemingway

    5. Re:Too many studies to keep track of? by gringer · · Score: 1

      There are lots of studies on studies, and in general they are a good idea. Here's my take on that (from SoylentNews), slightly paraphrased to hopefully demonstrate why meta-studies can be good:

      Keeping track of information is difficult, and journals generally don't like people to pepper their articles with too many citations. If the same information gets spread around, then the chance of citation drops for any particular article that contains that information. This is a problem, even with Watson-level recall, and even the very best papers will suffer from this issue.

      Let's say there's a wonderful paper published in a journal that reviews a whole bunch of things. It survives for about 6 months with citations ramping up, but then someone discovers something new and interesting about one of those things. Then, people who would previously cite the big paper and therefore let others know about it, might decide that in their particular area, the new paper is a more appropriate citation.

      About 6 months after that, the paper has hit its "peak citation rate", as the popularity of the paper is eroded in many different areas by the smaller, newer papers. Pick any one of those new papers, and you could easily say the earlier paper is better. However, pick any one of those many things, and you can probably find a better paper for the that particular area of study. Funding sources encourage this behaviour — being better than some previous paper, and fragmenting the research knowledge as much as possible.

      People could read the single big paper and get a great overview, but over time they become more likely to know about the smaller papers which give excellent detail, but are very specific. Over time, the general knowledge of readers is reduced, and they lose track of related work outside their area of expertise.

      --
      Ask me about repetitive DNA
    6. Re:Too many studies to keep track of? by Firethorn · · Score: 2

      I probably read less than 1% of the papers published in my field, but if there is a specific topic I need to research, I often can't find enough papers that focus on what I need. So, from my point of view, there aren't enough studies.

      I took a writing class where I had to write 'scientific' papers. On my chosen topic, admittedly very narrow(but that's kind of what the teacher wanted), I found myself having to kind of 'circle' my chosen target with tangential studies. I ended up writing into the review paper that here's my hypothesis, it's supported, at least in theory, by the results of these studies(and the same 3 names popped up in quite a few of them), but that the topic itself doesn't appear to be directly studied, so it would be useful to get some direct measurements.

      Oh, and as the AC mentioned, I had to discard a couple studies where I couldn't access more than the abstract, even using my university credentials.

      --
      I don't read AC A human right
    7. Re:Too many studies to keep track of? by Anonymous Coward · · Score: 0

      The solution is simple. Throw out studies that sound "too meta".

      Another solution would be to shoot idiotic journalists that misrepresent what studies say. The actual study does not say there are "too many" studies. What it says is that, since there are more studies, individual studies are cited less frequently, and may be read by fewer people. But nobody expects every scientist to read every paper published in their field. I probably read less than 1% of the papers published in my field, but if there is a specific topic I need to research, I often can't find enough papers that focus on what I need. So, from my point of view, there aren't enough studies.

      Also, there are too many books published. Proof: Amazon lists over a million titles, and there is no way that one person can read them all.

      There ARE too many studies being published, or rather too many piss poor studies. I remember when I used to be in research (plastic solar cells) and would read tens of journals all studying the same phenomenon, all trying to modify the same variables (take polymer and anneal, see boost in efficiency) and basically competing for the best sounding paper (we got the bestestest efficiency so far).

      Meanwhile none of them actually tried to come up with anything ground-breaking.

    8. Re:Too many studies to keep track of? by ranton · · Score: 1

      There ARE too many studies being published, or rather too many piss poor studies. I remember when I used to be in research (plastic solar cells) and would read tens of journals all studying the same phenomenon, all trying to modify the same variables (take polymer and anneal, see boost in efficiency) and basically competing for the best sounding paper (we got the bestestest efficiency so far).

      Meanwhile none of them actually tried to come up with anything ground-breaking.

      This just sounds like a problem which can be solved by opening up research to the search industry instead of behind paywalls. I have used IEEE and ACM recently and find their searching capabilities to be horrible. Give a company like Google full access to all research papers by both organizations and research productivity would be greatly increased.

      --
      -- All that is necessary for the triumph of evil is that good men do nothing. -- Edmund Burke
    9. Re:Too many studies to keep track of? by Pseudonym · · Score: 1

      If only we had the technology to be able to search the available research for specific items of interest, so we wouldn't have to rely upon poorly-written study titles and could narrow down the available research to the items that apply to our own narrow subject of interest.

      Ironically, I used to do research in this very area. It's a little easier in the biomedical field because most papers are tagged in ontologies like MeSH and ICD, but if you're trying to find the latest research on algorithms to solve a problem that you have, you're pretty much screwed.

      --
      sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
    10. Re:Too many studies to keep track of? by gweihir · · Score: 1

      Not at all. One of the few areas where science still works is well done meta-studies.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    11. Re:Too many studies to keep track of? by Anonymous Coward · · Score: 1

      "I probably read less than 1% of the papers published in my field, but if there is a specific topic I need to research, I often can't find enough papers that focus on what I need. So, from my point of view, there aren't enough studies."

      Exactly. As a biomedical engineer I don't read (and don't want to read) every single thing that is published in the biomedical engineering field (which can be anything from drug infusion devices, dumb implants, smart implants, prosthetics, clinical devices etc). But when I'm looking at something specific, I go after specific things, and then like you, struggle to find papers which mention want in a way that is not just a one or two sentence mention-in-passing.

    12. Re:Too many studies to keep track of? by Anonymous Coward · · Score: 0

      >there is no way that one person can read them all

      You may have missed the study showing the effectiveness, but you can take in far more input using a technology called BLIPVERTS.

    13. Re:Too many studies to keep track of? by amber_of_luxor · · Score: 1

      >the technology is here, the problem is paywalls.

      Whilst both indexing and accurate bibliographic citation in those paywalls has improved, for fields of study that are way off the mainstream, the only way to ensure that all relevant articles from a journal are correctly indexed, for that non-mainstream field of study, is to go through each article, in each volume of the journal, doing the appropriate indexing it yourself.

      > Probably No scientific institution in the world has access to all the journals that cover the relevant fields of the institution.

      Even by using Inter-library loan, accessing the articles is an issue. Especially since libraries started dumping their hard copy journals.

      --
      Wind Beneath Thy Wings
    14. Re:Too many studies to keep track of? by ericloewe · · Score: 1

      If it's "too meta", it's not well-done.

    15. Re:Too many studies to keep track of? by Anonymous Coward · · Score: 0

      The solution is simple. Throw out studies that sound "too meta".

      I may be misinterpreting this point, but it seems the need for more "meta" studies increases if there are too many publications for researchers to keep up with. i.e., it's very helpful when someone takes the time to synthesize everything that's been published in a certain area. Of course, by the time a meta-analysis is published it may be outdated, but at least it moves the baseline forward.

    16. Re:Too many studies to keep track of? by gweihir · · Score: 1

      Or alternatively, the reviewer was too stupid or to lazy to understand what it is about.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  3. 80% of respondents by Anonymous Coward · · Score: 0

    claimed they dont respond to polls.

    1. Re:80% of respondents by Earthquake+Retrofit · · Score: 1
      Both the headline and this are from the SimCity 2000 new ticker.

      P.S. More like 90% or more. You just have keep calling.

      --
      Fifty years of Yippie! 1968-2018
  4. Wow by WGFCrafty · · Score: 5, Funny

    Where's the study which examines studies about studies and found that 50% of them are fueled by irony.

    Further study is needed to confirm that number.

    1. Re:Wow by AikonMGB · · Score: 3, Interesting

      Try digging into this list, you might be able to find something relevant through there.

    2. Re:Wow by WGFCrafty · · Score: 1

      Awesome!

      I love Wikipedia's lists, but lists of lists of lists is definitely new to me!

    3. Re:Wow by Anonymous Coward · · Score: 0

      Someone should do a study about that study, what if it needs further study?

    4. Re:Wow by canadiannomad · · Score: 1

      I do believe we will need a study about studies of studies about studies. And if that study about studies of studies about studies gets repeated we will need to do a study of studies about studies of studies about studies..........

      --
      Hmm, the humour and sarcasm seem to have been be lost on you.
  5. Obvious question... by axlash · · Score: 1

    Does that include this one?

    --
    Deal with reality - the world as it is - rather than ideality - the world as you would like it to be.
    1. Re: Obvious question... by Anonymous Coward · · Score: 0

      Specifically, does the study of all studies include the study? (The answer is "yes, obviously", although I've never understood why. Something about infinity.)

  6. I know what to do. by 140Mandak262Jamuna · · Score: 1

    I am going to write up a project proposal to do a scientific study about why scientific studies are exploding at exponential rate. But calling it exponential before the doing the study would be prejudicial, so I am going to have to do a prelim study to determine whether or not it *is* exponential.

    --
    sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
  7. Not necessarily a bad thing by Anonymous Coward · · Score: 5, Informative

    As Slashdot patrons are eager to point out every time this sort of story gets published, the phenomenon described is not necessarily a bad thing.

    Many physical chemists these days are investigating ways to build nanostructures that can demonstrate interesting phenomena. For example, chemists have known for a long time that certain molecules will scatter light in the visible range but decrease its frequency by a molecule-specific constant. This process is called Raman scattering. These molecules are often dissolved in water, and it was recently shown that the adding metal nanoparticles to the solution will dramatically increase the amount of observed Raman scattering. Suddenly there's a lot of new research to do: How does the increase depend on the nanoparticles' sizes? On their shapes? On the particular metal of which they're made? On whether their surfaces are smooth or rough? What if the nanoparticles are hollow, or composed of layers of different materials? What are the theoretical explanations for the observed behaviors? And do any phenomena *other* than Raman scattering benefit from the presence of these nanoparticles?

    Many papers have been (and are still being!) published on all the clever things people have tried with these nanoparticles. Ten years from now, we'll have a pretty understanding of all the properties of surface-enhanced Raman scattering and most of these papers will be "forgotten" as researchers consolidate their knowledge into a couple of good textbooks. But that's perfectly fine---in fact, that's the whole point of scientific progress. Science is the process of observing a lot of complicated stuff and finding the most compact explanations for everything that was observed. It's nice that eighty years ago one researcher could sometimes discover a new phenomenon and provide a complete explanation for it before publishing his knowledge to the world. Today we have more researchers exploring a larger space of possible experiments, and the things they're studying are much more complicated. So they publish more papers as "scratch work" to help other researchers who are investigating the same phenomena, and eventually these papers are replaced by books. Again, that's perfectly fine.

  8. Duh by Registered+Coward+v2 · · Score: 1

    People are surprised that their are two many studies when when we keep pumping out academics who need to publish in the field lest they perish? That's like saying we're installing too many toilets because most only get used a handful of times.

    --
    I'm a consultant - I convert gibberish into cash-flow.
    1. Re:Duh by K.+S.+Kyosuke · · Score: 1

      Not publishing is obviously not the answer. But a part of the answer could involve improved information retrieval and NLP software technology...

      --
      Ezekiel 23:20
    2. Re:Duh by BFDTweets · · Score: 1

      Agree completely.

    3. Re:Duh by Anonymous Coward · · Score: 0

      People are surprised that their are two many studies when when we keep pumping out academics who need to publish in the field lest they perish? That's like saying we're installing too many toilets because most only get used a handful of times.

      I have read too many papers that are simplistic overviews of a topic yet they seem to proliferate. Is there a difference between published studies and papers versus studies and papers accepted by the various journals?

  9. Even scientists want thei 15 minutes of fame by msobkow · · Score: 1

    Everyone wants their 15 minutes of fame, even scientists. Perhaps more so than others, because their pay scales and tenure often depend on being published and cited as often as possible.

    The sad thing is that even a plethora of citations does not demonstrate the quality of a given paper. It just means it had one or a few quotable paragraphs; not that it's methodology or conclusions were necessarily stellar.

    When I worked on some research back in the university days, the prof in charge of publishing the paper insisted on citing a whole bunch of papers that neither I nor my cohort had ever read. Although the professor was only supposed to be the guide for the research, he'd read those papers so he insisted they had to be cited.

    I've always thought that was just "citation bloat" to try to make our own paper look more "researched" than it was.

    --
    I do not fail; I succeed at finding out what does not work.
  10. While publish or perish has problems... by Karmashock · · Score: 1

    ... we do need some way to separate good scientists that are working really hard and shit ones that are slacking off.

    So... do we have another method besides demanding that they be in various journals at some interval?

    Why do these studies need to be in journals at all? why not just have publications put out by every university where they internally audit every paper and if it is valid... publish it.

    Sure, you're going to have a lot of boring studies but so what? Science doesn't have to be exciting to be useful. And possibly if there was less bullshit in the studies they'd be a better resource.

    --
    I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
    1. Re:While publish or perish has problems... by godrik · · Score: 2

      Of course we are publishing more. We are pushed to publish, so obviously paper get forgotten. But it is not clear to me that it is a bad thing. What pulish or perish accomplished is that we are communicating more. So clearly we are communicating smaller ideas, smaller experiements, smaller contributions but we are also communicating earlier in the process.

      It is frequent nowadays that one idea is spinned into 3 papers, one preliminary workshop, one conference and one journal. Clearly once the journal is published, the workshop and conference version will not receive much citation. But does that mean that they were not useful? The citation they got mean that some people read these papers and that the knowdledge/insight contained in them was spread. This might not be a bad thing.

      Now, if you were using number of publication/citation as a metric of how good people have, the metric is probably ruined now. But that was a terrible metric to being with.

    2. Re:While publish or perish has problems... by prefec2 · · Score: 1

      Any metric you use which is not directly linked to the quality of the publication will cause side effects, as people optimize towards the metric. The problem with scientific quality is that it is almost impossible to come up with a solid metric. It is even impossible to come up with a ordinal ranking. Many brilliant scientists had their work rejected first time but later it was a breakthrough. So obviously the reviewers were in error when they rejected those papers. In some fields people reject papers when they are researching in the same area to get rid off the competition. However, only people who research in the same area are able to understand your work at all. So even with the imperfect metric used today the measurement taking process itself is flawed. In history this was not such a big problem, as you would not drop out of academia for a low publication index. You could just continue and some day you get so convincing results and no adversary is in the way and your paper gets through in a conference proceeding or a journal.

    3. Re:While publish or perish has problems... by prefec2 · · Score: 1

      When publish or perish would be about communication, we could just post all our preliminary results in our personal blog or in a forum and discuss it there. Instead we write papers which represent a minimal increment of knowledge. We then evaluate that minimal increment with often non standardized case studies and get through with it. For every topic I have to enter there are tons of publications, but most are rubbish. Some rubbish gets even published on ICSE, ASE or Models. The metric is ruining the scientific process. I read papers from another researcher in the past who is a great master of recycling. At first you do not recognize that, but after some papers out of the same office you see that large parts are duplicated and that not only in foundations (where this is understandable), but in the core part as well.

    4. Re:While publish or perish has problems... by Karmashock · · Score: 1

      There aren't enough academic jobs to accept all applicants. Therefore some sort of selection process has to be in effect.

      I guess I could just break a pool cue in half, throw each one a sharp cue shard, and tell them there is one opening.
      https://www.youtube.com/watch?...

      eh?

      We have to have something.

      --
      I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
    5. Re:While publish or perish has problems... by Karmashock · · Score: 1

      That's good to hear, laymen like myself only hear this expressed in negative terms.

      At some level we have to take the scientist's word for it. Though of course... they're people and people lie. Just as Dr House said "everybody lies". So people issuing grants or auditing or whatever... they have to rely on metrics and independent reviews and independent reviews of independent reviews.

      There's fishy things that go on in any organization. And the health of that organization is dependent on review least the whole thing turn into people slacking off and making the few people working hard feel like idiots because they're working harder and getting paid the same.

      So its good to know the whole publish or perish thing isn't all bad.

      --
      I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
  11. i have studied slashdot by Anonymous Coward · · Score: 0

    And find there are too many first posts.

    1. Re:i have studied slashdot by riverat1 · · Score: 2

      No there are just the right number of first posts. But there are way to many posts proclaiming "First Post!!!!!" or variants that aren't.

  12. AKA as Database Syndrome by Sir+Holo · · Score: 2

    The crop of PhDs from the last 10 or so years are either unable or unwilling to 'hit the books'. If they can't find it in an electronic database AND easily download a PDF, they will ignore the existence of the work.

    Such work often includes seminal publications, REVIEW articles of a field, and things like conference proceedings before 'everything-PDF' – all of which contain a wealth of information.

    It really bugs me when I see cited references from "whoever did something like that most recently," rather than drilling down to the original source. Unfortunately, there seems little we can do about it, aside from good scientists not referencing lazy scientists.

    1. Re:AKA as Database Syndrome by K.+S.+Kyosuke · · Score: 1

      And how do you find the books? Personally, I'd start with the latest papers that I could find online, and then I'd follow the DAG. There might be some room for topical proximity searches of unreferenced works, though.

      --
      Ezekiel 23:20
    2. Re:AKA as Database Syndrome by rsmith-mac · · Score: 2

      The crop of PhDs from the last 10 or so years are either unable or unwilling to 'hit the books'. If they can't find it in an electronic database AND easily download a PDF, they will ignore the existence of the work.

      One of the primary reasons we even have computers is to help organize and locate information. Meanwhile, because computers are so good at it and we now have so much information to process, information that is not available to a computer in 2015 is not useful information.

    3. Re:AKA as Database Syndrome by Anonymous Coward · · Score: 0

      Unless I'm writing a review, why would I give a shit about referencing original sources for everything?
      Hell, many publishers even limit the number of references you can have in a review.

      It is more important that science moves forward than wasting time to make sure everyone's ass is kissed. If you care so much, then write a fucking history of science book.

    4. Re:AKA as Database Syndrome by Anonymous Coward · · Score: 0

      >either unable or unwilling to 'hit the books'

      As a current PhD, my experience is that it is usually "unable"
      My university in the UK has stopped getting paper copies of journals and it turned out tossed a lot of the old archives because they weren't being accessed enough.
      Inter-library loans are possible, but come at a cost.

      The electronic subscription is no treat either: We are only allowed to look at one article per issue per journal. So if Journal of Synchrotron Science has 2 article I want to read that month, I have to get a friend to download it for me.

      If it is old science (say from before 1990) it is easier to refer to a book that has been written since that not only has the observation in it but gives context to it.

    5. Re:AKA as Database Syndrome by prefec2 · · Score: 2

      As in CS every invention gets reinvented every 10-20 years it is important that the old stuff cannot be found anymore. Beside that for key ideas I start with any publication I can find and start to find from there the most recent publications in the field and then try to find the original contributions again backward in time. However, this is a time consuming process and if it is only to document a minor argument, I will stop much earlier for instance with a survey on that topic.

    6. Re:AKA as Database Syndrome by Anonymous Coward · · Score: 0

      Unfortunately, there seems little we can do about it, aside from good scientists not referencing lazy scientists.

      Good news, your wish came through already! Scientists too lazy to make sure that their paper is online and easily accessible won't get cited by anyone at all. :)

    7. Re:AKA as Database Syndrome by Beck_Neard · · Score: 2

      This is extremely and wildly not true. The most basic part of doing literature review is following original sources and everyone I know does this. You have to, because reviewers pick this stuff up. Even when I couldn't find a pdf or physical copy of an original source, I'd still cite it. Also, you're fooling yourself if you think that just because something was done 30 years ago, there's no point in citing more recent sources. A lot of more recent work is nothing more than just repeating old ideas but with slight modifications that nevertheless reveal new insights. Finally, when writing a paper, there is no need to cite everything that has been done right back to ancient Greece. The audience of a scientific paper is assumed to be the scientific community which is already familiar with the body of work.

      --
      A fool and his hard drive are soon parted.
    8. Re:AKA as Database Syndrome by Sir+Holo · · Score: 1

      And how do you find the books? Personally, I'd start with the latest papers that I could find online, and then I'd follow the DAG. There might be some room for topical proximity searches of unreferenced works, though.

      It helps to get to know the guys who did the original work. They often have books they will share –books that had tiny print runs.

      And yes, good scientists trace back via References in articles. Lazy ones don't/

    9. Re:AKA as Database Syndrome by Anonymous Coward · · Score: 0

      This is true... ish. The problem is two-fold:

      - students are driven to finish far too quickly. When I started my PhD 3 years was the official length, but unofficially 4-5 years was fine, 6+ tacitly accepted under certain circumstances if that was what was needed to do a proper job. Now students at my old uni get churned through in 2 years, with a practical hard-limit at 3 years. There simply isn't time to delve deeply into a subject, so it's hardly surprising that students do a rush job.
      - Libraries have pretty much completely transitioned away from physically accessible journals. Even old journals and proceedings without electronic access are often locked away "in storage" where you need to fill out forms and wait for weeks to gain access. No acknowledgement is made of the fact that in practice you typically need to browse n articles, following citation trails from journal to journal, before choosing the most relevant articles for deeper study; which is just not possible when every other step in the search involves such delays and form-filling.

    10. Re:AKA as Database Syndrome by JanneM · · Score: 4, Insightful

      For citations central to your argument, sure, you need to track down the main papers. It's not that difficult - just look at what papers everybody else is citing. But most citations are just fulfilling the [citation needed] reqs for facts you use in your work. Any one of dozens, sometimes hundreds, of papers would easily fill in for that role.

      You find two references about the same thing. As far as citing the fact you need they're essentially equivalent. One will take three weeks and thirty dollars - and half a day of arguing to make the lab pay those thirty dollars - to get, and half the time your thirty bucks will give you a badly printed paper copy. The other you can download into your paper manager and read right now. Guess which one almost everybody will use?

      --
      Trust the Computer. The Computer is your friend.
    11. Re:AKA as Database Syndrome by gzuckier · · Score: 2

      For citations central to your argument, sure, you need to track down the main papers. It's not that difficult - just look at what papers everybody else is citing. But most citations are just fulfilling the [citation needed] reqs for facts you use in your work. Any one of dozens, sometimes hundreds, of papers would easily fill in for that role.

      You find two references about the same thing. As far as citing the fact you need they're essentially equivalent. One will take three weeks and thirty dollars - and half a day of arguing to make the lab pay those thirty dollars - to get, and half the time your thirty bucks will give you a badly printed paper copy. The other you can download into your paper manager and read right now. Guess which one almost everybody will use?

      The most frequently cited papers are not great steps forward, but method papers; somebody does some doofy study of fish farts, but it includes a great method for analyzing exhaust gases, so every gas analysis paper ever afterwards references it
      I read a study that showed that, ironically.

      --
      Star Trek transporters are just 3d printers.
    12. Re:AKA as Database Syndrome by Sir+Holo · · Score: 1

      I shouldn't feed the trolls, but just for the record:

      This is extremely and wildly not true. The most basic part of doing literature review is following original sources and everyone I know does this. You have to, because reviewers pick this stuff up.

      Actually, what I said is true. As a reviewer I DO pick this stuff up. And manuscripts with inadequate citations are rejected. Many submissions come in lacking any citation to a source (say, from 25 years ago). They will instead cite one of their buddies who parroted the primary 2-7 years ago. If it is a new interpretation or whatever, of course the more recent (primary source) who did so should be referenced.

      Also, you're fooling yourself if you think that just because something was done 30 years ago, there's no point in citing more recent sources. A lot of more recent work is nothing more than just repeating old ideas but with slight modifications that nevertheless reveal new insights.

      Whoever FIRST reported a specific observation, measurement, or hypothesis should be cited. Credit where it's due. All science is built upon previous work, so intelligent researchers cite the appropriate sources.

      Finally, when writing a paper, there is no need to cite everything that has been done right back to ancient Greece. The audience of a scientific paper is assumed to be the scientific community which is already familiar with the body of work.

      Ancient Greece? Thanks for the straw-man. RULE OF THUMB: If it is in books, there is no requirement to cite the originator.

      If, OTOH, you are extending the theory of so-and-so, you had better cite the primaries (or the most recent in the Ref. chain who modified it). Otherwise your manuscript WILL be rejected, by me or by anyone other referee.

      Last, if you're writing for submission to Science or Nature, targeting a broader audience than your specific field, it is imperative that you cite proper sources. I have a feeling that you've never published in either place, in any other prestigious journal, or probably never in any archival, peer-reviewed journal.

      If you ever get to the point in your career of being asked to referee, you will see the huge volume of 'minimal-effort' submissions that must be screened-out to maintain the quality of a given journal. Copying and failing to cite is the hallmark of bad (rejected) journal submissions.

  13. Aaron Swartz by Anonymous Coward · · Score: 0

    Pay Walled Knowledge Is Not Useful

  14. Publication cycle by Anonymous Coward · · Score: 0

    This is most likely an effect of new studies replacing older studies more rapidly when more studies are published in a field. The expected time until a study gets obsoleted by newer, superior studies thus gets shorter. Citations to older studies thus decline faster than before.

    1. Re:Publication cycle by Anonymous Coward · · Score: 0

      Bingo! Mod parent up!

      A decrease in the "half-life" of a research paper is like a decrease in the amount of time it takes to fix a bug in a piece of software. We're figuring stuff out more quickly. This is a good thing.

    2. Re:Publication cycle by amber_of_luxor · · Score: 1

      >until a study gets obsoleted by newer, superior studies thus gets shorter

      In my field, most of the research from the last two decades is pure unmitigated crap. Basic errors in research protocol are so common, that the few people who read, and review everything, remark when there are no research protocol errors.

      (It is pathetic to see a study by an author of a university textbook or research protocol, do a study that fails to adhere to what is in the book on research protocol that has their name on it.)

      --
      Wind Beneath Thy Wings
  15. Too many scientists by Anonymous Coward · · Score: 0

    being a scientist these days has become something like a job option. then work productivity is measured by number of publications.

  16. So many terrible posts by Anonymous Coward · · Score: 5, Insightful

    It saddens me to see so many sarcastic and cynical posts which fail to demonstrate that the poster has given the issue any thought whatsoever. Does the Slashdot community really consider it self-evident that scientific research is a failed enterprise? And does the Slashdot community really have no idea how scientific research works?

    Scientific papers aren't published for your benefit, you silly Slashdot reader. They're published for the benefit of other researchers. Suppose that some meta-researcher studied email patterns at your place of employment and found that this year a smaller percentage of your emails are replies to other messages [as compared to last year]; that is, a higher percentage of this year's emails are about new subjects. Then this paper gets referenced on Slashdot and someone (the author of the original article, the Slashdot submitter, or the editor) suggests that the lower reply percentage implies that intelligent discussion must be on the decline at your workplace because discussion requires people talking back-and-forth about the same topics. Then imagine that a bunch of people make short sarcastic posts that agree with that interpretation and variously lament about the decline of society as a whole or of your workplace in particular.

    Let us now make the biggest assumption of all and suppose that you have enough self-respect to be offended by this challenge to your intelligence. What would be the most mature contribution you could make to this discussion? I suppose it could be something like, "Your statistical analysis of my company's email habits is interesting, but your interpretation seems a bit misguided; it seems like a pretty big jump to go from 'percentage of emails which are replies to other emails' to 'abundance of intelligent discussion.' "

    So too it is with research papers. A statistical analysis has shown that researchers in various fields are more likely to cite recent papers than older papers, and the "half-life" of the typical paper (in the author's own words) has decreased somewhat over the last couple of decades. What conclusion should we draw from this? If scientists are less likely to cite a ten-year-old paper today than they were a decade or two ago, does that mean that there are "too many papers" and they're just swamped with recent stuff? Or does it mean that they're sufficiently well-organized that problems that used to take fifteen years to work out now only take five, and the investigations are moving on to new things?

    To paraphrase an old joke: I don't go to where you work and statistically analyze all the dicks in your mouth. So stop doing the same to scientists.

    1. Re:So many terrible posts by Anonymous Coward · · Score: 0

      Pointing out a challenge is a different from pointing out a failure. So much information is published in a typical field that a typical researcher in a small university simply can't keep up with it all. It's soon time for the superheroish "Ontologies and AI to the rescue!" proclamation.

    2. Re:So many terrible posts by Anonymous Coward · · Score: 0

      DUDE... on a different note, you have made the best post of all time by instantiating the phrase, "Hey - I don't go to where you work and statistically analyze all the dicks in YOUR mouth."

    3. Re:So many terrible posts by Anonymous Coward · · Score: 0

      I imagine a lot of the response is underpinned by a disquieting "information overload" and loss of ROI for mental effort across the economy in general, not simply in the sciences.

      It's said that Thomas Young circa 1800 was "the last man who knew everything", that is, the last man at the point of history where one could, as a genius in the sciences, still have a sufficient depth of knowledge across the sciences to be generally useful across its entirety. We're now 200 years later, and scientists study increasingly niche topics, because of the direct factors of the inevitable specialization of the sciences as they develop and the economic factors deciding where efforts are directed.

      There's the potential issue with the motivations and "moral hazard" of scientists in the present-day educational and economic environment, sure, but I think the wider issue is whether it is inevitable, even given genius scientists of spotless virtue, that the system ultimately cannot be sustained, or at least sustained in a way that anyone would want to participate. Sixty years ago, you could get a high school diploma, and the job you could get from that was sufficient for you (even as the sole parent working!) to buy a house and raise a family, and even put your children through their further education. Today, we have masses of PhD's that can't find a job, that is, demonstrate economic value within our system. What does that transition of incentives from "I'm getting paid doing something that anyone can understand as of value, particularly my family" to "I'm attempting to make a living doing something so incredibly obscure almost nobody understands it, including maybe myself" do to our society when extrapolated exponentially out to the indefinite future?

      We found it laughable when hearing the (supposed) quote of the Commissioner of the Patent Office in 1899 that "everything that can be invented has been invented". In another 20 years that may not be so amusing. Not because it will have become true as a literal question, but as a practical and economic one.

  17. Re:Not exactly by Anonymous Coward · · Score: 0

    People are surprised that their are two [sic!] many studies when when we keep pumping out academics who need to publish in the field lest they perish? That's like saying we're installing too many toilets because most only get used a handful of times.

    The problem is not "too many academics", but "too many studies" - and there is a difference: many academics with fewer studies -each involving more academics- may produce better studies (and even... better academics?!).
    Each toilet of your example can only be used -and requires the butt of- one academic at a time, but each study can be served by more than one academic at a time.
    I am a Greek; there is so much to be "discoved" by modern researchers in the studies of my ancestors that i often wish to stop "modern research" for a long period and just study... the discoveries of ancient Greeks!

  18. yea by Anonymous Coward · · Score: 0

    Maybe because most studies are put out before even verified. Seems like the studies are either stupid, and a lot of them are later disproved, and many are paid to get the desired results. (See climate change)

    1. Re:yea by gzuckier · · Score: 1

      Maybe because most studies are put out before even verified. Seems like the studies are either stupid, and a lot of them are later disproved, and many are paid to get the desired results. (See climate change)

      How do you verify the study before it is published? Ask the same guy to do it again, making all the same mistakes, and see it if comes out the same? Or make peer review include repeating the study?

      --
      Star Trek transporters are just 3d printers.
  19. Ran this story too quick by Anonymous Coward · · Score: 0

    I'm just gonna go ahead and assume this headline wasn't supposed to come out for 17 more days.

  20. A good thing? by excelsior_gr · · Score: 2

    Although this could be due to the "publish or perish" mentality, that often forces researchers to break down their work in several publications of lesser impact than make a single publication of larger impact, the fact that the "lifetime" of publications is getting shorter may also mean that the research is speeding up. Knowledge moves faster from papers, then to books, and then to being "common", and before you know it you don't really have to cite someone every freaking time anymore because everyone knows what you're talking about (I'm talking about things that are considered "common knowledge" here; you surely don't cite Newton every time you mention that white light can be broken up using a prism). More commonly, somebody will sum the "state of the art" into a book or in a good introductory chapter of a doctoral dissertation and people will cite that, instead of all the papers. Also, books keep getting cited for decades after their publication, so maybe a follow-up study could check whether there is a similar trend in the citation of books?

    While the plurality of journals has made publishing quite easy nowadays, I don't think this is the reason for the observation that papers get forgotten faster. A bad paper will not even get noticed and will probably get cited only by its own authors in subsequent publications. Since we are talking about papers that do get cited here, this means that they have managed to attract some attention, and can therefore not be too crappy.

    1. Re:A good thing? by Anonymous Coward · · Score: 0

      It seems another study of studies is needed!

  21. Obligatory by Anonymous Coward · · Score: 0

    Has science gone too far?

    1. Re:Obligatory by gzuckier · · Score: 1

      Has science gone too far?

      Stupid GPS.

      --
      Star Trek transporters are just 3d printers.
  22. Article progression by gringer · · Score: 1

    For a change, this is something that appeared on SoylentNews before Slashdot. It has been interesting tracking this article through the social media sites that I frequent:

    Reddit — Submitted Wed, Mar 11; 211 comments at the time of writing this comment

    SoylentNews — Submitted Sunday, Mar 15; 16 comments at the time of writing this comment

    Slashdot — Posted Monday, Mar 16; 30 comments at the time of writing this comment

    --
    Ask me about repetitive DNA
  23. Its a given by Anonymous Coward · · Score: 0

    When you see ads on TV telling you about a "study" proving Ab-Something works x% better than Ab-Otherthing - its sort of given how "studies" work.

  24. We need more meta science by duck_rifted · · Score: 1

    With the "publish or perish" culture thriving, more studies that help to determine what needs to be studied would be great. I fear that this doesn't happen enough because people feel like it gives away ideas that others might complete. But if the researcher who can show the need for a study can actually follow up, then they can get a head start on that work. If not, then why not let someone else do it? In academia, this is supposed to be about the good of the species and not some misguided desire to become the next science pop star.

    And that's point two. If you can't do, teach. If you can't do and you can't teach, become a celebrity. Help the public understand science in layperson's terms, without the need to teach the full rigorous skill set. We need that. The step before celebrity is journalist. So, that can come full circle back to doing and teaching. If you can crystallize concepts well, and you can communicate them in a tone others respond to, then we need you helping the public. What good does it do to answer humanity's questions if nobody can understand the answer?

    Unfortunately, none of this is going to change until boomers finally pass the torch once and for all. And ladies and gents, I hate to be the bearer of bad news but if we don't prevent this kind of problem in the future then we're going to be far worse than boomers. Life expectancy tends to increase as the species ages.

  25. These authors are a shoe-in for the by jpellino · · Score: 1

    next IgNobel awards.

    --
    "Win treats sysadmins better than users. Mac treats users better than sysadmins. Linux treats everyone like sysadmins."
  26. The problem with less is more by Anonymous Coward · · Score: 0

    is that others want to decide what you get less of.

  27. It's like anonymous comments by Anonymous Coward · · Score: 0

    There is too many of them.

    1. Re:It's like anonymous comments by gzuckier · · Score: 1

      anonyments.

      --
      Star Trek transporters are just 3d printers.
  28. data mining? by roc97007 · · Score: 1

    It seems like this (too many scientific papers) is a problem that could be solved by data mining. I know that concept is considered evil these days, but it does have it's practical, non-evil uses.

    It was inevitable, really, that at some point there would be more science going on than could conveniently be published.

    --
    Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
  29. Or even more... by gwolf · · Score: 1

    Having so many publishing venues available right now, with (thankfully) every day more of them available under open access licensing schemes, we can get to much more research in our field.

    That, however, means that when I start reading on a subject related to my area of study, there are too many documents fighting for my attention. And I will undoubtedly miss many among them, just because of sheer probability.

    Of course, the same will happen to my published works: They will no longer be _so_ unique, they will also depend on my luck for you to read them.

    1. Re:Or even more... by Anonymous Coward · · Score: 0

      Having so many publishing venues available right now, with (thankfully) every day more of them available under open access licensing schemes, we can get to much more research in our field.

      Depends on the field. I like open access as much as the next guy, but in my field 99% of the open access licensed stuff is crap.

    2. Re:Or even more... by C0R1D4N · · Score: 4, Funny

      Oblig XKCD

  30. Obvious from just looking at your own network by CptJeanLuc · · Score: 1

    Of course there are too many (useless or only marginally useful) scientific studies. Just look at the people who are working as scienctists that you know personally or that you otherwise vaguely know, how smart they are (everyone cannot be an Einstein) and your estimate of the quality of knowledge artifacts that they would produce, and what is the research they do, not just limited to your own field of schooling or expertise. And what do your friends and connection who are researchers have to say about the publish or perish regime, and whether they are happy about how they are able to go about their research. There is no need for some scientific study to tell us what most people who are not working as scientists ourselves - and thus have no need for convincing ourselves that the world of science is so fantastic and perfectly objective - can plainly see bright as day if we just open our eyes.

  31. Faster rate of scientific progress by Mostly+a+lurker · · Score: 1

    As progress in most scientific fields occurs at an ever faster rate, it is logical that previous research is more quickly supplanted by more relevant recent papers. Are we supposed to be surprised by this phenomenon?

  32. Meetings at work by Akaihiryuu · · Score: 1

    This reminds me of my last job. There was a BIG concern (and it was justified) that managers had no time to actually manage their departments or their people because they were doing nothing but sitting in meetings 8-9 hours/day. I didn't see my manager for over a month once because he was SOLIDLY in meetings from the time he got there until the time he left. Upper management's response to this: "Let's have a meeting to discuss that".

    1. Re:Meetings at work by Anonymous Coward · · Score: 0

      This reminds me of my last job. There was a BIG concern (and it was justified) that managers had no time to actually manage their departments or their people because they were doing nothing but sitting in meetings 8-9 hours/day. I didn't see my manager for over a month once because he was SOLIDLY in meetings from the time he got there until the time he left. Upper management's response to this: "Let's have a meeting to discuss that".

      Bad analogy. No one is forced to read all the studies, but your managers were forced to attend all the meetings.

  33. thanks for this by Anonymous Coward · · Score: 0

    you just made my whole evening. I love to wander among wiki articles, and now you have given me a map! I'm happier than a tie dye teddy bear jumping and laughing sparkling rainbows with confetti

    1. Re:thanks for this by Anonymous Coward · · Score: 0

      ... a tie dye teddy bear jumping and laughing sparkling rainbows with confetti

      That imagery has brought me closer to a psychotic break.

  34. filtration is key by swell · · Score: 1

    Eliminate biased studies and the rest can see the light of day.

    'Scientific Studies' today are a creation of a Marketing department in many cases. There is a product to be sold and it needs support and affirmative publicity. A company may do several studies in hope that one or two will be useful in their advertising. The others tend to disappear.

    The US government (and other governments and non-profits) conducted studies for many years with the intention of proving that smoking and second hand smoke were dangerous. When the statistical validity of their second hand smoke studies was not sufficient, they simply redefined the term 'statistical significance'. They are the government, after all.

    Any study that begins with the premise of proving some theory is flawed. They should clearly state the theory and try every possible means to disprove it. If they can't disprove it, they present their findings to their peers so that they may attempt to disprove it. Failure to disprove the theory over time can lead to general acceptance of it. The scientific method at work. Most studies do it backward.

    Big bold letters at the top of every study should reveal who paid for it and the financial interest of every contributor. It's a start, but still subject to corruption.

    --
    ...omphaloskepsis often...
    1. Re:filtration is key by amber_of_luxor · · Score: 1

      >Eliminate biased studies and the rest can see the light of day.

      If it wasn't for some researchers fudging data, breaking every rule in the book, about research design, there never would have been any pilots from Tuskegee, during WW2.

      In this case, the bias of the researchers, and the funders, was a goodness.

      --
      Wind Beneath Thy Wings
    2. Re:filtration is key by david_thornley · · Score: 2

      Seems to me that Einstein was rather big on proving his ideas correct, and therefore was biased.

      --
      "When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
  35. XKCD? by Anonymous Coward · · Score: 0

    I just know there is a relevant XKCD to back this up.
    Someone needs to do a study on XKCD comics and their ability to predict future slashdot headlines.

  36. Obligatory xkcd by Anonymous Coward · · Score: 0

    Competing standards is almost spot-on concerning a study about too many studies. But there is also Meta-Analysis.

  37. yo dawg... by tresstatus · · Score: 1

    ...we heard you like scientific studies. so we did a scientific study of scientific studies so you can science while you science.

    --
    stephen
  38. Salvageable through open science? by fygment · · Score: 1

    If everyone must make their data available, then a paper will be judged on the strength of its:

    a) academic contribution; and
    b) quality/usefulness of the data.

    So you might not be the author of the greatest paper, yet your impact might be the quality of the experimentation and resulting data.

    Right now, papers appear and the data is just hearsay. In that environment, anybody can publish anything ... and today, there's is a strong incentive to do just that.

    --
    "Consensus" in science is _always_ a political construct.
  39. Department of Redundancy Department by Anonymous Coward · · Score: 0

    Department of Redundancy Department

  40. Chris who? by jimbob6 · · Score: 1

    What they need to do is rank the studies based on the number of vowels in the authors name.

  41. dimly remembered cartoon strip from grad school by gzuckier · · Score: 1

    "My sister says all the good thesis topics have been taken. She's doing hers on how a dust speck bounces when it hits the table"
    "My brother is doing his on the letter G".

    --
    Star Trek transporters are just 3d printers.
  42. Is it also possible... by BadPirate · · Score: 1

    That rather than shorter attention spans, or more useless papers, papers are not staying relevant as long simply because the rate of technological progress continues to increase?

    For example, a paper on VHS would have been cited during a longer period than a paper on DVD, which would have been cited more than a paper on Blu-Ray... The rate of innovation has increased, and thus the duration of the usefulness of the discoveries as compared to updated versions of the same has gone down.

    --
    - Holy crap, I've got MOD points! Who thought that was a good idea.
  43. The amazing power of Survey papers. by Anonymous Coward · · Score: 0

    The problem is NOT too many studies. The problem is too few Survey papers.

    When entering a new field of knowledge (or getting back up to speed in an old one), most folks will start with Wikipedia (to get a quick handle on the domain's terminology), then get the best text books in the field (pedagogical approach), and finally move on to tools like Google Scholar to climb the citation tree.

    That's skipping a vital step: Reading the relevant "survey" papers.

    Writing a survey paper must be one of the most thankless tasks in research, looking at everything published within a limited scope, comparing and contrasting the papers, and summing them up with a view toward the future. The best survey papers I've read tend to be written by three types of folks: 1) Those at the start of their careers, paving their own way into a field. 2) Those at the end of their careers, desiring to sum things up. 3) Extraordinary (and possibly deranged) individuals who just like writing survey papers.

    Survey papers don't discuss only the knowledge. They can, and often do, also discuss writing, presentation, and organization. Most importantly, they also discuss technique, both theoretical and experimental.

    One of my favorite papers was a pure rant about the misuse of statistics in medical trials, covering not only gross errors, popular misconceptions, and actual fraud, but also revealing lower-level biases and bad historical traditions. Several highly-cited papers were torn to shreds, and a few hidden gems were revealed.

    Survey papers don't just boil down the content of a bunch of papers into a more conveniently digestible form. They also show how papers on identical topics can have vastly different impact in the field due to the clarity of exposition. Survey papers can also reveal which papers created new terminology that spread throughout the field, or techniques that changed how the field was explored. They can reveal how a field evolved and matured.

    The primary negative aspect of survey papers is simple: They are always historical, and never up-to-the-moment. But that's also their greatest virtue: After digesting the relevant survey papers, the number of individual papers left to read is reduced by orders of magnitude.

    So, as I said at the start: The problem is NOT too many studies. The problem is too few Survey papers!