Slashdot Mirror
Scientific Study Finds There Are Too Many Scientific Studies
HughPickens.com writes: Chris Matyszczyk reports at Cnet that a new scientific study concludes there are too many scientific studies — scientists simply can't keep track of all the studies in their field. The paper, titled "Attention Decay in Science," looked at all publications (articles and reviews) written in English till the end of 2010 within the database of the Thomson Reuters (TR) Web of Science. For each publication they extracted its year of publication, the subject category of the journal in which it is published and the corresponding citations to that publication. The 'decay' the researchers investigated is how quickly a piece of research is discarded measured by establishing the initial publication, the peak in its popularity and, ultimately, its disappearance from citations in subsequent publications.
"Nowadays papers are forgotten more quickly. Attention, measured by the number and lifetime of citations, is the main currency of the scientific community, and along with other forms of recognition forms the basis for promotions and the reputation of scientists," says the study. "Typically, the citation rate of a paper increases up to a few years after its publication, reaches a peak and then decreases rapidly. This decay can be described by an exponential or a power law behavior, as in ultradiffusive processes, with exponential fitting better than power law for the majority of cases (PDF). The decay is also becoming faster over the years, signaling that nowadays papers are forgotten more quickly." Matyszczyk says,"If publication has become too easy, there will be more and more of it."
"Nowadays papers are forgotten more quickly. Attention, measured by the number and lifetime of citations, is the main currency of the scientific community, and along with other forms of recognition forms the basis for promotions and the reputation of scientists," says the study. "Typically, the citation rate of a paper increases up to a few years after its publication, reaches a peak and then decreases rapidly. This decay can be described by an exponential or a power law behavior, as in ultradiffusive processes, with exponential fitting better than power law for the majority of cases (PDF). The decay is also becoming faster over the years, signaling that nowadays papers are forgotten more quickly." Matyszczyk says,"If publication has become too easy, there will be more and more of it."
Predictable, but sad outcome of the popularity contest that our lives have been converted to. Now mandatory for nearly all lifestyles and incomes.
Where's the study which examines studies about studies and found that 50% of them are fueled by irony.
Further study is needed to confirm that number.
As Slashdot patrons are eager to point out every time this sort of story gets published, the phenomenon described is not necessarily a bad thing.
Many physical chemists these days are investigating ways to build nanostructures that can demonstrate interesting phenomena. For example, chemists have known for a long time that certain molecules will scatter light in the visible range but decrease its frequency by a molecule-specific constant. This process is called Raman scattering. These molecules are often dissolved in water, and it was recently shown that the adding metal nanoparticles to the solution will dramatically increase the amount of observed Raman scattering. Suddenly there's a lot of new research to do: How does the increase depend on the nanoparticles' sizes? On their shapes? On the particular metal of which they're made? On whether their surfaces are smooth or rough? What if the nanoparticles are hollow, or composed of layers of different materials? What are the theoretical explanations for the observed behaviors? And do any phenomena *other* than Raman scattering benefit from the presence of these nanoparticles?
Many papers have been (and are still being!) published on all the clever things people have tried with these nanoparticles. Ten years from now, we'll have a pretty understanding of all the properties of surface-enhanced Raman scattering and most of these papers will be "forgotten" as researchers consolidate their knowledge into a couple of good textbooks. But that's perfectly fine---in fact, that's the whole point of scientific progress. Science is the process of observing a lot of complicated stuff and finding the most compact explanations for everything that was observed. It's nice that eighty years ago one researcher could sometimes discover a new phenomenon and provide a complete explanation for it before publishing his knowledge to the world. Today we have more researchers exploring a larger space of possible experiments, and the things they're studying are much more complicated. So they publish more papers as "scratch work" to help other researchers who are investigating the same phenomena, and eventually these papers are replaced by books. Again, that's perfectly fine.
The solution is simple. Throw out studies that sound "too meta".
Another solution would be to shoot idiotic journalists that misrepresent what studies say. The actual study does not say there are "too many" studies. What it says is that, since there are more studies, individual studies are cited less frequently, and may be read by fewer people. But nobody expects every scientist to read every paper published in their field. I probably read less than 1% of the papers published in my field, but if there is a specific topic I need to research, I often can't find enough papers that focus on what I need. So, from my point of view, there aren't enough studies.
Also, there are too many books published. Proof: Amazon lists over a million titles, and there is no way that one person can read them all.
Of course we are publishing more. We are pushed to publish, so obviously paper get forgotten. But it is not clear to me that it is a bad thing. What pulish or perish accomplished is that we are communicating more. So clearly we are communicating smaller ideas, smaller experiements, smaller contributions but we are also communicating earlier in the process.
It is frequent nowadays that one idea is spinned into 3 papers, one preliminary workshop, one conference and one journal. Clearly once the journal is published, the workshop and conference version will not receive much citation. But does that mean that they were not useful? The citation they got mean that some people read these papers and that the knowdledge/insight contained in them was spread. This might not be a bad thing.
Now, if you were using number of publication/citation as a metric of how good people have, the metric is probably ruined now. But that was a terrible metric to being with.
The crop of PhDs from the last 10 or so years are either unable or unwilling to 'hit the books'. If they can't find it in an electronic database AND easily download a PDF, they will ignore the existence of the work.
Such work often includes seminal publications, REVIEW articles of a field, and things like conference proceedings before 'everything-PDF' – all of which contain a wealth of information.
It really bugs me when I see cited references from "whoever did something like that most recently," rather than drilling down to the original source. Unfortunately, there seems little we can do about it, aside from good scientists not referencing lazy scientists.
It saddens me to see so many sarcastic and cynical posts which fail to demonstrate that the poster has given the issue any thought whatsoever. Does the Slashdot community really consider it self-evident that scientific research is a failed enterprise? And does the Slashdot community really have no idea how scientific research works?
Scientific papers aren't published for your benefit, you silly Slashdot reader. They're published for the benefit of other researchers. Suppose that some meta-researcher studied email patterns at your place of employment and found that this year a smaller percentage of your emails are replies to other messages [as compared to last year]; that is, a higher percentage of this year's emails are about new subjects. Then this paper gets referenced on Slashdot and someone (the author of the original article, the Slashdot submitter, or the editor) suggests that the lower reply percentage implies that intelligent discussion must be on the decline at your workplace because discussion requires people talking back-and-forth about the same topics. Then imagine that a bunch of people make short sarcastic posts that agree with that interpretation and variously lament about the decline of society as a whole or of your workplace in particular.
Let us now make the biggest assumption of all and suppose that you have enough self-respect to be offended by this challenge to your intelligence. What would be the most mature contribution you could make to this discussion? I suppose it could be something like, "Your statistical analysis of my company's email habits is interesting, but your interpretation seems a bit misguided; it seems like a pretty big jump to go from 'percentage of emails which are replies to other emails' to 'abundance of intelligent discussion.' "
So too it is with research papers. A statistical analysis has shown that researchers in various fields are more likely to cite recent papers than older papers, and the "half-life" of the typical paper (in the author's own words) has decreased somewhat over the last couple of decades. What conclusion should we draw from this? If scientists are less likely to cite a ten-year-old paper today than they were a decade or two ago, does that mean that there are "too many papers" and they're just swamped with recent stuff? Or does it mean that they're sufficiently well-organized that problems that used to take fifteen years to work out now only take five, and the investigations are moving on to new things?
To paraphrase an old joke: I don't go to where you work and statistically analyze all the dicks in your mouth. So stop doing the same to scientists.
If only we had the technology to be able to search the available research for specific items of interest [...]
Actualy, the technology is here, the problem is in the paywalls. Probably No scientific institution in the world has access to all the journals that cover the relevant fields of the institution.
Big problem.
Although this could be due to the "publish or perish" mentality, that often forces researchers to break down their work in several publications of lesser impact than make a single publication of larger impact, the fact that the "lifetime" of publications is getting shorter may also mean that the research is speeding up. Knowledge moves faster from papers, then to books, and then to being "common", and before you know it you don't really have to cite someone every freaking time anymore because everyone knows what you're talking about (I'm talking about things that are considered "common knowledge" here; you surely don't cite Newton every time you mention that white light can be broken up using a prism). More commonly, somebody will sum the "state of the art" into a book or in a good introductory chapter of a doctoral dissertation and people will cite that, instead of all the papers. Also, books keep getting cited for decades after their publication, so maybe a follow-up study could check whether there is a similar trend in the citation of books?
While the plurality of journals has made publishing quite easy nowadays, I don't think this is the reason for the observation that papers get forgotten faster. A bad paper will not even get noticed and will probably get cited only by its own authors in subsequent publications. Since we are talking about papers that do get cited here, this means that they have managed to attract some attention, and can therefore not be too crappy.
Oblig XKCD
I probably read less than 1% of the papers published in my field, but if there is a specific topic I need to research, I often can't find enough papers that focus on what I need. So, from my point of view, there aren't enough studies.
I took a writing class where I had to write 'scientific' papers. On my chosen topic, admittedly very narrow(but that's kind of what the teacher wanted), I found myself having to kind of 'circle' my chosen target with tangential studies. I ended up writing into the review paper that here's my hypothesis, it's supported, at least in theory, by the results of these studies(and the same 3 names popped up in quite a few of them), but that the topic itself doesn't appear to be directly studied, so it would be useful to get some direct measurements.
Oh, and as the AC mentioned, I had to discard a couple studies where I couldn't access more than the abstract, even using my university credentials.
I don't read AC A human right
No there are just the right number of first posts. But there are way to many posts proclaiming "First Post!!!!!" or variants that aren't.
Seems to me that Einstein was rather big on proving his ideas correct, and therefore was biased.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes