Slashdot Mirror
Researchers Feel Pressure To Cite Superfluous Papers
ananyo writes "One in five academics in a variety of social science and business fields say they have been asked to pad their papers with superfluous references in order to get published. The figures, from a survey published in the journal Science (abstract), also suggest that journal editors strategically target junior faculty, who in turn were more willing to acquiesce. The controversial practice is not new: those studying publication ethics have for many years noted that some editors encourage extra references in order to boost a journal's impact factor (a measure of the average number of citations an article in the journal receives over two years). But the survey is the first to try to quantify what it calls 'coercive citation,' and shows that this is 'uncomfortably common.' Perhaps the most striking finding of the survey was that although 86% of the respondents said that coercion was inappropriate, and 81% thought it damaged a journal's prestige, 57% said they would add superfluous citations to a paper before submitting it to a journal known to coerce. However, figures from Thomson Reuters suggest that social-science journals tend to have more self-citations than basic-science journals."
The surest way to get something on Wikipedia, is get something published then cite it. Accuracy notwithstanding.
I think we know the number one culprit here, ahem [citation needed] ahem.
This is what happens when you have metrics. You create a metric like "impact factor", and before long people will figure out ways to maximize "impact factor" that have nothing to do what the metric was originally supposed to measure. Hyperfocusing on metrics like that ends up undermining the things you really value in favor of increasing your scores.
This happens all over the place. Games in every game find ways to increase their score in ways that the game designers wouldn't really consider valid. Universities do things simply to make their "US News" ratings go up, not because they will make themselves better. Students figure out ways to raise their grades that have nothing to do with mastering the material of the course. Heck, the entire US (and world?) economy suffers from this; the most reliably rich people are the ones who manipulate money transactions, and do absolutely nothing with the underlying reality that money is supposed to be an abstract representation of.
People strive to improve the things that they are rewarded for and that they are evaluated on. When you focus too much on the wrong thing, people will do the wrong things in response.
I have bad news for you ... dig deep enough, and you find out all the other 'sciences' suffer from exactly the same problems.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
Here is one that might be relevant:
Reason 33. There is too much academic publishing.
http://100rsns.blogspot.com/2010/11/33-there-is-too-much-academic.html
When one's job is on the line (i.e. tenure-track faculty), people will do almost anything.
New Economic Perspectives
dig deep enough, and you find out all the other 'sciences' suffer from exactly the same problems.
As a recovering Mathematician I take issue with that.
Hint: There is one science founded (mostly) on logic that doesn't let reality get in the way of quality navel gazing.
There's a tricky balance between saying we build on this work and so necessarily reference it, and it is related to this other work (or may incorporate results from it, even if I'm not directly involved in that research) which is trying to give credit where it is due, not too much.
Science struggles how to cite widely disbursed facts. What is the speed of light? Right. You *can* cite the people who precisely measure it, but most research that relies on that data doesn't really need to cite it, because they aren't building on it. If I'm looking at spectra from a gas or a star the speed of light is really really really important to my work, but how the actual number was arrived at isn't that important. If you're a young researcher you want your work to look like it is related to a lot of things (think resume padding) so you cite more than you need to, and if you're new to the field you want your work cited as a measure how much impact it has - but judging what counts as impact is not always clear and you're pretty easy to persuade that it is better to over cite than under cite and risk being called out for plagiarism.
Another example. I'm working on a current project. One of the relevant facts to the work is the history of the Bismarck battleship (the nazi one). This is because the documentation about the history is relevant to how to quantify the statistics of the ship. But what is a valid reference for that? If anything? How about the mere existence of the ship? What factual information am I digging for that isn't sufficiently well known to be on wikipedia? Do I cite comparable information about the dozen or so other ships and aircraft she actually fought with, or do I just sort of take for granted that the ship had 38 cm guns (which directly maps to the problem I'm talking about which is balancing the relative combat power of the ship). If it was a history paper it's sort of obvious that the historical work is to be cited - discovering (or rediscovering) that information would be a worthwhile historical paper, but what about something that is only tangentially related, which is trying to define those statistics in a game?
It hasn't been uncommon for scientists to base research on 3 or 4 papers (possibly one or two of which was from their own group), and then when they're getting ready to publish to look for papers in the target journal that are related that can be cited as well (this is like self censorship, or self coercion, rather than be asked to do it, you do it on your own first0. It's not really a good practice, but I'm not sure it's as bad as the article tries to make it out to be. You really are legitimately looking for work that might be related to what you did, especially if you didn't do your literature search very well (which is harder than it sounds sometimes), and you are, as you say, citing authors you're extrapolating from (or at the very least doing related work to).
Shocked [3][12][21], I tell you! [4][7]! Studies [14][17][31] have shown [11][15] that this [26] never [21][22] happens [25] with reputable [5][14][24] papers [19]! How could this [32] have happened? [12][16]
Check your premises.
I put together an economics paper and sent it around to a few PhDs I know. Two of them came back with the exact response that this article indicates; "It needs more references if you want to get published." I asked if the math, logic, or conclusions were off, both responded they were not, but that was not the point. They made it clear that to get published it had to have more references to existing work, regardless of the content.
I can come up with arguments why such a policy has some merit -- keeping wacky stuff like modern monetary theory's hypothesis that there is no such thing as too much debt from distracting researchers, for example -- but good, bad, or indifferent; the fact that there is a barrier to papers which do not pay homage to existing academics is very real.
Stop-Prism.org: Opt Out of Surveillance
I did RTFA. The authors of the paper surveyed 54,000 academics, and about 1,300 responded to say, "Yes we felt pressured." That's 2.5%. Only 1/3 of those named a single journal that pressured them. Another 2.5% said, "We've heard that others have been pressured, but never us." 7.5% said, "We've never heard of it." And 87.5% didn't respond. The survey shows extreme self-selection as 7 of 8 academics did not respond. So before someone gets excited that 20% of academics are pressured, note that under 13% of academics responded.
Yeah, but if you go down that path, you wind up with mathematics being nothing but a tool, more a part of the process of science than science itself. Mathematics alone tells us nothing about the universe, other than that mathematics can be derived in it (and then you get back to whether or not you've really derived anything since the foundation is ultimately belief).
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
Not all economists habitually claim that wealth inequality and deregulation are good things. Also there is a strong movement to get economics away from simply math-based models and starting to use more empirical data. For example - if you take any decent class on international trade, you should find countless examples that disprove the basic models that have been around for ages. The Heckscher-Ohlin and Ricardian models (two of the most basic models upon which trade is based) have been tested and found wanting, especially H-O. Ricardian has been modified and updated somewhat but it still contains flaws. Newer models such as the New Trade Theory (inventive naming schemes abound! there's also a New New Trade Theory) and the Gravity Model have come into vogue, especially with the advent of FTAs in the past 20 years and the WTO.
As for monetary policy, it has certainly been the unfortunate case that in America and the UK (and to a certain degree Canada - although with Harper and his cronies with their goddamned majority it's sure to increase) the Austrian school has been prominent. However, people are starting to see that regulation needs to be enforced. If we look at the differences in banking between Canada and the US for example, we can see that Canada has fewer regulations, but enforcement is stricter and penalties more severe. I'm not, by the way, advocating that the Canadian banking system is without flaws, just that it tends to be less volatile, and their practices are less risky due to the regulations being observed. Now obviously the US is not going to adopt the Canadian banking system wholesale, but there are lessons to be learned.
There is a growing understanding that economists need to look at bigger pictures, and so political economy and comparative governance are big fields of study now.
I know that being on a board dominated by engineers and "hard" scientists this post will probably get downmodded, but it's important to recognize that there is a great diversity in the opinions of professional economists, and many of the younger generation will not buy into the simple ideas of "deregulate, lower taxes, and eliminate minimum wage".
Researchers Misunderstand Confidence Intervals and Standard Error Bars.
Belia, Sarah;Fidler, Fiona;Williams, Jennifer;Cumming, Geoff
Psychological Methods, Vol 10(4), Dec 2005, 389-396.
Little is known about researchers' understanding of confidence intervals (CIs) and standard error (SE) bars. Authors of journal articles in psychology, behavioral neuroscience, and medicine were invited to visit a Web site where they adjusted a figure until they judged 2 means, with error bars, to be just statistically significantly different (p .05). Results from 473 respondents suggest that many leading researchers have severe misconceptions about how error bars relate to statistical significance, do not adequately distinguish CIs and SE bars, and do not appreciate the importance of whether the 2 means are independent or come from a repeated measures design. Better guidelines for researchers and less ambiguous graphical conventions are needed before the advantages of CIs for research communication can be realized.
(http://psycnet.apa.org/journals/met/10/4/389/)
Who modded this flamebait? It's right on point.
[citation needed]
rewriting history since 2109
At Griffith University, Australia, we took a "Philosophy of Science" subject as part of the degree - mostly based on the philosophy of Karl Popper.
Basically, Science is:
Observable
Repeatable
Falsifiable and
Communicated.
I've always found this definition useful.
Thus proving that it's not science, but simply people pushing their personal opinions draped in the prestige of science.
Oh I forgot there are no differing opinions in science. I guess Stephen Jay Gould and Richard Dawkins agree completely on how evolution works.
The overwhelming majority of papers are read once and then never read again. I know that much.
This is especially true of people drawing conclusions rather then reporting data.
It's actually odd that they focus on asking graduate students to draw novel conclusions when it probably more useful to ask them to discover novel data. No interpretation. Just come up with an experiment or find something that has never been measured before. Then report in detail everything so it can be repeated or remeasured.
Most reports would probably be more useful.
There's nothing wrong with spending your life collecting dots for other people to connect. It's an absolutely vital portion of science. Too often sciences get too theoretical... too full of conjecture. Science is supposed to be about empiricism. Which requires 99 percent data and 1 percent conclusion.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
In my (former researcher who left to industry) opinion/experience its not the editors who put the pressure, but the possibility that you ignored a work of somebody who is important enough to referee for Nature or Science. There are some components of these phenomena:
a) Maybe the work really is important, and you did not know it because it's too long ago. There is usually nothing wrong with a referee saying "hey that is similar to what [xyz]" did, even if they are on the list of authors on the reference in question.
b) some referees dont react positively to not getting cited and will shoot down any paper not referring to *their* theory for other reasons (i believe that happened to me once)
c) In the abstract (which is the part really read by the editors before the refereeing process) you compare your paper to the previous publications. Authors are under the impression that comparing your work to previous important papers makes a better impression. How far this is true i cant judge. I found the editor stage *before* the refereeing in Nature and Science the most intransparent thing I have experienced as an author. Unlike the refereeing process there is no way to appeal, there is not information on what the editors disliked so much to refuse directly. (There is the saying that once you had Nature/Science papers it gets more likely to pass this stage, and i have seen at least one example of a paper being passed to Nature which for sure would have been rejected by the editors had it come from a less important group in the field)
Because an economy is a lot like an ecosystem. You eliminate the regulators (like, say the predators), in an ecosystem and you get populations booms followed by population collapse when the resources are depleted. Sometimes to extinction. Regulation is good in that it prevents this boom and bust scenario. Guess what the economic system advocated by conventional economists looks like? Boom and bust the whole length. The periods of stability in between have usually come as governments start enforcing regulations after a pariticularly boom and bust event has left a good part of the population in hardship. Then over time people forget. Little by little regulations are dropped, and of course so arrives a period of apparent prosperity (the boom phase) during which we are lured into dropping more and more regulations because obviously regulations are hampering this prosperity. Of course, it can't last. Without the moderation provided by regulation we enter this boom phase, that once all the suckers at the bottom of this pyramidal scheme have contributed can only end in a bust. Then we start talking of regulation again (which is needed).
Until we get rid of the models that advocate the "deregulate" point of view, we have no chance of getting out of this history of boom and bust. Current economic models encourage waste (we've dilapidated in a couple of generations all the energy wealth stored up over millions of years). That's hardly "economic" if it's wasteful. Boom and bust cycles, and chronic waste are a sure proof that accepted economic models are broken.
I like my dinosaurs feathery, and my pterosaurs hairy (or is it pycnofibery?)