Slashdot Mirror
Researchers Feel Pressure To Cite Superfluous Papers
ananyo writes "One in five academics in a variety of social science and business fields say they have been asked to pad their papers with superfluous references in order to get published. The figures, from a survey published in the journal Science (abstract), also suggest that journal editors strategically target junior faculty, who in turn were more willing to acquiesce. The controversial practice is not new: those studying publication ethics have for many years noted that some editors encourage extra references in order to boost a journal's impact factor (a measure of the average number of citations an article in the journal receives over two years). But the survey is the first to try to quantify what it calls 'coercive citation,' and shows that this is 'uncomfortably common.' Perhaps the most striking finding of the survey was that although 86% of the respondents said that coercion was inappropriate, and 81% thought it damaged a journal's prestige, 57% said they would add superfluous citations to a paper before submitting it to a journal known to coerce. However, figures from Thomson Reuters suggest that social-science journals tend to have more self-citations than basic-science journals."
The surest way to get something on Wikipedia, is get something published then cite it. Accuracy notwithstanding.
I think we know the number one culprit here, ahem [citation needed] ahem.
Social Sciences are not real sciences. Their only connection to science is that they attempt to use statistics, often improperly. After all, if social scientists were good with math they would have been real scientists.
This is what happens when you have metrics. You create a metric like "impact factor", and before long people will figure out ways to maximize "impact factor" that have nothing to do what the metric was originally supposed to measure. Hyperfocusing on metrics like that ends up undermining the things you really value in favor of increasing your scores.
This happens all over the place. Games in every game find ways to increase their score in ways that the game designers wouldn't really consider valid. Universities do things simply to make their "US News" ratings go up, not because they will make themselves better. Students figure out ways to raise their grades that have nothing to do with mastering the material of the course. Heck, the entire US (and world?) economy suffers from this; the most reliably rich people are the ones who manipulate money transactions, and do absolutely nothing with the underlying reality that money is supposed to be an abstract representation of.
People strive to improve the things that they are rewarded for and that they are evaluated on. When you focus too much on the wrong thing, people will do the wrong things in response.
Not exactly on topic but nevertheless an interesting read:
http://100rsns.blogspot.com/
Twitter: @dainsanefh
Who modded this flamebait? It's right on point.
But to ignore previous authors might be disrespectful. But isn't the citing of authors used for extrapolation?
... in the last sentence of the summary. I think the word you're looking for is "Naturally."
When the start referencing Brittany Spears song for their social relevance then I'll take exception.
What goes around, comes around. Considering this is basically the norm for student essays, it was only a matter of time until the students became the professors, and the professors fully saturated the journal editorial boards. It is just a promotion of the status quo to a level it is visibly a bad thing, really.
Great Intellect...
Shocked [3][12][21], I tell you! [4][7]! Studies [14][17][31] have shown [11][15] that this [26] never [21][22] happens [25] with reputable [5][14][24] papers [19]! How could this [32] have happened? [12][16]
Check your premises.
I put together an economics paper and sent it around to a few PhDs I know. Two of them came back with the exact response that this article indicates; "It needs more references if you want to get published." I asked if the math, logic, or conclusions were off, both responded they were not, but that was not the point. They made it clear that to get published it had to have more references to existing work, regardless of the content.
I can come up with arguments why such a policy has some merit -- keeping wacky stuff like modern monetary theory's hypothesis that there is no such thing as too much debt from distracting researchers, for example -- but good, bad, or indifferent; the fact that there is a barrier to papers which do not pay homage to existing academics is very real.
Stop-Prism.org: Opt Out of Surveillance
I did RTFA. The authors of the paper surveyed 54,000 academics, and about 1,300 responded to say, "Yes we felt pressured." That's 2.5%. Only 1/3 of those named a single journal that pressured them. Another 2.5% said, "We've heard that others have been pressured, but never us." 7.5% said, "We've never heard of it." And 87.5% didn't respond. The survey shows extreme self-selection as 7 of 8 academics did not respond. So before someone gets excited that 20% of academics are pressured, note that under 13% of academics responded.
They're being asked to pad their paper because the actual evidence being cited might not look that convincing on it's own right. And many of the conclusions aren't properly supported. Come on, we all had that experience writing papers. You've got a deadline, you're trying to get from point A to point B and you just don't have enough to make it all the way. So you make a statement you don't have support for and then link it to source material you know no one will read. So it looks like your conclusion is supported when in fact it isn't. You don't care though because the point is to get from point A to point B... and the only person you have to fool is the teacher or in this case the peer review that probably doesn't care that much anyway. Also... everyone else is doing it... and for the teacher to actually verify all those citations would be pretty much impossible. The only thing you have to be careful of is to not say something the teacher knows is false or will think is false. If you do that they might check the citation. But if you go outside of their knowledge forcing them to basically check everything or nothing... or stick closely to whatever the teacher is likely to believe anyway... you can get away with about 99 times out of a hundred. And the time you're caught... slap on the wrist or a small hit to your grade.
Now I have no experience with what happens when you actually start publishing things. I fully admit my ignorance here. But I'd be surprised if an academic history conditioned by this environment didn't predispose graduates to try the same thing. And really, who is going to stop them? They've had their whole academic career to perfect the best ways to scam the system. All those years they weren't just learning the subject but they were also learning how the subject is taught, how it is graded, the social characteristics of their judges, human psychology as it relates to auditing, etc. We learn all this stuff naturally.
Anyway, that there is fraud in academia isn't shocking. All human interactions involve fraud. If there's a benefit in deceiving someone then we probably do it and we get very good at it. This is indifferent to morality. It has more to do with intelligence. If you're clever whether you're a good person or a bad person... you learn to lie. Even if you don't use it for evil it's just a skill you acquire.
If there is anything I find bothersome here it is the conspiratorial aspect where someone is encouraged to decieve others. This sort of thing is marginally less offensive when it's kept isolated to individuals even if everyone is doing it. And really what people SHOULD be doing rather then finding bogus sources is find ACTUAL sources.
It's actually not useful to anyone if it's fifty percent bullshit. I don't care if it's half brilliant and half bullshit. Even ten percent bullshit isn't acceptable. Strip out everything that isn't backed by bullshit. If you can't get from point A to point B without using bullshit sources then maybe those two dots don't actually connect. I know you need to make a connection and maybe you are even required to make that specific connection because your peers won't tolerate anything short of it. But that isn't science and it isn't academically useful. Sure you get your grade or you get your degree or you get your job or you get paid. You get what you want. But you do it at the expense of system's integrity.
I don't know... it's hard to audit this stuff without investing unreasonable numbers of man hours.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
Science is now about the money like every other religion. Zealots will disagree but it is all but impossible to recreate experiments for yourself and thus people just have faith in scientists.
Maybe there is some sort of Internet Reference Counter worming its way through the Web. It looks at papers, and gives points to people who get referenced a lot. People who reference you are tallied as your friends, so you will know to reference them. People who publish in your area, but don't reference you, are foes, and get negative points. You can buy or sell references or points on eBay and pay for them with Bitcoins. People with lots of points are "Blue Chip" in the points futures markets. Points can be used to suppress rival research.
Hey, doesn't Facebook or Google do this already . . . ? . . . for an extra fee ? . . . ?
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
Researchers Misunderstand Confidence Intervals and Standard Error Bars.
Belia, Sarah;Fidler, Fiona;Williams, Jennifer;Cumming, Geoff
Psychological Methods, Vol 10(4), Dec 2005, 389-396.
Little is known about researchers' understanding of confidence intervals (CIs) and standard error (SE) bars. Authors of journal articles in psychology, behavioral neuroscience, and medicine were invited to visit a Web site where they adjusted a figure until they judged 2 means, with error bars, to be just statistically significantly different (p .05). Results from 473 respondents suggest that many leading researchers have severe misconceptions about how error bars relate to statistical significance, do not adequately distinguish CIs and SE bars, and do not appreciate the importance of whether the 2 means are independent or come from a repeated measures design. Better guidelines for researchers and less ambiguous graphical conventions are needed before the advantages of CIs for research communication can be realized.
(http://psycnet.apa.org/journals/met/10/4/389/)
money and career depend on what a brainless bean-counter adds up in a spread sheet.
if you want to publish the bleeding obvious, you'd better cite my prior work in the area.
In my (former researcher who left to industry) opinion/experience its not the editors who put the pressure, but the possibility that you ignored a work of somebody who is important enough to referee for Nature or Science. There are some components of these phenomena:
a) Maybe the work really is important, and you did not know it because it's too long ago. There is usually nothing wrong with a referee saying "hey that is similar to what [xyz]" did, even if they are on the list of authors on the reference in question.
b) some referees dont react positively to not getting cited and will shoot down any paper not referring to *their* theory for other reasons (i believe that happened to me once)
c) In the abstract (which is the part really read by the editors before the refereeing process) you compare your paper to the previous publications. Authors are under the impression that comparing your work to previous important papers makes a better impression. How far this is true i cant judge. I found the editor stage *before* the refereeing in Nature and Science the most intransparent thing I have experienced as an author. Unlike the refereeing process there is no way to appeal, there is not information on what the editors disliked so much to refuse directly. (There is the saying that once you had Nature/Science papers it gets more likely to pass this stage, and i have seen at least one example of a paper being passed to Nature which for sure would have been rejected by the editors had it come from a less important group in the field)
in fact, these kids got their research published without any references *at all*. http://rsbl.royalsocietypublishing.org/site/misc/BlackawtonBees.xhtml
i particularly like the section headings "once upon a time" and "the puzzle duh duh duhhhhh". i think however in the context of this article, the following exerpt from the background puts the corruption that has been highlighted by TFA to shame:
"So what follows is a novel study (scientifically and conceptually) in ‘kids speak’ without references to past literature, which is a challenge. Although the historical context of any study is of course important, including references in this instance would be disingenuous for two reasons. First, given the way scientific data are naturally reported, the relevant information is simply inaccessible to the literate ability of 8- to 10-year-old children, and second, the true motivation for any scientific study (at least one of integrity) is one's own curiousity, which for the children was not inspired by the scientific literature, but their own observations of the world. This lack of historical, scientific context does not diminish the resulting data, scientific methodology or merit of the discovery for the scientific and ‘non-scientific’ audience. On the contrary, it reveals science in its truest (most naive) form, and in this way makes explicit the commonality between science, art and indeed all creative activities."
wow, congratulations, after 150+ years, this 'science' is starting to maybe finally think about using the idea that you should use actual data from the real world in your formulation of theory. brilliant concept. let me know how it works out.
evaluate its theories, and whether or not they match reality. because, as mentioned elsewhere, economics is the only 'science' where nobody cares about actually measuring reality. they come up with a bunch of theories, get payed fat 'consultant' bonuses by corrupt leaders, and live in a state of suspended, deluded animation.
a great example is from Ferguson's film "Inside Job", where he flat out proves that professors took money to lie about the state of Iceland's banking system right before it fell into the abyss circa 2008. in what other field of science do 'researchers' get payed hundreds of thousands of dollars to write up theories proving the conclusions of their benefactors? it would be like if electrical engineers were payed to say that 8 bit register can hold 500 different numbers. no, they cant.
In the researcher world, it's all about publishing, publishing, publishing. 90% of the papers is crap, and at leat 75% of the conferences is crap. People fake results, publish even before they have the actual results, only to get noticed. I'm in the middle of it, and I sometimes feel so ashamed by the quality of what I'm forced to publish that I feel like stepping out.
There will also be some bias in terms of what makes a citation "superfluous." A good editor is concerned about insuring that the submitter's statements are accurate and well-sourced. On good legal journals, editors actually go and look at every cited source (hundreds per article) to see if it contains the proposition it is cited for. When a journal is well-sourced, it is more reliable as a source for practitioners and as a research tool for academics, as well as being a better stepping-off point for further research. Forcing an author to show that his or her statements are actually accurate can be useful. More junior faculty obviously get away with less blustering because their reputations are not already built, and journals have an interest in getting people who are better-known in the field.
-- IANAL, this isn't legal advice, and definitely isn't legal advice for you. Also, Squee!
It's discussing *superfluous* citations, not irrelevant ones. The authors are the ones deciding if they are superfluous. Authors will always think that having to add a citation is superfluous. That doesn't mean that they should be able to ramble on for a forty page paper with less than an absolute minimum of, say, 120 footnotes. Is it possible that the paper is perfectly correct without them? Sure. But if I'm putting a journal's name on them, and I'm responsible for the journal's reputation, I'd like them to be more reliable than a note someone wrote on the back of a napkin.
-- IANAL, this isn't legal advice, and definitely isn't legal advice for you. Also, Squee!
In my field, I usually end up struggling prior to submission to cut references, because many of the journals I submit to have limits on the numbers of citations. Often, this means citing a review rather than the primary literature (because one review can take the place of multiple primary papers), or citing a recent work using the most current methods and dropping citations of the earlier ground-breaking work in the field
I am an author and an editor of a journal that could use a higher impact factor to get noticed. But I have never been "encouraged" to add a reference that was not clearly missing (there have been one or two of those, due to inadequate research on my part), and as an editor I have never asked for additional references except in cases where there was clearly prior work that the authors should have been aware of and should have cited, usually because the missing references actually showed the results the authors were claiming as new contributions. So I think this is a case of extreme self-selection, and perhaps a particular field or journal where some practices need to be examined. I just don't see it in Computer Science, Economics, or related fields where I read and publish.