Slashdot Mirror
New Scientific Journal To Publish "Discrete Observations Rather Than Complete Stories" (sciencemag.org)
sciencehabit writes: Is the pressure to publish tempting scientists to improperly tweak their findings in order to create more cohesive stories? If researchers could report just the one finding they felt comfortable with, perhaps there would be no need to be dishonest. That thinking has spurred the creation of a new scientific journal, Matters. The open-access publication aims to boost integrity and speed the communication of science by allowing researchers to publish discrete observations rather than complete stories. "Observations, not stories, are the pillars of good science," the journal's editors write on Matters' website. "Today's journals however, favor story-telling over observations, and congruency over complexity Moreover, incentives associated with publishing in high-impact journals lead to loss of scientifically and ethically sound observations that do not fit the storyline, and in some unfortunate cases also to fraudulence."
I think a concerning matter is that journalists (not science journals necessarily) also destroy the credibility of science by taking these observations ("according to a recent study...") and running with the "results" as news. A recent one that comes to mind is that researchers noticed that the diabetes medication Metformin seemed to have effects on life expectancy. Of course news outlets are currently running with the story that we might have found the miracle anti-aging pill. You can turn up a bunch of articles by googling the drug. It's usually later found that the claims are hugely inflated by the media and further research really goes nowhere. I suspect that the fatigue of constantly hearing these kind of false-hope and misleading reporting articles might hurt the image of legitimate scientific research. I wonder if this will have an effect on this issue. I suspect researchers may be complicit in providing journalists with these stories that they love to run with. Keeping that kind of speculation to a minimum might help.
This would be great if it actually reduced the pressure on scientists somehow. When hiring decisions are based on publications in the pressure inducing top journals, this isn't going to help anyone who wants to be hired.
As they say in their website ``Once a group of authors has accumulated a sufficient body of linked, peer reviewed publications at Matters (reaching a minimal network size), we encourage them to submit a narrative integration of their observations to Mattersconsilience, the third journal of Sciencematters.'' [1].
Plus, if there is no story behind... how do we, scientists and people, know if an observation is important or not? Fact: ''I added nutrient X, the expression level of gene Y decreased''. Good, but... what's the deal with that? If you put it into context (story) then you can say 'meh' or "holy sh*t".
I think this is a bit of non-sense...
[1] https://www.sciencematters.io/what-is-matters
nearly in a tailspin over our greed fear ego based neglect of ourselves & others? undefeated since/until forever? creation provides more than enough of everything with no personal gain motives... get real? hand in hand we stand? watching fauxking nazi zion made for tv wmd on credit genocides? read the teepeeleaks etchings... teargassing etc... moms & kids does almost irreversible damage to us all? thanks again.... almost all the moms are still crying...wake up
"If researchers could report just the one finding they felt comfortable with, perhaps there would be no need to be dishonest."
Scientist speaking here. One finding in no finding. It's luck or mistake. If there's just one "finding" you're "comfortable with", it's not publication you should think about, it's changing what you do and how you do it.
"incentives associated with publishing in high-impact journals lead to loss of scientifically and ethically sound observations"
Bullcrap. And "that's all I have to say about that"
"Today's journals [...] favor [...] congruency over complexity"
Uhmm, sorry, what now? Why would one exclude the other? On the other hand, would they want journals that prefer complexity over congruency? Now, that would be a doozy.
"There are few, if any, places to publish one-off experiments that arenâ(TM)t part of a bigger story but might still be informative. So unless the researcher âoeinvests in a series of additional experiments to package the failed reproduction, that result will languish in laboratory notebooks,â"
Well, I don't think I could be convinced we should value un-reproducible one-off experimental "results". Ever. However, there's nothing stopping you people publish such "results", you know, there's the Internet and whatnot.
"a researcher who is able to show, with proper controls and statistics, that an extract from eucalyptus bark relieves pain under certain conditions. âoeIn todayâ(TM)s world, you canâ(TM)t publish that in a good journal,â Rajendran says. âoeYou would need to know which molecule it is"
Hell, good that it is so. There are still some people out there who actually like to know what the hell it is they put into their bodies and how it works (and that it actually works).
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
(Speaking from the perspective of a UK academic, may vary between countries) There is no pressure to publish, as an abstraction. There is pressure to demonstrate impact. The easiest way to demonstrate impact is to publish in top-tier publications. Publishing in a new journal or conference is always a big gamble - if the journal does well later then you may retroactively benefit from a later assessment of its impact, but typically it's in the noise of all of the spammy journals.
I am TheRaven on Soylent News
Ideally a paper has enough information that you can either recreate, apply and or expand upon the work. Saying here's some observations have at it with the curve fitting tools would be just idiocy. Don't even know how you would publish for any field that generates large amounts of data.
Then you have fluke events such as the apparent one time observation of a magnetic monopole. It's meaningless without context.
IMHO, the problem starts in school. As an example: you do a chemistry experiment, get some weird results, which aren't the ones you should have been getting, now you have two options, which are either to write up and conclude what you observed or bullshit and write up what was expected, as if it had worked. The first risks getting you low marks, while the second top marks. What do you think most people under pressure to perform would do?
The way I would like to see things done: you write things up as you observed, but add an in the conclusion an analysis of why you think your results varied from expected results. For example, did you put in too much of substance A or substance B, and why would that impacted things. It may put extra work on the teachers, but if we want students who can think and not cover up their tracks, then this may be worth it. A healthy workplace depends on this.
Jumpstart the tartan drive.
And this is what happens when people raised without the ability to concentrate for more than a few minutes at a time come into positions of power and authority.
I read the article. Kudos to the editors to try and further speed the process of publication (and for promising to pay editors and peer reviewers), but the basic premise is flawed. The only benefit to a publication like Matters will be to increase the publication count of its authors. Individual observations, without the scholarly research to provide a framework, or without a full line of inquiry to provide rationale and interpretation, does not move the field forward. To draw an exaggerated analogy, it's the equivalent to tweeting about what you had for lunch.
Put my fist through my alarm clock with its ding-dong death inside my ear. - The Blackjacks.
...is a Left Cheek Sneak that you blame on the dog.
My favorite observation:
Shut yourself up with some friend in the main cabin below decks on some large ship, and have with you there some flies, butterflies, and other small flying animals. Have a large bowl of water with some fish in it; hang up a bottle that empties drop by drop into a wide vessel beneath it. With the ship standing still, observe carefully how the little animals fly with equal speed to all sides of the cabin. The fish swim indifferently in all directions; the drops fall into the vessel beneath; and, in throwing something to your friend, you need throw it no more strongly in one direction than another, the distances being equal; jumping with your feet together, you pass equal spaces in every direction. When you have observed all these things carefully (though doubtless when the ship is standing still everything must happen in this way), have the ship proceed with any speed you like, so long as the motion is uniform and not fluctuating this way and that. You will discover not the least change in all the effects named, nor could you tell from any of them whether the ship was moving or standing still. In jumping, you will pass on the floor the same spaces as before, nor will you make larger jumps toward the stern than toward the prow even though the ship is moving quite rapidly, despite the fact that during the time that you are in the air the floor under you will be going in a direction opposite to your jump. In throwing something to your companion, you will need no more force to get it to him whether he is in the direction of the bow or the stern, with yourself situated opposite. The droplets will fall as before into the vessel beneath without dropping toward the stern, although while the drops are in the air the ship runs many spans. The fish in their water will swim toward the front of their bowl with no more effort than toward the back, and will go with equal ease to bait placed anywhere around the edges of the bowl. Finally the butterflies and flies will continue their flights indifferently toward every side, nor will it ever happen that they are concentrated toward the stern, as if tired out from keeping up with the course of the ship, from which they will have been separated during long intervals by keeping themselves in the air. And if smoke is made by burning some incense, it will be seen going up in the form of a little cloud, remaining still and moving no more toward one side than the other. The cause of all these correspondences of effects is the fact that the ship's motion is common to all the things contained in it, and to the air also. That is why I said you should be below decks; for if this took place above in the open air, which would not follow the course of the ship, more or less noticeable differences would be seen in some of the effects noted.
Instagram or Twitter
This is crazy. If you can't make sense of your observations and connect them to our understanding, then they are unlikely to be useful. Existing journals will publish observations that are not explained if they are accompanied by a careful explanation of what is and is not understood about the problem. We definitely do not need more publication of observations disconnected from understanding.
Great, so it's the headline --> comment section 2ms attention span for science. I'm sure this won't lead to isolated observations being turned into headlines and massively misconstrued as ammunition for people's agendas. I look forward to when this journal replaces Wikipedia as the most laughable thing one could possibly supply as a source to back up their lunatic ravings on /.
Somebody has reinvented the usenet and correspondence. Open correspondence with peer review sounds like a nice, inclusive way of pursuing the knowledge. Lets hope everyone behaves.
The most prominent motivation for this proposal lies in prominent failures and retractions in medical and psychological research. As a recent meta-study showed, most psychological studies are not reproducible (probably because their pool of subjects consisted of university students, a very weird bunch of people ;-). Also, many drug studies are influenced by pharmaceutical industry funding.
But the article's proposal won't work. It assumes, at some level, that there are fundamental facts, and that it's possible to discover these facts, without a theory. That's why they are proposing publishing discrete observations, without any "story" that observations fit into. But philosophers have thought about this already. Kant's theory of categories explains that you can't perceive facts "raw", but always see the world through some mental model you carry with you, wether you know it or not. So you always have a model of the world, which colours your perceptions.
I would argue, further, that thinking itself is impossible without a model. You need a structure to hang your ideas onto. You can't stand fully outside your own biases and mental preconceptions, and see things are they "really are". Your model may change over time, or someone else's model may become accepted as better, and observations will then fit into a different "story". That's what a scientific revolution is: a change of model to explain the same phenomena.
Facts need to published within the context of a "story". There's no way around this. At most, we can try to be aware of the story we are caught inside of.
This is great! I've worked in biology for a number of years now and just about everyone has reams of this kind of data waiting to be worked on. It's well done and reproducible but just fell outside of the scope of the project or funding. For whatever reason it was never followed up on.
While a lot of people might scoff at publishing this kind of data, I would remind them that it is exactly this kind of data that usually acts as the seed for a PhD project or grant application. Sure it's not the full story, but it's not pretending to be. This is just one more way for us to share ideas and build a better understanding of our respective subjects.
While claiming to be a new approach, a glance at the early Philosophical Transactions of the Royal Society indicates that this idea is very old. The modern journal's move to "cohesive stories" was in many ways a reaction to the initial idea of listing observations and discoveries. Hence, the table of contents of the first issue (March 6, 1667) includes:
... of a Spot on one of the Belts of the Planet Jupiter
...
- An Account of the Improvement of Optick Glasses at Rome
- Observations
- Motions of the late Comet predicted
- Relations of a very odd Monstrous Calf
- A Peculiar Lead-Ore in Germany
- the New American Whale
Perhaps the final item before the list of new books and "lately dead" anticipates the change to cohesiveness: - Narrative concerning the success of Pendulum-Watches at Sea
Yay! Those guys were a bunch of assholes anyway.
"2015/12/03 20:57:89.523 - Test rat 1591 consumed a pellet from dispenser A."
You don't get worthless results published because the authors spent too much time trying to track down all the issues with their data. The quality is low enough without getting the pre-alpha version.
Science and the public would be better served by forcing a waiting period before the stuff actually gets released.
How many premature "discoveries" made the newspapers this year? And more than that on slashdot.