Negative Irreproducible Tweets For Science Publishing

New submitter mwolfam writes "Every scientist has at least one paper or graph tucked in a folder that lies in a dusty corner of the hard drive next to that dancing baby that used to be all the rage. The data is interesting, but doesn't lend itself to the creation of the grand narrative you must have for a traditional publication. It's time to expand traditional scientific publication to include a place for the data that normally falls through the cracks: short but interesting bits of data, negative results, and irreproducible results."

Comments

Korea?

[Anonymous Coward]

Sounds like a place for the data to reside.

Like Arxiv?

[sackbut]

Maybe scientific Facebook pages that link to the Arxiv page?

what the hell are you talking about?

[Anonymous Coward]

"Negative" results are published all the time, as are irreproducible results. They just don't get the airtime for science-fans to notice, only scientists.

Re:what the hell are you talking about?

[crutchy]

i agree. scientists who ignore outliers are potentially tampering with the outcome of their paper. outliers aren't necessarily correct, but they can help a reader judge the reliability of the method used or the competence of the person performing the method. you don't necessarily draw conclusions from outliers when writing your paper, but ALL results should be included in the report somewhere (often as an appendix), regardless of their validity or contribution to the results reduction and analysis.

Re:what the hell are you talking about?

[RockDoctor]

There are whole journals for "Irreproducable Results". And prizes too!

Re:what the hell are you talking about?

[__aaltlg1547]

JIR focusses on weird experiments, not unexpected results from normal experiments.

Re:what the hell are you talking about?

[RockDoctor]

And how many times a year do you look at the results of a normal experiment (in my case, a batch test of numerous hypotheses concerning the stratigraphy, rock properties and fluid charge history of volumes of rock, based on well-analysed remote sensing data and well-founded models ; a.k.a. drilling an oil exploration well), and look at the results, and say "that's weird".

And then Bean-Counter Central comes back and says "We can't afford to reproduce those tests.

Where do you draw the line?

Late night AM radio fodder

[Anonymous Coward]

They'll have a field day with it, along with the moon landing and JFK conspiracies.

Re:Late night AM radio fodder

[Anonymous Coward]

And someone calling in as Gordan Freeman.

"AM Coast to Coast" is a riot... for about 10 seconds before I remember I can't stand rambling nutters.

arXiv.org?

[Anonymous Coward]

We've already got it... arXiv.org

Re:arXiv.org?

[buchner.johannes]

arXiv stores papers, not data. It is expected that what you put there has been accepted in a journal.

Traditional journals already do this.

[bkaul01]

Such a thing already exists: many journals (at least in my field) accept submissions for "technical notes" that aren't full-fledged papers, but merely describe a brief, interesting bit of data, etc. It's more a question of whether the researcher has any incentive to put the time into writing them up and submitting them than a problem of a lack of venues for us to do so.

Re:Traditional journals already do this.

[kharchenko]

Tweets are an abomination. You still have to describe what you've done properly, otherwise the reported result is of no value.
There are journals that were created specifically to report negative results. Irreproducible results, on the other hand, are not a scientific matter.

Re:Traditional journals already do this.

[dgatwood]

Nah. Irreproducible results are a scientific matter. Often one irreproducible result later turns out to be reproducible after controlling for something that you weren't looking for before.

Re:Traditional journals already do this.

[grqb]

I think the tweet idea is slightly different. For example, a lot of work that a scientist does is collecting data to make sure equipment is working properly. Usually these experiments aren't worth publishing and probably wouldn't make it past a peer review because 1) they're usually not novel experiments 2) they don't tell a story or add much value, but I think it could be useful to share this type of data. I mean, if you've collected it, why not share it?

Re:Traditional journals already do this.

[Anonymous Coward]

Often one irreproducible result later turns out to be reproducible after controlling for something that you weren't looking for before.

Irreproducible results rarely become reproducible. If that change was often, then science would be insanely productive and scary.

Re:Traditional journals already do this.

[dgatwood]

In established scientific fields, that might be true, but on the bleeding edge of research, initial failures quite frequently lead to subsequent successes when the hypothesis is correct, but the test equipment, methodology, or sample size is insufficient. Similarly, initial successes often stop working.

One great example of this is medical research. Frequently, things that aren't initially reproducible in studies later turn out to occur far more reliably in the real world. Half the recalls in medicine occurred because some rare negative side effect that only showed up once in the first test and wasn't reproducible in subsequent small scale tests turned out to be fairly common once the product went out into the wild.

For another great example, consider the cat's whisker radio. Although the initial experiment was a success, later attempts to reproduce the same results were fraught with failure. In spite of that, eventually they figured out how to make it work fairly reliably, and that design became the earliest semiconductor-based diode. Were it not for someone having the courage to try to reproduce those early experiments repeatedly until they figured out what the difference was, the computer you used to type your comment would likely not exist.

All it takes is a single critical difference in an experiment to make the difference between successfully reproducing an outcome and not reproducing it. Since humans are far from perfect, basic logic dictates that a fair number of irreproducible experiments would probably be reproducible if someone hadn't forgotten to document some very subtle detail of the experiment (or worse, some very subtle detail of the test subjects). If you aren't seeing this in practice, it probably indicates that there is a wealth of knowledge waiting for those with the courage and insight to try harder to reproduce experiments that initially seemed unreproducible.

In other words, either you're wrong, or we're missing out on a lot of important discoveries. Take your pick.

What the hell?

[Anonymous Coward]

Unless they are referring to working with phenomena that were one-offs like a particular passing comet, irreproducible results are by definition not science.

Of limited use

[Gemeinhardt]

While not a horribly bad idea, it would be of limited use. The reason science doesn't dwell on the odd irregular result, and especially on results that can't be reproduced, is that you cannot draw any conclusions from them.

Re:Of limited use

[tlhIngan]

While not a horribly bad idea, it would be of limited use. The reason science doesn't dwell on the odd irregular result, and especially on results that can't be reproduced, is that you cannot draw any conclusions from them.

Maybe not by itself, but sometimes interesting correlations pop up because of strange combinations. Or more likely, someone gets the results they were expecting, but sees an odd variance they can't explain. Perhaps if it was seen elsewhere, the odd data correllation may have some merit in investigation.

It's like an odd bug you find when using some software. You don't think it's important (perhaps it happens occasionally), but someone else decides to just mention it in passing, and then others chime in as it happened to them, and then hey, perhaps it's a bigger bug than expected.

Just putting it out there may bring others to notice they see the same thing as well and then provide incentive to do proper research in it.

Re:Of limited use

[crutchy]

odd irregular results still affect the R value of a curve fit, which helps determine the overall validity of the curve. they should not be ignored, at most they can be omitted from curve fitting if they can be explained

Useful

[Anonymous Coward]

I can see this being really useful, especially if the raw data could be easily accessed and manipulated. On the other hand, I, as a researcher, would be loath to simply give away data, even data for which I can forsee little use, just on the off chance that it could be used in a future publication, or form the basis of further work. A rather ignoble attitude, I'll admit, but one which I'm sure many others would share, and I think this would be a huge obstacle for the idea.

Re:Useful

[marcosdumay]

That is because giving that data away won't contribute to your career, but holding it until publishing will.

This problem is an easy to fix one.

More Tweets in a cacophony

[ackthpt]

Well, if it gets it out there, but why Twitter? It's going to have to compete with all the usual garbage which is trending.

Brett6565 Vampires in yet another TV show :P #fail #bloodsuckers

Wignut Yankees sign another pitcher #goyanks

Waddleduck Another show about lawyers #fail #bloodsuckers

Cherbonevski sci.fi/fd98guyrr Nucleotides enzymolgy in e. nemtodii #science #wowwee #knowledge

yellomello Moar lolcat pictures of my kitty! bit.ly/r9d8gns9ds #LOL #CATS #LOLCATS

cityfied Tevez to Milan! Good-bye and don't let the door hit you on the arse on the way out! #MCFC #TEVEZ
 

Other journals that also already do this:

[Anonymous Coward]

Fortean Times
Annals of Improbable Research
Journal of Irreproducible Results
. ... etc, etc.

OMG.

[tenco]

Prepare to be drowned in irrelevant data.

Asimov's

[gmuslera]

The most exciting phrase to hear in science, the one that heralds the new discoveries, is not 'Eureka!' (I found it) but 'That's funny...'

Re:Asimov's

[perpenso]

The most exciting phrase to hear in science, the one that heralds the new discoveries, is not 'Eureka!' (I found it) but 'That's funny...'

Unless that scientist is your physician and looking at your test results. :-)

Re:Asimov's

[__aaltlg1547]

I think it may still be pretty intriguing to your doctor.

It's been done

[SoftwareArtist]

Journal of Failed Crystallization Experiments

Ok, some of the humor is a bit esoteric for those who don't know much molecular biology. You'll just have to take my word for it that it's really funny!

Crack

[SEWilco]

I have the most logically organized and beautifully poetic reply to this. It would bring tears of joy to your grandchildren's eyes, but it will not fit in this space. I'll just tweet about it.

"irreproducible results"

[John Hasler]

There is a journal entirely devoted to exactly that.

Re:"irreproducible results"

[notreez]

And if I remember correctly it was called "The Journal of Irreproducible Results." I probably still have some laying around.

Isn't this what the web is for?

[perpenso]

Isn't this what the web is for? Put it on the web, let google index it, and it will be far more accessible that anything else ... "national firewalls" permitting of course.

Already exists

[Anonymous Coward]

Journal for Irreproducible results:

http://www.jir.com/

Lets measure increase and publish a paper ...

[perpenso]

Prepare to be drowned in irrelevant data.

People reading these posts are online. They are already drowned in irrelevant data. The content being described probably wouldn't amount to a noticeable increase. On the other hand it might, perhaps we could quantify the increase and publish a paper. ;-)

Re:Lets measure increase and publish a paper ...

[marcosdumay]

Compared to the scientific data out there, this content will be orders of magnitude bigger. And, yes, researchers are already drowned in useless data (several kinds of those, like irrelevant, unreliable, undecipherable).

It would be in everybody's interest to keep that new data separated. Then, people could use some search algorithms to walk through it. How to traverse that amount of data would be reason to publish several papers.

Article Titles

[bill_mcgonigle]

The purpose of a title is to give the reader some inkling of what might come next.

"Negative Irreproducible Tweets For Science Publishing" may be the worst [not incorrect] Slashdot article title ever.

Something like "A Plan for Publishing Minor Science Results on the Web" (or do better, you're the submitter) would at least not leave readers perplexed.

arXiv papers do not have to be in a journal

[ODBOL]

I am a moderator for arXiv, and I am quite sure that submissions are filtered only for prima facie relevance, and do not have to be "accepted in a journal." The format of arXiv is probably not suitable for all sorts of data, but lots of data can be presented as text and can be placed in arXiv.

Re:arXiv papers do not have to be in a journal

[definate]

That's cool, but text is not an optimal format for this. Some sort of large online database or code repository, would be much better suited to it. Like those Google has.

Re:result later turns out to be reproducible

[Anonymous Coward]

Bingo.

I am a bit of a bio-fluke. I have no sense of smell and I get no headaches on hangovers. Reproduce that! Oh wait - I specifically took up Milk based Cocktails to reduce hangover headaches, based on some other irreproducible result I read...

Oh dear, this one goes AC as well...

JSUR

[Anonymous Coward]

It's time to expand traditional scientific publication to include a place for the data that normally falls through the cracks: short but interesting bits of data, negative results, and irreproducible results.

You mean a bit like this journal?
http://www.jsur.org/
No issues yet, but it's a journal that welcomes failure, and discoveries of things that just shouldn't happen.

That's what ...

[PPH]

... Slashdot Idle is for.

All of this has been done / possible / etc.

[oneiros27]

For #1, there was The Journal of Earth Science Phenomena (hasn't had anything new in over a year), where they'd publish what they called 'micro-articles', which was mostly just a picture and a short description. Unlike a tweet, it actually had some peer-review, and enough information to make the item useful in its own regard. In solar physics, it's not a journal, but there's the Heliophysics Event Registry, where scientists can submit events/features/phenomena, but it's not peer reviewed. (and some are submitted via pipeline processing, so there might not've been any human involved in the detection other than writing the software)

For the negative results, there are plenty of dedicated journals in various fields, and if there isn't, there's always PLoS ONE. It's possible that they might take the irreproducable stuff, too. In their description, they say they'll take anything that's 'technically sound'. They do use a model that's different from other peer-reviewed journals, and go with the author-pays approach, which many of the other journals claim makes them invalid (yet, those same journals charge even more to make your article 'open access' if it gets accepted)

A Scientific "Source Control" Repository?

[definate]

Perhaps the best idea, would be a science orientated source control repository, where the data and a brief could be "dumped", and others that wanted to work with it, or incorporate it into their work, could either create a new branch (if they join the repository), or they could create their own fork.

Of course it would need to have a good interface like GitHub, but more orientated for publishing in TeX.

Who wants irreproducible results?

[Anonymous Coward]

"but interesting bits of data, negative results, and irreproducible results"

Maybe I am missing something here, but who wants irreproducible results?

It is tough enough with "sciences" like history, which goddammit never repeats itself, and ground water research.

Sadly, the Marimekko clad women in social research frequently giggle when they inform you that each individual is unique and that each event in their research has been unique and as such cannot be repeated. Still, they haven't studied statistics and the term "irreproducible" might give them the idea they were correct... I "Irreproducible" may have some relevant mean somewhere I am not aware of.

Why give your competitors an advantage?

[cyn1c77]

Journal of negative results will never become widespread because the scientific funding system does not favor it. It's a fantasy of a wide-eyed postdoc, who has yet to experience the cutthroat realities of the scientific funding system in the US. (I know this because I, and may others, used to have the same idea.) Here's why the concept is fatally flawed:

The OP notes that your scientific colleagues are also your competitors. He then notes that if you don't report your scientific failures, that your colleagues will likely repeat them. When completing for limited funding, why would you give your competitors an advantage? Wouldn't you rather they spend 2 months of their resources and $50K doing something you will set them back?

Also, let's say that you had 20 articles published in the Journal of Negative Results and two published in a "real" journal. Is that going to look good on your resume? Or on your next funding proposal?

Successful scientists become good at recasting their negative results as positive ones and getting them published in a journal. No unethical behavior is required. A negative result can become a positive one with a simple change of hypothesis! Ideally, you plan for that outcome when designing your experiment, so that you always get a useful result. Or you limit the devoted resources (scoping tests) until you have an indication that you are onto something good.

Negative trial database in beta

[ananyo]

A database of negative results is actually already in beta: http://figshare.com/ Psychology professor Jonathan Schooler also called for a negative trials database in Nature in February last year. He says it's possible such results could explain the 'decline effect' that plagues science http://www.nature.com/news/2011/110223/full/470437a.html

Or it wins a Nobel Prize (Chemistry 2011)

[oneiros27]

Of course, it was ~30 years later after Shechtman had been ridiculed for his 'quasicrystal' discovery.

So even if 99.9% of the time, the 'odd, irregular result' is worthless ... there's also the chance that it's revolutionary, and we want to make sure that those get preserved. It might be that 5 people have similar 'odd, irregular' results, but they can then compare notes amongst themselves and figure out what might be the significant factor and make it reproducable.

If they were real scientists...

[unitron]

...they'd know that data takes plural verbs, as it itself is the plural of datum.

Re:If they were real scientists...

[Anonymous Coward]

...they'd know that data takes plural verbs, as it itself is the plural of datum.

I know that at the journal Nature, whether 'data' should treated as singular or plural is still a hot topic of debate. http://www.gpuss.co.uk/english_usage/data_plural_singular.htm

Re:If they were real scientists...

[unitron]

...they'd know that data takes plural verbs, as it itself is the plural of datum.

I know that at the journal Nature, whether 'data' should treated as singular or plural is still a hot topic of debate.
http://www.gpuss.co.uk/english_usage/data_plural_singular.htm

If the singular form is data, what, pray tell, is the plural?

Will someone in the dismal future be saying "The datas are clear"?

Journal of Serendipitous and Unexpected Results

[Anonymous Coward]

http://www.jsur.org/

Good Idea

[Anonymous Coward]

This is a good idea precisely because unlikely or odd "facts" and "results" may be of great interest to random folks in other fields of interest, and have no immediate value to the original source at all. It is precisely when our society is allowed to be very granular and decentralized, that the greatest opportunities and breakthroughs occur. Even if nothing were to come of it at all, the mere acceptance of a pointless act of truth telling has inherent merit.

JJ

what about /.?

[Anonymous Coward]

why not slashdot? its already full of useless shit... more can only make it better, right?