Peer Review Ring Broken - 60 Articles Retracted

blackbeak (1227080) writes The Washington Post reports that the Journal of Vibration and Control's review system was hijacked by a ring of reviewers. 60 articles have been retracted as a result. "After a 14-month investigation, JVC determined the ring involved “aliases” and fake e-mail addresses of reviewers — up to 130 of them — in an apparently successful effort to get friendly reviews of submissions and as many articles published as possible by Chen and his friends.'On at least one occasion, the author Peter Chen reviewed his own paper under one of the aliases he created,' according to the SAGE announcement."

Comments

The Good News?

[KibibyteBrain]

Peter holds a very high standard for himself, I'm sure.

Re:The Good News?

[Cryacin]

It's just the new strategy employed to increase the speed of scientific research and development. It's called the self-peer-review.

Amazingly articles can get released on the same day as submission with this method.

Re:The Good News?

[marcello_dl]

Besides, who's more peer with respect to the author than the author itself? I tell you, it can't get more peery than this.

Re:The Good News?

[m00sh]

Peter holds a very high standard for himself, I'm sure.

The standard practice is to form a unspoken agreement between several reviewers that they will all favorably review each others papers.

Peter couldn't find his circle and created a self-circle.

Re:The Good News?

[thunderclap]

Peter holds a very high standard for himself, I'm sure.

The standard practice is to form a unspoken agreement between several reviewers that they will all favorably review each others papers.

Peter couldn't find his circle and created a self-circle.

Otherwise known as a circle jerk.

Re:The Good News?

[Anonymous Coward]

One minor query if I may:
What of the so called "ring"?
Does a lone protagonist now qualify as a "ring" in the new world order?
Sensationalistic hyperbole?
Perhaps TFA need more stringent peer review in itself?

Re:The Good News?

[niftymitch]

It's just the new strategy ........ It's called the self-peer-review.

Amazingly articles can get released on the same day as submission with this method.

Not unheard of here on /. as well.

Multiple accounts on multiple virtual machines at multiple coffee shops
perhaps gatewayed via VPN thanks to co conspirators to present
a global view.

Watch how quickly someone, not I, mods this up and down...

Wish I could say I was surprised

[Crashmarik]

We live in a day and age where you can make a pretty decent living as a scientist without actually advancing science, or doing very much technologically related labor, only natural people would game the system. While science should be immune to this sort of thing, just how many unimportant not particularly interesting results do people actually try to reproduce ?

Re:Wish I could say I was surprised

[Anonymous Coward]

You have it backwards. the fault is not that not every scientist has a breakthrough.

the fault is that in academia its pretty much "publish or die". The incentive to publish over anything else pushes the unscrupulous to do things like this.

the system itself creates this sort of situation.

Re:Wish I could say I was surprised

[Karmashock]

Wrong. The issue is that publishing is considered sufficient.

It should be publish or die. How do you know they're doing anything if they don't publish? they could be watching tv all day for all you know otherwise.

But as is made clear here, simply publishing and getting it through peer review is clearly not good enough. We need to increase what they have to do to avoid this situation.

For example... maybe one scientist pays another scientist to reproduce his work.

Maybe you have big collections of graduate students that as part of their process of getting a degree get assigned some random papers submitted by scientists in their field and they have to reproduce the work.

Obviously this isn't always possible... but whenever it isn't possible that needs to be put as a giant red asterisk on the paper saying "this work has not been reproduced"...

Do that and you're not going to get as much fraud or laziness.

Re:Wish I could say I was surprised

[Penguinisto]

It should be publish or die. How do you know they're doing anything if they don't publish?

Dude, seriously? Look up Hendrik Schön; he published... a LOT.

Re:Wish I could say I was surprised

[Penguinisto]

...and I skipped over a bit you posted, my bad.

Re:Wish I could say I was surprised

[Karmashock]

If you had read what I said, then you'd see my post addressed your criticism of that one out of context quote.

Re:Wish I could say I was surprised

[Anonymous Coward]

Wrong. [...] It should be publish or die.

I belive the phrase you're looking for is "publish or perish."

Re:Wish I could say I was surprised

[ShanghaiBill]

We need to increase what they have to do to avoid this situation.

Alternatively (or in addition), we could increase the penalties for those caught cheating.

Re:Wish I could say I was surprised

[TapeCutter]

we could increase the penalties for those caught cheating

No thanks, keep the lawyers out of it unless a genuine crime has been commited, the last thing we want is politicians regulating peer-review. There is no system that is totally incorruptable, the fact that these frauds were exposed means the system is working in this case. The fact that the scientific and academic communities will ostrasize the fauds for the rest of their lives is natural justice, anything more crosses the line between natural justice and revenge

Re:Wish I could say I was surprised

[antifoidulus]

No, "publish or perish" really dis-incentivizes novel research because guess what, often times really novel research fails. All "publish or perish" really does is incentivize either cheating or the lowest risk research imaginable. There are other mechanisms for making sure a researcher is actually doing their work, punishing them for taking risks shouldn't be among them.

Re:Wish I could say I was surprised

[Karmashock]

If you pay scientists to do science and they are contracted to do it... they fraudulently do not do science yet continue to cash your checks... that is a crime.

Re:Wish I could say I was surprised

[Kazoo the Clown]

No, "publish or perish" really dis-incentivizes novel research because guess what, often times really novel research fails. All "publish or perish" really does is incentivize either cheating or the lowest risk research imaginable. There are other mechanisms for making sure a researcher is actually doing their work, punishing them for taking risks shouldn't be among them.

If novel research is failing peer-review, I don't see that not publishing is a good answer to that. A convenient one, no doubt.

Re: Wish I could say I was surprised

[Anonymous Coward]

Woah there. Reference, please!

Re:Wish I could say I was surprised

[Brulath]

The problem there, then, is that research papers which analyse a failure aren't accepted often enough, which probably leads to other people redundantly repeating the same fruitless efforts. Failures aren't as flashy, but they're surely still useful.

Re:Wish I could say I was surprised

[Anonymous Coward]

The publish or perish mentality specifically contributes to an unwillingness to reproduce results, though; the likely outcome of doing such is not publishable and therefore effectively worthless in career terms. Why on earth would someone waste time reproducing results knowing full well it isn't going to get them anything (and in fact will set them back, as many of their colleagues will be doing original research).

Re:Wish I could say I was surprised

[Karmashock]

I was quoting someone else... please correct them instead... *yawn*

Re:Wish I could say I was surprised

[Karmashock]

Okay... I am going to assume you're reasonable and not a loony... and that what we have here is a failure to communicate.

Here is what I want... please throw out your existing notions of what is currently going on or whatever talking point score card you're reading from here...

1. It is reasonable for scientists in the pay of the public to be required at intervals to publish the results or at least what they were currently doing over the past few months or year or whatever interval is deemed reasonable.

2. Works thus published should be subjected to reasonable audits to detect fraud, laziness, waste, or incompetence.

3. The nature of audits should make it difficult or impossible for conflicts of interest to corrupt the auditing process.

4. The auditing process should be sufficient to determine what is and is not valid science.

5. Reproduction of work obviously cannot be done with all papers however, they should be done with all significant work deemed significant.

6. The deeming of significant or insignificant work could be down to collective or crowd sourced choices made by other scientists to cite a given work or say they found it interesting or significant. When X number of scientists say its significant then someone in the community should be tasked with verifying it through reproduction.

Do you have a problem with any of the above?

I do not claim my notions above are perfect and am open to modification. However, the basic gist of my post I think is defensible and if challenged, I will defend it.

Re:Wish I could say I was surprised

[Karmashock]

I think I answered this point in this post:

http://slashdot.org/comments.p...

In summary, my point is not a defense of any specific method of auditing work and ensuring people aren't just screwing around.

Rather, my point is a defense of auditing in general.

If you don't like publish or perish then please suggest an alternative that doesn't just let scientists wake up at the crack of 4pm, drink until they pass out, and then do the same tomorrow.

I'm not saying they would do that or they are doing it... I'm saying they are being paid and people being paid have an obligation to the people paying them to give reasonable assurances that they're not just screwing around on paid time.

Publish or Perish is one of the ways scientists show they're not dead weight. If you have other methods then I will of course be open to them.

I await your alternatives.

Re: Wish I could say I was surprised

[Anonymous Coward]

Publish I do not think that word means what you think it means.

They need a full peer reviewed paper in a recognized journal.
Not a progress report.

The system worked better in the past since there were less papers published. Now finding something unique is harder every year.

Re:Wish I could say I was surprised

[Anonymous Coward]

Scientifically useful negative results don't merely fail peer review, they are simply unpublishable in a major journal.

Re:Wish I could say I was surprised

[AK Marc]

If I proved that cats claws grow proportionally to zinc in their system (assuming otherwise healthy), up to zinc overdoses, it may be valid, presumably interesting, but I doubt that it would be picked up by a "major" paper. But if I proved that human IQ was proportional to the temperature of the blankets one slept under, that would be published many places and gain me much fame. Even if the results were faked and peers paid off.

The solution would seem to be the university publishing all the unpublished articles, peer reviewed by others in the same system. They'd be published, and if they can't pass a peer review, that should come out.

Re:Wish I could say I was surprised

[mwvdlee]

Which is why there is a growing movement of scientists who promote publishing failed research results.
Scientists, out of anybody, should know that failure is when you learn the most.

Re:Wish I could say I was surprised

[alvinrod]

It's not a matter of failing peer review, it's a general disinterest in publishing negative results. If you find a cure for cancer it's a big deal, but if you just found one more thing that doesn't work any better than a sugar pill, none of the journals are going to care about publishing it even if it's the most well-run study in the history of the world.

If someone starts doing some novel research that's going to take five years to possibly produce results and nothing pans out, they aren't going to get anyone to publish the findings.

Re:Wish I could say I was surprised

[guises]

The problem with "publish or perish" isn't the fact that scientists have to eventually share their results, it's the volume of publishing that's expected which gets in the way of actual work. When a scientist has a data set and the first thought is "How many papers can I get out of this?" it's an indication that something is wrong.

Re: Wish I could say I was surprised

[Karmashock]

A progress report TO the public that is audit-able is what we've always wanted.

Re:Wish I could say I was surprised

[Anonymous Coward]

1. It is reasonable for scientists in the pay of the public to be required at intervals to publish the results or at least what they were currently doing over the past few months or year or whatever interval is deemed reasonable.

The basic problem here is that you seem to have at best a shaky grasp of what "publish" means in academia. No journal is going to just publish "what they were currently doing over the past few months" unless that includes a significant result. I'm honestly not sure of where to recommend that you go to get a better understanding of how academic journals work, but I suppose the Wiki article couldn't be the worst place to start.

2. Works thus published should be subjected to reasonable audits to detect fraud, laziness, waste, or incompetence.

Using what money? Peer review currently is done on a voluntary basis; no journal that I'm aware of pays its reviewers. You seem to think that just because it would be nice if all published research was reproduced that it *should* be reproduced, without concern for where the researcher-time and money will come from to accomplish this. The current reality is that any professor or research scientist who devoted significant amounts of time to reproducing already-published science would quickly find himself out on his ass, because publishing *original* research is the first and foremost factor in maintaining/advancing a career in academia. To put it in software development terms, it would be like expecting a programmer to spend a large chunk of his time on the clock refactoring code while his bosses are telling him to leave it alone and work on implementing new features.

3. The nature of audits should make it difficult or impossible for conflicts of interest to corrupt the auditing process.

Agreed, the journal from this article should absolutely have done a better job of verifying the identities of its peer reviewers.

4. The auditing process should be sufficient to determine what is and is not valid science.

In what sense? If someone publishes a paper based on years of astronomical observations, is the peer reviewer obligated to spend years making his own observations to see if he can find (more or less) the same result? If "Yes", then the simple reality is that no one will volunteer to peer review such work, and you'll end up in a situation worse than the present one. If "No", then you're back to admitting that at some point the reviewer has to trust the article's author(s).

5. Reproduction of work obviously cannot be done with all papers however, they should be done with all significant work deemed significant.

I would argue that, certainly in my own field, "significant results" intrinsically draw more attention once published, and thus any mistake or malfeasance is more likely to be caught.

6. The deeming of significant or insignificant work could be down to collective or crowd sourced choices made by other scientists to cite a given work or say they found it interesting or significant. When X number of scientists say its significant then someone in the community should be tasked with verifying it through reproduction.

And, again, whence comes the money and time to reproduce it? This is a point that most scientists, I believe, would agree with -- but no one is going to sacrifice their own career to help accomplish it. And the "publish or perish" mentality contributes to this problem, as journals do not publish articles which simply say "yes, this other article seems to be good science".

Overall, I don't see anything too objectionable in what you want -- but it is basically a list of demands without any suggestion of how they could be accomplished or any understanding of why they are *not* being accomplished (to the extent that they aren't) in the current syst

Re:Wish I could say I was surprised

[Anonymous Coward]

It should be publish or die.

Life is hard. People fail all the time. Sometimes the best you can do is keep on trying.

Should everyone who has ever been in a car accident be permanently banned from driving? If a single student fails a class should the instructor be permanently banned from teaching? If someone commits a crime does that mean that all the police should all be fired for failing to prevent that one crime? If someone dies of cancer at a hospital should the hospital be shut down for failing to cure the patient? Should we no longer "support the troops" because Iraq is still a mess? Should we shut down governments because people are still living in poverty?

Sometimes the best you can do is buy the lottery ticket and hope - particularly when you're doing real science - exploring the unknown in the hope of doing something that no one has ever been able to do in the history of the planet.

So how do you get accountability in something as complex as science? I'd argue that there isn't a simple easy formula - fundamentally you have to build up communities where people know each other and that are united by a desire to advance scientific knowledge.

Re:Wish I could say I was surprised

[Karmashock]

1. I'm not interested in being brow beaten by some fool more interested in winning an argument then in addressing the argument.

If you're going to keep attempting an ad hominem then I'm going to simply not talk to you. And then what will you have accomplished?

This is the internet... be nice or you're getting a flame war.

2. Scientists are paid. You missed the whole point about not assuming that i am defending the current system but rather talking about the problem that the current system was set up to deal with and that will need to be dealt with if we get rid of the current system.

The point is that money goes to scientists and the people that provide that money have a right to expect something be done with it.

Furthermore, they must share information... you don't like the term publish or you're going to get asinine on the issue? fine... We'll start using other words. I'll speak chinese if I need to get you to stop trying to make this a semantics debate.

3. I'm not talking about any journal in particular. They're all suspect because there's no uniform way of doing this and the journals themselves are not audited which is another problem.

4. I specifically said that if it were not practical then the paper gets a big red asterisk next to it's name that says "unverified" or "not reproduced" or whatever.

By all means... put out as much research as you want that no one could possibly verify or reproduce. Make my fucking day. But it gets the red asterisk.

Peers will take that in stride because it won't be that uncommon. But laymen will at least understand what has and has not be verified. That is important. Science cannot be something only scientists understand any more then the law can be something only lawyers understand.

You do that and you create a situation where everyone has to walk around telling each other to "trust" them... and guess what... humans don't work that way.

I don't trust the guy at the bank when he says "oh this home loan is the best for you... I swear"... I don't trust the lawyer that says "oh this is a good contract, sign it"... I don't trust the doctor that says "you don't need a second opinion, get this operation."

And I'm certainly not going to let the scientists get away with a similar argument.

its not acceptable.

So things need to be structured in a reasonable way so that scientists find the process reasonable, those providing funds know that their money isn't being wasted, and the public can use the resulting science without having to just take on faith something some guy said.

doubtless you're going to try and argue that we have to trust them because its just too complicated. I'm not interested in that discussion. My opinion is no. End of that tangent.

6. As to the money to reproduce it, that can be provided by the same institutions that hire the scientists in the first place as part of their quality control policy. Which is in large part what all of this in the first place.

Would that money go to the same scientist or the same type of scientists? Probably not. We might have specialists that ONLY reproduce other people's work. That might be literally all they do. And they might be paid by the scientists that produced the paper who are themselves taking the money from their grants or working budgets as a cost of publishing.

here you'll tell me they don't have enough money to do that... well obviously not because they didn't need to do that before so they weren't given the money to do that.

I'm anticipating the circular arguments from you because you demonstrated several of them above and its making my head throb.

Re:Wish I could say I was surprised

[Karmashock]

Yeah that's what I was advocating captain strawman... everyone that ever stumbles should be shot in the face with a chainsawgun.

*yawn*

Either read what people write before coming up with a opinion about it or don't press send.

Re:Wish I could say I was surprised

[Anonymous Coward]

Stop trying to find technical solutions to social problems, you dullard.

If you don't like publish or perish then please suggest an alternative that doesn't just let scientists wake up at the crack of 4pm, drink until they pass out, and then do the same tomorrow.

We start with a society which teaches good values from a young age and which ostracises those who neglect those values. You go to America and you're way more likely to find people trying the minimum for the maximum than in another culture like, say, Japan, where laziness brings shame but hard work brings celebration and honour.

A group of good scientists knows who the bag eggs are. They tend to self-regulate as long as they're treated well by their own superiors. If they don't, they'll close ranks and protect each other regardless of competence. That's how labour always organises itself: for efficiency, if well-managed; for mere self-preservation, if harmed.

Re:Wish I could say I was surprised

[Maow]

1. I'm not interested in being brow beaten by some fool more interested in winning an argument then in addressing the argument.

If you're going to keep attempting an ad hominem then I'm going to simply not talk to you. And then what will you have accomplished? ...you're going to get asinine...

Jeez, pot meet kettle.

To top it off, he addressed your points quite well and it appears that it's you that seems intent upon winning an argument with your long-winded reply, which, of course, doesn't specifically and concretely address the issues raised by the person you're replying to.

Funding to reproduce coming from same institution? So they'll have half the money for original research then. And the suckers tasked with the reproduction won't be advancing their own careers under the Publish (original, ground breaking work) Or Perish model used today.

Like it was stated, in a fairly appropriate analogy, reproducing others' work is akin to re-writing a new software project - in software dev, it's a losing game.

In science it's important, but like in software dev, the boss isn't interested. And while the result may be beneficial, it's hard to convince people that it's a rewarding career move to play catch-up to others' work.

Having said all that, I think we all agree that reproducibility is important -- question is, how to go about it as the current system kinda disfavours it in all but the most important projects.

We need to implement specific, concrete changes -- having grad students do some of that is a good idea, but not sure if it'll completely solve the issue.

But laymen will at least understand what has and has not be verified. That is important. Science cannot be something only scientists understand any more then the law can be something only lawyers understand.

Laymen will never understand cutting edge science (unless they're quite keen on the topic at hand - a miniscule minority), and any layman that thinks they understand the law as well as lawyers generally get their arses handed to them should they attempt pro se representation.

Specialization in complex fields is natural.

Re:Wish I could say I was surprised

[Rich0]

And this is part of why all the drug development work ends up happening in private industry.

A scientist will come up with a molecule that inhibits some enzyme and get some publishable result. At that point they issue the typical "possible cure for cancer" press release and move on to the next thing. 5 years and $10M later a pharma company figures out that it causes heart valve degeneration or that inhibiting the enzyme isn't the magic bullet everybody hoped for. They don't bother publishing it, but none of their scientists get paid by the publication anyway. The companies interest is that if it eventually works out they make billions.

So, in that sense you actually have an example of a way in which industrial research is actually less risk-averse than academia, which should be shocking.

That said, when it comes to the basic research side of things pharma companies do tend to let the academics do the work for them.

Re:Wish I could say I was surprised

[Karmashock]

So until your utopian society comes along I should just accept bullshit?

No. When you hire people, you set up mechanisms to monitor their work and if they're not doing their jobs you fire them.

Otherwise why am I paying these people?

Same deal in pretty much everything.

Your solution is to completely re-engineer the entire society to correct this one issue.

Great plan... totally practical...

You call me a dullard? Do you have a plan that isn't completely halfbaked and impractical or is that all you're good for?

Re:Wish I could say I was surprised

[Anonymous Coward]

... everyone that ever stumbles should be shot in the face with a chainsawgun.

A non-trivial hypothesis in science can easily take 5-10 years to explore. When you start you have no idea whether the hypothesis is correct - which is the whole point of science: exploring the unknown. But if the hypothesis turns out to be wrong that's a 10 year "stumble". In the current environment it's very hard to publish a failure. And if you don't publish then it can be hard to feed your family - not quite as bad as a chainsaw gun but pretty scary.

You want to live in a simple world where accountability is easy - like counting output on an assembly line. But you just really don't get just how hard science is. I mean, modern scientists are trying discover things that even Einstien couldn't discover but all they get is simple-minded complaining about accountability.

Re:Wish I could say I was surprised

[Rockoon]

I was quoting someone else... please correct them instead... *yawn*

So you plagiarize too?

Re:Wish I could say I was surprised

[Karmashock]

I swear to god... were are there so many fucking illiterate people commenting on this thread?

I already said... and at this point it is four times... "if its impractical to reproduce the research then it gets a red asterisk to a little disclaimer that says "no reproduced"."

Read, motherfucker.

Re:Wish I could say I was surprised

[serviscope_minor]

Is something special about science?

If you pay someone to do a job and they don't do it... you fire them. That's it, no worrying about crime and so on. If they're really contracting, then at worst it's a breach of contract and you could sue to get your money back.

Re:Wish I could say I was surprised

[Karmashock]

That's fine so long as their fraud doesn't have additional damages to the institutions that employ them.

It is in the interest of such organizations to be harsh with people that take their coin and then try to cheat them.

Re:Wish I could say I was surprised

[PvtVoid]

.It is reasonable for scientists in the pay of the public to be required at intervals to publish tts or at least what they were currently doing over the past few months or year or whatever interval is deemed reasonable.

Dear NEJM: For the last six months, I mostly sat around in my office, read Slashdot, and mined Bitcoin on the cluster.

Re:Wish I could say I was surprised

[serviscope_minor]

Well, when it comes to doing browbeating, you're doing a bang up job. You are assuming that your solution is right and will not accept any criticism of it no matter how much of a bad idea it is. Your solution is unfortunately unworkable.

Is the current system perfect? No, not even slightly? Is your solution actually a solution? No, yours is a cure worse than the disease, or at least unworkable.

The point is that money goes to scientists and the people that provide that money have a right to expect something be done with it.

That's what publish or perish is! The money is largely provided by the public and not unreasonably, they want to see that the money is being used. Spending that money going in circles replicating results for the sake of it doesn't yield much of use, sadly. So, you'll have to convince people that it's a good thing that now (say) half the amount of research is getting done.

Furthermore, they must share information... you don't like the term publish or you're going to get asinine on the issue? fine... We'll start using other words. I'll speak chinese if I need to get you to stop trying to make this a semantics debate.

The point about publishing is that as soon as you have a result worth sharing, you share it. In fact if anything, publish or perish encourages that. The opposite: dumping out stuff because there's a reporting deadline does actually happen. For example any EC funded projects of which there are many hav mandatory reporting deliverables. I think a few of the RCUK bunch do as well.

Guess what? No one ever reads the damn things, mostly because any results worth sharing are published as papers. So in fact what you advocate does happen and is demonstrably useless. You can in fact go and request copies of these documents if you wish. I believe they are a matter of public record.

By all means... put out as much research as you want that no one could possibly verify or reproduce. Make my fucking day. But it gets the red asterisk.

I don't see what function that would serve. All research is already considered to have a big red asterisk by default. You might weigh the liklihood of correctness by the content of the paper, the believability of the result and the track record of the researchers, but new publications are generally taken to be unverified.

So your red asterisk would not serve any purpose.

You claim you want to make it easier for "laymen" but the red asterisk won't help, because it's already effectively there. The best thing you can do is stump up the money to educate them instead.

6. As to the money to reproduce it, that can be provided by the same institutions that hire the scientists in the first place as part of their quality control policy. Which is in large part what all of this in the first place.

And where does that money come from? Who wants to double the research costs of universities just to do what's already being done by a less formal method? Whether it's privately or publicly run, the people stumping up the hard cash are going to want to know why the output has halved or the cost has doubled.

Would that money go to the same scientist or the same type of scientists? Probably not. We might have specialists that ONLY reproduce other people's work. That might be literally all they do. And they might be paid by the scientists that produced the paper who are themselves taking the money from their grants or working budgets as a cost of publishing.

Well that would shoot a huge amount of research in the foot. I've published a fair chunk. All of my most major publications were done either on the side with no budget at all or in a small research group that had enough money for 3 PhD students (and this was way back when we were 10k per year) and almost no equipment (seriously we were on hand-me-down computers that the better funded groups junked due to obsolescence).

Come to think of it quite a lot of people I know got their big breaks by looki

Re:Wish I could say I was surprised

[Karmashock]

Sure, referencing the person you're talking to in a discussion on the internet is plagiarism... if you have no brain at all.

Re:Wish I could say I was surprised

[Karmashock]

Actually I said repeatedly that I'd accept criticism. However, not reading my post before commenting, taking me out of context, using various straw man arguments, etc is not constructive or valid.

So yeah... i'm going to flame, turn it to ash, and crush the ashes under my boot.

No mercy. No hesitation. No remorse.

As to the point of red asterisk, I pointed out that this was mostly for the lay community that gets a lot of their science news from the media that is full of a lot of people that don't know any better.

You need to help these people out by making clear whether given findings were reproduced or not.

What I think you'll find is that in some cases the media if they have that point will fixate on it and basically undermine the finding until it is reproduced.

And then the scientists if only out of irritation because the lay community that ultimately pays for everything keeps bring it up... will have it reproduced somewhere thus removing the red asterisk and moving on.

I'm not asking for every little thing to be reproduced. I just think its reasonable for the lay community to not get so easily duped by bad science.

As to it already "effectively" being there... but its not. The lay community is generally totally clueless as to whether a given finding has been verified or not.

So effectively it isn't there FOR THEM.

As to not having money... then don't do it and put the asterisk on your paper. Write the whole thing in crayon on napkins and mail it to your mother with carrier pidgins. Post it as a rhyming blog on facebook... Whatever floats your personal boat.

My point was that if you want to avoid confusion you should hold to certain standards. But who needs them, am I right? Lets just do whatever the hell because its just too expensive.

No really... do that. But then put a disclaimer somewhere... In crayon if that's all you've got... and that way when it hits the media the poor journalists that don't know anything can have a chance at not overstating things.

Here you say but all of that is redundant... its obvious... except it isn't for the laymen. So it isn't redundant. It isn't obvious. Put it on the paper.

If you want to scrawl that in crayon... go for it... just put it there. I'm sure Crayola has a wide selection of red crayons to choose from.

As to money and taxes... you do realize that most of your funding problems stem from the public and politicians not seeing tangible results... right? If you told the people, give us X dollars and we'll produce research that will yield everyone X*100 then you'd get all the money. ALL OF IT.

This is a big factor in a lot of spending. Now you can't ever make those sorts of promises. I appreciate that. But giving people better reporting that is understood and can be turned toward something practical means your funding will flow a lot easier.

Here you're going to tell me it doesn't work that way or I don't understand or something along those lines. Well, that's circular logic. You can always say that isn't how it works. If I advocated a republic type government 4000 years ago you could sit there and say "you don't understand, we have these peasants, and these nobles, and these priests... these guys rule everything and... etc"... I know that. You can do things that way forever if that's what you want.

But then don't bitch when the funding gets tight because the "trust us" argument is only good for limited funding.

If you want the money to flow... you have to give us something more. You have to make us understand.

Actually. Not stupid condescending cartoons. We're not stupid. We're not children. There are a lot of things laymen understand about a lot of things that scientists of whatever description know nothing about.

Lets not treat each other like garbage and instead do our best to help each other come to a common understanding and from that move forward together.

Re:Wish I could say I was surprised

[TheRaven64]

Reproducing work is often a good thing to set for first-year PhD students to do. If they reproduce something successfully, then they've learned about the state of the art and are in a good position to start original research. If they can't reproduce it, then they've got a paper for one of the debunking workshops that are increasingly attached to major conferences and that's their first publication done...

Re:Wish I could say I was surprised

[shrewdsheep]

Wrong. The issue is that publishing is considered sufficient.

It should be publish or die. How do you know they're doing anything if they don't publish? they could be reading slashdot all day for all you know otherwise.

FTFY

But as is made clear here, simply publishing and getting it through peer review is clearly not good enough. We need to increase what they have to do to avoid this situation.

For example... maybe one scientist pays another scientist to reproduce his work.

Maybe you have big collections of graduate students that as part of their process of getting a degree get assigned some random papers submitted by scientists in their field and they have to reproduce the work.

Obviously this isn't always possible... but whenever it isn't possible that needs to be put as a giant red asterisk on the paper saying "this work has not been reproduced"...

Do that and you're not going to get as much fraud or laziness.

I think that there is a common misunderstanding about the function of a publication. First and foremost it is a progress report of the scientist. This creates a lot of published noise - no doubt - OTOH it creates something that can be measured. This is absolutely critical to keep the scientific circus running (in a positive way). There are different ways to measure quality (which journal, reading an abstract, reading an article, asking by email) and scientific progress/quality is somewhat orthogonal to the publishing process. If you want to be sure of something be sure you have your act together to judge publications. The system can be gamed but it is not a problem in itself.

Re:Wish I could say I was surprised

[TheRaven64]

In the UK, university research departments are assessed base on the Research Excellence Framework (REF, formerly the Research Assessment Exercise [RAE]). Each faculty member is required to submit 4 things demonstrating impact. These are typically top-tier conference or journal papers, but can also be artefacts or examples of successful technology transfer. The exercise happens every four years, so to get the top ranking you need to write one good paper a year. The only incentive for publishing in second-tier venues is meeting other people who might lead to interesting collaborations.

Re:Wish I could say I was surprised

[serviscope_minor]

Here is what I want...

OK, you can want what you like. Hoewver for it to be a practical solution it has to be better than what we currently have and if more expensive, then the money has to be raised from somewhere. It also has to produce usefult results. I do not think your solution meets those criteria.

Therefore I think that what you want is counterproductive.

1. It is reasonable for scientists in the pay of the public to be required at intervals to publish the results or at least what they were currently doing over the past few months or year or whatever interval is deemed reasonable.

Reasonable? Yeah I guess. Pointless? You bet. The EC and RCUK already require this. The documents disappear into a black hole of woe never to be read again. Have you ever requested one? I believe they are a matter of public record and if not, a FOI request would surely work.

Most of what's worth writing already gets put into papers. Not everything (sometimes we abandon papers which ae too hard to get through review or for lack of time), but some of them wind up on arXiv and whatever anyway.

I do not know if it is a coincidence that the EC funding has the strongest reporting requirements and the smallest output pre unit of currency invested.

2. Works thus published should be subjected to reasonable audits to detect fraud, laziness, waste, or incompetence.

Audits are already done on the spending by some of the major funding councils, such as the EC Framework grants. If you meet the deliverables you agreed with the money they gave you're fine.

Laziness? Well, if people are lazy they don't have much to publish. That's easy enough to find by looking at research output.

Fraud is much harder. How do you propose to do these audits?

Incompetence is generally covered reasonably well by peer review. It's not perfect and doesn't have 100% success, but no system is ever prefect.

3. The nature of audits should make it difficult or impossible for conflicts of interest to corrupt the auditing process.

Well, that's just wishful thinking. I don't think it is possible to design an auditing process for use by humans which is corruprion free. Never mind in science just look around at all the other auditing that goes on. Sure you can want this if you like but that doesn't make it possible never mind practical.

Unless you can propose a practical solution for such audits...?

4. The auditing process should be sufficient to determine what is and is not valid science.

5. Reproduction of work obviously cannot be done with all papers however, they should be done with all significant work deemed significant.

6. The deeming of significant or insignificant work could be down to collective or crowd sourced choices made by other scientists to cite a given work or say they found it interesting or significant. When X number of scientists say its significant then someone in the community should be tasked with verifying it through reproduction.

You know this is already how it works right, except for a minor change in the last sentance? If work is deemed significant and interesting, then others will try to build on it. To do that they will naturally replicate it. There's no formal process, but it is nonetheless what happens. No one gets "tasked", they just do it anyway.

And really significant results do attract interest. High temperature superconductivity. Gigantic magnetoresistance. Cold fusion. GFP transfection. Treating cells with acid to gt stem cells.

All of those examples were really interestind and had huge potential for lots of interesting new work. As a result they got a lot of attention which hadthe effect of determining correctness or not. They're just some of the more well known example. In any field there are examples of the same sort of thing. Papers which are significant attract attention because they point the way forward (or not).

Do you have a problem with any of the above?

1 already happens to some extent and is generally pointeless. 2 and 3 are unworkable. 4, 5 and 6 already happen in an informal manner.

Re:Wish I could say I was surprised

[Anonymous Coward]

like, say, Japan, where laziness brings shame but hard work brings celebration and honour.

Seriously? Japan? Because what we say happening in Fukushima shows we should trust Japan to do the right thing... it was a botched thing from top to bottom. Both at the highest levels in the company masking all the crap that was happening, and at the low levels with all the workers who, must have known things were going spectacularly wrong, who simply said nothing. Honor you say... I don't think you know what that means, or the Japanese don't, at least.

And at least in academia what I've heard from Japan was that indeed hard work is appreciated. So, people come to the lab very early and leave very late, but few of those hours are indeed of any decent productivity. Wake up man, Japan isn't what you think it is.

Re:Wish I could say I was surprised

[hxnwix]

Look at the moderation your posts have received. Nobody likes you; nobody thinks you're right.

Re:Wish I could say I was surprised

[Anonymous Coward]

I think what the poster you quoted wanted to say is that often to make major contributions you have to do something that has never been done before, and not just follow up on previous research. Pushing on current trends is not difficult, at all, and is basically guaranteed to get you a publication in a decent journal. A lab head can do several dozens of these papers a year if he has a few handfuls of people in his group and decides to have his focus on this. Now doing this more than guarantees a comfortable living as an academic. Quite often this research can even be wrong: It's middling at best, and nobody really cares, so nobody will notice (and yes, the STAP scandal with Obokata et al. is really a special case and not what usually happens; their claims were "too" interesting to the general public and in any case outright fraudulent). Many fields are saturated by this type of managers (rather than scientists), and they have been rewarded for their noninnovative research for so long I doubt they would even recognize a basic flaw in a paper when they saw one.

This means that peer review has become useless (when your peers are managers rather than scientists) and in fact every month I spot papers from my field in some of the top journals that have zero scientific contribution: their methods are only borderline correct, and the conclusions known for decades. But they have nice pictures and peer reviewers are probably their manager-friends or manager-somebodyelses who did not have a clue what was done and well-known 50 years ago (and indeed, why care, if by ignoring old research you can accidentally redo them and get more papers!). Try publishing a paper showing that their experiment must be wrong as it violates the second law of thermodynamics and you will be shot down and now they know your name. Good luck with grants and peer reviews.

I got a bit derailed above, but no, I am not bitter nor is the above a completely accurate presentation of my personal experience. This said, it is obvious that many scientists are afraid of speaking their mind and criticizing others even when others are wrong, and that this is corrupting the entire system where one is supposed to be able to trust one's peers.

Back to the topic: Coming up with a totally new idea, trying it, and failing at it will never get written up. You say that this is the right thing to do, if you don't publish, you ought to perish. Now is failed research "wrong"? Should you have known beforehand that your idea is stupid and not even test it? Not being able to publish this failed idea and only regarding publications as a measure of your success would certainly imply this.

Hack a Day publishes fails of the week. They are not meant as articles where we laugh at someone's stupidity or bad luck, but are informative writeups about new ideas where something in the implementation went wrong, or serve as examples of how even experienced people can fail to consider some basic (or advanced) principles. Related to this, perhaps my favourite TED talk is that by Eddie Obeng. He talks about business, not research. And I remind you that the only reason university research exists is that otherwise fundamental or high risk projects would not get funded as you might not have a direct way to make money off of them, or you might lose a lot, which makes them unattractive for business. Surely Obeng will then tell you that as a business manager, do the safe projects, punish those whose ideas don't work. Well this is what he says: "You're doing something new that nobody's done before, you get it completely wrong. How should you be treated? Well, free pizzas! You should be treated better than the people who succeed. It's called smart failure. Why? Because you can't put it on your CV." Companies can treat their employees with pizzas when they fail at something new, but academia is not a structured system where you could get different kinds of rewards: it is only about publish or perish. This is why it is a horrible system.

Re:Wish I could say I was surprised

[Ol Olsoc]

If you pay scientists to do science and they are contracted to do it... they fraudulently do not do science yet continue to cash your checks... that is a crime.

Then again, so is your fractured syntax.

Re:Wish I could say I was surprised

[Anonymous Coward]

I forgot to add this recent article to my post. It goes to show that the problems I am talking about are not just my personal anecdotes or limited to my field.

Re:Wish I could say I was surprised

[Anonymous Coward]

On the other hand, if we setup a system where pretty much all that matters is how many studies have been accepted, this kind of thing is not surprising.

Re:Wish I could say I was surprised

[Ol Olsoc]

Is something special about science?

If you pay someone to do a job and they don't do it... you fire them. That's it, no worrying about crime and so on. If they're really contracting, then at worst it's a breach of contract and you could sue to get your money back.

Agreed. One small point to add though is that the people they are working for put a lot of pressure on them to publish.

This is just stage one of a process. It's been known that the "publish or perish" culture has produced some problems. Self plagiarism is a biggie, but they are weeding out the fraud at the same time.

So while the more politically inclined might wring their hands and moan "we have to do something!" this is all happening because we are doing something The system is working. Tools are in place now to catch the bad boys and girls. Soon they'll know they can't plagiarize themselves, come up with bogus peer review, or doctor the data. It's good, what is happening.

It's a science solution. If it were a political solution, the fraudsters would just end up winning, and would become more powerful.

Re:Wish I could say I was surprised

[StripedCow]

It should be publish or die (...)

You might want to read this:
http://www.theguardian.com/sci...

Re:Wish I could say I was surprised

[Anonymous Coward]

The red asterisk idea is pretty lame. For interesting discoveries, rebroadcasters of the announcement wouldn't feel obligated to use it and would generate clickbait. Who decides what is impracticable to be reproduced, is it subjective? If the researcher thinks it could be easily reproduced as long as you rented xyz equipment that costs millions of dollar and a supercollider, he could drop the red asterisk even though to a layman that's ridiculously not reproducible. Would it be in a scientist's best interest to avoid a scarlet asterisk by subjectively saying his/her research is reproducible? What are the chances that grants would start saying you can't have a red-yellow star if you want funding?

I get the feeling you think scientists and associated institutions should eat the costs of this research being done twice by different teams. Pay the scientist $50,000 for research that has to be done twice and thus cost $100,000.

Re:Wish I could say I was surprised

[serviscope_minor]

However, not reading my post before commenting, taking me out of context, using various straw man arguments, etc is not constructive or valid.

I don't believe I have. I can only go on what you've written in each individual post. You seem to take offence awfully easily and jump straight from holding forth to flaming everyone who has criticised you. Come to think of it, now I've read the whole thread, there have been many carefully thoughtout critiques of your post and you've jumped straight to flaming and ad homenim there too.

As to the point of red asterisk, I pointed out that this was mostly for the lay community that gets a lot of their science news from the media that is full of a lot of people that don't know any better.

Yep, I got that, and if you took the time to read my reply you'd have read my reply, for starters.

I don't think it would help. The thing you need to do is educate people on what the red asterisk means, not just put it there. Once you've done that, you can simplyrealise that all work is de-facto red asterisked. That makes it redundant.

And then the scientists if only out of irritation because the lay community that ultimately pays for everything keeps bring it up... will have it reproduced somewhere thus removing the red asterisk and moving on.

The scientists do this anyway and not out of irritation. And instead of a red asterisk, you get a list of things citing the current article---most journals track that information. Of course you have to read the citations to see if they're reproducing it or simply padding a lit review.

But that information is already there. The trouble is you have to have the education to understand it. But you'd need that to understand the red asterisk as well.

Here you say but all of that is redundant... its obvious... except it isn't for the laymen. So it isn't redundant. It isn't obvious. Put it on the paper.

No it's not obvious but I think the red asterisk would be less obvious than you expected as well. Sometimes they'll get removed.

If you told the people, give us X dollars and we'll produce research that will yield everyone X*100 then you'd get all the money. ALL OF IT.

I doubt that: first, you can't tell which research results will pay off financially, or how much, or over what time scale. That's impossible to know. Secondly why would they want to give you money to tell them something they already believe they know?

This is a big factor in a lot of spending. Now you can't ever make those sorts of promises. I appreciate that. But giving people better reporting that is understood and can be turned toward something practical means your funding will flow a lot easier.

OK, so you pointed out my criticim with that already. I don't get it then. That's impossible. But I don't get how the red asterisk will solve the problem of turning science towards something practical. The people who do that are usually the scientists involved. They know if it works or not and understand enough about the field to beable to deduce red astrixes themselves.

But then don't bitch when the funding gets tight because the "trust us" argument is only good for limited funding.

And you accuse me of circular arguments? The mind boggles. Your solution to trust is to say "no honestly trust us, we've put a red star next to it". In terms of trust you're back to square 1.

If you want the money to flow... you have to give us something more. You have to make us understand.

You ask the impossible. No one can force someone to understand something.

Actually. Not stupid condescending cartoons. We're not stupid. We're not children. There are a lot of things laymen understand about a lot of things that scientists of whatever description know nothing about.

So your point is that people have specialisations and it's unrealistic to (for example) expect a scientists to understand the deep things that some laymen know about th

Re:Wish I could say I was surprised

[Anonymous Coward]

Your figure for how much cash a failed drug candidate can burn through in 5 years of clinical trials needs another zero or two, depending on which phase of development it fails during.

Re:Wish I could say I was surprised

[Anonymous Coward]

Failures are publishable. Knowing what doesn't work and why is valuable to future researchers, but your own research still needs to be of sufficient quality to be accepted.

Re:Wish I could say I was surprised

[ilsaloving]

I'll just leave this here... :)

I <3 Tom Lehrer.

https://www.youtube.com/watch?...

Re:Wish I could say I was surprised

[Karmashock]

Okay, why is your program acceptable when clearly we could do something even cheaper.

Why have journals at all? It would be cheaper not to have them.

Why use trained scientists? It would be cheaper not to have them. We could have everything done by robots or trained sea monkeys. What could possibly go wrong?

Look, the bullshit science is not acceptable. You want to talk about money? This shit means you get less money. It creates the impression that many of you are selling bullshit on a regular basis... do not respect the public... do not respect the institutions... do not respect your own profession... and lack all integrity.

And worse... it sends the impression that if you're doing all those things we have no ability to know one way or the other.

Now... under those conditions... how many zeros am I going to write on this check?

Please tell me how sad you are about science not getting properly funded while at the same time telling me you can't verify anything.

As others have noted, many science grants include requirements of status updates and information sharing as a requirement of the grant.

That's good. And it means there are already people thinking along the lines I've laid out.

And because I'm sure you're going to say you don't like one thing or another. That's fine. I offered a giant loophole... just say somewhere on the submission that you didn't do any of those things.

That's all. Say it hasn't been reproduced. Say it hasn't been vetted in detail. Just say that.

And then when the laypeople see it they know that everything hangs on the credibility of a couple scientists that could be some flavor of wrong.

Re:Wish I could say I was surprised

[Karmashock]

I'm often ranked pretty highly... my score is excellent and has been for a long time... does that mean anything to you or does this sort of thing only mean something when it is in your favor?

What is more, lets say I said something horrible... but lots of people up voted me... would you then conclude that my argument was good or that for some reason there were just a lot of sick people upvoting me?

Son, what you just threw at me was a logical fallacy. Look it up...

And it is ironic that in a discussion about science and integrity there in someone thinks a logical fallacy disproves or undermines my position.

Try again.

Re:Wish I could say I was surprised

[Karmashock]

I read it. Do you care what I thought about it or did you just want me to read it?

Re:Wish I could say I was surprised

[Anonymous Coward]

You are simply wrong. RNA-Seq experiments take $1000 of reagents to setup and $1000 to run on the sequencer whether you're the first to isolate this library or the second. Then you're paying a post-doc to redo the bioinformatics which will take almost the same amount of time as it did the original authors, maybe even more if your lab is not specialized in their analysis techniques. The cost is the same, and nothing is learned.

However, papers are checked indirectly. You take someone else result, you use it to advance theory, and you use this advanced theory for your own experimental design. If their results were wrong, you'd notice. That's science.

Re:Wish I could say I was surprised

[Karmashock]

But as you keep saying, the scientists already assume this stuff. So for them, the red asterisk is redundant.

My point is that it is not for the laymen. Telling them that something has or has not been reproduced is important information that either might not occur to them or they might assume one way or the other.

Its good for people to be careful with science that has not been reproduced. Laypeople clearly need some help with that.

If the works had a disclaimer on them when they were unverified it would make it less likely and less forgivable when a newspaper went off half cocked with a report.

This is a good thing.

Re:Wish I could say I was surprised

[Karmashock]

Again... this is probably the tenth time I've said this...

Read it, asshole.

If you don't want to reproduce the work. FINE. Don't have anything verified.

Hell, I don't even care if you get it peer reviewed. Have the whole thing written up by a bipolar crackhead for all I care.

All I want is that you specify all the things you have not done to back up your work.

if you've not been peer reviewed, say that.
if your work has not been reproduced, say that.
if the people that did the work were not actually scientists, say that.

etc.

That's the loophole I offered to the last ten people that have said variations of the same argument.

And I expect that an 11th or 12th person is going to do it as well... because who needs reading comprehension when you've got elbows and a keyboard. Just mash away and press send, am I right?

I am beyond tired of repeating myself on this issue...

You don't want to go through the expense of having your work reproduced or it isn't practical or whatever? Fine.

When its reported or publish or both... it will be said somewhere in there, "the work has yet to be verified"... and I am happy. I can then largely ignore the whole thing until it has been verified. Scientists can of course spend as much time as they like going over it... that's their profession. The general public has no reason to give a damn until the work has been verified. We shouldn't be troubled with anything less.

Re:Wish I could say I was surprised

[narcc]

No. When you hire people, you set up mechanisms to monitor their work

And the best mechanism you could come up with is "wait around a few years to see if their name shows up at the top of a journal article"?

Re:Wish I could say I was surprised

[golden age villain]

For example... maybe one scientist pays another scientist to reproduce his work. Maybe you have big collections of graduate students that as part of their process of getting a degree get assigned some random papers submitted by scientists in their field and they have to reproduce the work.

You don't work in science do you? Being paid for reproducing someone else's work means you are not producing anything original of your own. It doesn't advance your career. Then with respect to your second point, being a graduate student means to perform original research. If your PhD is about reproducing someone else's work, you won't be able to publish anything of significance.

The problem is the system globally: journals, which push for high impact sexy stories; promotion committees, which only look at how many high impact papers scientists have published and at how much funding they have attracted; and finally funding bodies, which only look at publications. If you are not lucky enough to get into a big lab which automatically publishes in high impact journals based on the labhead's reputation, the incentive to game the system is high. You just need to look at all the scandals that have come to light in the last years. You can even buy authorship on papers to which you have contributed nothing (http://www.sciencemag.org/content/342/6162/1035.summary).

Re:Wish I could say I was surprised

[narcc]

The auditing process should be sufficient to determine what is and is not valid science.

Do you know how I know you haven't heard of the demarcation problem?

Re:Wish I could say I was surprised

[narcc]

If the works had a disclaimer on them when they were unverified it would make it less likely and less forgivable when a newspaper went off half cocked with a report.

Journalists are reporting from press releases, not published articles.

Re:Wish I could say I was surprised

[narcc]

I presume he expected you to gain some insight from the article. It's a shame he didn't know that your beliefs are unshakable.

Re:Wish I could say I was surprised

[the gnat]

Alternatively (or in addition), we could increase the penalties for those caught cheating.

FYI, cheating like this is already a guaranteed career-ender. People who do things like this aren't rationally weighing the cost of getting caught against the career advancement that comes from publishing; they simply don't expect to get caught.

Re:Wish I could say I was surprised

[Anonymous Coward]

Good PR and marketing can overcome any science or the lack of science any day of the week. Look how well the specialty dog food industry is doing.

Re:Wish I could say I was surprised

[Anonymous Coward]

And this is part of why all the drug development work ends up happening in private industry.

You're joking, right?

Pharmaceutical Companies Spent 19 Times More On Self-Promotion Than Basic Research: Report

Pharmaceutical research and development: what do we get for all that money?

Why Pharma Needs the NIH: Basic Biology Drives the Industry, Says Genentech VP

I love research about research

Re: Wish I could say I was surprised

[HornWumpus]

The public doesn't know enough to read the progress report. Much less would they have the time to read all of them.

Re:Wish I could say I was surprised

[Anonymous Coward]

Are you 12 or just naturally obnoxious?

Re:Wish I could say I was surprised

[Anonymous Coward]

Relying on a set of uniform morals or values for a system to work is to design a system built to collapse.

Design a system which recognizes and accounts for human behavior and the pursuit of self-interest that that entails.

Re:Wish I could say I was surprised

[interkin3tic]

I'd argue that a lot of scientists are actually paid to bring in grants, teach, and publish. It's just a lucky coinicidence that doing science is helpful to at least two of those goals.

Re:Wish I could say I was surprised

[interkin3tic]

That's true for large clinical trials, but clinical trials aren't all or even most research. For basic research like what this journal seems to publish on, no. It's rare that you'd have an experiment which would take months, let alone 5 years, and it would only be at the very end that you'd get a yes or a no.

For example, a study in that journal is entitled "Predicting blast-induced ground vibration using general regression neural network." The abstract is

Blasting is still an economical and viable method for rock excavation in mining and civil works projects. Ground vibration generated due to blasting is an undesirable phenomenon which is harmful for the nearby inhabitants and dwellings and should be prevented. In this study, an attempt has been made to predict the blast-induced ground vibration and frequency by incorporating rock properties, blast design and explosive parameters using the general regression neural network (GRNN) technique. To validate this methodology, the predictions obtained were compared with those obtained using the artificial neural network (ANN) model as well as by multivariate regression analysis (MVRA). Among all the methods, GRNN provides excellent predictions with a high degree of correlation.

Emphasis mine: they're testing if they can predict how the ground will shake after an explosion.

They're not going to spend five years dreaming up a model for it completely on a dry-erase board, set off a stick of dynamite, realize they were wrong, and throw out five years of work. They get preliminary results that are positive and encourage the project to go forward. They make up a model based on data, then refine it with subsequent bombs. If one were completely unable to use the method they propose to figure out how the ground shakes during explosions, they'd likely find that out before months had gone by.

I don't know what the breakdown is between clinical trials and basic research. I'd bet there's more research money spent on clinical trials than basic research (necessary, given certain realities about clinical trials) but that there are more scientists in basic research. So I'd bet that most research out there is actually not really affected a whole lot by the negative results issue: if you get a negative result, you get it before you commit yourself to it, and you move your reserarch in a different direction.

Re:Wish I could say I was surprised

[Ol Olsoc]

Are you 12 or just naturally obnoxious?

Probably

Re:Wish I could say I was surprised

[bitingduck]

But as you keep saying, the scientists already assume this stuff. So for them, the red asterisk is redundant.

My point is that it is not for the laymen. Telling them that something has or has not been reproduced is important information that either might not occur to them or they might assume one way or the other.

Its good for people to be careful with science that has not been reproduced. Laypeople clearly need some help with that.

At risk of entering a mild flamewar...

You keep advocating a thing (your red asterisk) and people keep telling you it's already there. You're asking for the wrong thing. After reading a bunch of your replies, it sounds like what you really want is better science reporting. The red asterisk is (as described by many other commenters) already there. Actually putting a note on it that says "this is unverified *red star*" you add a whole bunch of problems without actually addressing the real issue. Problems that are added, among others, are: who decides when it comes off, if you find an old version of something with the scarlet letter on it, how do you find out definitively if it's been removed, what if parts of a paper are verified and parts aren't (the most common case), and many more that others have no doubt already mentioned. And on top of that, you're doing it for lay people who aren't going to look at the original literature anyway-- it's generally interpreted for lay audiences by "science writers" (where I use quotes because they may or may not know anything about science). Bad science writers will ignore the asterisk and go for sensationalism whether it's implicit (as it is now) or explicit (as you want). Good science writers will discuss the strengths and weaknesses of the research and probably point out where more work needs to be done and who's doing it. Improving the quality of science writing is something that is unfortunately more in the hands of popular publishers and their customers than it is in the hands of researchers-- if good science writing doesn't sell, it doesn't matter how big and red your asterisks are, they're going to be ignored.

Re:Wish I could say I was surprised

[bitingduck]

Reproducing work is often a good thing to set for first-year PhD students to do. If they reproduce something successfully, then they've learned about the state of the art and are in a good position to start original research. If they can't reproduce it, then they've got a paper for one of the debunking workshops that are increasingly attached to major conferences and that's their first publication done...

In many fields there's no way a first year PhD student is going to be able to reproduce anything, and the cost can be very prohibitive. If a lab doesn't already have the expensive infrastructure that's necessary and have it configured to replicate the desired experiment, then it's going to be $$$$ and take a long time. And then the expensive equipment is tied up doing work that's relatively low value (replication).

Re:Wish I could say I was surprised

[Anonymous Coward]

No scientists are not paid "by the publication" anywhere, I have no idea where you got that idea.

Drug development is not "less risk averse" than basic research, because they're already starting with an idea that works. The rate of success for drug development is something on the order of 25% (there are published studies on this, oddly enough). From my experience, the rate of success for basic research is in the single digits, simply because most ideas you come up with as a research are wrong, and you have to try a lot of wrong ideas to get one right one. You just don't see all the risks that don't pan out, but there are tons of them. Some take days, some take weeks, some take years to not work. You wrap it up and move on to the next thing. That's how science works, and that's why no one publishes negative results, because there are thousands of times more negative results than positive results. That said, in some fields, truly interesting negative results *do* get published. It's just that most negative results are not interesting.

Re:Wish I could say I was surprised

[Anonymous Coward]

It depends a lot on the field. In many cases, what you said is true, and it's certainly considered best practice in science to "fail early." But in other fields, it really does take years to figure out that what seemed like a very reasonable idea just doesn't work.

Re:Wish I could say I was surprised

[david_thornley]

I've never perceived scientists as all that well-paid, given their education and the amount of work that they appear to be put in. If somebody is smart enough to become a scientist, there's got to be more lucrative things they could do.

Re:Wish I could say I was surprised

[david_thornley]

But as is made clear here, simply publishing and getting it through peer review is clearly not good enough.

I thought they weren't good enough anyway, that a paper in a peer-reviewed journal didn't prove anything by itself. There's been plenty that turned out to be wrong.

After the publication, people are likely to want to build on that work (if it's interesting), and they'll wind up replicating parts of it. If it isn't solid, for whatever reasons, it'll get found then. If it holds up, there will soon be other corroborating papers.

It seems to me that things are working well here. Some people use underhanded techniques to get published, their papers are retracted when they're found out. The net result seems to be pretty much what you want.

Re:Wish I could say I was surprised

[david_thornley]

You're advocating one specific quantitative performance metric: the length of the publication list. If I can get into a realm I'm more familiar with, people have been trying for years to come up with quantitative metrics to tell how good programmers are. It doesn't work. First, any such metric I'm familiar with fails to reward the right things. Second, any such metric gets gamed.

What you want a scientist to do is come up with good, innovative, science. The publication list penalizes innovation, since innovations don't always pan out, and rewards scientists who only undertake things they're pretty sure will work. A scientist simply may not be able to afford to investigate something that will take time and might not produce a positive result.

In this case, we saw the system get gamed by outright cheating. There are ways to game it that don't involve actual cheating. I haven't seen anybody suffer from the "least publishable unit" approach, and I remember one paper that left me awestruck at the precision in which the authors pulled out one result from a research program and got a peer-reviewed journal to publish the paper. It was an interesting result, but it was being precisely 1.000 LPU.

In my brief journey through academia, it seemed that the pub list was used a whole lot. There are variations, such as the citation list and journal impact factors, but these are closely related to the list length. This isn't healthy.

We're talking about a whole lot of highly intelligent and creative people here. Why aren't there other measures? Why are you expecting a non-scientist to come up with one?

Re:Wish I could say I was surprised

[david_thornley]

Since we're talking about peer-reviewed journals here, all papers have been peer-reviewed. The papers I've seen have usually had some discussion on what's come before, so you can tell if it reproduces or relies on previous work. (For seeing if it's been reproduced later, use a citation index.) They also list the credentials of the authors. I don't know what your problem is there.

However, these papers aren't progress reports, and aren't meant to be. They are findings of sufficient significance that it's worth keeping them around for later reference. This means that, if you don't get a significant result, you don't get published. If you get a lot of barely significant results, you get a big publication list.

This is a fine method to advance science, but it is also used to judge scientists, and that's where the problem lies. You think scientists should submit progress reports, which I think everybody agrees with, but even useful progress reports don't help the scientist's career unless they result in a finding worthy of a peer-reviewed publication. The difference between these things is where I think you're wrong.

Re:Wish I could say I was surprised

[thunderclap]

Wrong. The issue is that publishing is considered sufficient.

It should be publish or die. How do you know they're doing anything if they don't publish? they could be watching tv all day for all you know otherwise.

But as is made clear here, simply publishing and getting it through peer review is clearly not good enough. We need to increase what they have to do to avoid this situation.

For example... maybe one scientist pays another scientist to reproduce his work.

Maybe you have big collections of graduate students that as part of their process of getting a degree get assigned some random papers submitted by scientists in their field and they have to reproduce the work.

Obviously this isn't always possible... but whenever it isn't possible that needs to be put as a giant red asterisk on the paper saying "this work has not been reproduced"...

Do that and you're not going to get as much fraud or laziness.

I agree with this for one reason. It would finally dispense with the climate change insanity. Since its actually near impossible to get data that is verifiable about climate older than 50yrs that isn't tainted or limited and subject to interpretation(like core samples or tree rings), we could finally put al gore were he belongs. Out of sight and mind. Each day passing I am more thankful that the supreme court gave the victory to Bush. I only wish it could have happened in 2012 for Romney.

Re:Wish I could say I was surprised

[Karmashock]

the point is to make whether something was or was not reproduced/verified so obvious and easily obtainable and implicit in the report itself that any science writer that fails to note it will be grammar nazied into oblivion by his science writer peers.

Look, what you're saying is that its hopeless, there is nothing we can do, and we should just accept the status quo.

I refuse to do that. We didn't get as far as we have as a species by giving that mentality and respect... and I give it none.

I appreciate that the problem we're dealing with here might be a complicated one... that's fine. Then we can come up with solutions that are functional either because they interact with its complexity or because they bypass it via some elegant loophole.

Re:Wish I could say I was surprised

[bitingduck]

Look, what you're saying is that its hopeless, there is nothing we can do, and we should just accept the status quo.

No, people are saying that your proposed method of doing this won't work for reasons that have been reasonably well explained.

You'd be better off hiring teams of underemployed post-docs to beat science reporters senseless if they don't report science accurately. It would cost less, be more effective, and do a better job of addressing both the science reporting process and the employment issues in some fields.

 

Re:Wish I could say I was surprised

[Karmashock]

Absent constructive counter proposals you are advocating the status quo.

Do you have constructive counter proposals or are you advocating the status quo?

Re:Wish I could say I was surprised

[Anonymous Coward]

Its a waste of time. worse, you've got nothing original or novel to bring to the discussion. Why would anyone find your comment to be even a little worth reading? Its got nothing.

Spoken like a publisher.

Re:Wish I could say I was surprised

[Karmashock]

how is that an insult?... and what have you done here but waste my time with it?

Again... nothing novel or interesting... your post has no reason to exist.

Re:Wish I could say I was surprised

[serviscope_minor]

Okay, why is your program acceptable when clearly we could do something even cheaper.

You hadn't adovcated anything on the grounds of cost until now. Everything below seems like a new suggestion.

Why have journals at all? It would be cheaper not to have them.

Well that could work, except in High Energy Physics, where, they're mostly free, open, volunteer run and the papers are available at no cost. Seriously, HEP has had its shit together for years, making use of the web. Not surprising seeing as HEP was an offshoot or CERN, but it's funny that their online life is so much better than engineering or computer science.

So it goes.

Look, the bullshit science is not acceptable. You want to talk about money? This shit means you get less money. It creates the impression that many of you are selling bullshit on a regular basis... do not respect the public... do not respect the institutions... do not respect your own profession... and lack all integrity.

I don't see how having another set of scientists verifying the work would help. If there's a lack of trust, then having two untrustworthy people tell you the exact same thing won't help. And does this mean we get less money?

Apparently scientists are considered some of the most trustworthy professionals:

http://www.gponline.com/poll-r...

So, I'm not convinced teh trust gap exists. And as for money, is it costing us? I don't know. It's easy to claim either way.

Please tell me how sad you are about science not getting properly funded while at the same time telling me you can't verify anything.

Well, now you're setting up another straw man, or you simply never bothered to read my post, something I note that you accused me of. The interesting, world changing stuff already gets verified just fine using the current informal system. I don't think that formalising it would help.

As others have noted, many science grants include requirements of status updates and information sharing as a requirement of the grant.

That was me.

That's good. And it means there are already people thinking along the lines I've laid out.

You missed the part where the strictest requirements also correlate well with the lowest research output per euro invested. You also missed the bit where basically no one ever gives a flying fuck and I've never, ever heard of one of these documents being read outside of a project review meeting.

And then when the laypeople see it they know that everything hangs on the credibility of a couple scientists that could be some flavor of wrong.

You are trying to provide a solution for lack of education where the solution is in fact education.

Re:Wish I could say I was surprised

[Areyoukiddingme]

...reproducing others' work is akin to re-writing a new software project - in software dev, it's a losing game.

Is it? Because this happens all the time, for both commercial and open source software. Especially in open source. Somebody gets a bug up their butt and decides to reimplement, from scratch, a duplicate of some perfectly usable, functional software. Because they didn't like the language it was written in. Or they didn't like the style it was written in. They want a functional implementation, not object oriented. Or whatever. It's rampant.

The analog when it comes to scientific studies would be to reproduce a result not to follow a procedure. Any scientist worthy of the chair he's holding down should have a deep enough understanding of his field to come up with a way to reproduce a particular result using a different approach. I think this should serve both purposes: it validates the result, and the process followed, being different, should qualify for publication in its own right. It should be obvious that such an approach is considerably more valuable than "do the same thing over again."

If that isn't possible, either scientists aren't thinking creatively enough, or the analogy is actually fairly poor.

Re:Wish I could say I was surprised

[bitingduck]

If you read my last post, I suggested something likely to cost less and be more effective.

Better science education in general is the only real solution. I was just listening to NPR a few hours ago and they had their staff take a "mental status exam" that consisted of spelling "world" backward and counting backward from 100 by 7's. These are educated people, probably mostly with degrees in journalism. Most of them could spell world backward. None of them could count backward from 100 by 7's. You can put all the asterisks you want on science publications, but if the reporters can't even be counted on to be able to subtract 7 from any number less than 100, you're not going to get more reliable reporting of results in the press. I've had reporter friends who were pretty smart try to describe scientific concepts and get them *completely* wrong, and they had to get a short tutorial to get it right. Education of reporters is a better answer, not adding a bunch of bookkeeping.

Your proposal is fraught with difficulties that will be expensive to resolve and which won't provide any information better than the status quo. Currently, *all* scientific publications are tentative. They're generally peer reviewed for gross errors, but the assumption is that the authors are being honest. Only in cases of wildly sexy possible breakthroughs are experiments directly repeated. More commonly, other workers in the field do a bunch of related work, some of which depends on what was reported in other papers. They reference that work. Stuff that gets referenced a lot and is shown to be useful ends up in textbooks as "currently accepted science". If you want to know what's currently accepted, get a recent textbook. Stuff that isn't mostly disappears, except to a narrow group of people who are in the same field, and is maybe occasionally summarized in a review paper if it's useful. If it's useless or wrong, it gets ignored for eternity. No additional bookkeeping necessary, and it's all crowdsourced. With your proposal, somebody has to go back and find all the damn asterisks and erase them. And say "hmm, part of what was in this paper was verified, and part wasn't. what do I do about the asterisk?" And who do you get to do that? Are they qualified? Most scientists are already too busy scrambling for money to be willing to spend more (probably unpaid) time expanding bureaucracy.

Re:Wish I could say I was surprised

[niftymitch]

Wrong. [...] It should be publish or die.

I belive the phrase you're looking for is "publish or perish."

Either gets the point across.

I would like to see some data that outlines the potential
number of authors and the potential number of papers
as limited by page count.

It seems to me that this is a rigged game with rules
drawn from childhood agony playing musical chairs only
to the extreme.

With the modern internet page count is no longer the issue
but it is because that is how the game rules are written.

Qualified reviewers are few and far between as science,
literature, history and all of the academic world have carved
thing up into such fine narrow specialized fields that only
one researcher in the universe has any knowledge of the
topic.

Compound that by the rampant insertion of tenured staff names
in the author list of all papers coming out of institutions that
new science is all done by Mr Et Al.

The only process in the US that comes close to this foolishness is the process
in place for US patents where the contents of a whiteboard can be edited never
implemented and turned into a process patent. There is however overlap
where the whiteboard might be a class project or lecture note taken off line
and refactored into something apparently new but stolen outright.

Consider that if you are in a design meeting, and make a suggestion and
are not later credited as an inventor you are the victim of intellectual and
professional theft. Keep a notebook....

Re: Wish I could say I was surprised

[cwsumner]

The public doesn't know enough to read the progress report. Much less would they have the time to read all of them.

An individual, chosen at random, probably doesn't know enough.
But "the public" is a -lot- of people, and some of them are smarter than you or I.

Re:Wish I could say I was surprised

[Rich0]

Maybe another zero. A single drug candidate doesn't have to be all that expensive. The problem is that there are so many of them for each one that succeeds.

Re:Wish I could say I was surprised

[Rich0]

And this is part of why all the drug development work ends up happening in private industry.

You're joking, right?

Pharmaceutical Companies Spent 19 Times More On Self-Promotion Than Basic Research: Report

Has nothing to do with the relative spending of academia vs industry on development.

Pharmaceutical research and development: what do we get for all that money?

Also has nothing to do with the relative spending of academia vs industry on development.

Why Pharma Needs the NIH: Basic Biology Drives the Industry, Says Genentech VP

This is about basic research, not development. As I said in my post, "That said, when it comes to the basic research side of things pharma companies do tend to let the academics do the work for them."

I love research about research

Also has nothing to do with the relative spending of academia vs industry on development.

My point was that most drug DEVELOPMENT costs are incurred by private industry, because it isn't a low-risk publication environment.

I wasn't saying that the drug industry didn't have problems.

I remember Journal of Vibration and Control!

[RevWaldo]

That was one high class bondage mag, right up there with Bizzare and Exotique.

I don't think "peer review" means what WaPo thinks it means...

.

Re:I remember Journal of Vibration and Control!

[sd4f]

As a mechanical engineer, I've used it. Great cure for insomnia if you need one.

Re:I remember Journal of Vibration and Control!

[plover]

[Citation needed]

Re:I remember Journal of Vibration and Control!

[gargleblast]

Now that's discipline.

Re:I remember Journal of Vibration and Control!

[Anonymous Coward]

I will never be able to read Wikipedia the same way ever again.

And lonely women the world over revolt.

[Anonymous Coward]

Demand class action status.

Re:Chen? Sounds Chinese

[ShanghaiBill]

So this shouldn't surprise you at all. The Chinese are always cheating the system, bribing people, etc.

He is from Taiwan, not China.

In other news,

[Zomalaja]

There actually is a Journal of Vibration and Control. Must be some thrilling stuff to read.

Re:In other news,

[Anonymous Coward]

There actually is a Journal of Vibration and Control. Must be some thrilling stuff to read.

Dull. I prefer GPS World; now that's some timely and relevant stuff.

Re:Chen? Sounds Chinese

[Anonymous Coward]

Go say that in China. See if they agree with the distinction.

Dildos and bondage!

[Anonymous Coward]

Yay!

Re:Chen? Sounds Chinese

[Tablizer]

He is from Taiwan, not China.

You sure about that? Maybe he lied on that also. Hell, maybe he's not even Asian, or a "he".

Chen-Yuan Chen

[Anonymous Coward]

There's a lot of weirdness about this story. Firstly, guy's name is Chen-Yuan Chen, not "Peter". Secondly, he works at a teachers' college. Thirdly, he's supposed to be a researcher in methods for using electronics to help people learn, so why would he suddenly start writing a bunch of papers about mechanical systems? In addition to spamming 60 fraudulent papers in a few years, he also had each of the 60 papers cite all the other papers!

And the weirdest thing is that a bunch of right-wing crackpots are coming out of the woodwork to argue that this has some implication for climate change research. The fuck are these people smoking?

Re:Chen-Yuan Chen

[AK Marc]

My first job was for a guy named Marshall Marshall. So it's not only the Chinese that re-use names. That, and it's likely that the two Chen's aren't the same word, but phonetically similar.

Re:Chen-Yuan Chen

[Anonymous Coward]

There's a lot of weirdness about this story. Firstly, guy's name is Chen-Yuan Chen, not "Peter".

That's not weird at all.
Many Chinese who move to the west adopt "western" first names to make it easier for others to remember and say their names. I'm not sure they use it officially (probably not), but as a researcher you can choose how you sign your publications. You can even sign with a pseudonym if you like.

It sounds a bit weird to us that a grown person would just choose a name an have everyone call them that, but I think in the Chinese culture your first name isn't that important anyway - it's the family name that counts.
Also, it's quite convenient. In my previous job I had a Chinese colleague who hadn't adopted a western name, and a lot of us had trouble remembering his name. Even when we did remember it, I'm pretty sure our pronunciation was completely off.
He didn't care. I guess he was just used to it and he was a pretty cool guy to begin with but yeah, it'd make all of our lives easier if he'd just gone with "Peter" like the fraud in the article.

Re:Chen-Yuan Chen

[Anonymous Coward]

And the weirdest thing is that a bunch of right-wing crackpots are coming out of the woodwork to argue that this has some implication for climate change research. The fuck are these people smoking?

David Koch's cock.

Re:Chen-Yuan Chen

[Anonymous Coward]

A fellow grad student from Taiwan told me his name. Oh, I said "Aaron?" No he says and tries again. "Arin?" No, he says and writes down "Alan", conceding that, in retrospect, it was not such a good name choice for him...

Re:Chen-Yuan Chen

[Anonymous Coward]

What's also weird is none of the "official" artifacts used the name "Peter" anywhere (always Chen, C-Y).
Where did "Peter" so prominently paraded by the lynch mob come from?
If anything, any official release should have used the name that's splashed all over offending articles listed.

Unrelated problem is, you can't get anything published in the trashiest tabloids without a figurative full cavity search.
JVC is suppose to be a proper academic journal, but they seem to have nobody with bullshit radar on the editorial board?
Does Ali H. Nayfeh do anything for the said journal beyond collecting a paycheck?

Re:Chen-Yuan Chen

[Anonymous Coward]

What's also weird is none of the "official" artifacts used the name "Peter" anywhere (always Chen, C-Y). Where did "Peter" so prominently paraded by the lynch mob come from? If anything, any official release should have used the name that's splashed all over offending articles listed.

Yes! Thank you. All the other commenters seem to think I'm some kind of idiot who doesn't know Asians sometimes adopt Western-style aliases. My point is just that: he doesn't seem to actually use that alias anywhere that would make it relevant to this story.

Re:Chen? Sounds Chinese

[sexconker]

Go say that in China. See if they agree with the distinction.

Lisa, soon you'll have a Chinese baby sister who will surpass you academically!

I don't know about that, I'm considered preeeetty smart.

Well Tibet was considered pretty independent, how'd that work out?

s/Tibet/Taiwan
China won't even let MS push out the Taiwanese IME unless you have a specially-flagged build of Windows (which I've never been able to find).

on perfecting the science of sockpuppetry

[Anonymous Coward]

There's a paper to be written, I'm sure.

Re:on perfecting the science of sockpuppetry

[Anonymous Coward]

You're the best, AC! Let me coauthor your paper and I'll suck your sock!

Re:Chen? Sounds Chinese

[Anonymous Coward]

If you -- you personally, the country China, or the rest of the world -- want to claim that Taiwan is a part of China, then it cuts both ways. So yes, he is "Chinese". And technically, the majority of Taiwanese people are ethnically Chinese.

So yeah, don't try to differentiate only when it suits your needs. When Taiwan does something good, I'm sure you're the first to jump out and say they're "Chinese".

Re:Chen? Sounds Chinese

[AK Marc]

China won't even let MS push out the Taiwanese IME unless you have a specially-flagged build of Windows (which I've never been able to find).

What's the difference between the Taiwanese IME and the Chinese one (assuming you can select Traditional)?

And Taiwan is "protected" by the US, so they'll likely do better than Tibet did.

Wikipedia

[Anonymous Coward]

Funny, Wikipedia does not mention it yet: http://en.wikipedia.org/wiki/Peter_Chen
Must have been peer reviewed out.

Re:Wikipedia

[Mike Frett]

Change something and watch what happens, It'll be gone in five minutes. Same with the Blu-ray Wiki, post anything true but condemning and check back five minutes later -- poof. "They" get paid good money to keep things in the companies best interest. Which is also why I don't donate anymore to Wikipedia.

Re:Wikipedia

[Nonac]

I don't think that is the same Peter Chen.

"You've got the wrong guy. I'm the Dude, man"

Re:Chen? Sounds Chinese

[backslashdot]

And what about Hendrik Schön, where was he from?

Or maybe it's not just the Chinese, your brain locks onto fake patterns.

Someone mod this up

[Puff_Of_Hot_Air]

An insightful post, I'd love to hear if you had an ideas on how the system could be improved?

Re:Someone mod this up

[Anonymous Coward]

Unfortunately, I can't say that I have any brilliant ideas on how the system could be fixed. It is fundamentally a cultural problem, and will only really be addressed when the cultural perception of research changes. But, it's quite difficult to sell people on the idea that the very boring work of reproducing research (to ensure that what we think we know is actually true) is worthwhile. Comparatively, selling people on a new particle accelerator or a new cancer treatment or whatever is much easier.

There is already a sort of class divide in academia between the tenured or tenure-track professors and the non-tenure-track lecturers and adjuncts and it would be great if this could be mapped to original research vs reproduction. Unfortunately, though, the non-tenure-track is absolute shit as a career path and the only way to get off of it is to publish, publish, publish.

So basically, we just need to convince academia that reproducing research is respectable work and that producing significant original research should not be the only viable career path, and society at large that funding such reproduction is a good use of money. While we're at it, maybe we can solve world hunger and negotiate peace in the middle east. :)

Web of Trust

[Dr_Barnowl]

People should cryptographically sign peer reviews (and their papers). And journals should only trust signing keys that themselves have been signed by respected experts. The more respected you get, the more signatures your keys and papers get.

Re:Web of Trust

[Nonac]

That sounds more like a popularity contest than a peer review system.

Re:Web of Trust

[WindBourne]

Actually, that is a good idea. BUT, you have to have vetted signers.

Re:Web of Trust

[WindBourne]

I will say that I have long thought that USPO should start offering vetted keys. Just as they do passports, they would be ideal for doing state IDs and vetted keys. If more govs. offered up such things, then it would make it possible for publications and other groups to require a truly vetted key.

Re:scientists are deceitful shits

[KeensMustard]

Well, feel free to not use any of the things developed from scientific advances. I hear that caves are comfortable year round, and herbs and grasses picked from the mountainside can make a fine salad!

Re:scientists are deceitful shits

[Anonymous Coward]

No one tell KeensMustard that homestead agriculture and legacy farming was doing quite well until the 20th century, when "scientists" discovered how to improve yields and make it a profitable industry.

Another ring: Method Engineering

[Frans Faase]

There are whole fields within Computer Science, one being "Method Engineering", that basically are one big ring. For your information, "Method Engineering" is about methods for developing software.

Re:Another ring: Method Engineering

[HornWumpus]

Equally true if you use the British definition of 'ring'.

See also Dr. David Goodstein's 1990s predictions

[Paul Fernhout]

You make good points. See also: http://www.its.caltech.edu/~dg...
"The public and the scientific community have both been shocked in recent years by an increasing number of cases of fraud committed by scientists. There is little doubt that the perpetrators in these cases felt themselves under intense pressure to compete for scarce resources, even by cheating if necessary. As the pressure increases, this kind of dishonesty is almost sure to become more common.
    Other kinds of dishonesty will also become more common. For example, peer review, one of the crucial pillars of the whole edifice, is in critical danger. Peer review is used by scientific journals to decide what papers to publish, and by granting agencies such as the National Science Foundation to decide what research to support. Journals in most cases, and agencies in some cases operate by sending manuscripts or research proposals to referees who are recognized experts on the scientific issues in question, and whose identity will not be revealed to the authors of the papers or proposals. Obviously, good decisions on what research should be supported and what results should be published are crucial to the proper functioning of science.
    Peer review is usually quite a good way to identify valid science. Of course, a referee will occasionally fail to appreciate a truly visionary or revolutionary idea, but by and large, peer review works pretty well so long as scientific validity is the only issue at stake. However, it is not at all suited to arbitrate an intense competition for research funds or for editorial space in prestigious journals. There are many reasons for this, not the least being the fact that the referees have an obvious conflict of interest, since they are themselves competitors for the same resources. This point seems to be another one of those relativistic anomalies, obvious to any outside observer, but invisible to those of us who are falling into the black hole. It would take impossibly high ethical standards for referees to avoid taking advantage of their privileged anonymity to advance their own interests, but as time goes on, more and more referees have their ethical standards eroded as a consequence of having themselves been victimized by unfair reviews when they were authors. Peer review is thus one among many examples of practices that were well suited to the time of exponential expansion, but will become increasingly dysfunctional in the difficult future we face.
    We must find a radically different social structure to organize research and education in science after The Big Crunch. That is not meant to be an exhortation. It is meant simply to be a statement of a fact known to be true with mathematical certainty, if science is to survive at all. The new structure will come about by evolution rather than design, because, for one thing, neither I nor anyone else has the faintest idea of what it will turn out to be, and for another, even if we did know where we are going to end up, we scientists have never been very good at guiding our own destiny. Only this much is sure: the era of exponential expansion will be replaced by an era of constraint. Because it will be unplanned, the transition is likely to be messy and painful for the participants. In fact, as we have seen, it already is. Ignoring the pain for the moment, however, I would like to look ahead and speculate on some conditions that must be met if science is to have a future as well as a past."

I think a "basic income" for all could be part of the solution, because a BI would make it possible for anyone to live like a graduate student and do independent research if they wanted.

It's money.

[Anonymous Coward]

The problem is not that researchers have to publish per se. It is the fact that their monetary rewards (e.g., salary) are primarily based on the amount of papers published. Also, having published more gives you a better chance for a assistant / associate professorship.

Re:scientists are deceitful shits

[m00sh]

Well, feel free to not use any of the things developed from scientific advances. I hear that caves are comfortable year round, and herbs and grasses picked from the mountainside can make a fine salad!

Scientific advances happened long before peer review and scientific advances are happening in spite of peer review.

Peer review is just a fancy concept invented to make it look like scientific publication is blind to "politics". It is quire the opposite. Scientists are asked to volunteer their time to review peer papers without pay or compensation. The hidden compensation is that they get to push their friends and colleague's work ahead of the pack and the favor is returned.

If you check journals and publications, you will find the same group of people publishing over and over again in the same set of journals and publications. If you read through them, you can tell they are churned out papers aimed at a publication rather than some scientific advancement.

How many...

[Anonymous Coward]

...aliases does Micheal Mann have I wonder?

And a large cache of hockey sticks was seized

[Applehu Akbar]

No, wait - that's a different peer review ring.

Re:Chen? Sounds Chinese

[Anonymous Coward]

The official position of the United States does not recognize Taiwan as a separate country.

no different than many stories here

[WindBourne]

Seriously, we read many stories here in which big deals are made of them, but as soon as I check that it has lead by Chinese Academicians (even if they are now working in the USA), I discount it. WHy? Because over and over, I see fraud in the publications, and here, I notice that many of these stories are being pushed by ACs. In a nutshell, these ppl are putting together fraudulent publications (generally, leaving out the negatives that they came across), and then marketing them to make themselves look good.

Yes, some of you will scream that I am racists, and yet, over and over and over, this occurs.

Re:no different than many stories here

[Anonymous Coward]

That's funny, as soon as I see WindBourne and the ring of Elon Musk cocksuckers I discount them too.
WHy? Because over and over, I see fraud in the publications, and here, I notice that many of these stories are being pushed by ACs. In a nutshell, these ppl are putting together fraudulent publications (generally, leaving out the negatives that they came across), and then marketing them to make themselves look good.
Yes, some of you will scream that I am Elon Musk hater, and yet, over and over and over, this occurs.

Re:no different than many stories here

[Anonymous Coward]

Really? That is the best that you can do? Sad. Not an intellectual thing about you. Worse, you plagiarize other's work and do not give credit. Oh wait.... that is what the article is about as well. So, you must be Chen, or an American conservative.

No reference checks

[Monoman]

If only the journals could run some kind of check to determine if "peers" are who they claim to be.... and only them.

Not surprising

[teakillsnoopy]

I've been proofreading engineering/medical papers for universities in Taiwan for over 7 years and this is not surprising in the least. There is almost no stigma regarding plagiarism in this region (I've done work for Malaysian, Vietnamese, Indonesian, etc. authors). When I alert an author about copy/pasted text, their reaction is one you would get if you told someone that their reference format needs to be change. "Oh, ok. I guess I'll change it.". The universities here never seriously investigate plagiarism because all the big fish at the top did it themselves to get to the top.

Re:Chen? Sounds Chinese

[WindBourne]

No, he is teaching at Taiwan. There are many Chinese natives that teach at Taiwan (and elsewhere for that matter).

Chen is Chinese, and that worries me

[Anonymous Coward]

As someone who has many Chinese American friends, some born in the USA some from abroad, a high number of them seem to be of the culture and mindset of cheating is OK. Mostly the ones with closer ties to mainland China, as in born there or parents were born there. What one friend told me was that in her competitive high school in Shanghai, practically all males cheated during tests.

I think it is up to the Chinese to set a better example than what was set before them. Cheating is not OK, especially when your cheating results affect the lives of others so directly.

Re:Chen? Sounds Chinese

[Anonymous Coward]

They're also an island with a substantial tech-heavy economy, a sophisticated army and democratic institutions. In other words, unlike Tibet, they are (fingers-in-the-ears la-la-la'ing by the UN aside) a real country.

Re:climate science, conspiracy, scientists

[BergZ]

I see you've decided to use an article that has nothing to do with climate change as an excuse to make snide comments about climate science and the people who advocate it.
I usually associate that kind of behavior with people who have a "____ derangement syndrome" (they make everything about the topic/person they hate most: Obama or Bush, Liberals or Conservatives, Communists or Capitalists, Secularism or Religion, etc).

Re:Chen? Sounds Chinese

[Anonymous Coward]

Tell that to Ukraine

Re:Chen? Sounds Chinese

[HornWumpus]

Ukraine, next time you have the chance, lease out a navel base to the USA.

Would have changed everything, but too late now. If you take Crimea back, least the navel base to the US navy, day 1, hour 1.

The article has it all wrong...

[SplawnDarts]

That's not a ring - that's "scientific consensus" at work.

Re:The article has it all wrong...

[david_thornley]

Really? The scientific consensus seems to be to retract the papers.

Re:Chen? Sounds Chinese

[AK Marc]

They don't even claim to be a country. Taiwan and Beijing both claim to the the "true" capital of China. Beijing has a bigger army. Both claim each other as territory. It's a cease-fire in a civil war, not a separate country.

Re:Chen? Sounds Chinese

[AK Marc]

What close ties and treaties do they have with the USA? None. Oh, there's your failure. Taiwan was founded by anti-comunistic capitalists. The US backed them, but unlike the White Army, the US didn't serve them up to be killed after false promised of help, but intervened in a civil war to create a new country.

The equivalent is if the English had backed the South, and the US Civil War ended with Louisiana being the Confederacy, under the protectorate of the English. Louisiana isn't going to win an "invasion" of the USA, but it's not worth a second Revolutionary War to finish them off, so you wall them off, and mostly ignore them, while keeping the "claim" on the territory. Though the LA governemnt is formed mostly of former US Senators and Representatives from around the country, so they "claim" to be the head of the government, but can't (and don't) do anything about it.

The Ukraine is like Somalia or Yugoslavia. We don't know. We don't care. We didn't start it, and we will try to not get involved, and when we do, we'll accuse the president of treason for defending non-Americans.

Re:climate science, conspiracy, scientists

[Layzej]

Are you suggesting that all scientists who study signal analysis are colluding to deceive policy makers and that the entire field is actually bogus propaganda? Cuz that's what those other guys are suggesting. If there is a small group of bad actors they will be rooted out (as was seen in this case) and as has been seen in climate science as well: http://science.slashdot.org/st... .

Typical reaction : Discrediting new competition.

[Anonymous Coward]

The traditional publicists are having a rough time desperately trying to portray new ways of publishing as not serious. Why? Because with new free publishing and open peer-review, they are scared down their pants that their way of doing it is on the way out...