Scientific Research That Could Have Been Avoided

indian_rediff writes "An article from Friday's Wall Street Journal (reprinted in the Pittsburgh Post-Gazette) details how some of the research being done by scientists ends up simply stating the obvious. Their observations make for some interesting and hilarious reading." From the article: "Want job satisfaction? A 'careful choice of career is the key,' researchers concluded in a paper this spring in the Journal of Economic Psychology. Choosing a career based on a well-lubricated encounter at a bar, it turns out, may not be the most promising route to career satisfaction. People who choose their jobs carefully are more likely to be satisfied with them than those who take a flying leap into the great unknown."

Comments

Simple

[rm999]

They do the research, and if they find nothing interesting to say, they say something that isn't interesting. That is how they get more money to do more research.

Re:Simple

[jadavis]

Also, many of these things have to do with degree. Like the vision study: we all know that as you move away it's harder to see the small detail. But how much detail is lost? In a criminal trial, the eyewitness testimony could fall into three categories:
(1) Positive ID: you saw the whites of their eyes, and they were wearing their driver's license as a badge.
(2) Corroberating evidence: Someone matching that description was seen running away.
(3) Ruling out suspects: a small woman was seen running away, but the suspect is a big man.

And a study like that could help the jury categorize how to view the evidence.

Re:Simple

[nasor]

" Also, many of these things have to do with degree."

Exactly. When I was in college I was paid to participate in a study on how sleep deprivation affects people's ability to analyze information, follow instructions, and work in groups. A big group of us stayed awake for 48 hours. We periodically had to take short written tests, perform fairly complex tasks by following someone else's directions, and work in groups to try to solve problems. We were later told that the study was funded by the military, who wanted information on how people's performance degraded at various tasks as they stayed awake longer and longer.

The people funding the study wanted to get an idea about how long an enlisted infantryman could go without sleep and still be effective relative to an officer, helicopter mechanic, paramedic, etc. Now, obviously if you summed the results of this study up as "Sleep deprivation decreases your capacity to solve problems, follow instructions, and work in groups" it would sound pointlessly obvious.

A subtle distinction...

[FlyByPC]

Sometimes, science advances by asking questions about things that, on the surface, seem "obvious." For instance, at one time, everyone "knew" that:

* The Earth was flat;

* Objects slowly came to a stop unless a force was exerted on them;

* Matter and energy were always conserved;

* Time was a universal constant;

...etc. Perhaps the problem is, too much attention is paid when these questions come back with the expected answer, rather than the fact that these questions are being asked.

Question everything, but sometimes the answer is "yes, that's correct."

Re:A subtle distinction...

[MoralHazard]

I was thinking along similar lines while I read the article. The point of doing research is to confirm OR deny a hypothesis. This article seems to assume that scientists should be able to know in advance what's true and what's false... Guess what, WSJ, THAT'S THE POINT OF THE EXPERIMENTS.

And like the parent poster says, you can't just go around saying "Why research that? It's obvious?" We get proved wrong on "obvious" shit all the time.

Re:A subtle distinction...

[Scrameustache]

* The Earth was flat;

Odly enough, it had been scientifically demonstrated that the earth was round at a time when Alexandria was a place with a really cool library, and that got obscured by this religious sect that had a holy book that implied otherwise.

Fortunatly, these days we don't suffer from crazy religious groups getting political power and subverting scien... well, ok, we DO, but at least we don't get burned alive as part of it no more... that's a kind of progress.

Re:A subtle distinction...

[NanoGator]

"And like the parent poster says, you can't just go around saying "Why research that? It's obvious?" We get proved wrong on "obvious" shit all the time."

Part of the problem is that some people assume they know stuff when they really don't. They believe, but they don't actually know. For example, somebody was saying the other day that he knew all along that Iraq didn't have WMDs. Um, no, he didn't. He hadn't ever even been to Iraq. Heh.

Anyway, the point is that 'obvious' isn't scientific. I can't help but think the dude who wrote this story was inspired by watching Myth Busters. Sometimes they 'bust' myths that seem pretty darn obvious. They had an episode not too long ago that was about whether or not a frozen chicken could do more damage to a plane than a thawed one. I remember thinking "duh" through the whole thing. It seemed pretty obvious to me that yes, of course, a frozen chicken will do more damage. Still, the experiments were surprising. They had a seriously difficult time proving it. In the end, they did, but not before falsely declaring TWICE that freezing the chicken had no change in effect. Since I had never experimented with this, what call do I have shouting 'duh!'? Heck, one of them thought it was 'duh!' that a frozen chicken wouldn't do any more damage than a thawed one.

I'm a big fan of the "there's no such thing as a stupid question" philosophy. Given how much we base our lives on assumptions and 'conventional wisdom', I'm not very eager to shake my pitchfork at conducting an obvious or redundant experiment. Sometimes it's worth it just to refine the testing process. Going back to my Myth Busters example, that's exactly what happened. They cooked up a better test despite thinking they had come to a solid conclusion before.

Only stupid on the surface.

[pavon]

Seeing as how they didn't link to or even cite any of these studies, I think I'll reserve judgment. Half the time the problem with "stupid science" is really stupid journalism. You'll have this perfectly good biology research that looks at how a specific enzyme facilitates a particular aspect of the metabolic system that wasn't completely understood before, and is a good step in the direction of understanding how our bodies work. And how does the news report on it? - "eating fat makes you gain weight". Well no duh. It wasn't that that was interesting, it is the details of how it causes you to gain weight that were meaningfull.

Furthermore, "common sense" can be used to explain all sorts of conflicting ideas. If the study had come out the other way, everyone would be saying that it's obvious that people are happier if they jump in and try out all sorts of things before settling on what you really want to do - "life is a journey", "you need time to find yourself". Psychology is the study of scientifically testing what common sense ideas about ourselves actually are true, to what extent, and in what situations. Of course some psychologists are better than others, but just because you could have guessed the answer doesn't mean it's not worth finding out for sure.

There is also a problem with papers written for the sole purpose of getting published, and I don't like that. I wish that more universities would wise up to the fact that knowledge is becoming more and more in-depth and specialized, and therefore it will take longer till someone is far enough along in thier specialty that they can begin doing research that is new and meaningfull. If you force people to write thesis earlier, 90% of them will be rediscovering somthing that has already been discovered.

But I stand by my statement that stupid jounalism is more of the problem than stupid research, and that knowing something is better than thinking it.

Re:Only stupid on the surface.

[theguyfromsaturn]

"common sense" can be used to explain all sorts of conflicting ideas.

I couldn't agree more. A prof of mine told us a story about the dangers of "common sense" once. As a structural engineer, he had been contracted to design a cover for a terrasse for a cafe somewhere. The owner wanted open space, and so he designed it as a cantilever, only supported on the building side, with no column at the other end. In such a configuration the beams of the cantilever will be in tension on the top part, and compression in the bottom part. Concrete doesn't do very well in tension so you put more reinforcement in the top part. Case closed.

The contractor who was doing the job, had no experience with cantilevers. He managed to convince the owner that it would be safer with columns at the other end. And the owner agreed. Who could argue that a roof would be stronger with columns at both ends of the span? It's common sense, right?

What they failed to take into account, is that now the stress patterns were the opposite of the design stresses, with tension on the bottom of the beam, and compression at the top since the beam was now supported at both ends. They never consulted with the engineer who had made the original design, so of course after construction the beams shortly started cracking in the underside and the roof slowly sagged. It was forturnate that some steel reinforcement had been included for the compression part, or it might have failed without warning and there might have been victims.

The point of the story was that "common sense" was very often based on our limited experience. Unless you know what mechanisms make "common sense" true, you might be dead wrong. It never hurts to be sure; to have proof.

And of course, as ou pointed out, the details of the study, the mechanism of what is happening is often what the research is all about, but also what the journalists like to skip.

Reminded of a Quote

[Comatose51]

When I took cogntive science, my professor liked to stress this qote:

"Ordinary people marvel at extraordinary things. Extraordinary people marvel at ordinary things." -Confucius

Why in cog. sci? Let's think about seeing the color black for a minute. Pretty ordinary. If I told you that I did my Ph.D. on our ability to see the color black, what would you think? "For this you got a Ph.D.?" If I stopped there, you could probably write a short, shallow article about how scientists wasted time and money doing research on things that's mundane. But let's think about it for a minute. How do we see? Light entering our eyes. What color is a projector screen? White. So how is it that we are able to see black on a movie screen during a movie? If we see because of lights entering our eyes, where is the black coming from? The projector shines light, the white screen reflects it back, the portions that we see as black has no light, how do we end up seeing black on a white screen? Maybe not everything we see comes from the outside world. In fact, black is something our brain creates, which then really makes you wonder about shadows. Don't believe me? Check this out.

So we've gone from something that seems really ordinary to a startling discovery. In fact, it's usually the deeper truth behind ordinary things that surprise us and make us go "wow" and inspires us. Stars from the ground are nothing more than specks of light. I guess we can call astronomy look at specks of light through glass and mirrors. Sounds pretty boring too.

this is not scientific research

[eh2o]

rtfa;

this is an editorial piece, nothing scientific about it. there are no references to the supposed studies they quote, there are not even second opinions from authors or other scientific researchers. pure speculation.

and many those research problems cited are *not* obvious, but the author belittles the studies they quote to make the research questions sound more obvious than they really are.

* on a finding that men over 55 are a high risk group for digit loss due to power tools... this requires good statistics to know, its *not* obvious.

* that workers are less efficient in a cold environment... again, not obvious, and many workplaces (imho) keep the thermostat too low.

* that asthma and smoking aggregate worse than either, again not obvious... many people falsely believe this is not true.

* that doctor-patient communication is critical for reducing harmful effects of mixing drugs, also not obvious -- now we know that communication skills are an important part of medical training.

granted, many of the studies conclude with obvious recommendations, e.g., "be careful with power tools", and the author makes great fun of these "obvious conclusions" when in fact, they are not the substantiative conclusion (i.e., factual finding) of the research, just a recommendation for how to interpret the finding.

Re:Teen Pledges Barely Cut STD Rates, Study Says

[benna]

You have to laugh at the whole abstinence-based sex education thing, or else you are liable to cry. It would be like if in drivers ed they told people not to drive, because they might get into a car accident. That would be absurd. Just about everyone is going to drive anyway. Instead, the rational thing to do is to teach people to be safe drivers, wear seatbelts, etc. This is how sex education classes should operate as well.

Re:counterpoint cabal

[mollymoo]

Can you tell me one logical reason why anyone might think that people with stiff, cold fingers would not make more typing errors than people with normal fingers? That's the point of the whole thing: only an idiot would need to test that hypothesis.

The WSJ kindly didn't give us a reference, so we don't know what was actualy in that paper. However, having read a few papers in my time I bet you my left arm that it didn't consist of "we made this guy's fingers really cold and his typing sucked". I bet it says precisely how poor the typing was. Do you know exactly how badly cold affects typing? Just how cold is "chilly"? Is it 280K or 290K? How is the error rate correlated with temperature? How is it correlated with age and sex?

If it's so futile to perform the experiment, if the answers are so obvious, you won't mind telling us the temperature at which a healthy 30-year old male experiences a 10% temperature-induced error rate.

I'm sure you wouldn't mind using your guess when deciding when you should shut down air-traffic control if the heating malfunctions, but I'd rather the people making decisions like that have some hard data to work from.

Re:counterpoint cabal

[billdar]

well, the first link returned from a google search "temperature typing errors" gives a link to a Cornel University study

From the article: "When the office temperature in a month-long study increased from 68 to 77 degrees Fahrenheit, typing errors fell by 44 percent and typing output jumped 150 percent...raising the temperature to a more comfortable thermal zone saves employers about $2 per worker, per hour"

If I employed only 5 typists, thats $400/wk. More than enough to pay the increased heating and start saving for my end of year bonus for being clever and commissioning the study.