Slashdot Mirror


How Scientists Know An Idea Is a Good One

Physicist Chris Lee explains one of the toughest judgment calls scientists have to make: figuring out if their crazy ideas are worth pursuing. He says: "Research takes resources. I don't mean money—all right, I do mean money—but it also requires time and people and lab space and support. There is a human and physical infrastructure that I have to make use of. I may be part of a research organization, but I have no automatic right of access to any of this infrastructure. ... This also has implications for scale. A PhD student has the right to expect a project that generates a decent body of work within those four years. A project that is going to take eight years of construction work before it produces any scientific results cannot and should not be built by a PhD student. On the other hand, a project that dries up in two years is equally bad. ... the core idea also needs to be structured so, should certain experiments not work, they still build something that can lead to experiments which do work. Or, if the cool new instrument we want to build can't measure exactly what I intended, there are other things it can measure. One of those other things must be fairly certain of success. To put it bluntly: all paths must lead to results of some form."

12 of 140 comments (clear)

  1. For certain values of "good" by LordLucless · · Score: 5, Insightful

    That's not a description of a good idea. That's a description of an idea that fits into an arbitrary 4-year timescale that fits with a PhD program's average length.

    --
    Just because you're paranoid doesn't mean there isn't an invisible demon about to eat your face
    1. Re:For certain values of "good" by FairAndHateful · · Score: 4, Funny

      read the article, maybe?

      No time! I need to post within an arbitrary slashdot timescale that fits with getting modded up!

  2. Failures are very necessary part of science by prasadsurve · · Score: 4, Insightful

    Science as a process is like Natural selection and just as in Natural selection, one may come with the dead end. This is not necessarily bad.
    To quote Thomas A. Edison, "If I find 10,000 ways something won't work, I haven't failed. I am not discouraged, because every wrong attempt discarded is another step forward".

    1. Re:Failures are very necessary part of science by jasnw · · Score: 5, Insightful

      While you are theoretically correct, you are real-world dead-in-the-water. A big problem with getting science funding these days is what I'll call the Golden Fleece Award Effect (for Sen. William Proxmire's Golden Fleece Award - wikipdeida it). While funding organizations are well aware that a solid negative result in a difficult research area is just as pertinent and useful as a positive one, Congress (the source of all funding) doesn't understand it and doesn't like it. Money out needs to be balanced by succes in. I know many researchers who do 90% of the research needed for a given NSF (or NASA) proposal before they propose it so they can (a) show it will indeed result in success, and (b) it will succeed so they can get more NSF funding. Nothing breeds lack of funding like failure. This is a dumb-ass way to do science, but since all funding comes from the Kingdom of the Dumbasses you get what you'd expect.

  3. 4 years.. by dlenmn · · Score: 4, Informative

    A PhD student has the right to expect a project that generates a decent body of work within those four years.

    Four years? Ha! That's a good one!

    1. Re:4 years.. by Anonymous Coward · · Score: 5, Insightful

      Four years? Ha! That's a good one!

      The easiest way to enforce that is for the awarding institution to say that if it isn't done in 4 years, it will be taken as a complete failure.

      No, that rule would result in a lot of thesis committees approving completely crap theses. Believe it or not, thesis committee members are human and have a lot of difficulty telling kids that their last four (or five, or eight) years of work are worth no recognition and please leave. Thesis advisors become emotionally attached to their students and want to see the succeed/graduate, even if those students are incompetent. Sometimes you can compensate for the incompetence with time. Only rarely will a thesis committee 'over-rule' the advisor, with their input generally taking the form of 'this would become acceptable if the student adds [foo] over the next year or so.' Mandated time to completion is a recipe for diminishing the quality of theses and migrating a PhD from someone prepared for reasonably independent work to a glorified MS. Probably already moving in that direction, as many 'PhD's aren't really ready to work independently until they've finished two or more post-doctoral internships.

  4. The Persian method by Anonymous Coward · · Score: 5, Funny

    Ancient Persians would debate ideas twice - once sober and once drunk. It had to sound feasible in both states to be a good idea.

  5. Re:What? by show+me+altoids · · Score: 5, Funny

    It is obvious that you're a mathematician. Your equation is dimensionally wrong.

    No, it's correct. Let's do the analysis: $= (time + obtanium) / desire * beer
    time is in seconds
    obtanium is in seconds (how long to obtain it)
    desire is in seconds/liter (the longer you wait, the more you want)
    beer is in dollars/liter
    so we have (seconds + seconds)/(seconds/liter) * (dollars/liter) = dollars
    Q.E.D.

    --
    I feel sorry for people that don't drink, because when they get up in the morning, that's as good as they're gonna feel
  6. Good ideas are discovered after the fact! by SirAstral · · Score: 4, Interesting

    Good ideas are hard to determine, and sometimes you find out something was actually a really bad idea only after several years like trans fats, or saccharin.

    The results of scientific discovery are diminished by classifying them as success/failure. The only 2 classifications should be "A Truth Discovered" or "Pseudo Science".

    Any lab experiment which is conducted to seek the truth even if it does not yield a commercially viable result is still a truth discovered. A so-called failed experiment still is a success at discovering a method which does not work to achieve desired results, and discovering what does not work in some cases can be more important then finding out what does and is an actual truth discovered.

    Any experiment performed to skew results in a particular direction, or where evidence is tossed that does not agree with your idea's is nothing but pure Pseudo Science. Unfortunately we have so much of this it has made people distrust scientists because they have proven that they are just as opportunistic as normal people and will do just about any dishonest thing for a buck! True Science be damned!

  7. Basically they don't. At least they shouldn't by 140Mandak262Jamuna · · Score: 4, Interesting
    If you know `a priori whether an idea is "good" or "bad" it will bring prejudice that will taint the results. One of the famous example is a naive Indian astrophysicist on his first trip outside India met Eddington on eve of his big presentation, in 1929. That guy explained in great detail his idea and Eddington not only dismissed it, he was scheduled to present just before this paper from that young man. He trashed the idea so much that the young man abandoned that field altogether and chose to pursue a different field [*]. Most others dismissed that paper too. It eventually garnered that young man a Nobel Prize in physics, and is the foundation of what is known as Chandrashekar Limit that tells you if a star is big enough to go supernova. That paper was discovered about 15 years later, after WW II. So in theory they should not know if an idea is good or bad.

    But that is theory. In practice, having some realistic goals based on available resources of money and time is common to all fields, not just science.

    [*] Chandrashekar was not bitter about Eddington, he credits being forced to change fields in his late 20s, taught him how to learn and he deliberately abandoned his field of study about every ten years, he continued to be productive into his late 70s. If you find the spoof paper written in his style The Imperturbability of Elevator Operators, by S Candlestickmaker, by one of his grad students, it makes hilarious reading for the geeks. ]

    --
    sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
  8. Hindsight by naroom · · Score: 5, Insightful

    The only way to know if an idea was good, is after you've already done it. Future prediction is always a crapshoot. People who claim to be good at it were typically just lucky, and are deluding themselves into thinking it was all skill.

  9. Re:Luck... by ColdWetDog · · Score: 4, Interesting

    But seriously, if you work with old-school biologists, do the world a favour, and teach them that a Gaussian error on a number of cells is dumb and wrong.

    I think that entry into either medicine or the biological sciences should require a passing grade on a graduate level statistics course. Only then do you stand a chance in hell to start moving away from a century of misconstrued numbers. In medicine, it's still painfully obvious that most researchers couldn't get past Stats 101. And that is even after they have the manuscript reviewed by a biostatistician (who is probably shivering in a basement closet hoping that the next group of researchers gives up looking for him and goes to a bar.)

    Of course, I'd still be fixing cars for a living, but that might have been a better outcome for myself and society....

    --
    Faster! Faster! Faster would be better!