Slashdot Mirror
How Scientists Know An Idea Is a Good One
Physicist Chris Lee explains one of the toughest judgment calls scientists have to make: figuring out if their crazy ideas are worth pursuing. He says:
"Research takes resources. I don't mean money—all right, I do mean money—but it also requires time and people and lab space and support. There is a human and physical infrastructure that I have to make use of. I may be part of a research organization, but I have no automatic right of access to any of this infrastructure. ... This also has implications for scale. A PhD student has the right to expect a project that generates a decent body of work within those four years. A project that is going to take eight years of construction work before it produces any scientific results cannot and should not be built by a PhD student. On the other hand, a project that dries up in two years is equally bad. ... the core idea also needs to be structured so, should certain experiments not work, they still build something that can lead to experiments which do work. Or, if the cool new instrument we want to build can't measure exactly what I intended, there are other things it can measure. One of those other things must be fairly certain of success. To put it bluntly: all paths must lead to results of some form."
What do you want to tell us with that?
mathematicians only need chalk
(apropos: http://xkcd.com/435/ )
That's not a description of a good idea. That's a description of an idea that fits into an arbitrary 4-year timescale that fits with a PhD program's average length.
Just because you're paranoid doesn't mean there isn't an invisible demon about to eat your face
intuition, foresee and application area is required, before getting your hands dirty or diving, foremost imp. is scope, problem definition and then title.
Science as a process is like Natural selection and just as in Natural selection, one may come with the dead end. This is not necessarily bad.
To quote Thomas A. Edison, "If I find 10,000 ways something won't work, I haven't failed. I am not discouraged, because every wrong attempt discarded is another step forward".
A PhD student has the right to expect a project that generates a decent body of work within those four years.
Four years? Ha! That's a good one!
A big part of the problem is that there are few negative results in scientific literature. Ever found a paper with a clear negative outcome? I didn't. This "positive bias" in scientific publications is probably causing a major blow to the efficiency of scientific research.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
"A PhD student has the right to expect a project that generates a decent body of work within those four years."
For a Masters degree, this is acceptable. For a PhD, they had better be coming up with their own idea, a plan, funding, and then have their advisor and committee evaluate during the prospectus defense. Having their topic/project dropped in their lap so they can turn the crank is not what a PhD is all about.
...and the ability to think on your feet.
It is not possible to plan 4 years ahead to ensure success. What you get instead is a PhD project plan that's wrapped in a set of general concepts (AKA escape routes) in case you hit a dead end. I'm currently doing a life science PhD and have changed tack at the half way point. A number of my colleagues have also, often quite drastically, whether for reasons of practical feasibility or time constraints.
If we know accurately what we were going to work on that far in advance, it has probably already been done.
Python coder | PyQt Applications | Writer
Chris Lee may be a physicist when he dons that hat, but in TFA he speaks as a people/resource manager, not as a physicist. In the science of physics, the only thing that determines whether an idea has merit is the scientific method, and that's very well documented.
Resource management is much more about cost and "return on investment" than about physics, even in the hard sciences, He wasn't speaking as a physicist in any way that's relevant to the science of his field.
Ancient Persians would debate ideas twice - once sober and once drunk. It had to sound feasible in both states to be a good idea.
I'm afraid the title of your note is misleading. Good science, much more than good engineering, involves testing new or old theories, to find how they work in previously untested ways, or to make sure that the previous test was really valid and caught all the important factors. A good graduate school project, involves a constrained project that can be reasonably tested in a few years, that does involve something of interest to the adviser, and that with good luck can be turned into a career of related questions.
The key is to make the initial question relatively simple, so that the concept can be expanded into tests or other related fields as time and funding permits. This isn't asking the "right size" of question, it's asking a question with enough related, interesting implications but that still has relevance if only the simplest parts can be addressed. Let me take an example of something I'd love to find a good thesis for: the cost of using different sorting algorithms.
The maximum computational costs of complex sorting algorithms is well understood (and well described at Wikipedia). But the additional computational cost of maintaining registers is not factored in, especially for small or modest data sets, and the cost of comparison _itself_ between different formats, or between positive and negative numbers, is not factored in to those computational costs. Neither is the cost of a partial sort that has to be started over from scratch or the benefit of algorithms that can be used when it is partially sorted. There is _wonderful_ material for a thesis in that kind of question, and even material for almost immediate application to industry. The preliminary survey and testing work with computational models can be done within a year by someone competent, but testing it against different CPU or software environments would be even more valuable and could easily fill out the rest of a graduate program, even leading to a creer in optimization of computational algorithms.
and is always an option
See how Nikola Tesla did it, but do the same without the likes of Edison, big money, banksters, corrupt politicians, in other words, you're doomed!
The mind conceives, the body achieves, the spirit manifests.
Good ideas are hard to determine, and sometimes you find out something was actually a really bad idea only after several years like trans fats, or saccharin.
The results of scientific discovery are diminished by classifying them as success/failure. The only 2 classifications should be "A Truth Discovered" or "Pseudo Science".
Any lab experiment which is conducted to seek the truth even if it does not yield a commercially viable result is still a truth discovered. A so-called failed experiment still is a success at discovering a method which does not work to achieve desired results, and discovering what does not work in some cases can be more important then finding out what does and is an actual truth discovered.
Any experiment performed to skew results in a particular direction, or where evidence is tossed that does not agree with your idea's is nothing but pure Pseudo Science. Unfortunately we have so much of this it has made people distrust scientists because they have proven that they are just as opportunistic as normal people and will do just about any dishonest thing for a buck! True Science be damned!
But that is theory. In practice, having some realistic goals based on available resources of money and time is common to all fields, not just science.
[*] Chandrashekar was not bitter about Eddington, he credits being forced to change fields in his late 20s, taught him how to learn and he deliberately abandoned his field of study about every ten years, he continued to be productive into his late 70s. If you find the spoof paper written in his style The Imperturbability of Elevator Operators, by S Candlestickmaker, by one of his grad students, it makes hilarious reading for the geeks. ]
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
To continue the CKXD comic,
Math is applied Logic
Logic is applied Philosophy
Philosophy is applied Sociology
and "the circle is now complete."
...that he has no clue about particle physics. Building the ATLAS detector took well over 8 years and PhD students were involved. They worked on detector R&D, performance studies etc. as well as simulation of the physics the detector would eventually be able to detect. Not everyone can do their physics in their own lab with their own group of students and postdocs. Some physics requires huge machines and detectors and that necessitates long construction times...and if this guy poked his head outside his lab once or twice he would know that!
Four years? Not in Canada - and presumably not in the US either. The department average in my program was more like 6 (I took about 6.5), and I've known people who have taken as long as 10 to complete their PhD.
From some document I found on startpage: http://careerchem.com/CAREER-INFO-ACADEMIC/Frank-Elgar.pdf
"Median time-to-completion of the PhD has nearly doubled during the last three 2 decades (from 6.5 to 11 years). "
This is why it's so important in biology to know people, or to have a PI who does. Friends tell friends their negative results, and that's how word gets around.
The only way to know if an idea was good, is after you've already done it. Future prediction is always a crapshoot. People who claim to be good at it were typically just lucky, and are deluding themselves into thinking it was all skill.
Or, a series of small interrelated projects.
That is the customary approach I've seen the last decade.
Todays so called "Scientists" are mostly after one thing, money. Whether it be grant money, donations from private or government funding, governmental tax breaks or whatever that's the sign you have a good idea.
And that's all science is anymore, good ideas. That is to say, ideas that sound good.
All you need to be a scientist and get money is to come up an idea that sounds good. You don't need facts, science or proof, if you sound like you know what youre talking about them the money and attention will come. All scientists are for the most part now are salesmen pitching an idea. If you look hard enough and are a good speaker you can convince anyone of anything really.
Youre an especially good scientist if you can scare people as well. Like tell them how their faucets are causing heart disease, that cheese leads to cancer, and sleeping at night causes strokes in 1:100,000,000,000 test subjects.
I love science and Im all about it, but science for the most part now is a damn sad joke played on the weak minded and stupid.
Good idea - one that people mock and refuse to accept or try.
"Good idea" - one that scientists get federal, uni, or private funding for but produce no meaningful output for 20 years, if ever.
"Scientists" isn't some coherent group that "knows" something. Some take guesses, some succeed, some fail. Many get it wrong too, and quite frequently.
Scientists tell if an idea is a good one by trying to prove it wrong, over-and-over-again and in as logical a thought-out way as possible, til they give up. This is known as "science", and the fact that they do it this way is why we call them "scientists".
Once again we see that one can determine with decent success value of a scientific effort in the near future, not just centuries down the road. This is quite relevant to the funding of science. If the scientists themselves are trying to figure out what activities will be more fruitful, then that's a strong indication that society ought to be doing this as well.
A PhD is a several year apprenticeship in some area where you learn to do right-size research projects. The largest error of many new graduate students is to choose a project that has already been done, one that is too trivial to get publications out of, or one that is too large to finish in 3 years of work. There are fields outside of my PhD where i think I know pretty much the basic knowledge,e.g. computer science. But I would not be able to choose a "right size" R&D project without help.
If you do a PhD with the objective of landing a job in academia, you should also worry about your PhD advisor and his publication record. He/she can have a zilion briliant ideas, you will be fucked if you don't publish. Moreover, in finding a good advisor there are factual criteria, whereas determining which ideas would require prescience (and I don`t believe in it).
Er, how many?
The triangle of supply and demand, where one of the sides isn't supply and neither of the others is demand?
Don't repeat stuff you've misheard while listening to the grownups. It makes you look stupid.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Ha ha.
A real scientist must develop and rely on instinct. This makes me laugh because it demands a recognition and respect for the forces beyond the immediately measurable material realm. (Instinct, when I use the term, is not the same part of your mind used to tackle the Monty Hall question, btw).
If you fail to do this, you're just going to explore a lot of boring, safe science, and never do anything truly special or illuminating. Exploration is not for the faint of heart.
The great scientists of note followed their hearts and minds toward and through those problems which most fascinated them, and they trusted that the universe would provide.
Nothing worthwhile in life comes without a cost, and that cost isn't just counted in physical and monetary resources. That stuff is easy. Faith is hard, and most can't deal with it.
It's a dangerous practice. If the same practice was used in Einstein's time we would still not be aware of special and general relativity.
Great discoveries in science are fundamentally not predictable by the current scientific community.
... it gets even worse: http://www.pdfernhout.net/to-james-randi-on-skepticism-about-mainstream-science.html#Some_quotes_on_social_problems_in_science
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
If I had a dollar for every time some moron with a really stupid idea was able to get other people to part with their money for it, I'd be a rich man. George Carlin sums it up nicely when he said "You nail two pieces of wood together that have never been nailed together before and some schmuck will buy it from you." I would extend that further by saying "If you have charisma, you are able to convince people that the words coming out of your mouth are pure gold." In my experience, as with role-playing games, if someone with high charisma, chances is also a moron but people won't see it until it's too late. Chances are also high that such a person has the ability to blame their miserable failures on something or someone else, often a smart person with low charisma.
Funnily enough I'm putting the finishing touches on my PhD dissertation tonight after 9 years of work. I've worked on 3 projects, of which only the final one worked. The American Institute of Physics keeps stats (you can find them easily on the web) and the average physics PhD is just over 6 years, with considerable spread in either direction. Everyone thinks they can get their PhD done in 4 years, but only ~5% do, the same number that get theirs done in 8+ years.
In my wizened opinion, the time it takes is a crapshoot within about a factor of 2.
During my PhD I learned that the only thing truly required for a PhD is time (at least in science). While it is nice to get a great result and generate tons of papers, what really comes from the PhD is learning how to do research (organize plan, collect data, analyze, write, etc). Though disappointing, a negative result does not hinder this. In fact, being able to confidently say "this will not work" rather than "I couldn't get it to work" means you have really deeply investigated the problem and have sufficient skills to assess the problem, data, and results.
Most PhD students start thinking they will discover the next great thing. Some do, but most "simply" complete a nice project and get a few conference or journal papers. This is the nature of science, not everything is ground breaking. Attend any large conference and you will see several thousand ideas, only a very very small number will be truly transformative.