Slashdot Mirror
Study Attempts To Predict Scientists' Career Success
First time accepted submitter nerdyalien writes "In the academic world, it's publish or perish; getting papers accepted by the right journals can make or break a researcher's career. But beyond a cushy tenured position, it's difficult to measure success. In 2005, physicist Jorge Hurst suggested the h-index, a quantitative way to measure the success of scientists via their publication record. This score takes into account both the number and the quality of papers a researcher has published, with quality measured as the number of times each paper has been cited in peer-reviewed journals. H-indices are commonly considered in tenure decisions, making this measure an important one, especially for scientists early in their career. However, this index only measures the success a researcher achieved so far; it doesn't predict their future career trajectory. Some scientists stall out after a few big papers; others become breakthrough stars after a slow start. So how we estimate what a scientist's career will look like several years down the road? A recent article in Nature suggests that we can predict scientific success, but that we need to take into account several attributes of the researcher (such as the breadth of their research)."
Ah, nah, what was I thinking. Whether someone produces future scientists or students who know science, doesn't matter one bit. Let's continue to fetishize publication, and the system of duchies it rests on!
I am interested in how anyone would predict the successfull contributions of people who have been hiding in the patent office for several years being denied promotions for their lack of credentials.
Exceptions are exceptionally hard to predict.
Even if they start successfully predicting individuals careers, wouldn't the system eventually break down since professors would probably change based on the results of the prediction?
Yes if your had the top thesis advisor, went to the best schools and work in a lab with good funding you do well. What a surprise! This would probably ignore patent clerks that discover Relativity however. I recall one paper that claimed to be able to predict your whereabouts by some kind of cell phone info. I can predict it without any data. %90 of the population spends %90 of the of their time within 1/4 mile of their place of residence or employment/school etc. Wow that was hard. Can I get a grant for that?
The summary is pretty inaccurate. The h-index was proposed by Jorge Hirsch, not Jorge Hurst. Rather than give a vague description, why not simply provide an exact definition? The h-index of a scientist is the largest number h, such that he/she has at least h papers each of which have received h or more papers.This is easier to understand if you look at the picture in the Wikipedia entry for h-index.
This reminds me of some research about artists which found that you could divide the most 'successful' artists into two rough categories: those who made a big splash right away and those whose classic work did not emerge until much later.
http://www.nber.org/papers/w8368
It should be noted that the usefulness of h-indices varies from field to field. For example, in various branches of pure mathematics, a heavily-referenced paper is one that, maybe, garners 25 to 100 citations. In applied mathematics and certain subsets of statistics, the threshold would be a factor of magnitude larger.
Also, as a preference, I tend to ignore metrics like h-indices when evaluating a researcher, as they provide very little evidence for his her her capabilities, let alone the quality of the work.
To elaborate, at least from my own experiences, in certain portions of applied mathematics that bleed over into computer vision, machine learning, and pattern recognition, I've seen papers that are relatively mathematically prosaic, but possibly easy to understand or where the code is made available, be heralded and heavily cited for a period. In contrast, I've come across papers that provide a much more sound, but complicated, framework along with better results, possibly after the topic is no longer in vogue, and go unnoticed or scarcely acknowledged.
In a different vein, there are times when a problem is so niche, but nevertheless important to someone or some funding agency, that there would be little reason for anyone else to cite it.
Touching on an almost completely opposite factor, there are times when the generality of the papers, coupled with the subject area, artificially inflates certain scores. For instance, if a researcher spends his or her career developing general tools, e.g., in the context of computer vision, things like optical flow estimators, object recognition schemes, etc., those papers will likely end up more heavily cited, as they can be applied in many ways, than those dealing with a specific, non-niche problem, e.g., car detection in traffic videos. Furthermore, the propensity for certain disciplines to favor almost-complete bibliographies can skew things too.
Finally, albeit rare, are those papers that introduce and find the "best" way to solve a problem that no other discussion is warranted.
Unless you need publishing cred for your job, I can't see why anyone would bother going that route.
It's only really useful for tenure in a teaching position, and *slightly* useful for other job prospects. If you're not pursuing either of those, why bother?
1) Your information is owned by the publisher, you can't reprint or send copies to friends.
2) You make no money from having done the work.
3) The work gets restricted to a small audience - the ones who can afford the access fees
4) It's rife with politics and petty, spiteful people
5) The standard format is cripplingly small, confining, and constrained.
6) The standard format requires jargonized cant to promote exclusion.
A website or blog serves much better as a means to disseminate the information. It allows the author to bypass all of the disadvantages, and uses the world as a referee.
Alternately, you could write a book (cf: Quantum Electrodynamics by Feynman). There's no better way to tell if your ideas are good than by writing a book and submitting it to the world for review.
Alternately, you could just not bother. For the vast majority of people, even if they discover a new process or idea publishing it makes no sense. There's perhaps some value in patenting, but otherwise there's no real value in making it public.
Today's scientific publishing is just a made-up barrier with made-up benefits. In the modern world it's been supplanted by better technology.
... quality is now going to be measured by popularity?
I see the little narcissists are growing up and setting policy now...