Slashdot Mirror
Peer Review Highly Sensitive To Poor Refereeing
$RANDOMLUSER writes "A new study described at Physicsworld.com claims that a small percentage of shoddy or self-interested referees can have a drastic effect on published article quality. The research shows that article quality can drop as much as one standard deviation when just 10% of referees do not behave 'correctly.' At high levels of self-serving or random behavior, 'the peer-review system will not perform much better than by accepting papers by throwing (an unbiased) coin.' The model also includes calculations for 'friendship networks' (nepotism) between authors and reviewers. The original paper, by a pair of complex systems researchers, is available at arXiv.org. No word on when we can expect it to be peer reviewed."
I can't quite remember what it was, but I seem to remember seeing it everywhere. It was exactly like TFA article, though. Damn, what was that place called again?
Living With a Nerd
I saw that last sentence coming. I can imagine $RANDOMLUSER's supervisor won't accept his excuse for said supervisor's paper failing peer review.
This is precisely what the global warming skeptics say is happening with the global warming alarmist community. ie. scientists review each others' papers, in a 'co-operative' manner as it were.
I think I'll point some skeptics at this paper and then sit back with a bowl of popcorn and watch what happens.
"The system provides an opportunity for referees to try to avoid embarrassment for themselves, which is not the goal at all," he says.
So, if a reviewer sees a paper that has actual data and a conclusion that goes against the consensus of the scientific community, the reviewer may reject it for fear of appearing foolish? Or rejecting someone just because of their publicized personal beliefs?
Here's a hypothetical, a climate scientist who's an openly devout Christian finds data that sheds doubt on human caused global warming will be rejected because someone's afraid of looking foolish.
That's the way I'm interpreting this study.
RIP America
July 4, 1776 - September 11, 2001
Peer Review Highly Sensitive To Poor Refereeing
Much like
Trolling is a art,
When you're talking about scientific papers, a "bad apple" reviewer may be able to skew the record in terms of 1-10 scales, but reviewers also do a qualitative write-up of the material. That's really the only important part and if one or two people fall outside the line of general consensus, they'll just be ignored.
Since the scientific community is so very obsessed with peer review, will this study actually modify the standard procedure?
Of course, all I can think of is: gee, I wonder if this has had any impact on all the climate change studies that are constantly contradicting each other...
I can't decide if I should care whether this was peer reviewed or not.
Almost always - no, that's not a scientific deduction, it's just coming from skewed subjective personal experience - the ones who most complain about problems with article peer review systems are those who have the most problems publishing decent articles at decent places. Also, nepotism? Ever heard of [single/double] blind reviews? I guess this must be one of those slow news days.
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
I mean scientists who publish among themselves, i.e. inside their narrow specialty, in their own journals, without checking whether the problem at hand has been solved elsewhere. This is more and more common as people get more specialized, and can lead to very basic errors propagated inside the whole community, like rheologists believing in the existence of pure elongational flow (a trivial misunderstanding of tensor algebra). Since the peers reviewing the papers are members of the same community, those errors usually get unnoticed.
Just this week, I was asked to peer review a paper in which I was mentioned in the Acknowledgments. The request was sent out automatically -- the journal has records of all their authors, and the keywords for this paper matched the keywords in my profile, so I was picked to review it.
I recused myself, but really I should never have been asked. If they're going to handle the peer review process automatically, the artificial intelligence that makes the decisions needs to be improved.
Genocide Man -- Life is funny. Death is funnier. Mass murder can be hilarious.
The great anecdote demeaning peer-reviewed journals is The Social Text Affair, where a prominent peer-reviewed journal published with enthusiasm the article "Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity", only to be informed it was, in fact, computer-generated gibberish submitted as a joke.
Can we get a "-1 Wrong" moderation option?
There are a couple of significant and important limitations in the model:
a) It assumes only two reviews per paper, and that the reviews are pure boolean, and that reviewer types are also pure and reviewers are randomly selected (when two of the classes of reviewers, 'mythantropes' (always reject) and 'altruists' (always accept) are specifically selected against by editors and PC chairs based on reputation).
b) It does not consider the cases (such as conferences) where there is a program committee meeting and the papers are not just considered on their own, but gone through a relative ranking process.
Test your net with Netalyzr
Slashdot let's you publish first, and be reviewed later. The peer-review system used by scientists forces them to work on their papers until someone finally "mods" it acceptable. Imagine how much faster science could advance if we had a system that actually let scientists focus on research, let people trained in technical writing do the reporting, and let Google design a post-publication moderation system to sort out the useful advances from the career posturing. Science could learn a lot from Slasdot. It is simply ignorant that we continue to put up huge barriers to publication.
just tweak arXiv so that reviews of posted articles are linked to in the abstract page. or something like that. why the need for anonymous reviewers? as long as the discussion is objective and to the point, there's no need for anonymity. and we are far past the point where it costs money to attach comments to an article...
new sig
After reading the article, you can add these terms to George Carlin's list of oxymoronic phrases.
"Flyin' in just a sweet place,
Never been known to fail..."
Each reviewer produces a binary recommendation within the same timestep: ’accept’ or ’reject’. If a paper gets 2 ’accept’ it is accepted, if it gets 2 ’reject’, it is rejected, if there is a tie (1 ’accept’ and 1 ’reject’) it gets accepted with a probability of 0.5.
If a single 'bad' reviewer (i.e. one that gives the 'wrong' answer as determined by the 'correct' method of reviewing mentioned as a control in the paper) can cause a paper to have a 50:50 chance of acceptance or rejection it doesn't seem too suprising to me that a relatively small number of them could cause the process to become '[not] much better than by accepting papers by throwing (an unbiased) coin' - because in their model, in the case of a reviewer disagreement, that's exactly what is happening!
Broken record time, but yes. Such subversion of the peer review process did show up. The culprits weren't the ones you expect.
In general however, I think that this study is rather pessimistic. And anyway, it hasn't been peer reviewed, so who knows... ;-)
(yes, I did read TFA, but not the paper
The peer review system is great for regulation, standardization and unification. However, all scientists that I've worked with/researched with/spoken with much about this topic admit that the system can be annoyingly flawed by group think and conformity. One bad apple ruins the bunch, right?
The good news? While this part of the scientific community is not immune to problems, the slack is picked up elsewhere: As long as methods, data and results are transparent, reproducible and published we can actually have quality science.
I often speak to people about scientific research and they're shocked that it's not full proof. This is kind of like buying software (perhaps even a Microsoft product) and finding that it's not perfect. Science is done by committee and progresses slowly. "If we know what we were doing, it wouldn't be called research" ~Albert Einstein
Then again, I'm an idiot....
...... and idiots rule the world....
I actually read the comments on TFA, and down at , there's a particularly interesting one:
This study overlooks not only the role of the editor, but also the process in which the authors are able to answer the referees' objections. When the referees are competent, this leads to better papers through useful suggestions. On the other hand, when they aren't, overcoming the exasperation of the authors, their objections are easily brushed away, and the paper eventually gets through. Also, when the case is particularly contentious, there's still the option of calling for an adjudicator. In summary, the peer-review process is far more complex than this simulation might suggest. On the dark side, I’ve also noticed that referees are sometimes reluctant to object papers from certain renowned authors. The human factor is hard to remove. I guess many people will agree that there’s a need to look for better approval systems, specially today, when there’s an explosion of submissions. However we must also acknowledge that the present system has served its purpose of maintaining a certain quality.
There's actually a reasonably intelligent discussion going on in there...
My dad (has PhD in a scientific field from Cornell) told me that when submitting a thesis to a review board of professors, it really doesn't matter how "Tough" a professor is as long as that professor in your committee has a rival. Take advantage of their ego with an equally assertive ego. You purposefully choose the rival professor to join your committee as well. Then they'll spend all the review board discussions and presentations contradicting and arguing one with another, and in the end they'll both be so incensed, that they cancel each other out, and it doesn't matter what you presented... I guess the TFA is only pointing out that this occurs at the publishing level as well.
http://www.beanleafpress.com
The outliers might not be due to conscious suppression of competing research. People just have some ways of thinking that make their subjective opinions sometimes contrast with what an objective observer would think.
My first anonymous coward...
My boss (a leading authority in his area, a serious person) has found around 10 errors in epidemiological modelling papers (papers that, by they way, influence WHO policies that influences public health). These are proven errors (in mathematical formulas). They vary from minor to massive.
He informed all authors of the respective problems. How many corrected or retracted the manuscripts? ZERO.
Will he write a paper documenting all the errors (as they are hard-science type of errors, in formulas)? Are you crazy? Remember that the other authors in the area peer review his papers. Therefore we will not risk their wrath.
But note that this varies a lot from area to area. I work in another area (related to genetics) and 90%+ of the people that I know are honest.
But yes, some areas need a cleaning up.
Welcome to the real world intellectuals....this is how things are done. I'm not saying its right, but knowing the right people is how things function. I know the University and intellectual world like to think that they are removed from such things but that is just not the case.
I remember a news show highlighting a "famous" professor that did some amazing work in his early carrier and then proceeded to build a bigger carrier while reusing fake data and bogus information. He WOULD NOT have been able to do so if he had not known the right people and had the "right amount of fame". No one doubted him...however if his papers where published by a "nobody" they would have been caught QUICKLY.
-- Disclaimer: I can't really back up anything I post on
Peer review only works if the reviewers can be trusted and don't form a clique to get their work in and keep other people out. Surely anyone with even basic knowledge of human psychology would understand this?
Yeah Slashdot is a peer-reviewed website, and has exactly the same flaws as described in the article. You've been assigned a (0). Slashdot's mod system is basically just "+1 I Agree" or "-1 I Disagree". I don't see why it even exists, because the same purpose kind be achieved via posting.
"I disapprove of what you say, but I will defend to the death your right to say it." - historian Evelyn Beatrice Hall
Because this is an important question for serious people, but has no bearing on why various cranks (Intelligent Design people, climate change "skeptics", Time Cube, etc.) may have trouble getting their work in print. Papers by such people generally don't end up in the peer review phase - they aren't sent out for evaluation by the journal, so peer review doesn't matter.
That said, peer review provides substantially the same benefit as those "shoplifters will be prosecuted" signs you see in department stores.
Shoplifters are very seldom if ever actually prosecuted - but the threat, even the vaguest menace - of public scrutiny has an impact on behavior. I'm not talking about scientific fraud (which peer review will seldom catch,) but about quality of reasoning, doing the needed controls, etc. We may have a system that rewards good research little-better than an unbiased coin, but the <b>perception</b> that it works, or that it might work for you, motivates people to do the work needed to survive peer-review.
The good and new comes from no quarter where it is looked for, and is always something different from what is expected.
Letting technical writing people doing the writing won't work. A large part of the scientific writing is the discussion of the experiment, which not only helps the scientist clarify his or her own thoughts and gives insight into future experiments, but also really only is worth reading if the scientist or members of the experimental team do it themselves. Technical writers really only would have the ability to write the experimental procedure, and even then it would be hard. Since science is so specialized you'd have to have technical writers for thousands of subdisciplines, etc. This goes especially true for mathematics, where the writing procedure is very closely related to doing mathematics.
Already because of this, no time for the scientist would be saved. A Google moderation system would have two problems. First, it wouldn't save any time because you still have to have some person doing the reviewing, and secondly you have to have someone qualified doing the reviewing whom you can trust to some extent to review in confidence, for otherwise if there are certain major problems with the paper but a few good ideas, they can be "stolen" by others, which may become a problem.
"What lies behind us, and what lies before us are tiny matters compared to what lies within us." Ralph Waldo Emerson
You mean whine when a POS paper is printed.
G&T's paper in E&E refuses to accept that the presence of another object nearby at temperature will cause the temperature of a hotter object to be higher than it would without. Making a mockery of the laws of thermodynamics which REQUIRE that this be true.
E.g. If you have one end at 300C and the other end at 200C, at some time later, say, 1 minute, the hotter end will have cooled to, say, 280C. If the other end was at 100C rather than 200C, G&T would have the temperature of the hotter end at 280C still.
But that makes a mockery of the thermal conduction laws which include the temperature DIFFERENCE between two ends.
It also makes a mockery of the radiative loss equations for an object in a hot enclosure:
t1 = temperature of hot body
t2 = temperature of enclosure
power loss = sigma * (t1^4 - t2^4)
Yet G&T's paper in E&E would say that this is impossible! Since the temperature drop of the hot body cannot, to them, change if the enclosure is hotter.
Think also that according to G&T, you end up with the hot end colder than the cool end, because it cannot be affected by the temperature of the cool end, therefore it must cool to the same temperature as the situation of absolute zero at the other end. That means it cools to absolute zero. Which is colder than the other end.
How does G&T's paper work even at teenager school physics level of understanding?
It doesn't.
But it got through to be published?
Yet so many are bleating here about how it must be the IPCC "warmists" suppressing papers opposing them. Here we have
a) no opression, the paper is printed
b) the paper is OBVIOUSLY crap
yet somehow, (b) doesn't mean that denialists promote crap papers to bolster their position. According to some, anyway.
How does THAT work?
The government uses peer review to evaluate proposals for science and engineering grants. The same issues probably apply to those evaluations.
I have experienced a situation in which one reviewer recommended turning down a grant for reasons that could be considered as biased, although the bias was groupthink rather than individual. The other reviewers were enthusiastic about funding the grant and regarded it as a potential game-changer. It didn't get funded. A few years later the game-changing nature of the technology was recognized, but it was too late for the original applicant.
the problems of nepotism and tribalism are everywhere.
from internet message boards and professional office environments to national government and international politics.
here's a paradox for you,
someone could do a study on how to eleminate nepotism and tribalism.
then they can put it up for peer review.
In my humanities field, where the vast majority of publications in the last 100+ years were not peer-reviewed, we're seeing pressure from administrators to publish in "peer-reviewed" journals. And, yes, we're seeing the predictable results:
A. Any half-competent referee can figure out pretty quickly the author.
B. While there are data and interpretation in the articles, the author and referee are supposedly skilled at argumentation, and, even when they're not, "nothing new here" will sink any article the ref. doesn't like.
C. Nobody pays us to referee articles; it takes a lot of time, and does nothing for our personal careers.
We had a fun case recently, where one professor (known for shooting down his enemies with the "nothing new" response) ironically turned out to have built his career on plagiarizing other's (non-peer-reviewed, usually) material and submitting them to peer-reviewed journals, with a first page of his own that was sufficient to identify him as the author. He made professor pretty quick. In one case, he plagiarized some guy's published Ph.D. thesis, and submitted it to a journal. The journal sent it to (rumour has it) the victim's Ph.D. advisor to be refereed, and this advisor recommended it for publication without noticing that it was a wholesale rip-off.
In another case, as relatively unknown scholar complained to a peer-reviewed journal that this dude had plagiarized his thesis. From what I understand, the journal submitted the guy's complaint, the plagiarizer's response, and the evidence (which was conclusive) to blind peer review, and the referee sided with the plagiarist. After the extent of the professor's copying has been published, I suppose that journal will bypass peer review and retract the article.
In the sciences, you can argue that a "few bad apples" will cause problems, but in the humanities it's outright broken, and basing hiring and promotions purely on the prestige of the journal or the presence of peer review demonstrably leads to failure.
One problem I see with the approach you suggested is that work is already visible to many but may or may not be accepted in the end. Publications are often rejected not due to a lack of innovation but because of other aspects (evaluation, comparison to other work, ...). A new idea appearing in this paper will then be available to the world while the author is not (yet) credited for his work. Someone else might just pick up his idea, fix the problems, and publish it himself.
This is more or less how it works in Physics, and an increasing number of other projects. The paper in TFA is an example of this. It's published on the preprint archive (arXiv.org - the X is meant to be a chi), where it can be read and commented on. A final version, incorporating feedback, may later by submitted to a more traditional journal.
I am TheRaven on Soylent News
...and then curse again. Let the recursion begin!
Set your phasers on "funky"!
Yup, I've seen exactly the same thing in several sub-fields of computer science. You read about solutions in textbooks published in the '80s, describing established knowledge, and then see brand new research papers in a different area where they ignore all of this prior research and come up with inferior solutions. About 90% of the papers I've read on Grid Computing fit in this description.
I am TheRaven on Soylent News
Anyone having an open eye in academia could confirm this.
The short version of your linked article, for those who don't want to read:
1. Not peer-reviewed => generally not credible science
2. Peer-reviewed => not necessarily credible science
Of course, any scientist already knows this. It's the mainstream press that loves to take crackpot claims at face value, because it's sensational.
I am officially gone from
Someone else might just pick up his idea, fix the problems, and publish it himself.
I don't see the problem. The first publication will always have the earliest time stamp. If people really try to fudge time-stamps, we could check the time-stamps in Google's cache and see who is lying, or in the worst case resort to some formal time-stamping system for publications. If scientist A presents an idea, but it contains a technical flaw, and scientist B fixes it, then everything is working perfectly. Let the scientific community decide which of them deserves acclaim. Scientist A is still free to ask trusted colleagues to review his paper before he publishes it. All I'm saying is that he shouldn't be forced to.
And yet at some point I thought that the emergence of a huge, readily-accessible, and often high-quality literature on the internet would somehow make for the growing lack of general scientific culture among students and researchers. Indeed, it's amazing to see that (at least in my field) a large proportion of new and recent papers are now directly posted online in spite of desperate efforts by Wiley, Springer and such. I think this is quite recent and hugely beneficial to everyone. Provided people really use that information, of course...
"a good paper of theirs gets turned down, or when a bad paper they disagree with gets published"
This basically says it all. No one ever submits a bad paper, in their own opinion, and bad papers are limited to the ones that they disagree with.
In case you missed it, peer review is dead. Get over it. Don't be sad, you can still have your journals and pomp and circumstances. You can drag out the rotted corpse and parade around with your scientist friends. It just carries no weight with the public. We've seen how twisted it can get.
That is why mob review has supplanted peer review. You can call it crowd sourcing or whatever, but it is not going to go away. Any paper that makes it thru peer review now has to make it past the non-peer reviewers, that is, everyone else. If you thought your peers were tough reviewers, then you are going to really hate mob review.
Using a model? Guess what, there are people who love to dissect models. Using statistics, lots of statisticians running around that are going to poke at yours.
My advice, don't fight it. The majority of people just want to get to the truth.
Peer review is often well established scientists shitting on competition to their method, or dissenting views. Each topic has it's fiefdoms battling to protect their own interests. Time after time I have seen good papers sunk because they are competition, and mediocre papers published after the authors simply add citations suggested by the referee – of course, those papers are the referees' and their friends'. I haven't directly been involved in the granting process, but I get the impression that the review boards operate in a similar manner. And through all of this science progresses. Bizarre.
46 & 2
"Imagine how much faster science could advance if we had a system that actually let scientists focus on research, let people trained in technical writing do the reporting, and let Google design a post-publication moderation system to sort out the useful advances from the career posturing."
/. about some massive new breakthrough that is going to cure cancer and halitosis but instead is just another small-medium technical advance and everyone demands to know who the jackass scientists were that overinflated their results, how dare those overfunded eltist blowhard frauds? Yeah, that's what almost happened to me. How exactly will your system avoid massively amplifying this?
A few things:
1. Where is this money to hire 10,000 technical writers going to come from when, excluding tenured faculty, all academic scientists are threatened with losing their jobs due to lack of funds at least one year in three?
2. What exactly is a technical writer going to be doing? They will never have the necessary background to write a review article (seriously--a review article can have anywhere from 150 to well over 300 referenced papers, selected from an even larger pool). A research paper is the description of what 1-30 people have spent the last couple of years doing, plus a small literature review, plus discussion and future directions. Is the technical writer going to go through a few to a few dozen notebooks from all of these people and assemble it, despite not knowing what's going on and somehow predict where it's going to go? Or are they there to check over a rough draft and polish it? If they're the polisher, how much time do I have to spend getting them to understand the terminology, which can be and often is extremely precise? How much time do I have to spend going over the paper, which will have my name on it and not theirs, and which could (especially in the case of a very poorly written or wrong paper) have a huge impact on my career, checking to make sure that the wording is correct? Nuance can be critical in a scientific paper, especially if you're attacking somebody else's results. Communities are small and egos can be large.
3. Your "technical writers write it and stick it on Google model" sounds like science reporting to me. I've had research described by the local paper. Thankfully the reporter was extremely diligent and emailed us what was about to be final copy. The boss was livid with the changes introduced by the reporter's editor to make the work "punchier." Had it gone out as it was it would have been a major embarrassment to the lab but we managed to do triage on it. You know all those stories that get written up on
4. Google-based moderation will increase the incidence of posturing. Ever seen a website pushed to the top of the search heap by artificial means?
5. Lots of scientists read slashdot. We're well aware of the crapfest that is the slashdot moderation system. See any post having anything to do with global warming or evolution. Hell see any post having anything to do with biology and there's some asshole tagging it "whatcouldpossiblygowrong" and at least a dozen highly modded comments demanding that the scientist should be prevented from playing God...when it's poking about in a few systems in a highly benign fashion.
6. There are barriers to publication. Some of them are a good thing. Any ignorant crank can spew crap online, as is their right. A paper in a peer-reviewed journal ideally, and in fact normally, means more. It means that it has been read by peers of the authors who should, and usually do, spot outright bullshit. It means that it has gone through a process of criticism--not necessarily 100% conductive but papers are usually made a little bit better; think add/remove/modify figure X or do this one extra experiment. Is peer review perfect? Hell no! Anyone who's published more than two papers has gotten back comments that are useless and we're all familiar with the occasional 100% bull
They are established scientists with their own self interest. If a small number of referees can have a large effect, imagine what a small number of poor editors could do.
46 & 2
Small correction if I may: Peer reviewed => not necessarily credible science, but you never ever base conclusions on only one paper.
In most cases, a good rebuttal or three will counter the effects of any completely wacko paper that slipped through. In the case I cited, not only were there rebuttals, many of the other editors involved resigned from the journal because they felt that it was a wrong that that paper got published. I think the (larger) system more or less corrected itself there, at least until people started quote-mining stolen e-mails.
Is this study peer reviewed?
In what way were they made one body? If they were merged, then that's simple.
Or do you not manage to cool your bath down by pouring some cold water into the too-hot water?
Your assertion that there is no average temperature is itself a howler.
I have several publications that were significantly improved through the peer review process. When I review papers my goal is not to shoot down the work, rather I try find ways to improve it. Of course there are 'bad' reviewers, who think that reviewing a paper is shredding it to pieces. These are actually easy to spot, because they rarely suggest anything useful and are often ignored by the journal editors. Speaking of which, journal editors are yet another part of the peer review process that is missing from their model
There is a variant on this, when some researchers dig up physics, biology, etc journals, take models/approaches from them and apply them to CS problems. Because in the CS field these results are not really known, the paper is usually considered interesting. Another effect is that while all of the theories are properly referenced (so no plagiarism happens) the reviewers usually assume more work done by the researcher than what actually was done (reviewers rarely read the cited papers).
Imagine how much faster science could advance if we had a system that actually let scientists focus on research, let people trained in technical writing do the reporting
Hell YES! Papers I've read are gawdoffal, seizure inducing bores of things that are incredibly fascinating once digested. There was one paper I had to suffer through at work ten or fifteen years ago that used the word "enumerate" fifteen times in a single paragraph without once using the word "count."
It's a shame we don't have more researchers like the lat Dr. Asimov, who was a published science fiction writer well before obtaining his PhD in biochemistry and becoming a cancer researcher. His nonfiction was clear, precise, unambiguous, educational, while at the same time being every bit as readable and interesting as his fiction.
But lacking scientists like Asimov, we need intelligent, educated writers that can actually construct a coherent sentence that won't induce narcolepsy to write for the scientists. You don't want an astronomer doing biology, why do we have astronomers writing? Writing is not, after all, their field.
Free Martian Whores!
The Precogs sent a ball rolling. They want their Minority Report back!
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
People like sience journalists?
Rethinking email
Some ignoramus put forward a regressive edit on an article lacking citations as evidence of process problems at Wikipedia. Yes, the edit was poorly judged, but an article lacking citations is a cork in a windstorm.
I tend to defend Wikipedia since the interesting question is "Why does it work at all?" One lesson we've learned is that metrics of irreproachability constitute a poor utility function with respect to what most people need most of the time. It falls short when the aim is to slap your name on a document purporting to contain original work adding value to the community.
So yes, Wikipedia falls vastly short of providing a credible foundation for promulgating reputation. The niche it properly occupies is halfway between the card catalog and the dusty tomes of eminence. It's a secondary transfer station in the subway system of knowledge.
When I was eight years old my father showed me how to wind a wire around a large nail and make an electromagnet. It wasn't long before I looked up electromagnetism in an edition of the Encyclopedia Britanica from the 1960s. This was heavy going for a nine year old. I learned practically nothing, the eminence and authority of the text flying completely over my head.
Sugata Mitra: The child-driven education
How did it go? SPOILER ALERT
Going back to my early experience with Britannica, "apart from understanding that the ratio of windings in a transformer established the ratio of voltage and the inverse ratio for current, I've understood hardly anything else." That's not high praise. I'd have made more progress with an internet full of unreliable napkin diagrams explicated in a foreign language.
The problem is that all too often irreproachable equates to inaccessible. You can even find this relationship within the cloisters of the peer review process.
From the font of no knowledge:
My recent insight into peer review is that it operates at two distinct time scales. The short time scale is career advancement with tau = 5 years. The longer time scale is reliable scientific consensus which a time scale of tau = 30 years. (It sometimes takes seven tau periods before the last ripple is smoothed out.) There is no question that peer review is a excellent mechanism to achieve scientific consensus over century time scales.
The scientists themselves are caught up in the dual function of peer view to gate career advancement and to establish the accurate, long term consensus, and thereby sometimes fall into the trap of conflating power with authority.
Within the context of scientific peer review, over the generational time frame, the cliques and rivalries fall by the wayside, the old arguments lose their sex appeal, and the bad actors move onto more topical debates.
Peer review fails to transcend human politi
That's because the period is too short. Statistical significance means "unlikely to have been just "luck"" and that requires a significant number of values.
30 years for climate, though sometimes 15 years is (barely) enough.
So "no significant trend" != "no trend" and 4 years does not a trend make.