"The classic 737 didn't have the 'nose up' problem that the MAX has due to it's new engine placement."
It did actually. Pretty much any aircraft with engines mounted beneath low wings is going to have the issue. Mounting the engines low means you're going to have off-axis thrust which will generally have a positive pitch contribution. Where that can get dangerous is if you're in a low speed stall, a situation in which you have less aerodynamic control authority and your instinct is to add power. If you do that, the increased thrust will push your nose up, making the stall worse, and you won't have the control authority to compensate. 737 pilots are specifically trained *not* to add power in a stall.
The engine placement on the MAX makes the problem worse, to the point where Boeing decided the usual stall recovery procedure wasn't sufficient and some automatic software augmentation was required.
I don't think that's the case. Every report I've seen, including by pilots, has specified that disabling the MCAS system was *not* part of the standard procedures.
There is an existing (and existing since the original 737) procedure for dealing with runaway trim. If something goes wrong and your electric trim starts spooling off, you flip a couple of switches and disable it. There are videos on YouTube. The thing is, that's not what MCAS does. It dials in a bit of trim, then stops. A bit later, if the problem isn't resolved, it dials in some more.
Yes, it seems like a fairly small difference, but pilots are trained to deal quickly with specific problems using specific procedures. If the issue is a bit different, generally you don't want to be following a rote procedure that may not be appropriate. Boeing didn't update the training to specify that the solution to this new type of runaway trim was the same as the old one.
Here's a blog (with a link to a published paper) discussing the error and it's incidence in neuroscience: https://www.theguardian.com/co...
Basically, imagine you've got a control group and two different treatments. You determine that treatment group A is not significantly different than control, but treatment group B is. So you conclude that treatment B works better than treatment A. Implicitly, you've assumed that the non-significance of group B means "no difference" or at least "less difference." Both of these options are fallacies.
That *seems* like a silly example, except that it's present in so many papers. But the error can be much more subtle. Have you ever seen a pair of fMRI images side by side, with blobs on one that aren't on the other? fMRI "scans" are actually statistical maps (a t-test per pixel, hopefully mostly corrected for multiple comparisons) with the coloured blobs indicating p some threshold like 0.05. Comparing two of them is committing the difference of differences sin on a massive scale.
No, there's a reason most people learn how to do Gaussian stats and then stop. MOST measurements have Gaussian error because that's what you get when you average or add up a bunch of random variables. Most measurements are really composites like that, and the noise is quite Gaussian.
It's very important to recognize situations where that's not true though. Counts, surveys, ordinal scales, data that's been transformed, etc. People are legitimately terrible at doing that.
Nope. What needs to happen is we need to give up on this idea that papers must be True. Scientific papers evolved from personal letters and presentations at scientific society meetings. A published paper is basically "hey guys, I think I found this thing that might be cool. Take a look?"
The key being the last part. Have a look and see if you see the same thing. If a bunch of us do, we might be onto something. If, instead, you get a reputation for finding random crap nobody else can replicate, well....
They are absolutely right on that score. People do that ALL the time. I tell my students it is the first sin of statistics because I'm sure it's responsible for the vast majority of committed statistical fallacies.
It's the root of the "difference of difference" error, which is apparently present in 50% of neuroscience papers that have the opportunity to make it.
Meh, you just have to call it something else. Confidence interval. Likelihood ratio. Bayes factor. The dirty little secret is that these things are all mathematically equivalent, or very nearly so, for the vast majority of analyses that are actually conducted.
People like simple solutions. A demon to exorcise. P-values fit that. The real problem is lazy interpretation. Any single result is questionable, no matter how well the data is collected and analyzed. A journal article is not truth, it's an observation. But that uncertainty bothers people.
The publication of inconclusive results is a problem outside physics. Particle physics does have an advantage though: the data and analyses tend to be from only a few places. In parts of physics where Joe Anybody can ask a few undergrads a handful of questions and then write a paper, there's likely less publication of all those inconclusive results.
"That is actually one of the problems with statistical significance. It's only relevant if you're reporting one single result."
No, it's not. That's one of the problems with not knowing what you're doing. You're *supposed to* formulate a detailed hypothesis and analysis plan. That plan should include criteria for deciding what tests you'll do, and what combination of tests you will judge to be supporting each part of the hypothesis. Then you perform multiple comparisons correction based on the number of individual tests that contribute to your conclusions.
Just because nobody bothers to do it doesn't mean the procedure doesn't exist.
Yes. They basically want people to stop saying p 0.05 and instead say p = 0.xxxxx. It's a great idea. As far as I can tell, it mostly happened twenty years ago when people learned how to use computers. Every once in a while I review a paper by someone who didn't get the memo and make them include their actual p-values.
No it's not. *Data* in medicine is inherently messier. That makes good statistics more important. In most cases the actual stats are easier: measurements in medicine tend to be so much crap averaged together that the central limit theorem works quite well, Gaussian assumptions are valid, and the t-test reigns supreme.
One time a particularly annoying research assistant came running down the hall all excited about two recently published papers that showed exactly the situation you mentioned. Look! Contradictory results! Who's wrong? Uh, those results are compatible with each other. One of them had a confidence interval that completely included the other.
p 0.05 is supposed to be kind of a minimum threshold. Higher than that and you really can't draw any conclusions. Less than that, and you might have something. Maybe. It's basically a first level filter.
Does that mean you get some false negatives? Absolutely. And you also get lots of false positives.
There seem to be a bunch of people who want to look at confidence intervals and say "well, a good part of my confidence interval is over here, which is interesting, so this is important!" There are also a bunch of people who think that too many false positives get published and want to clean things up. These two things are at odds with each other.
Judges are at least trained in law, and in most places do tend to specialize in different areas, so they have some professional ability to become familiar with the kinds of evidence that tend to be presented in their field.
I think a mandatory review by independent, court appointed scientists of any scientific evidence is a great idea though.
The science linking glyphosate to lymphoma seems to be a bit iffy. But I don't think that's even relevant. The question should be whether Monsanto covered it up. If they did, I agree, a corporate death penalty is in order. Otherwise, they made a product, which was approved for use according to the regulatory requirements of society. If new evidence shows it doesn't meet those standards then it should be withdrawn with no fault.
Bacon causes cancer. But we deem bacon an acceptable risk. We also deem tobacco and sugar an acceptable risk. Causing cancer or not isn't the issue.
I'm not saying Monsanto is innocent, or shouldn't be punished. I'm saying selecting twelve people randomly from the populace and asking them to decide isn't a great way of establishing guilt, or reasonable consequences.
Well, the bribery is contemporary. They bribed a government official in Indonesia responsible for assessing one of their GMO crops, and covered it up in their books. The US justice department went after them but they got a deferred prosecution agreement; the same kind of deal that's causing a big scandal in Canada at the moment, except ours is a big construction company that seems to have bribed everyone except me (dammit, am I not good enough?).
Monsanto is usually associated with evil for their patented crop shenanigans. I'm sure there was lots of ethically questionable stuff going on there as well, but most of it that's actually documented seems to be legal, if maybe a bit aggressive. The highest profile cases of suing individual farmers really do seem to be the farmers being smartasses, like the guy who had a few GMO seeds fall on his field, then got caught growing 97% roundup ready crops.
They don't seem to deserve their status as *most* hated corporation, but still lots of reasons to dislike a big chemical company that graduated from manufacturing chemical weapons to pushing the boundaries with gene patents and ended up taking over a good portion of the world's food supply.
But that's pretty much my point. A justice system isn't supposed to be about convincing twelve people to hate someone or something. It's supposed to be about evidence and consequences that serve society.
Well, Monsanto used to be the largest manufacturer of PCBs in the US, a fair amount of which apparently got dumped into some rivers. They paid out $700 million to some people in Alabama as a settlement. Something along the same lines in Wales. They were also involved in making agent orange for the US to use in Vietnam, and then denied a connection between exposure and US veterans' medical problems. They settled that one too. They've also admitted to illegal bribery and accounting fraud.
Yup! And the great thing is that with ridiculous liability payouts in the US, everybody (you eat food, right?) gets to pay lots of money for those random people's decision.
Monsanto is hated for good reasons, but it should be held to account based on real evidence.
The US jury system is an interesting little historical oddity. It's the only country that uses juries extensively in civil cases.
Boeing didn't want pilots to have to go through a full training course for the new 737 as they would for a new aircraft, so they had to convince the FAA that it was just a variant, meaning only abbreviated training was required. The MCAS system is designed to reduce the differences in flying characteristics between the new and old planes, and intervene only when the plane is approaching a dangerous part of its flight envelope. That should never happen unless the pilot has screwed up, and if it does happen, the MCAS just gives a little tap and makes it all better. If something were to go wrong, the fix is the same as an existing procedure anyway.
It's a nice little chain of small misjudgements and unforeseen consequences, putting bandaids on little problems and then forgetting how they can add up. I imagine the whole thing will be a case study in systems integration and engineering courses in the future.
Slashdot Mirror
Trying to read the summary was bad enough.
There was no stall. The MCAS system was engaged due to a malfunctioning angle of attack system.
The incidents happened in Indonesia and Ethiopia, with Indonesia having first crack.
"The classic 737 didn't have the 'nose up' problem that the MAX has due to it's new engine placement."
It did actually. Pretty much any aircraft with engines mounted beneath low wings is going to have the issue. Mounting the engines low means you're going to have off-axis thrust which will generally have a positive pitch contribution. Where that can get dangerous is if you're in a low speed stall, a situation in which you have less aerodynamic control authority and your instinct is to add power. If you do that, the increased thrust will push your nose up, making the stall worse, and you won't have the control authority to compensate. 737 pilots are specifically trained *not* to add power in a stall.
The engine placement on the MAX makes the problem worse, to the point where Boeing decided the usual stall recovery procedure wasn't sufficient and some automatic software augmentation was required.
I don't think that's the case. Every report I've seen, including by pilots, has specified that disabling the MCAS system was *not* part of the standard procedures.
There is an existing (and existing since the original 737) procedure for dealing with runaway trim. If something goes wrong and your electric trim starts spooling off, you flip a couple of switches and disable it. There are videos on YouTube. The thing is, that's not what MCAS does. It dials in a bit of trim, then stops. A bit later, if the problem isn't resolved, it dials in some more.
Yes, it seems like a fairly small difference, but pilots are trained to deal quickly with specific problems using specific procedures. If the issue is a bit different, generally you don't want to be following a rote procedure that may not be appropriate. Boeing didn't update the training to specify that the solution to this new type of runaway trim was the same as the old one.
Probably for the same reason the light to indicate that there's a disagreement is optional. Money.
I absolutely include designing the study and gathering data under the heading of statistics.
Here's a blog (with a link to a published paper) discussing the error and it's incidence in neuroscience: https://www.theguardian.com/co...
Basically, imagine you've got a control group and two different treatments. You determine that treatment group A is not significantly different than control, but treatment group B is. So you conclude that treatment B works better than treatment A. Implicitly, you've assumed that the non-significance of group B means "no difference" or at least "less difference." Both of these options are fallacies.
That *seems* like a silly example, except that it's present in so many papers. But the error can be much more subtle. Have you ever seen a pair of fMRI images side by side, with blobs on one that aren't on the other? fMRI "scans" are actually statistical maps (a t-test per pixel, hopefully mostly corrected for multiple comparisons) with the coloured blobs indicating p some threshold like 0.05. Comparing two of them is committing the difference of differences sin on a massive scale.
No, there's a reason most people learn how to do Gaussian stats and then stop. MOST measurements have Gaussian error because that's what you get when you average or add up a bunch of random variables. Most measurements are really composites like that, and the noise is quite Gaussian.
It's very important to recognize situations where that's not true though. Counts, surveys, ordinal scales, data that's been transformed, etc. People are legitimately terrible at doing that.
Nope. What needs to happen is we need to give up on this idea that papers must be True. Scientific papers evolved from personal letters and presentations at scientific society meetings. A published paper is basically "hey guys, I think I found this thing that might be cool. Take a look?"
The key being the last part. Have a look and see if you see the same thing. If a bunch of us do, we might be onto something. If, instead, you get a reputation for finding random crap nobody else can replicate, well....
They are absolutely right on that score. People do that ALL the time. I tell my students it is the first sin of statistics because I'm sure it's responsible for the vast majority of committed statistical fallacies.
It's the root of the "difference of difference" error, which is apparently present in 50% of neuroscience papers that have the opportunity to make it.
Meh, you just have to call it something else. Confidence interval. Likelihood ratio. Bayes factor. The dirty little secret is that these things are all mathematically equivalent, or very nearly so, for the vast majority of analyses that are actually conducted.
People like simple solutions. A demon to exorcise. P-values fit that. The real problem is lazy interpretation. Any single result is questionable, no matter how well the data is collected and analyzed. A journal article is not truth, it's an observation. But that uncertainty bothers people.
The publication of inconclusive results is a problem outside physics. Particle physics does have an advantage though: the data and analyses tend to be from only a few places. In parts of physics where Joe Anybody can ask a few undergrads a handful of questions and then write a paper, there's likely less publication of all those inconclusive results.
"That is actually one of the problems with statistical significance. It's only relevant if you're reporting one single result."
No, it's not. That's one of the problems with not knowing what you're doing. You're *supposed to* formulate a detailed hypothesis and analysis plan. That plan should include criteria for deciding what tests you'll do, and what combination of tests you will judge to be supporting each part of the hypothesis. Then you perform multiple comparisons correction based on the number of individual tests that contribute to your conclusions.
Just because nobody bothers to do it doesn't mean the procedure doesn't exist.
Yes. They basically want people to stop saying p 0.05 and instead say p = 0.xxxxx. It's a great idea. As far as I can tell, it mostly happened twenty years ago when people learned how to use computers. Every once in a while I review a paper by someone who didn't get the memo and make them include their actual p-values.
No it's not. *Data* in medicine is inherently messier. That makes good statistics more important. In most cases the actual stats are easier: measurements in medicine tend to be so much crap averaged together that the central limit theorem works quite well, Gaussian assumptions are valid, and the t-test reigns supreme.
Sad but true. And I do medical research.
One time a particularly annoying research assistant came running down the hall all excited about two recently published papers that showed exactly the situation you mentioned. Look! Contradictory results! Who's wrong? Uh, those results are compatible with each other. One of them had a confidence interval that completely included the other.
I much prefer Bayes factor hacking. Sounds way fancier.
p 0.05 is supposed to be kind of a minimum threshold. Higher than that and you really can't draw any conclusions. Less than that, and you might have something. Maybe. It's basically a first level filter.
Does that mean you get some false negatives? Absolutely. And you also get lots of false positives.
There seem to be a bunch of people who want to look at confidence intervals and say "well, a good part of my confidence interval is over here, which is interesting, so this is important!" There are also a bunch of people who think that too many false positives get published and want to clean things up. These two things are at odds with each other.
Judges are at least trained in law, and in most places do tend to specialize in different areas, so they have some professional ability to become familiar with the kinds of evidence that tend to be presented in their field.
I think a mandatory review by independent, court appointed scientists of any scientific evidence is a great idea though.
The science linking glyphosate to lymphoma seems to be a bit iffy. But I don't think that's even relevant. The question should be whether Monsanto covered it up. If they did, I agree, a corporate death penalty is in order. Otherwise, they made a product, which was approved for use according to the regulatory requirements of society. If new evidence shows it doesn't meet those standards then it should be withdrawn with no fault.
Bacon causes cancer. But we deem bacon an acceptable risk. We also deem tobacco and sugar an acceptable risk. Causing cancer or not isn't the issue.
I'm not saying Monsanto is innocent, or shouldn't be punished. I'm saying selecting twelve people randomly from the populace and asking them to decide isn't a great way of establishing guilt, or reasonable consequences.
Well, the bribery is contemporary. They bribed a government official in Indonesia responsible for assessing one of their GMO crops, and covered it up in their books. The US justice department went after them but they got a deferred prosecution agreement; the same kind of deal that's causing a big scandal in Canada at the moment, except ours is a big construction company that seems to have bribed everyone except me (dammit, am I not good enough?).
Monsanto is usually associated with evil for their patented crop shenanigans. I'm sure there was lots of ethically questionable stuff going on there as well, but most of it that's actually documented seems to be legal, if maybe a bit aggressive. The highest profile cases of suing individual farmers really do seem to be the farmers being smartasses, like the guy who had a few GMO seeds fall on his field, then got caught growing 97% roundup ready crops.
They don't seem to deserve their status as *most* hated corporation, but still lots of reasons to dislike a big chemical company that graduated from manufacturing chemical weapons to pushing the boundaries with gene patents and ended up taking over a good portion of the world's food supply.
But that's pretty much my point. A justice system isn't supposed to be about convincing twelve people to hate someone or something. It's supposed to be about evidence and consequences that serve society.
Well, Monsanto used to be the largest manufacturer of PCBs in the US, a fair amount of which apparently got dumped into some rivers. They paid out $700 million to some people in Alabama as a settlement. Something along the same lines in Wales. They were also involved in making agent orange for the US to use in Vietnam, and then denied a connection between exposure and US veterans' medical problems. They settled that one too. They've also admitted to illegal bribery and accounting fraud.
Yup! And the great thing is that with ridiculous liability payouts in the US, everybody (you eat food, right?) gets to pay lots of money for those random people's decision.
Monsanto is hated for good reasons, but it should be held to account based on real evidence.
The US jury system is an interesting little historical oddity. It's the only country that uses juries extensively in civil cases.
They didn't want to confuse "the average pilot."
Boeing didn't want pilots to have to go through a full training course for the new 737 as they would for a new aircraft, so they had to convince the FAA that it was just a variant, meaning only abbreviated training was required. The MCAS system is designed to reduce the differences in flying characteristics between the new and old planes, and intervene only when the plane is approaching a dangerous part of its flight envelope. That should never happen unless the pilot has screwed up, and if it does happen, the MCAS just gives a little tap and makes it all better. If something were to go wrong, the fix is the same as an existing procedure anyway.
It's a nice little chain of small misjudgements and unforeseen consequences, putting bandaids on little problems and then forgetting how they can add up. I imagine the whole thing will be a case study in systems integration and engineering courses in the future.