Nuclear Missile Command Drops Grades From Tests To Discourage Cheating

An anonymous reader writes: Earlier this year, just over half of the military officers put in charge of U.S. nuclear launch facilities were implicated in an exam cheating scandal. The Air Force conducted regular exams to keep officers current on the protocols and skills required to operate some of the world's most dangerous weapons. But the way they graded the test caused problems. Anything below a 90% score was a fail, but the remaining 10% often dictated how a launch officer's career progressed. There might not be much functional difference between a 93% and a 95%, but the person scoring higher will get promoted disproportionately quicker. This inspired a ring of officers to cheat in order to meet the unrealistic expectations of the Air Force. Now, in an effort to clean up that Missile Wing, the Air Force is making the exams pass/fail. The officers still need to score 90% or higher (since it's important work with severe consequences for failure), but scores won't be recorded and used to compete for promotions anymore. The Air Force is also making an effort to replace or refurbish the aging equipment that runs these facilities.

Comments

Not sure how well this will stop cheating

[robstout]

You still need a 90% to pass. At least you no longer have folks clamoring for the top score.

Re:Not sure how well this will stop cheating

[ERJ]

They had a story on the radio last night about this. The issue is that everyone (well, most everyone) was getting a passing grade. When they came in and gave an unexpected test the average score was 95%. The problem is that promotions were based on the grades. So, people were not cheating to pass but instead to be "perfect" in order to look better for promotion.

Re:Not sure how well this will stop cheating

[misexistentialist]

Obviously promotions should go to the candidate who has launched the most missiles.

Re:Not sure how well this will stop cheating

[TWX]

If the average score was 95%, then wouldn't that mean that the general field of scores was falling somewhere in the 100%-90% range, possibly with disproportionately more above 95% to offset those falling below 90%?

Frankly, the danger is that we can't really know what the actual scores, without the pervasive cheating, would have been. There might well be 30% that passed that would have failed without cheating.

If over 50% of the participants were able to cheat, then it sounds like they need to work on their testing procedures in addition to their scoring metric. In this day and age it's not all that difficult to random-generate tests and source questions from sets so that one set may have 30 questions that apply to the same topic and three are randomly chosen; it means that for a 100 question tests there'd need to be probably a thousand questions grouped into sets, but if it's that important then it's not unrealistic to do the major work once and to maintain it properly from then on out.

Re:Not sure how well this will stop cheating

[Zeek40]

I don't remember a "Catholic State" , remind me again?

The Papal States, Vatican City, and arguably The Holy Roman Empire

The Air Force is also making an effort to replace

[Ronin Developer]

... or refurbish the aging equipment that runs these facilities.

Which equipment? The testing equipment, the launchers, missiles, terminators?

Sample Question

[PvtVoid]

Q: What is the launch code for all U.S. Minuteman missiles?

A. 00000000

Re:Sample Question

[geekoid]

I'll let you in on a secret.
Just having the code stopped the problem they created the code to fix. Since it fixed the problem, it makes sense for it to be an easy number. I suspect the rpesident will be under a little stress if he had to actually use it, so you want to minimize mistakes.

The security guy makes a classic security review mistake. Ignoring why and the practicality and the impact.

So that actual number is irrelevant, and no, you don't punch the code and then missiles launch.

Unrealistic expectaitions?

[bickerdyke]

Unrealistic expectations?

Not for the best of the Best of the BEST, SIR!

Re:What partisan wrote this?

[geekoid]

No. the difference between 93 and 95 percent is irrelevant. This isn't someone getting 80% and someone else getting 95%.

You're looking hard for an issue that isn't hear. I was in SAC. You, OTOH, should probably read my sig.

take the men out of the loop

[Joe_Dragon]

take the men out of the loop

Re:What partisan wrote this?

[dbrueck]

Maybe, but I read it it slightly differently. 'Disproportionate' means 'too large or too small in comparison with something else'.

So I took it to mean that someone who got two percentage points higher on their test ended up being promoted at a much higher rate than would generally be expected for that small of a difference in scores.

As a made up example, if you scored two percentage points higher on your final than me, and all else equal, as a result over the course of your career that single test caused you to get promoted at a rate that was double the rate at which I got promoted, then one could realistically say that the rate of promotion is disproportionately higher than expected because in many cases a two percentage point difference would not be statistically significant while a doubling would be. The two are so different that it doesn't seem unreasonable to call them disproportionate. To be clear, those aren't the numbers from the article, but the article was just suggesting that there was that type of mismatch - small test score difference leading to a large, long term difference.

And that long term effect is key because it magnifies the issue: a very small difference in a score on one test you take early in your career has large ramifications for potentially decades? That creates a large incentive to cheat. And that's just it - I'm not arguing what's fair or unfair or what's right or wrong in this scenario, just saying that it sounds like there were some pretty big incentives to cheat.

"unrealistic expectations of the Air Force" ?

[jimmifett]

Are you kidding me? 90% competency in protocol is unrealistic?
When it comes to a nation's nuclear weapons, I don't want a B or less, I want the person with the A managing the switch.

Has this country become so lazy and apologetic towards 'bad grades hurt feelings' pansies that they will pass everyone?
If I recall from ye olde school days:
A = 94-100%
B = 84-93%
C = 74-83%
D = 64-73%
F = 64%

C shouldn't even be a passing grade. It was never acceptable in my house. C's wait tables. D's are garbage collectors, F's live in government housing and vote for the people that continue to favor teacher's unions over children's educations and to take money from those the paid attention and succeeded to support them.

Everyone should be striving to be top in their desired field, whatever that be, from software dev to mechanic to entertainer. And if one isn't in their desired field, then they should be spending their free time to improve themselves to get into their desired fields.

90% is not unrealistic, it's a MINIMUM requirement for success.

Re:"unrealistic expectations of the Air Force" ?

[Kelbear]

Who are you talking to?

The summary states that 90% will continue to remain the minimum requirement for success, as it was before.

  The "unrealistic expectation" was making promotion decisions based solely on the difference between 93% vs 95% on the test score. A 90% was the equivalent of a "D". The problem was that to be promoted, the expectation was to hit that 2% difference (which may very well be a single question on the test) and that would mean the difference between being promoted or not being promoted (which means a host of different responsibilities). It's nice to have a firm metric you can point to in order to justify the decision that was made.

The problem is that the single question out of many, was the deciding factor between 2 candidates to take on a multitude of increased responsibilities, their qualification for which may not be accurately gauged by a single question out of many on a graded exam. For comparison, let's say you have 2 programmers take a test, programmer A gets 93%, and programmer B gets 95%. They both clearly have a very strong grasp of the requisite knowledge, which would you promote? The 95%? Well what if programmer B has excellent book-retention, but is lazy and disorganized in his personal and professional life? Maybe he has poor leadership skills over the people that he/she oversees? The idea of promotion based on a tiny difference in already-strong test scores starts to fall apart.

Re:"unrealistic expectations of the Air Force" ?

[tlhIngan]

I get that, I'm not commenting on that fact. I'm commenting on the 90% or better is an unrealistic expectation in the summary.

Now, as to 95 vs 93% affecting promotion speed? I don't see the problem there either. The person who studied harder (or better) than his rival should definitely have their higher score have more weight when considered for promotion.

Did both pass the test? Yes, but candidate x passed better than y. If all other things are equal, then X should get the promotion.

Except well, a 2% difference on an exam score really means you got one question wrong on a multiple-choice test. Unless the test is hundreds of questions long, 2% isn't that many questions and scores can vary even amongst the same person because of the questions asked. I mean, assuming the tests are generated uniquely for each person from a question pool, it can be expected that they may get 43 questions right on one exam, 42 on another. Luck of the draw, and yet has extreme career implications.

Hell, if you want to take this to the extreme, visit Asia. Where the exam you take right out of high school literally determines your future. Get 95% or higher and you'll get scholarships to study at the world's finest universities - US, UK, wherever you want. Get 90% and you'll go to an in-country university. Below that - you're a tradesperson and a shame on your family. Go kill yourself.

Yes, you'll actually find the rate of teen suicides disproportionally high - many succumb to the pressure before the exam, many immediately after (they didn't think they did well, might as well end it before shame of receiving the marks), and many after getting their marks rather than face their families. Yes, they'd rather commit suicide than explain to their parents that they got an 89%.

Oh yeah, pressure to cheat is high too, for obvious reasons.

Basically unless the exam is hundreds to say, a thousand questions long, the difference between a 93 and 95% is well within statistical error. But if you get lucky and get a test that had one easier question than another, that means you'll get promotions way quicker, well, there's a lot of incentives to cheat.

Move to pass/fail is the correct thing to do. The only way to make a 2% difference matter is to make sure each question counts for just a tiny bit so it averages out. If it's a 1000 question test, 93% means one person answered 930 questions correctly, while 95% means they answered 950 questions correctly. The 20 extra questions eliminate the chance it was a question that was slightly easier, and that someone actually did study harder, and practice harder.

Re:"unrealistic expectations of the Air Force" ?

[geekoid]

Do you know why they want to get promoted in missile command? to get out of missile command.
Look, I was in Missile Command(F.E. Warren*). You're whole view on this is wrong. Your view is based on a school math like grading system. That in no way applies here.

*Fuck Everybody Warren.

Re:What partisan wrote this?

[quantaman]

I disagree. I'd be surprised if the standard deviation for an individual test taker was less than 2%. If you take the office who scored 95% and the officer who scored 93%, then made them take another test on the same subject, I wouldn't be remotely surprised if the scores were reversed. This is a good rational to make the test pass/fail and drop the grades.