Slashdot Mirror
Cheaters Exposed Analyzing Statistical Anomalies
Hugh Pickens writes "Proctors and teachers can't watch everyone while they take tests — not when some students can text with their phones in their pockets, so with tests increasingly important in education — used to determine graduation, graduate school admission, and — the latest — merit pay and tenure for teachers, Trip Gabriel writes that schools are turning to 'data forensics' to catch cheaters, searching for data anomalies where the chances of random agreement are astronomical. In addition to looking for copying, statisticians hunt for illogical patterns, like test-takers who did better on harder questions than easy ones, a sign of advance knowledge of part of a test or look for unusually large score gains from a previous test by a student or class. Since Caveon Test Security, whose clients have included the College Board, the Law School Admission Council, and more than a dozen states and big city school districts, began working for the state of Mississippi in 2006, cheating has declined about 70 percent, says James Mason, director of the State Department of Education's Office of Student Assessment. 'People know that if you cheat there is an extremely high chance you're going to get caught,' says Mason."
...the lesson here is to cheat just barely enough to get by, or to consistently cheat all the way through. A better lock just makes a better lock picker. Not that I'm saying we shouldn't discourage cheating, but the issue is why students want to cheat rather than gain the knowledge. I understand that testing is one of the best ways to gauge knowledge, but we're too focused on testing and no where near focused enough on education. Tests are proven to decrease a students interest in subjects. The solution is to move to more objective based learning where students complete projects or applications showing their knowledge of the material. Obviously testing is easier, but if our goal is education, we need to change.
Gotta love this line:
Interesting. So their people have time to explain the methods to non-peers ... but not enough time to write them up for peer review.
And yet, there will also be false positives.
I did poorly on one test. Noticing this, I studied hard and greatly improved my grade in the next test. Would this flag up a warning that I'm a cheater?
Or for that matter, doing better on the 'harder' questions. Perhaps I decided to concentrate on doing those questions because they offered higher marks than the easier questions, or because I had a natural aptitude for some elements. I may have elected to study those materials harder.
Professors can't rely solely on 'statistical anomalies'. Illogical patterns may well have an explanation that has nothing at all to do with cheating or advanced knowledge of the test. Of course, we all know just how lazy a minority of our lecturers are.... and how likely they'd be to take the word of this agency as gospel.
So there I was, scribbling down some notes off the PC screen by hand, when I reached for the keyboard and Ctrl-S'd.
When I was in university I wrote a group paper with one guy whose wife was a professional editor, she helped us out by reviewing it and making suggestions, we had to fight not to get expelled because our paper was "too well written" to be our own work.
You had it proof read and edited by a professional. You did in fact "cheat", the work was not completely yours. This is essentially the same as buying your term paper on-line. You can rationalize it all you want, but the bottom line is your professor expected the work to be yours, not that of a "professional editor". I assume you went on to get an MBA?
If you want news from today, you have to come back tomorrow.
I disagree. Some portion of students will cheat as long as (in theory):
[benefit of higher grade] - [cost of honestly achieving higher grade] > {[benefit of cheating] - [cost of cheating]} * [ risk of getting caught cheating] * [value of punishment for cheating].
The real problem is that people are lazy and want to get the best return for the smallest investment. This cannot be fixed, it is human nature.
So we tip the equation in favor of not cheating, by either/and
1. Making the punishment so extreme (expulsion) that even if the risk of getting caught is low, cheating is not a good idea. The problem with this approach is that as the risks of getting caught decrease, people dismiss the risk as zero. This is a known problem with how humans interpret probability and risk dealing with VLNs and VSNs.
2. Increasing the chance of being caught. This is a problem because of the costs involved, as well as an "arms race" between proctors and students.
Note that the equation is also affected by the fact that cheating has become easier, and thus cheaper. There is also a factor for personal inhibitions against cheating, but I'm not sure how to fit it into the model.
PS. sorry for the messy formula.
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
I remember an example from my high school days. It was a statistics class and the teacher had three classes of the exact same subject. He would give the same test to all the classes. One would think that the situation was ripe for massive cheating and you would be correct.
When the first test was given, he purposely allowed the students to take their tests from a pile. Of course, there were more tests taken than there were students in the first class. These purloined tests made their way into the hands of students from the second and third classes who did unusually good on that test. By the way, the first class was composed of the best students. When the test scores were analyzed of course the second and third classes did much better than the first class. When the tests were handed back, everyone noticed that they had unexpectedly bad test scores. Here is what happened.
For the first class, the teacher lumped their scores in with the other two classes. Of course, this skewed their curve so that they received low marks. For the second and third classes, he did not do this and their average was so high that it was impossible to get a good mark. Their curve was "skewed" also. He then went into length about how he used statistics to teach us a lesson about cheating. He explained that the good students were aware of the cheating and did nothing, as such they were enabling the cheating. The students that distributed the stolen tests actually were damaging their test scores so they lost. The students that used the stolen tests also lost out because to the "skewed" curve. In the teachers words, he "SKEWED" all of us. Needless to say, we were all leery about cheating in his classes after that.
It's not a scenario, it's a model. Any real-world scenario will fit into that model... and if not, the model, like any other, can be adjusted.
Well, no shit. That's why there is a factor for risk of getting caught.
Are you sure about that? Do you really believe that deterrent punishments have no effect on the likelihood of people to break the rules? Deterrent punishments work, to a certain extent. This is why people who believe they are anonymous or hidden will break rules they'd never break if they thought they'd be held accountable. If the punishment is severe, it may be resented, but it acts a deterrent. Whether the deterrence effect is significant enough to overcome the benefits of the behavior depends on the actual scenario.
The rest of your post also fits into my model. That online exam? Perfect example... easy to cheat, not likely to get caught... thus, people will cheat as it is less costly than actually doing the work required to honestly get a good grade.
Exactly. If your goal is to reduce cheating, you need to consider the factors that make cheating worthwhile to students... which is described by the equation I wrote out.
I think we're basically stating the same thing... but approaching it from different angles.
The main place where I think we differ is that you seem to assign the cause of cheating to the class environment. I assign it to the student, but the decision each student makes is influenced by many factors: their perceived reward from cheating, the punishment for getting caught, the risk of getting caught, etc.
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
I did this type of work back when I was in college (the 1980's), let me share a bit about the process, or at least how we did it. What we did was count up the matching incorrect answers between people. For most people who entered in a number of incorrect answers, the same wrong answer count tended to be low. For those who were copying answers, the matching wrong count was high. However, this by itself was not sufficient to accuse someone of cheating. What we did was give the names to the exam proctors, who during the next exam would either ensure that these people where physically seperated, or who to keep an eye on. Note this was in the 80's, before the common availability of wireless communication. But who to keep an eye on would still work.
Some observations from the work we did. One: if your going to cheat, cheat off a smart person (duh!). With fewer wrong answers, you blend more into the crowed. Second: this technique would false positive on people who study together. That's why we never used it to accuse someone, but only who to seperate or keep an eye on. In addition, it would give no clue of who was cheating from whom. Three: to counteract this, some professors would give out several slightly different versions of exams. It would be very hard explaining how you got all the right answers for an exam you did not take, but all wrong for the one you did take.
The elephant in the room is that in American society, people are in general far more educated than they need to be. I have a bachelor's degree, none of the knowledge gained in pursuit of which[1] is of any help to me whatsoever in the course of my daily life, whether personally or professionally. And while I have no data, if the scuttlebutt is to be believed, I am very not alone in this. Furthermore, even a lot of the knowledge I gained in high school has proven completely useless to me[2]: outside of a class, I have never used any mathematics more advanced than the Pythagorean theorem.
As long as unreasonable academic credentials are required for jobs, though, there will be incentive for people to cheat—that is to say, cheating is not the problem; it's a symptom. Elminate the degree inflation in the job market, and you'll eliminate most of the cheating.
[1] Aside, possibly, from reading comprehension and writing skills, but those were not developed in college—I tested out of all the required English classes and all but one of the history classes—merely honed.
[2]The important words here are of course “to me.” I know lots of things which, objectively, are of no utilitarian use in my situation, but which I have sought out the knowledge of simply because it interested me; my enjoyment of them constitutes their usefulness.
Many people with ADHD consistently do better at "hard" things than at "easy" things. I'm a pretty decent programmer, I could do calculus (though not always very well) by about 4th grade... and even now I can't do single-digit arithmetic with complete reliability. Assuming that people who show the pattern I will show on basically any test of my ability in any field I work in are "cheating" is a poor tactic at best.
As a possible indicator, maybe useful. As a definite rule, hell no.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/