Slashdot Mirror
Cheaters Exposed Analyzing Statistical Anomalies
Hugh Pickens writes "Proctors and teachers can't watch everyone while they take tests — not when some students can text with their phones in their pockets, so with tests increasingly important in education — used to determine graduation, graduate school admission, and — the latest — merit pay and tenure for teachers, Trip Gabriel writes that schools are turning to 'data forensics' to catch cheaters, searching for data anomalies where the chances of random agreement are astronomical. In addition to looking for copying, statisticians hunt for illogical patterns, like test-takers who did better on harder questions than easy ones, a sign of advance knowledge of part of a test or look for unusually large score gains from a previous test by a student or class. Since Caveon Test Security, whose clients have included the College Board, the Law School Admission Council, and more than a dozen states and big city school districts, began working for the state of Mississippi in 2006, cheating has declined about 70 percent, says James Mason, director of the State Department of Education's Office of Student Assessment. 'People know that if you cheat there is an extremely high chance you're going to get caught,' says Mason."
The headline should be "Cheaters Exposed By Analyzing Statistical Anomalies"? I thought the cheaters themselves were doing the analyzing, to get ahead of the cheat detection.
This is a substitute for a clever sig that fits within the maximum number of characters.
If I fall into the anomaly category without cheating, I'll be screwed. What can I demonstrate in my defense? Not much. I find it hard to believe they can prove that you cheated without actually video-taping you cheating or something along those lines.
Anomalies are what they are, data anomalies, nothing more and nothing less.
...the lesson here is to cheat just barely enough to get by, or to consistently cheat all the way through. A better lock just makes a better lock picker. Not that I'm saying we shouldn't discourage cheating, but the issue is why students want to cheat rather than gain the knowledge. I understand that testing is one of the best ways to gauge knowledge, but we're too focused on testing and no where near focused enough on education. Tests are proven to decrease a students interest in subjects. The solution is to move to more objective based learning where students complete projects or applications showing their knowledge of the material. Obviously testing is easier, but if our goal is education, we need to change.
I would like to apply this idea to see when a politician is lying. But then I realized it would just overload. So then I figure we should try to see if it can detect when they are telling the truth. That way you work with a much smaller data set. Damn near zero. So it looks to be a total failure.. Nevermaind
For justice, we must go to Don Corleone
Gotta love this line:
Interesting. So their people have time to explain the methods to non-peers ... but not enough time to write them up for peer review.
so, what happens if the student doesn't have a cellphone, or has two, or is borrowing one from a friend?
Teacher: I'm sorry I am going to have to fail most of you for cheating.
Students: But we didn't cheat!
Teacher: Then how do you explain how so many of you came to the same conclusion that 2+2=4?
"A person is smart. People are dumb, panicky dangerous animals and you know it." - K
"Your goal is not to catch a bunch of people and hang them," Dr. Fremer said. "Your goal is to have fair and valid testing."
I hope administration agrees. When I was in university I wrote a group paper with one guy whose wife was a professional editor, she helped us out by reviewing it and making suggestions, we had to fight not to get expelled because our paper was "too well written" to be our own work.
They say a little knowledge is a dangerous thing, but it's not one half so bad as a lot of ignorance. - Terry Pratchett
And yet, there will also be false positives.
I did poorly on one test. Noticing this, I studied hard and greatly improved my grade in the next test. Would this flag up a warning that I'm a cheater?
Or for that matter, doing better on the 'harder' questions. Perhaps I decided to concentrate on doing those questions because they offered higher marks than the easier questions, or because I had a natural aptitude for some elements. I may have elected to study those materials harder.
Professors can't rely solely on 'statistical anomalies'. Illogical patterns may well have an explanation that has nothing at all to do with cheating or advanced knowledge of the test. Of course, we all know just how lazy a minority of our lecturers are.... and how likely they'd be to take the word of this agency as gospel.
So there I was, scribbling down some notes off the PC screen by hand, when I reached for the keyboard and Ctrl-S'd.
When the anomalies are highly unlikely -- their random occurrence, for example, is greater than one in one million -- Caveon flags the tests for further investigation by school administrators.
It does sound like they are acting responsibly and merely saying such marked tests need further scrutiny by the school, and only marking tests with a really low probability to be just lucky... but who knows what the individual schools do.
With more than 100,000 students tested, proctors could not watch everyone -- not when some teenagers can text with their phones in their pockets.
And how exactly did they read those text messages if their phone was in their pocket?
'The tyrant will always find pretext for his tyranny.' - Aesop's Fables
so, what happens if the student doesn't have a cellphone, or has two, or is borrowing one from a friend?
Clearly all students should have to take the test naked to ensure they don't have a hidden cellphone.
Can I volunteer to be a proctor at some all female university?
Some emotional disorders can cause lapses in concentration in which complex questions are solved and easy questions answered incorrectly. I would think that any accusation or action in regard to cheating best have one heck of a strong proof or the lawyers will have a feeding frenzy.
There seems to be an increasing emphasis by schools on "catching cheaters". This seems to be missing the point.
We send our kids to school not so they can pass tests. I honestly do not care if my kid gets an "A" or an "F" on the test; I care that he actually learns the material. Tests are a tool that educators can use to help them determine if a child is learning the material but passing grades shouldn't be the goal. If students are cheating on tests then you need to look at the reason why. Is the material being presented in a way that is too hard for the child to understand? Is it not being presented in a way that interests the student? If a student is intererested, he will learn. If he learns, then he has no need to cheat.
Stop spending money on anti-cheating technologies. Spend money on improving the methods of education.
When teacher pay was linked to student achievement in standardized tests, some Chicago teachers decided to 'help' students during tests, and were subsequently caught through statistical analysis
I remember an example from my high school days. It was a statistics class and the teacher had three classes of the exact same subject. He would give the same test to all the classes. One would think that the situation was ripe for massive cheating and you would be correct.
When the first test was given, he purposely allowed the students to take their tests from a pile. Of course, there were more tests taken than there were students in the first class. These purloined tests made their way into the hands of students from the second and third classes who did unusually good on that test. By the way, the first class was composed of the best students. When the test scores were analyzed of course the second and third classes did much better than the first class. When the tests were handed back, everyone noticed that they had unexpectedly bad test scores. Here is what happened.
For the first class, the teacher lumped their scores in with the other two classes. Of course, this skewed their curve so that they received low marks. For the second and third classes, he did not do this and their average was so high that it was impossible to get a good mark. Their curve was "skewed" also. He then went into length about how he used statistics to teach us a lesson about cheating. He explained that the good students were aware of the cheating and did nothing, as such they were enabling the cheating. The students that distributed the stolen tests actually were damaging their test scores so they lost. The students that used the stolen tests also lost out because to the "skewed" curve. In the teachers words, he "SKEWED" all of us. Needless to say, we were all leery about cheating in his classes after that.
I was disqualified from a competition run by FBLA (Future Business Leaders of America) when I was in high school because I scored so well I must have cheated.
It was multiple choice on 'Computer Concepts' I scored 98/100, second highest was 76/100.
That was pretty bad... but worse was the next year, I tried again... and was disqualified because I 'won' the previous year.
I ended up dropping out of school and getting a GED later because of the stress of it.
Hopefully this method is better
I have a feeling you'd be disappointed. Have you ever seen groups of women, focused on academia, who aren't competing over men?
It's somewhat like the nude beach dream... once you've been to one, you really never want to go back. The ratio of attractiveness to people who should never be seen in public without a full coat is far too low.
I am pretty sure that the last time I made up that statistic is was 45%, your made up 93% must be a statistical anomaly.....
The elephant in the room is that in American society, people are in general far more educated than they need to be. I have a bachelor's degree, none of the knowledge gained in pursuit of which[1] is of any help to me whatsoever in the course of my daily life, whether personally or professionally. And while I have no data, if the scuttlebutt is to be believed, I am very not alone in this. Furthermore, even a lot of the knowledge I gained in high school has proven completely useless to me[2]: outside of a class, I have never used any mathematics more advanced than the Pythagorean theorem.
As long as unreasonable academic credentials are required for jobs, though, there will be incentive for people to cheat—that is to say, cheating is not the problem; it's a symptom. Elminate the degree inflation in the job market, and you'll eliminate most of the cheating.
[1] Aside, possibly, from reading comprehension and writing skills, but those were not developed in college—I tested out of all the required English classes and all but one of the history classes—merely honed.
[2]The important words here are of course “to me.” I know lots of things which, objectively, are of no utilitarian use in my situation, but which I have sought out the knowledge of simply because it interested me; my enjoyment of them constitutes their usefulness.
Many people with ADHD consistently do better at "hard" things than at "easy" things. I'm a pretty decent programmer, I could do calculus (though not always very well) by about 4th grade... and even now I can't do single-digit arithmetic with complete reliability. Assuming that people who show the pattern I will show on basically any test of my ability in any field I work in are "cheating" is a poor tactic at best.
As a possible indicator, maybe useful. As a definite rule, hell no.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
The CISSP exam has special questions designed to catch cheaters or those who got a copy of the actual exam answers. At least a dozen questions are ambiguous and have more than one correct answer. The odds of two people answering all of those questions exactly the same, or exactly matching one of the the illicit copies of the exam answers is exceptionally low. The odds are low enough that you will get flagged for at least a manual audit of your test and test book.
Another dead giveaway is if your answers match almost exactly with the answers of someone else in the room. All the test books are not identical as they may have the questions in a different order or even different questions. If your answers to questions 1-40 exactly match the answers of your neighbor and he's using a different book, that would be suspicious too.
The irony is that there are cheaters for the CISSP exam, a certification that supposedly values honesty and ethics above actual knowledge.
I don't think that this system works for a normal (non multiple choice) test. At least that would require the teachers to spend a lot of time typing each students results in a nonambiguous way.
On the other hand it is a little more difficult to use cell phones to cheat in a non-multiple choice test, because the entire answer of an exercise is too long to fit into an SMS.
I would be flagged every single time. Though earlier in life I cheated (and was good at it, got 100% on some exams, thanks to weak lecturer passwords), these days I don't, because I really want to learn, and not just get a piece of paper. Though I sometimes wonder if a hybrid approach could be better, since I do want the extreme grades, since they sometimes count.
However, I digress. I'm ADD and Bipolar. While most people think this means "stupid" and can appear stupid, it's isn't necessarily. Basically, some things I get hyper focused and excited on, to the extreme point of spending days awake, working on the problem, sometimes forgetting to eat/drink/etc. On a softer scale, I have trouble retaining attention for "easy" problems, and have less trouble retaining attention for "hard" problems. Because of this, when doing tests, quizzes and exams, I often fuck up the easy ones, and do extremely well on the hard ones.
Based on their stated idea, I'd be flagged as cheating. This would happen in every exam. I wonder how much a university (with a strict no-cheating policy (like they all have)) would tolerate a student continually coming up flagged as cheating, regardless of their ability to prove it. If I get reprimanded for cheating, in my university, then I'm gone for a minimum of 3 years, and other universities in the area, might be warned of me (or at least that's the threat).
That would fucking destroy me. I'm already devastated by my results, as they're always 75%+, but they almost never reflect my competency in the subject. Some subjects it works in my favour, but mostly it works against me.
Side note: In some of my earlier statistics courses, we were privy to some analysis done by the statistics lecturers on various courses, and what variables explain the variation seen in students scores. Consistently in each course it was found that quizzes, attendance, having read the textbook, assignments, and many other variables, were all lousy explanatory variables for the final exam result, and as such the greater the weight of the final exam, the more likely your overall result wouldn't reflect you (working under the axiom that the other material/variables, better explained your competency).
This is quite interesting, at the very least.
* I find it impossible to remain consistent between the usage of the words subject and course, but they both mean the same thing.
This is my footer. There are many like it, but this one is mine.