Slashdot Mirror
Ask Slashdot: How To Allow Test Takers Internet Access, But Minimize Cheating?
New submitter linjaaho writes "I work as lecturer in a polytechnic. I think traditional exams are not measuring the problem-solving skills of engineering students, because in normal job you can access the internet and literature when solving problems. And it is frustrating to make equation collections and things like that. It would be much easier and more practical to just let the students use the internet to find information for solving problems. The problem: how can I let the students access the internet and at same time make sure that it is hard enough to cheat, e.g. ask for ready solution for a problem from a site like Openstudy, or help via IRC or similar tool from another student taking the exam? Of course, it is impossible to make it impossible to cheat, but how to make cheating as hard as in traditional exams?"
I remember being allowed to bring notes with me to class. Would just making this open book/open notes accomplish the same thing?
Ok, I give up, why you?
They don't need the whole internet; only a handful of sites. Set up a proxy that permits only GET requests to a few domains like Wikipedia, disable Javascript for good measure, and you're done.
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
If in a lab situation, use software that records sites visited, or is capable of viewing the student's screens. Make it known that this software is being used.
Make sure the questions are unique, change them between each exam, and carefully watch from the back of the room. You could also ask for a log of all the traffic through the WiFi point, and search for know chat domains.
In "real life" students will have access to all those things. Perhaps it isn't cheating but rather utilizing tools that they would have access to in "real life".
Assume they'll use every tool at their disposal- and write the tests in such a way that they can't copy the question into a search bar and google the answer.
"That's the way to do it" - Punch
Teaching means showing the way to solve problems. Nobody cares about correct solutions to school problems. It's all about the process of solving the problem, a scheme of thinking.
Block all traffic except port 80 http. (They don't need https, do they? They aren't checking bills online or using email, or some other security oriented task...)
Block all udp connections.
Dns filter a blacklist of known cheating sites.
Block bullshit sites like facebook, myspace and pals too. That's just good sense.
"And it is frustrating to make equation collections and things like that."
(A) Suck it up and do the work once.
(B) Use a textbook that comes with a premade formula card for use on tests.
(C) Find a premade formula card online and distribute that for tests.
Personally, I use option (B) for my math classes. Trying to make the internet non-communicable is like making water not wet.
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
When I was in grad school, in many classes we were allowed to use the internet on tests, as well as our notes, any spreadsheets/programs/scripts we had pre-made, etc. The caveat was that the tests were structured in a way that if you didn't already know what to do, you wouldn't have enough time to look it up and still finish the test. Googling things takes time. And the test really only provided enough time to actually do what you already knew.
You can also use random variables for each test, or groupings of tests, to prevent direct copying of answers. With a time limit, cheaters would have to wait for someone else taking the test to find the correct answer, send it out, and then modify it to match their own variables. If they can do all of that in a crunch, chances are they understand it pretty well on their own, even if they are lazy.
Give them access to a copy of wikipedia on disk. If they can't find the information there, they will be unlikely to find it elsewhere on the internet, but there should not be explicit answers to test questions.
http://meta.wikimedia.org/wiki/Wikipedia_on_CD/DVD
My school wouldn't even proctor the exam, they'd just expel you if you were ever caught cheating (no ifs ands or buts) , so getting an A instead of a gentleman's C by cheating didn't seem worth it. It did happen of course, and roughly 0.1% to 0.2% of the student base would get booted every year.
So what happens when all the "original content" makers die off? If we just search the web, we'll only get old information. Let people figure out how to create their own OC by searching within and solving/exploring on their own, so that the future internet will have new information. In the meantime, grade on the curve just to keep the education process moving.
If you're going to allow them unlimited research, then why not let them collaborate too? Give the whole class a set of problems big enough that they need to organize and split them up to get them all done in time. And if they can find the solution already completed elsewhere, so be it, that's what a good engineer is supposed to do. The whole point of working in the real world is that your performance depends on those around you, so the only way to measure the performance of students individually is to put them in an artificial problem solving situation like a traditional exam. That's why we still have paper, closed-book exams in theory classes, and why there are an increasing number of "project classes" where the entire class grade depends on the success of a hands-on group project.
Simple answer. Allow them to do whatever and then review what they visited. If there is any sign of going somewhere that might be questionable, call for a review.
Practical exercises another.
I'd say keep the exams closed book/no net, and the practicums open (you can't help but have them open). But then take 3-5 minutes per student and make sure that the practicum is at least fully understood by the student with an oral exam (TA's can handle that if too much workload).
Check your premises.
Ultimately, the cheater only hurts themselves. It shouldn't be your concern as to whether they are cheating. The only thing a lock does is keep an honest person honest. The cheaters will find a way, no matter what you do to restrict them, so the better solution is to make them take responsibility for their cheating by trusting them not to cheat.
My alma mater has an honor code, that is essentially this, on every assignment for credit, whether paper, exam, etc... you had to write a statement saying you upheld the honor code and sign it. In return, professors were hands off when we took a test, they weren't allowed to be on the same room (they had to be available if we had questions, but they were not allowed to watch us take the test. The school TRUSTED us to do the right thing, and the amazing thing is, most people did. Cheating was certainly not eliminated, but I'm willing to bet that there was far less cheating than the typical college.
I would argue that courses already have a component to them that is geared toward building research skills: essays. If a professor indeed wants to encorporate an "Advanced Google" portion to the course, simply weight the papers more, or do away with the exams entirely in favour of assignments.
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
If you allocate a *tight* amount of time for each problem, then students will find that it takes too long to cheat by googling. The downside is that you'll get complaints about your exam being too hard. In particular, students won't have time to make mistakes and correct them - they have to either know the material cold, or fail the question and move on.
Also, remember to change the questions every year.
Suppose it costs $20/hour to hire a student to help proctor a test.
Suppose students take four classes per semester, two semesters per year, four exams per class, two hours per exam. That's 64 exam hours per year per student.
Hire one proctor for each of ten students. So each group of ten students will have to pay for $20 * 64 proctor hours. That's $1280 per ten students, or $128 per student per year for exam proctoring.
Now, let them use the Internet as much as they want, and have one student-proctor monitoring each group of ten students for inappropriate behavior. That costs $128 per student per year.
Now, hire an additional set of proctor-proctors for another $128 to manage and oversee the first set of proctors. Hire students from the business school and give them half a credit of management.
With twice the estimated required number of proctors, that's still only $256 per student year to closely monitor the tests. That is not a large portion of college tuition.
This sounds like a very solvable problem -- if the institution is flexible enough to come up with interesting solutions. Seems like being able to come up with that kind of solution would also be a pretty good way of judging the quality of a university -- good PR opportunity.
Having grades align well with academic proficiency seems like a high-value line item for universities. Spending less than 10% of tuition to make exams more accurately test for subtle skills seems like a worthwhile investment to me.
Stop-Prism.org: Opt Out of Surveillance
So does the whole group get an A, if they have some rock star who knows the material cold while 4 of the other students contribute absolutely nothing, and should have normally failed the exam?
That's no different than one person doing their homework and letting their friends copy it.
Really. actually even pre 19th century - times in which where knowledge was more theory than practice.
Now, it should be practice. tests should be abolished. people should be given continuous assignments, projects and workshops, and instead learn things while doing them, as it should be - instead of memorizing stuff from a textbook and courses and to write them down when prompted.
Read radical news here
Tell us what school you teach at so I'm certain my daughter doesn't apply.
sorry bad formatting
My thought process:
block all ports other than port 80 - not effective, see tools like google chat
block port 80 + internal dns a records to make sure chat/email sites like gmail, hotmail, yahoo mail don't get resolved - still not foolproof, and a chat client that operates on LAN could get around it (engineering students are clever after all), alternately phones can sit in your pocket and be tethered and no one would know you're not on their firewalled connection.
use school-provided laptops? - too expensive How about make a program that the students are required to install to take the exam, and the program screenschots at random times what they are doing and uploads it to a LAN address so you can just see what they're doing? Maybe even get a programming class to write the apps and analysis software as one of their own final projects. - is definitely an invasion of privacy though (if students currently taking an exam can claim to have such a thing)
Or just make the exams so friggin hard that if they have to google every little thing, they won't get a good grade because they won't finish it. Ask for things like to sketch flowcharts that will not translate over text or chat in a meaningful way. (and if 20 students all submit exactly the same flowchart due to an email ring, it'd be easy to spot for the grader)
You could just allow blanket access, require everyone using a connection to get MAC address filtered access (so you know every device requesting access) and then log everything. Then provide stipulations that any live chat or forum use is forbidden. Anything except reputable / academic sources is forbidden. To make it extra fun, tail the log of the access point live (projector?) and grep it through a few good regex to weed out junk and find any terms associated with IRC, forums, etc etc. Allow them to ask for white listing sources, or provide your own (allow wikipedia, but not the discussions on each page which can be used to carry out conversations, etc). Or just allow all net access but restrict access to just the sites you think are of use (wikipedia, specific journals, publisher's reference information, google for unit converting on the search bar, etc).
You're never going to be able to make cheating as hard as it is on non-open tests as it is on open tests. That's an inherent problem in allowing access to outside information, particularly when you're dealing with worldwide communications.
What you can do is minimize the impact of cheating by working with the test itself: in particular, by setting a time limit based on its length. The idea here is to make it so that someone who constantly looks up outside information is highly likely to run out of time to finish the test. There's a delicate balance to be struck here, because you've said that some amount of going outside for information is not only to be expected but completely appropriate. But at the same time, you expect at least some knowledge to be "in-brain" (for lack of a better term), and so by using in-brain knowledge when it's there, a passing student will be able to finish the test quickly enough to beat the time limit. The trick is calibrating things, and I'm afraid I don't know a good solution for that.
The problem with a traditional teach-learn-test-forget-teach cycle is that students have to stuff as much of the lecture material into their brains as they can fit, pour it all out on the test and repeat the cycle. In my opinion, having tests that actually check for understanding rather than memorization capability would promote actual understanding of material instead of the repeated stuffing.
I've been out of school for a while, but I have recent anecdotal evidence -- vendor certification exams. Specifically, I took the VMWare exam recently. I passed, but it was quite difficult because I work with the product on an infrequent basis -- that is, I don't have the entire GUI memorized. More than half the questions would be easy to answer if you had the GUI in front of you and could just check the available options; the rest tested your knowledge of product architecture, limits and quite frankly trivia items. I've never done well on exams like these, because I'm just not a memorizer.
When I was in school a million years ago, with the Internet just becoming a viable research tool, some of my upper-division chemistry professors wouldn't give standard exams - we'd get "take home exams" which were actually mini-research projects that you could do pretty well if you were paying attention in class. The questions were just right in most cases...challenging enough to be a major pain to brute-force your way through, but made easier if you knew where to start looking (by knowing the material that was presented.) I'm not sure you can do this with a class of hundreds in freshman chemistry lectures, but when you have 20 or 30 students taking the class, and most are motivated to do well anyway, these are easier to do.
So the question isn't "how do I block Internet access for the test?" but more along the lines of "How do I make a challenging-enough test that can be finished in a finite amount of time, and doesn't just test student's lookup skills?"
Ask them questions that require an application of working knowledge / theory, as opposed to vocab / rote memorization style questions.
A little less "What does HTTP stand for?" a little more "I need to do some task using HTTP, show me how to make it do what I want it to do." That'll nuke using Google for an easy look up (for an answer), and potentially make anyone who copies off of another (via texting, emailing, cellphone, whatever) liable to fail the class (plagiarism ho!). See, by making it a non-trivial answer, you destroy the use of search engines for an easy answer, and by requiring some creativity (or even a fair amount), you can more accurately gauge a student's understanding, while also ensuring (via creativity) that no two student's answers should be identical. Of course, there are potential problems here, but it does, with a little tweaking, should help you identify the group-thinkers or no-thinkers with some ease. Plus, job security, as a teacher / professor, as you get to grade everyone's exams manually (the techs know you fear the machines, you need not be shy about it); just be sure to announce at the beginning of class that your style is that of the Athenians (Greek philosophers, focusing on thinking, etc.), or something to that effect.
The key here, to berate the point, is to ensure each answer is unique. Since simple answers cannot be unique, it's impossible to ensure that cheating has not occurred. Whereas with the greater increase in complexity (but not necessarily difficulty, mind you) of the answer, the more unlikely it is that two answers can be the same without one person copying another. When complexity increases enough, you have the effect of the Mona Lisa, where if 5 people turn the same or similar enough picture in, you have an extremely good idea that they were in communication with one another. It's not mathematically impossible, of course, that they should all create the same Mona Lisa, only hideously unlikely. Hell, if the solutions are unique enough, you might even learn something from them.
I am John Hurt.
Let the students tell you where they're going for answers.
Tests are suppose to show that the students are learning, right? Then monitor the internet traffic and see where they're going for answers, that will show you if they've really learned how to find answers to questions or not. And give real life type word questions, not just "1+1 = ?", stuff like "If you have one apple and someone gives you another apple, how many apples do you have?"
If they're going to sites like Openstudy to just ask someone to think for them then block access to that next test or live depending on your lab is setup, but remember sometimes going to forums and such are the best places for answers to real-life problems so I'd be careful trying to decide what sites to block.
Also you didn't really explain how they would have access so I assumed they would be in some sort of school computer lab, not on their personal laptops, and you have access to the network traffic and can restrict access at will.
my karma will be here long after I'm gone
I would say to avoid short answer questions like multiple choice or one word answers.
Essays are probably harder to cheat on without getting caught.
Or...
Make every question 20-multiple choice w/ different sets of 20 answers out of 100 on every test. Picking correct one by collusion is more difficult. To actually force the problem solving, interspersing questions where the correct answer is not listed and "none-of-the-above" is correct makes collusion even more difficult.
Bonus points for giving a test that where all the correct answers, but 1, on a 20 question multiple-choice test are "none-of-the-above". My high-school calculus teacher did that and I really, really had to think hard about that one question (which happened to be the last one) that wasn't none-of-the-above like all the other ones...
Many of my engineering classes allowed "formula sheets" or a "formula card", usually a single sheet of paper or a 4x6 index card, that the student was responsible for formulating themselves.
I used this to completely ace the exam in several of my EE classes where I otherwise would have had great difficulty. (Analog just wasn't my thing while becoming a CompE; I rocked my digital and computer classes.)
My tactic: Virtually all professors provide sets of review problems, and the answers to the review problems (along with all homework questions and mid-terms) were on file with the library. I'd go the library and make copies of those materials. I would then go back to my room and pass-through every single homework assignment, mid-term, and review question, and solve every problem to the point where the remainder of the solution was "busy-work." If, after much staring, I simply could not figure out how the professor got from point A to point B, I simply copied the entire solution to that problem (writing very small with a very sharp pencil if I was confined to a card, or just about 3 rounds of reducing on the copy machine if I wasn't) onto my formula sheet/card.
90% of the time, the problems where I had to copy the solutions wholesale onto the card ended up on the exam (with some trivial parts changed), and I was invariably one of the few people in the class to get it right, despite the fact that I had utterly no idea how the solution worked.
This problem is not amenable to technical solution. Trying to stop attackers from cheating via the Internet, by using some a filter or other form of limited access -- is as futile as trying to solve the halting problem, and enumerate the irrationals, at the same time.
The halting problem fails because it's too easy to craft countermeasures aimed deliberately at the scanner. Enumerating the irrationals fails because there is so much complexity, it's literally impossible to go throgh it all.
But just because you can't solve this problem technically doesn't mean it can't be solved. It's difficult, but I believe it might be possible. Don't bother beyond the basics. Get a computer lab set up with computers you control. Don't allow the students to bring in any USB sticks or CDs.
Then simply install tracking software on every PC. (You can also use a network sniffer to back this up.) The idea isn't that to prevent cheating technically; rather, you want to preserve the ability to tell that people have cheated, and simply punishing them under the existing rules.
You tell everyone in the class that you'll be monitoring their internet usage during the exam. Then tell everyone what you consider cheating. Have your grad students go through the logs manually; the difference should be fairly obvious.
Their are 2 way you could go about this.
1) Authoritarian route : State before the Exam starts that Key-logging and screen recording software has been installed on all machines, and no cell phones allowed.
With an automatic fail if key-loggger or screen capture software is disabled or caught using a cell phone (with a supervised resit later the same day with an automatic 1/3 drop in maximum score.)
2) Sneaky B'stard approach : Make the question so hard or badly worded that their is NO definitively correct answer, or that no 1 answer makes sense. Then it would be harder for "little Jimmy goggler" to find a correct answer on cheating sites, unless he's been taking notes and paying attention in class.
Either approach probably won't work but it's my suggestion ^_^
Laters Sol "Have you found the secrets of the universe? Asked Zebade "I'm sure I left them here somewhere"