Slashdot Mirror
Ask Slashdot: How To Allow Test Takers Internet Access, But Minimize Cheating?
New submitter linjaaho writes "I work as lecturer in a polytechnic. I think traditional exams are not measuring the problem-solving skills of engineering students, because in normal job you can access the internet and literature when solving problems. And it is frustrating to make equation collections and things like that. It would be much easier and more practical to just let the students use the internet to find information for solving problems. The problem: how can I let the students access the internet and at same time make sure that it is hard enough to cheat, e.g. ask for ready solution for a problem from a site like Openstudy, or help via IRC or similar tool from another student taking the exam? Of course, it is impossible to make it impossible to cheat, but how to make cheating as hard as in traditional exams?"
I remember being allowed to bring notes with me to class. Would just making this open book/open notes accomplish the same thing?
Ok, I give up, why you?
They don't need the whole internet; only a handful of sites. Set up a proxy that permits only GET requests to a few domains like Wikipedia, disable Javascript for good measure, and you're done.
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
If in a lab situation, use software that records sites visited, or is capable of viewing the student's screens. Make it known that this software is being used.
Make sure the questions are unique, change them between each exam, and carefully watch from the back of the room. You could also ask for a log of all the traffic through the WiFi point, and search for know chat domains.
In "real life" students will have access to all those things. Perhaps it isn't cheating but rather utilizing tools that they would have access to in "real life".
Assume they'll use every tool at their disposal- and write the tests in such a way that they can't copy the question into a search bar and google the answer.
"That's the way to do it" - Punch
Teaching means showing the way to solve problems. Nobody cares about correct solutions to school problems. It's all about the process of solving the problem, a scheme of thinking.
Block all traffic except port 80 http. (They don't need https, do they? They aren't checking bills online or using email, or some other security oriented task...)
Block all udp connections.
Dns filter a blacklist of known cheating sites.
Block bullshit sites like facebook, myspace and pals too. That's just good sense.
"And it is frustrating to make equation collections and things like that."
(A) Suck it up and do the work once.
(B) Use a textbook that comes with a premade formula card for use on tests.
(C) Find a premade formula card online and distribute that for tests.
Personally, I use option (B) for my math classes. Trying to make the internet non-communicable is like making water not wet.
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
When I was in grad school, in many classes we were allowed to use the internet on tests, as well as our notes, any spreadsheets/programs/scripts we had pre-made, etc. The caveat was that the tests were structured in a way that if you didn't already know what to do, you wouldn't have enough time to look it up and still finish the test. Googling things takes time. And the test really only provided enough time to actually do what you already knew.
You can also use random variables for each test, or groupings of tests, to prevent direct copying of answers. With a time limit, cheaters would have to wait for someone else taking the test to find the correct answer, send it out, and then modify it to match their own variables. If they can do all of that in a crunch, chances are they understand it pretty well on their own, even if they are lazy.
Give them access to a copy of wikipedia on disk. If they can't find the information there, they will be unlikely to find it elsewhere on the internet, but there should not be explicit answers to test questions.
http://meta.wikimedia.org/wiki/Wikipedia_on_CD/DVD
My school wouldn't even proctor the exam, they'd just expel you if you were ever caught cheating (no ifs ands or buts) , so getting an A instead of a gentleman's C by cheating didn't seem worth it. It did happen of course, and roughly 0.1% to 0.2% of the student base would get booted every year.
So what happens when all the "original content" makers die off? If we just search the web, we'll only get old information. Let people figure out how to create their own OC by searching within and solving/exploring on their own, so that the future internet will have new information. In the meantime, grade on the curve just to keep the education process moving.
I was an exams officer for a couple of years and we used e-testing for certain subjects. The golden rule from the awarding bodies was to treat it the same way you would any exam, have invigilators watching the students. Requirements would vary between different awarding bodies (HE & FE levels). For a Poly it would depend on what you are teaching and who actually issues the qualification, but it's the exams office that would be responsible for telling you.
From a technical standpoint lock the PC to a kiosk mode and firewall access to anything other than the exam site. Also remind students that if they cheat they fail and are removed from the course.
If you're going to allow them unlimited research, then why not let them collaborate too? Give the whole class a set of problems big enough that they need to organize and split them up to get them all done in time. And if they can find the solution already completed elsewhere, so be it, that's what a good engineer is supposed to do. The whole point of working in the real world is that your performance depends on those around you, so the only way to measure the performance of students individually is to put them in an artificial problem solving situation like a traditional exam. That's why we still have paper, closed-book exams in theory classes, and why there are an increasing number of "project classes" where the entire class grade depends on the success of a hands-on group project.
Why are you measuring problem solving skills of your engineering students? Are you teaching problem solving? Or are you teaching a subject? All of them passed your schools entrance requirements. You should be able to assume they have some minimum IQ. If you have a dumb student that has mastered the material are you not going to pass the student?
Simple answer. Allow them to do whatever and then review what they visited. If there is any sign of going somewhere that might be questionable, call for a review.
Practical exercises another.
I'd say keep the exams closed book/no net, and the practicums open (you can't help but have them open). But then take 3-5 minutes per student and make sure that the practicum is at least fully understood by the student with an oral exam (TA's can handle that if too much workload).
Check your premises.
Ultimately, the cheater only hurts themselves. It shouldn't be your concern as to whether they are cheating. The only thing a lock does is keep an honest person honest. The cheaters will find a way, no matter what you do to restrict them, so the better solution is to make them take responsibility for their cheating by trusting them not to cheat.
My alma mater has an honor code, that is essentially this, on every assignment for credit, whether paper, exam, etc... you had to write a statement saying you upheld the honor code and sign it. In return, professors were hands off when we took a test, they weren't allowed to be on the same room (they had to be available if we had questions, but they were not allowed to watch us take the test. The school TRUSTED us to do the right thing, and the amazing thing is, most people did. Cheating was certainly not eliminated, but I'm willing to bet that there was far less cheating than the typical college.
I was looking for a way to cheat, thanks for pointing me in the right direction with openstudy.com........
I'm in a program at NYU-Polytechnic. My professors have been fairly successful at building exams that they can complete in half the allotted time. Many of the students in my classes take most of the available time. It's not enough time to be able to do research unless you're a wizard at google hacking. Sure, they could ask someone or have someone take it, but the alternative is proctored exams, which is a ROYAL pain.
I would argue that courses already have a component to them that is geared toward building research skills: essays. If a professor indeed wants to encorporate an "Advanced Google" portion to the course, simply weight the papers more, or do away with the exams entirely in favour of assignments.
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
If you allocate a *tight* amount of time for each problem, then students will find that it takes too long to cheat by googling. The downside is that you'll get complaints about your exam being too hard. In particular, students won't have time to make mistakes and correct them - they have to either know the material cold, or fail the question and move on.
Also, remember to change the questions every year.
Suppose it costs $20/hour to hire a student to help proctor a test.
Suppose students take four classes per semester, two semesters per year, four exams per class, two hours per exam. That's 64 exam hours per year per student.
Hire one proctor for each of ten students. So each group of ten students will have to pay for $20 * 64 proctor hours. That's $1280 per ten students, or $128 per student per year for exam proctoring.
Now, let them use the Internet as much as they want, and have one student-proctor monitoring each group of ten students for inappropriate behavior. That costs $128 per student per year.
Now, hire an additional set of proctor-proctors for another $128 to manage and oversee the first set of proctors. Hire students from the business school and give them half a credit of management.
With twice the estimated required number of proctors, that's still only $256 per student year to closely monitor the tests. That is not a large portion of college tuition.
This sounds like a very solvable problem -- if the institution is flexible enough to come up with interesting solutions. Seems like being able to come up with that kind of solution would also be a pretty good way of judging the quality of a university -- good PR opportunity.
Having grades align well with academic proficiency seems like a high-value line item for universities. Spending less than 10% of tuition to make exams more accurately test for subtle skills seems like a worthwhile investment to me.
Stop-Prism.org: Opt Out of Surveillance
Then, quite simply, the professor better not write the exam that way! It's usually possible to avoid overwhelming students with minutiae when putting together an exam, even if it does take a little bit of extra effort. I've heard a few PhD candidates complaining about the challenges of doing so, but nevertheless the students will come away feeling they've been graded more fairly.
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
So does the whole group get an A, if they have some rock star who knows the material cold while 4 of the other students contribute absolutely nothing, and should have normally failed the exam?
That's no different than one person doing their homework and letting their friends copy it.
Really. actually even pre 19th century - times in which where knowledge was more theory than practice.
Now, it should be practice. tests should be abolished. people should be given continuous assignments, projects and workshops, and instead learn things while doing them, as it should be - instead of memorizing stuff from a textbook and courses and to write them down when prompted.
Read radical news here
I don't want to do the work to create unique tests, so how can I keep them from getting outside information?
Oh, I also want to allow them to get outside information.
Look, there training to be engineers, you can not prevent them from access 'part' of the internet.
Don't let them access the internet at all.
THEY are there to learn that subject. As such they should figure it out. ANY engineer that has to look up everything its a crappy engineer.
The Kruger Dunning explains most post on
If you're worrying about testing them on what you want to test them on, you may want to start looking at BLOOM's Taxonomy and try to write your test questions at a more difficult level if you want to go down the "test question" route.
http://en.wikipedia.org/wiki/Bloom%27s_Taxonomy
I have had to deal with test development and the difference between Knowledge and Comprehension level questions for a few years at one of my jobs. First off, Knowledge level questions for tests are fairly easy to develop. Comprehension and higher level questions are much more difficult in the long run because there is a lot more that is involved in the question development side of the house and also in the test question validation phase too.
Besides, if you are already asking these questions, you may want to look at the whole evaluation mechanism that you are using and maybe try to emulate it after something in the real world that would give you a better measurement of the Learning Objective(s) you are trying to accomplish with your students. To me, an overarching project with multiple stages to it over the time period of your course would be a much better method to accomplishing this versus doing a "test, quiz, etc." just to have some numbers, even though I'm sure your Stat-Addicts employers would love some kind of arbitrary measurement that they can use to justify buying more stuff or "why" they program is producing such wonderful students. Or even better, get them to work together in teams since that is how they do it in the real world anyway.
Unfortunately, the way testing is done nowadays does not EVEN come close what people are seeing when they work collaboratively in the real world. So I would suggest that you look at that maybe and readjust your testing mechanism to work better for your learning objectives. You'll come across as a much more objective instructor and actually get phenomenal feedback from students with something that will stick with them when they move out into the corporate world and start working together successfully in collaboration because they had the chance to do that in your class.
If you're at all worried about this, don't allow internet access. Either allow it or don't, but don't half-ass allow it. If you let them open any electronic device, you have to assume they have access to the full, unfiltered internet. Welcome to the 21st century, where we have cell phones with wireless tethering and all manner of wireless access dongles like 4G modems which are completely out of your control. I suppose if the classroom was surrounded by a Faraday cage and only wired internet to their desks was allowed you could try to filter it, but then you're putting yourself into an adversarial relationship with blackhat engineering students... not a great place to be. If you think you're smarter than them, you're probably right, but they still might try a thing or two you haven't considered.
If you're testing them individually, due to the problems mentioned above, don't allow internet access at all. Cheaters will talk amongst themselves, which means in the best case scenario you'll have a bunch of students to fail, and in the worst case, you won't even know, so they'll have artificially higher marks than everyone else.
It's better to vote for what you want and not get it than to vote for what you don't want and get it.
- E. Debs
You can setup which websites are allowed and block all others right in the Browser Options/Content. It's called Content Advisor, I've done this for me kids, works very well, and it can be password protected. Takes 5 min. to setup.
-- By all means let's be open-minded, but not so open-minded that our brains drop out.
Is that the cheaters are going to cheat, no matter what you do. Had a guy who listened to class material on his ipod in class. Wrote answers on their hands. Texted them to each other. Bluetooth micro-earpiece and mumbled questions under their breath. You can't stop them. You just have to let them know that when they get their dream job, with their fake resume and their unjust transcript, and they get fired within a month because they can't do it, that perhaps they would have been better off learning the material.
Tell us what school you teach at so I'm certain my daughter doesn't apply.
This is perfectly true. Part of a skillset is how to get help. Even, you could say, getting help through blackmail or cheating.
But there's only so many people to get help from.
Sure, those helpful people can set up a chat room during the exam and have everyone pass.
Which is great until the helpful people all retire or stop bothering and none of the students can solve hard problems on their own.
My thought process: block all ports other than port 80 - not effective, see tools like google chat block port 80 + internal dns a records to make sure chat/email sites like gmail, hotmail, yahoo mail don't get resolved - still not foolproof, and a chat client that operates on LAN could get around it (engineering students are clever after all), alternately phones can sit in your pocket and be tethered and no one would know you're not on their firewalled connection. use school-provided laptops? - too expensive How about make a program that the students are required to install to take the exam, and the program screenschots at random times what they are doing and uploads it to a LAN address so you can just see what they're doing? Maybe even get a programming class to write the apps and analysis software as one of their own final projects. - is definitely an invasion of privacy though (if students currently taking an exam can claim to have such a thing) Or just make the exams so friggin hard that if they have to google every little thing, they won't get a good grade because they won't finish it. Ask for things like to sketch flowcharts that will not translate over text or chat in a meaningful way. (and if 20 students all submit exactly the same flowchart due to an email ring, it'd be easy to spot for the grader)
Or it will train them not to think when wikipedia goes down...
Some of the best instructors I had taught the concepts and not the "units", and all the notes and cribsheets in the world were useless if you didn't understand the concept. Not every instructor can be one of the top-10, so maybe we do need to handicap those profs with internet access. /s
https://www.accountkiller.com/removal-requested
Only allow them to use Google cache to be sure they don't use some chat site.
so you want people who can cram to pass but not test people with the can doing the work for real. If you want to LEARN you want it to be like a paper mcse when you can pass by just craning for the test?
You could just allow blanket access, require everyone using a connection to get MAC address filtered access (so you know every device requesting access) and then log everything. Then provide stipulations that any live chat or forum use is forbidden. Anything except reputable / academic sources is forbidden. To make it extra fun, tail the log of the access point live (projector?) and grep it through a few good regex to weed out junk and find any terms associated with IRC, forums, etc etc. Allow them to ask for white listing sources, or provide your own (allow wikipedia, but not the discussions on each page which can be used to carry out conversations, etc). Or just allow all net access but restrict access to just the sites you think are of use (wikipedia, specific journals, publisher's reference information, google for unit converting on the search bar, etc).
You're never going to be able to make cheating as hard as it is on non-open tests as it is on open tests. That's an inherent problem in allowing access to outside information, particularly when you're dealing with worldwide communications.
What you can do is minimize the impact of cheating by working with the test itself: in particular, by setting a time limit based on its length. The idea here is to make it so that someone who constantly looks up outside information is highly likely to run out of time to finish the test. There's a delicate balance to be struck here, because you've said that some amount of going outside for information is not only to be expected but completely appropriate. But at the same time, you expect at least some knowledge to be "in-brain" (for lack of a better term), and so by using in-brain knowledge when it's there, a passing student will be able to finish the test quickly enough to beat the time limit. The trick is calibrating things, and I'm afraid I don't know a good solution for that.
The problem with a traditional teach-learn-test-forget-teach cycle is that students have to stuff as much of the lecture material into their brains as they can fit, pour it all out on the test and repeat the cycle. In my opinion, having tests that actually check for understanding rather than memorization capability would promote actual understanding of material instead of the repeated stuffing.
I've been out of school for a while, but I have recent anecdotal evidence -- vendor certification exams. Specifically, I took the VMWare exam recently. I passed, but it was quite difficult because I work with the product on an infrequent basis -- that is, I don't have the entire GUI memorized. More than half the questions would be easy to answer if you had the GUI in front of you and could just check the available options; the rest tested your knowledge of product architecture, limits and quite frankly trivia items. I've never done well on exams like these, because I'm just not a memorizer.
When I was in school a million years ago, with the Internet just becoming a viable research tool, some of my upper-division chemistry professors wouldn't give standard exams - we'd get "take home exams" which were actually mini-research projects that you could do pretty well if you were paying attention in class. The questions were just right in most cases...challenging enough to be a major pain to brute-force your way through, but made easier if you knew where to start looking (by knowing the material that was presented.) I'm not sure you can do this with a class of hundreds in freshman chemistry lectures, but when you have 20 or 30 students taking the class, and most are motivated to do well anyway, these are easier to do.
So the question isn't "how do I block Internet access for the test?" but more along the lines of "How do I make a challenging-enough test that can be finished in a finite amount of time, and doesn't just test student's lookup skills?"
You could always lock out IRC or any other places you don't want them to go. Also let them know that all sessions will be recorded in case any questioning of cheating comes up. Of course this is only if this takes place on classroom computer where you have control.
Ask them questions that require an application of working knowledge / theory, as opposed to vocab / rote memorization style questions.
A little less "What does HTTP stand for?" a little more "I need to do some task using HTTP, show me how to make it do what I want it to do." That'll nuke using Google for an easy look up (for an answer), and potentially make anyone who copies off of another (via texting, emailing, cellphone, whatever) liable to fail the class (plagiarism ho!). See, by making it a non-trivial answer, you destroy the use of search engines for an easy answer, and by requiring some creativity (or even a fair amount), you can more accurately gauge a student's understanding, while also ensuring (via creativity) that no two student's answers should be identical. Of course, there are potential problems here, but it does, with a little tweaking, should help you identify the group-thinkers or no-thinkers with some ease. Plus, job security, as a teacher / professor, as you get to grade everyone's exams manually (the techs know you fear the machines, you need not be shy about it); just be sure to announce at the beginning of class that your style is that of the Athenians (Greek philosophers, focusing on thinking, etc.), or something to that effect.
The key here, to berate the point, is to ensure each answer is unique. Since simple answers cannot be unique, it's impossible to ensure that cheating has not occurred. Whereas with the greater increase in complexity (but not necessarily difficulty, mind you) of the answer, the more unlikely it is that two answers can be the same without one person copying another. When complexity increases enough, you have the effect of the Mona Lisa, where if 5 people turn the same or similar enough picture in, you have an extremely good idea that they were in communication with one another. It's not mathematically impossible, of course, that they should all create the same Mona Lisa, only hideously unlikely. Hell, if the solutions are unique enough, you might even learn something from them.
I am John Hurt.
Let the students tell you where they're going for answers.
Tests are suppose to show that the students are learning, right? Then monitor the internet traffic and see where they're going for answers, that will show you if they've really learned how to find answers to questions or not. And give real life type word questions, not just "1+1 = ?", stuff like "If you have one apple and someone gives you another apple, how many apples do you have?"
If they're going to sites like Openstudy to just ask someone to think for them then block access to that next test or live depending on your lab is setup, but remember sometimes going to forums and such are the best places for answers to real-life problems so I'd be careful trying to decide what sites to block.
Also you didn't really explain how they would have access so I assumed they would be in some sort of school computer lab, not on their personal laptops, and you have access to the network traffic and can restrict access at will.
my karma will be here long after I'm gone
As you say, when they get into the workplace, if they need to know something, they're going to Google it. By limiting the resources they can use when answering exam questions, you are increasing the distance between education and the workplace - who does that help?
So set them realistic problems and let your students solve them by hook or by crook, just they way they'll solve them when they're on a payroll.
At least that way, the students who do well on the test will be the ones who will do well in industry, as opposed to the current situation, where we have engineers who can do CFD by hand, but can't tell the difference between Aluminium and Steel.
Ask questions that require the student to demonstrate synthesis of the various things you've wanted them to learn. They might be able to google individual steps to solve the larger problem, but they wouldn't be able to google the end result - they'd have to know what they're doing. Put a time limit on it.
For example, in an introductory programming class (a CS 100 level course at my university), the final for the class consisted of this:
"Take a sound file (speech.wav) consisting of several dozen words with brief pauses between them and create a program that will output a new sound file with a name of the user's choice. The new sound file will consist of the words from the original file reorganized into a random sequence. The code must be commented to explain your thinking at each step. You have 3 hours for this task."
That task would require that the student understand how to analyze a problem, break it into smaller problems, design a process to address each of those problems individually, and then combine those solutions into the finished program.
Oh, sure - the student could google "how do I resize an array in java" (which might be necessary to know to accomplish several steps in the task), but they would need to know so much more in order to even realize they needed to know that, and that stuff wouldn't be something they could just look up.
The only downside would be the ability of the students to send the question to another student who is more advanced and who would spit back an answer; to guard against that you'd need to actually have an idea of what your students are capable of individually.
Since I can't tell them apart, I treat all ACs as the same person.
So sure, in real life you can look stuff up. I'm in the computer field after going through a BSEE a gagillion years ago. I've seen desktop support folks try to step into the server admin role thinking they can get by with Google and the MS Knowledgebase. Invariably screw it all up. Plugging in a mouse, troubleshooting email on the client -- sure you can flip through menus or decide that the flat end of the mouse isn't going to fit into the round port. But for more complex things, some training and theory are necessary.
In terms of exams, you want your students, especially engineering students, to have enough theory that they can go out and design stuff: create it from scratch. If all they know is the right equation for this or that and when the most likely time to plug in a number is, then you've graduated a tech, not an engineer. Don't tell the physicists, but engineers can come up with ideas too. :-)
Followed up with lie detector test.
Assuming your .edu already has a content filtering firewall (e.g. Smoothwall + SmoothGuardian) just get a profile created specifically for exam purposes with rules to block all IM/Chat programs, blacklist cheating type sites and queries and log all non whitelisted activity.
If the hardware is .edu owned then you might also want to consider something like Deep Softwares "Activity Monitor" so you can audit the exams... or perhaps just insinuate that you have installed something along those lines in the hope / knowledge that most students won't risk it.
Seriously, that is the big issue. Come up with classes of questions/answers and then have them picked in random. Likewise, have the results checked by the computer. If somebody differs, then they are handchecked.
I prefer the "u" in honour as it seems to be missing these days.
It's fairly clear that engineering students likely have the skills to comb the Interwebs for information. Instead of giving tests that might require it, give an open book exam and make sure that all the references they need are in the materials allowed during the test.
This will test the students' ability to identify what information they need to solve the problem, without giving them access to information that could allow them to cheat. Creating a test that requires access to the Internet to gather the information required to take the test is laziness on the part of the test creator, IMHO.
No, no, you're not thinking; you're just being logical. --Niels Bohr
I would say to avoid short answer questions like multiple choice or one word answers.
Essays are probably harder to cheat on without getting caught.
Or...
Make every question 20-multiple choice w/ different sets of 20 answers out of 100 on every test. Picking correct one by collusion is more difficult. To actually force the problem solving, interspersing questions where the correct answer is not listed and "none-of-the-above" is correct makes collusion even more difficult.
Bonus points for giving a test that where all the correct answers, but 1, on a 20 question multiple-choice test are "none-of-the-above". My high-school calculus teacher did that and I really, really had to think hard about that one question (which happened to be the last one) that wasn't none-of-the-above like all the other ones...
If you design a good test, and grade it well, then there will be no way to cheat.
I would design questions that require a good understanding of the material to answer, let them do whatever they want online, and then talk individually with students that you think might be copy/pasting answers, whether from other classmates or from the internet.
A 2 or 3 minute conversation with them will tell you pretty easily whether they understand their answers or not.
If they understand their answers well enough to convince you they could have given them legitimately then it doesn't matter what actually happened, they are either learning what you are teaching, they already know it, or they have good enough grasp of the whole field of study that they can fake it.
Any of those three outcomes is sufficient for your purposes, at least in my opinion.
The hard part here, of course, is designing good questions. It's incredibly hard to design good tests, which explains how few of them you find out in the wild.
RUGBYRUGBYRUGBY
Many of my engineering classes allowed "formula sheets" or a "formula card", usually a single sheet of paper or a 4x6 index card, that the student was responsible for formulating themselves.
I used this to completely ace the exam in several of my EE classes where I otherwise would have had great difficulty. (Analog just wasn't my thing while becoming a CompE; I rocked my digital and computer classes.)
My tactic: Virtually all professors provide sets of review problems, and the answers to the review problems (along with all homework questions and mid-terms) were on file with the library. I'd go the library and make copies of those materials. I would then go back to my room and pass-through every single homework assignment, mid-term, and review question, and solve every problem to the point where the remainder of the solution was "busy-work." If, after much staring, I simply could not figure out how the professor got from point A to point B, I simply copied the entire solution to that problem (writing very small with a very sharp pencil if I was confined to a card, or just about 3 rounds of reducing on the copy machine if I wasn't) onto my formula sheet/card.
90% of the time, the problems where I had to copy the solutions wholesale onto the card ended up on the exam (with some trivial parts changed), and I was invariably one of the few people in the class to get it right, despite the fact that I had utterly no idea how the solution worked.
When I was screening new hires with a knowledge quiz, I would allow them Internet access - but only for the last third of the time, and after giving them a red pen. Sometimes it is knowing how to find an answer, not actually knowing the answer itself, that is meaningful. It was also a simultaneous stealth test of Internet search skills. The red answers, and ratio of red to black, was frequently interesting...
We've had this exact problem. Setting up a policy that is both fair and hard for engineering students to bypass, is absolutely non trivial.
Instead, tell the people watching the exam to watch for extended keyboard noise. Most people reading material will use the mouse and perhaps type a few keywords, while cheaters will be typing out full sentences.
Make the test truly collaborative. Indicate that anyone can ask for help, anyone can offer help (more like a normal work environment). But indicate that credit for help is required. Put a spot by each question for "person asked" and "help given" -- that way person A has to mark "B helped me" on his test, and person B marks "I helped A." on her test. It'd be best if you can also make the questions unique. You want the help to be more than "pssst! what's the answer to number 2?"
Of course, helping someone else will also reduce the amount of time you have to complete your OWN test, so there's a certain amount of selfish time-management that has to occur as well.
You could then give a certain number of collaboration points for the test: "Person B assisted in 6 questions, and asked for assistance on 4 questions. She gets an additional 6 points, and loses 4 points - for a net gain of 2 points."
This encourages helping others, and discourages asking for help. If the names don't match up, then someone may be cheating....
If your test is about problem solving, finding a way to solve a problem, then allowing internet access seems fine. However, if the test is about are they learning specific engineering techniques and principles, then it seems letting them search the internet for a solution would be inappropriate. While it is true that in the real world, they will have all of those online tools, a classroom setting is not a real world. Get your calculations wrong in a class and you flunk a test, in real life a bridge collapses and people die.
Give them the same sort of info they'd have if they were doing the job for real.
No one cares if the engineer looks things up on the internet while working on something. It's all about results. If his solutions are correct and reliable then no one cares.
So test them with real world problems giving them no more info then they'd have outside your classroom.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
Make the test so long that you can only finish it if you know the material.
that the flat end of the mouse isn't going to fit into the round port. that is what you get from some people with a BA in CS.
Now for server that should be some kind of tech school / learn on the job training.
and desktop support is a good starting point to get into sever now where did they mess it up?
also troubleshooting email on the client is likely more about how that client works vs the theory of email
... the candidates could ask the person sitting next to them. Or maybe get a contractor to do the work for them. Or... at some point, exams have to stop being exactly like the real world. Otherwise, you need an exam that looks like and lasts as long as the career the candidate might take up on passing the exam.
Virtually serving coffee
If you have to rely on removing all of my resources to 'test me', then your test is invalid. Unless you're training fighter pilots or some other 'split second decision' job, my ability to perform has less and less to do with my ability to memorize.
PS: I don't reply to ACs.
and use the jigsaw voice.
Also don't put macgyver in there.
1: Ask them this question, grade the responses on a curve and use the best answers for your method.
2: (non-silly method) Split the class into small groups of 4-8 students. Arrange a list of subjects and have half the group form test questions based off of the list. Make note of any preferred methods, or requirements to use in the questions (this is where you can guide them into asking the right kind of questions). Have the other half solve the questions. Then have them switch sides and repeat. Grade based on how challenging the questions are, scope of the answers, and use of requirements. Keep a DB of previous questions submitted to verify originality. PROS: gives students ample opportunities for failure, makes for some very interesting and engaging tests, helps to teach how to find a problem AND how to find a solution. CONS: Makes grading the tests a lot more time consuming.
Sometimes, life itself is sarcasm...
What about putting VNC or any remote desktop software on the PCs. Warn them in advance that they are permitted to research, but being caught outright asking anyone inside or outside the classroom is an automatic 0. Randomly pop into different students screens from your PC. (the benefit of remoting vs walking towards them, is by the time you connect in, it is too late to alt F4 as they have no idea who is about to be screened, and no ability to predict where you are looking, or even know whether you are in their screen or browsing slashdot.
NOOO we've all been tricked.... This is the askers Esay question for a test!
Provide the devices on which internet access is allowed. Visually record and log all activity on each device and let students know beforehand that the recording will be made.
Let them edit it throughout the semester, and then only give access to that site (hosted on a machine on a LAN, not connected to the internet). Then everyone would have the same information base, and they could search for information. You could even keep the information alive through semesters. Make sure it's read only during the test. And you can go through it quickly after writing the test to make sure no information is too helpful.
Even better, or also, you could host a stack exchange like service too, that people can record and exchange information for the projects they do during the semester. Teaching is the best way to learn, after all...
Get a few people to walk around and monitor students. It's hard to cheat if somebody is looking over your shoulder. That's what they did when we were allowed to use our own laptop on tests.
I encouraged, but did not require, collaboration.
As much as possible, I defined in advance how much accomplishment constituded C, B and A work, respectively. I was never able to make this perfectly objective, and the clarity varied according to the material and my own imagination in the project assignment.
To evaluate each student's individual accomplishment, I interacted with the students. Sometimes I did this in scheduled individual face-to-face interviews. It was OK for a student to talk about other people's work as well as her own. But I only gave credit for insight explained to me through description of work, not for work that just sat there on paper (or online). Once in many years of this practice I encountered a student who appeared for the interview and seemed unable to express himself in a clear sentence. I made a wild guess for that guy's grade, and vowed to have midterm interviews as well as final interviews as much as possible.
Later, when I had a Wiki server, I required projects to be displayed incrementally on the Wiki. I had a very difficult time convincing students not to wait until the last day and dump lots of material out of the blue (nobody who did so got a good grade). In (thankfully only) one case, a student who failed to attend class (attendance and participation in discussion was mandatory according to written posted instructions) sent me a "project" 1 hour after the absolute final deadline, and seemed shocked that I awarded a D (the only grade below C I ever had to assign in this system).
I never achieved 100% understanding of the ground rules (in spite of posting them in explicit instructions on the Web and discussing them in class). There were always students who failed to understand that only iteration could produce a good result on these sorts of projects, and a few who understood but failed to act (sometimes requesting incomplete grades at the last minute). I was working on intermediate deadlines to eliminate such behavior as much as possible, but stopped teaching before I worked it all out. One student who understood, but failed to perform, in one class signed up for another topic taught in the same way, and did quite well.
For the students who understood, and acted upon, the instructions, the results were very good. I could tell students as they went along what progress they had made toward C, B, A, and avoid the end-of-term suspense that I find counterproductive. Evaluations at the time of the classes varied a lot, from those who loved the system to those who couldn't stand it. A few students got in touch years later, and were very positive. These were all students who had done well, and it is very unusual to hear from dissatisfied students long after class, so I have insufficient information about students for whom my approach did not work.
The principle that I tried to follow was to avoid completely any requirement in which the possibility of cheating was relevant. I tried to design all assigned work so that the accomplishment required worthwhile learning while using all available resources, including fellow students and outside consultants. I believe that this principle is good for pedagogy, and not just a way to avoid worrying about cheating (although my experience in a few cases of cheating in classes with conventional assignments and exams was painful enough that I might choose to avoid the possibility even if it were not good pedagogy).
I found this principle to require a lot of rethinking, and the application to each topic, context, instructor, and type of student is different. I'm not convinced that conventional assignments and exams are better in this respect, I think that they only make it easier to avoid feeling bad about the failure by attributing it entirely to the student.
I also think that there are particular topics and contexts for which my approach is not appropriate, but I tend not to like teaching those topics in those contexts.
Mike O'Donnell http://people.cs.uchicago.edu/~odonnell/
It depends on what your course is, but if you want them to solve problems in a real world setting, you give them real world problems. Last time I checked, even with the vast amount of information that's available on the internet, there are still problems out there that need solving. Offer those up.
If you simply ask them to solve equations, that's not really solving problems in the real world. But if you ask them to design a bridge/circuit, that's something else. The problems should also be open-ended. If you have simple questions with only answer, that's easy to cheat. But having open-ended questions with multiple paths and/or multiple solutions makes it much more difficult to cheat. (And are probably better examples of real world situations.)
SHOW YOUR WORK! :)
The more steps that need to be taken, the more 'samples' you have to see if people are simply rote cheating or not. If I'm solving an equation and I need twenty steps, and someone else does it in the same twenty steps - guess what? One of us copied off of the other. Even if they are smart enough to not copy all twenty steps verbatim, that still requires some level of intelligence.
PUNISH HARD
As the likelihood of catching someone cheating decreases, the penalty should likewise increase. Just make it clear that if you're caught cheating, you get a zero on the exam, and it doesn't matter if you were the person doing the copying or being copied from.
Where the wind blows, the tumbleweed goes.
You can restrict access with a whitelist or a blacklist, as someone has already mentioned. Another option is to install software to track which websites the student accesses; you could factor their research process into the grading and penalize those who access forbidden sites. If you do this, be very explicit about which kinds of sites are OK and which aren't so you don't have too much of a battle on your hands after the exam.
>so you want people who can cram to pass but not test people with the [can?] doing the work for real.
Well this is one of the biggest problems that students from China face. Chinese academics tend to be based on a reward system for rote memorization, and often go as far as to literally punish attempts at creative thinking. So a student who has been brought up in that culture and then transported into an American university situation has a huge benefit and an even bigger liability. Asset, is that they are typically much better at straight memorization and cold recitation than the average American student. Liability is that it typically does not even occur to them to do any sort of lateral or particularly creative thinking. Better teachers and curriculum developers understand this and anticipate it and design mitigating controls into the curriculum. Poor teachers just assume that "Asian students are smarter" (because they can usually memorize like mad) and reward this to the detriment of others.
This problem is not amenable to technical solution. Trying to stop attackers from cheating via the Internet, by using some a filter or other form of limited access -- is as futile as trying to solve the halting problem, and enumerate the irrationals, at the same time.
The halting problem fails because it's too easy to craft countermeasures aimed deliberately at the scanner. Enumerating the irrationals fails because there is so much complexity, it's literally impossible to go throgh it all.
But just because you can't solve this problem technically doesn't mean it can't be solved. It's difficult, but I believe it might be possible. Don't bother beyond the basics. Get a computer lab set up with computers you control. Don't allow the students to bring in any USB sticks or CDs.
Then simply install tracking software on every PC. (You can also use a network sniffer to back this up.) The idea isn't that to prevent cheating technically; rather, you want to preserve the ability to tell that people have cheated, and simply punishing them under the existing rules.
You tell everyone in the class that you'll be monitoring their internet usage during the exam. Then tell everyone what you consider cheating. Have your grad students go through the logs manually; the difference should be fairly obvious.
Although my engineering degree is over 40 years old now, I still remember one exam in Fluid Mechanics where everything was open book and open notes. It didn't help one bit if you didn't understand the concepts. One of the final exam problems was a very complicated siphon system and the numbers that one needed to plug into the applicable equations to calculate flow rate were not too hard to find. However, the problem was rigged so that the highest point on the path of the siphon was more than 33 feet above the source so the equations did not apply. However, this was NOT very obvious. If you understood how a siphon worked, and took the trouble to see whether this one satisfied the conditions necessary, you got full credit for just writing down the flow is zero since the necessary conditions for flow are not met. If you tried to use the numbers that were more obvious to calculate the flow you would get a reasonable answer that was completely wrong. This was one of three very similar problems on the final exam.
Asian students also do solo work as group
I meant to say so you want people who can cram to pass but not test people if they can do the work for real.
well then degree should not be in the work place.
Let's say take 3 people all the same who is better for a IT / desktop / server job.
1 who if a lot cert tests (ones that can be passed by cramming by people who don't know what they are doing)
2 who did a lot of IT work on there then / maybe went to a tech school that has real job skills.
3 Some one with a mostly theory based CS with just about none of skills that you get at a tech school.
Well the real work place is not individual and with some things you need to test people on the job / in real work place conditions. So in some places testing individual may fit but most real work is group based with open books / open Google. Even before Google you had reference books in the work place.
As a High School Teacher, if you're going to open the Internet up to them, wouldn't short-term (1 or 2 week) assignments be easier? (Other than the marking of course.) It'd test the ability to problem solve and research more realistically, and you could use turnitin.com to ensure students aren't copying each other/the Internet.
On another point, at the University I did both a pure Science degree, and then an Education degree (with credited subjects). What I found interesting was that the Science degree was taught in centuries-old formal format - lectures/labs/exam, but the Education degree was taught in small-class format/group projects/presentation assignments/etc. Only towards the end of my degree did one of my old science lecturers announce that he was taking some of the Education subjects as a way to upskill his teaching methods.
The tests I give are all 'open book', which I define as allowing any books, notes, documents or other aids (e.g. little model chromosomes for genetics problems) but no electronics. Unfortunately this means no e-texts.
Their are 2 way you could go about this.
1) Authoritarian route : State before the Exam starts that Key-logging and screen recording software has been installed on all machines, and no cell phones allowed.
With an automatic fail if key-loggger or screen capture software is disabled or caught using a cell phone (with a supervised resit later the same day with an automatic 1/3 drop in maximum score.)
2) Sneaky B'stard approach : Make the question so hard or badly worded that their is NO definitively correct answer, or that no 1 answer makes sense. Then it would be harder for "little Jimmy goggler" to find a correct answer on cheating sites, unless he's been taking notes and paying attention in class.
Either approach probably won't work but it's my suggestion ^_^
Laters Sol "Have you found the secrets of the universe? Asked Zebade "I'm sure I left them here somewhere"
When I was lecturing, a much older (tenured) colleague came up with a series of multiple choice tests, and made the time limit very limiting - with the explanation that it was enough time to answer the questions if you knew the answers, but not enough time to look up the responses.
They field tested it, made sure that the time limit was reasonable (and removed some questions that were just too tough for the 3rd year course) and made an exam bank that was 3x the size of the number of questions. The students got 2 attempts, and because the questions were randomized, it was fairly difficult for an individual student to get the same test twice.
They found that students would print (to pdf) the questions and review them and pass them around, so they ALSO made the test/retest window quite small for the entire class. It was well received, used for several years with about a 30-40% question rotation/replacement every year, and scrapped when the professor left on an extended leave/sabbatical - no one else in the department felt it was a good use of their time to adapt their materials to the system - they all preferred to stick with standard university "test systems". What a joke. I've since adapted the system for myself and lost touch with the prof who set it up.
Quartz Extreme and Core Image. Are there any other real reasons to spend all that money on generic hardware?
...this answer. Or (D) let the students make up their own cards, like many posters are saying.
From the sound of your question, you don't care about students having all the formulas, etc., which is good. Many (most?) of my exams have been some combination of these suggestions, or open book. I think my undergrad university might've had a generic physics equation sheet, because I tended to get sheets with equations that I didn't even recognize. At least one of my exams included several pages of common VAX assembly instructions.
My more recent degrees have either had open book exams (those were the toughest) or "make your own notes". Usually restricted to how much you could bring into class.
The toughest exam I've had in the last four years (I'm halfway into my second graduate degree right now) was open book, open notes, you could look up your notes on your computer, etc. I even think you were allowed to get onto the internet. But the questions required sufficient understanding of the processes that if I'd tried to answer it by searching online I'd have run out of time before I was halfway through.
Anti-Cheat #1: If you are using university computers to do the exams, you can always use a key-logger/screen-logger - so you know exactly what information they search for. Though open information and privacy are huge concerns in every day life, try to find a way to explain the need of Internet privacy during an open Internet exam. You must of course, pre-warn them that if they access their own email or any site with passwords, that snapshots will be recorded, as will be their passwords, etc.
Anti-Cheat Backup: This should go with #1, but used separately if #1 proves too difficult or if there is somehow a privacy issue even during an exam (oh, i just thought of one - my fingers slipped 104 times in a row and sent an email to a friend with the exact question i needed answering and got a response quickly was pure coincidence and I couldn't possibly not write down that answer since that was what I was going to write anyway!). Anyway, the back-up: don't install one, but SAY it's installed (be sure to go on about the name of the software, developed by X-high-tech-company-just-for-universities, what kind of new technology that has gone into them, how it's code is updated quarterly, and be sure to emphasize that every single keystroke makes a difference and can be used as official evidence for expulsion in the case of cheating).
Ways to Cheat: Photo cryptography may go undetected in some situations, though it's difficult when you have a key-logger present in the case of two-way transmission. One way cryptography is possible, though also difficult (If the list of answers to all questions exist in photos somewhere good for them.) (two-way is not practical -> the student can't send without typing keys - unless he uses a Ceasar cipher or something that is relatively quickly broken. Any other cipher for that matter, though this is already a huge red-flag in the first place if they aren't typing an accepted language or are using obvious code wording (What does the Gangster mean daddy when he says "The duck is about to lose its feathers..."?) Another crack, if someone uses something like a wiretap (leaving a phone on or other device) so the listener can hear the question and post a page. student could write every question on the screen as well while another 'listener' is watching (which is relatively easy if there's a pre-agreed upon site or by listening in on the phone), post the site relatively hidden on another pre-agreed upon website after x-minutes - and do this in of a pre-agreed upon cipher (such as the first word of each sentence is the solution (or more obscure than this if they are hard-working enough)). I think that's a good start - and unless I'm missing something obvious, it is about as tough to cheat, if not tougher, than current exams (university computers are relatively secure, and no phones allowed during exams - oh i guess hidden phone may be another one, but that's general cheating related, not open Internet test). Someone may find another way, at which time that knowledge can be used to thwart future use of that method, or discontinue it's use if it's a devastating one.
-Ultimate Stickman Game Developer Infinite World Puzzler
Essays are probably harder to cheat on without getting caught.
Essays are easy if you have an accomplice outside the classroom that does completely your job. For example a student of a previous year.
I would say to avoid short answer questions like multiple choice or one word answers.
Short answers can be ok if it is take longer to copy the question to an accomplice outside the classroom than to solve it yourself. Keeping the problem on paper can help for that.
I was a secondary school teacher for many years. We used to allow open book tests. The students always found those to be the hardest. Many students wouldn't bother to study, since they could look up the answers. But looking up answers takes time, and exams are time-limited. I suspect that you will find exactly the same thing in your classes. Let them look up whatever they want, but structure the exam so that the questions are problems that require understanding of principles, not just recitation of facts, and you should be just fine.
Aah, the flaw of old style knowledge.
Supposing there is no problem that can't be solved by some juicy resources.
(Insert complicated test question.)
(Correct but suspicious answer comes back.)
"Congratulations! You have earned your B. S. in Business Management!"
"But I am an Engineer!"
"No, you clearly don't know the class material. But you have already set up a consulting business. Therefore I automatically forwarded your exam to the Business School!"
Problem Solved : )
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
Respondus Lockdown Browser will 'lock' the student's screen so that s/he can't go anywhere else but that test page. It doesn't minimize, alt/tab, or allow any access to new tabs/windows etc. Of course, that pre-supposes that the student doesn't have a cell or some other device to find answers with, even if it's just to text ChaCha with LOL
You don't have them use the Internet. At all. You ensure your engineers are trained to be able to still use their knowledge when the Internet goes down.
Cranky educator.
Set up or request a closed intranet. As in, no access to the outside internet. Perhaps have saved versions of the resources they might need. I know that digital versions of encyclopedias and textbooks do exist.
What you are asking for is very difficult to provide, even if you had the perfect AI-based proxy server of every librarian's dreams: the person simply is honest and does not cheat or access prohibited content. I have taken take-home tests where I explicitly avoided looking up the solution, but the Internet is crafty, and I eventually ended up reading a variant of the problem description that subtly provided me with hints without my knowing. When I reflected on my thinking, I realized that it was severely effected by what I found on the Internet and that some of the leaps I should have made on my own were actually provided to me. Was I cheating? Perhaps, but not on purpose. The line between 'related' and 'solution' is very blurry, and even humans have a hard time distinguishing, let alone some automated arbiter or policy.
--"You are your own God"--
I teach at a university, my course is about network protocols and IT security. I prefer to trust my students rather than use punishment as a way to influence them. My attempts to eliminate cheating are quite effective, because the results of the exams are always within my expectations, i.e. a mediocre student never got an A out of the blue.
Here's a review of my methods:
- The final grade is derived mostly from the practical assignments they get throughout the semester. In this context I get to talk to each of them and spend a lot of time interacting with everyone in my group; this is how I know what they know.
- The final grade is computed as 60% = practical assignments and 20+20% = midterm and final exam. This way, even if you cheat at the exam, it won't help you very much, unless you also worked hard during the _entire_ semester.
- Formulate questions that don't take answers that can be copy/pasted from a book, the lecture notes or the Internet. Any question must require analysis. One who thought about it in the past will easily deal with it, one who has never been exposed to the ideas of the course won't be able to construct a good answer in a reasonable amount of time.
- Give them more time than they need, to ensure that time is not a bottleneck of their performance.
When I mentioned social engineering, I relied on research by Daniel Ariely. You can influence people's behaviour in multiple ways:
- a written commitment not to cheat
- give them a moral problem to think of, before giving them the exam itself
- adjust the environment (in your case, tell them that all the Internet traffic is logged - so they know that they _can_ get caught)
For example, I used these tasks in the previous semesters:
- "write as many of the 10 commandments as you can remember" (taken "as is" from Ariely's experiment)
- "actually, there were 11 commandments, but one of them was lost. Think about it and write down a rule which is worthy of being listed as the 11th commandment"
- I once tried a written commitment too. Everyone who was in class signed it and smiled: https://www.facebook.com/photo.php?fbid=469536753019&set=a.453850808019.243204.739418019&type=3&theater Three years after that exam, people are still talking about it and are proud to be a part of that experience.
You may be interested in:
- "Predictably irrational" and "The upside of irrationality" by Daniel Ariely
- http://duke.edu/~dandan/Papers/BadApples.pdf - here's an example of a paper he wrote about cheating, there are other ones too.
You must also make sure the students care about the course and want to learn, rather than just get a passing grade. Have a look at my notes of a book about this, "Punished by rewards" by Alfie Kohn: https://www.facebook.com/media/set/?set=a.10150475760123020.375546.739418019&type=3&l=70e1f3712e
I tried to ensure my assignments are not only useful, but also interesting and fun to play with. A basic requirement is to make sure some humour is always involved, with some references to Futurama or Monty Python or some sci-fi book or movie. Here are some examples:
http://info.railean.net/index.php?title=Lab2_-_HTTP_crawler
http://info.railean.net/index.php?title=Lab1_-_simple_client/server_application
At the moment I'm in the process of devising a very short code of ethics (if it is long, no one reads it). You can read the draft: https://docs.google.com/document/pub?id=115bLhvMUisnw
The saddest poem