Slashdot Mirror
Ask Slashdot: How To Allow Test Takers Internet Access, But Minimize Cheating?
New submitter linjaaho writes "I work as lecturer in a polytechnic. I think traditional exams are not measuring the problem-solving skills of engineering students, because in normal job you can access the internet and literature when solving problems. And it is frustrating to make equation collections and things like that. It would be much easier and more practical to just let the students use the internet to find information for solving problems. The problem: how can I let the students access the internet and at same time make sure that it is hard enough to cheat, e.g. ask for ready solution for a problem from a site like Openstudy, or help via IRC or similar tool from another student taking the exam? Of course, it is impossible to make it impossible to cheat, but how to make cheating as hard as in traditional exams?"
I remember being allowed to bring notes with me to class. Would just making this open book/open notes accomplish the same thing?
Ok, I give up, why you?
They don't need the whole internet; only a handful of sites. Set up a proxy that permits only GET requests to a few domains like Wikipedia, disable Javascript for good measure, and you're done.
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
If in a lab situation, use software that records sites visited, or is capable of viewing the student's screens. Make it known that this software is being used.
Make sure the questions are unique, change them between each exam, and carefully watch from the back of the room. You could also ask for a log of all the traffic through the WiFi point, and search for know chat domains.
I would say to avoid short answer questions like multiple choice or one word answers.
Essays are probably harder to cheat on without getting caught.
Because there is no way to do it. However, you can set up a monitoring program to see who goes where.
If Jimmy goes to "FreeEssays.com", you know you should approach him about it.
In "real life" students will have access to all those things. Perhaps it isn't cheating but rather utilizing tools that they would have access to in "real life".
Assume they'll use every tool at their disposal- and write the tests in such a way that they can't copy the question into a search bar and google the answer.
"That's the way to do it" - Punch
The ability to effectively use resources, literature, and yes, web searches, is based on having an innate, intuitive sense of the field. Anyone can search. A person who has a base knowledge in his or her head can search better. If you want to replicate the job experience, do an internship. If you want to LEARN, keep the Internet out of the testing.
Teaching means showing the way to solve problems. Nobody cares about correct solutions to school problems. It's all about the process of solving the problem, a scheme of thinking.
Block all traffic except port 80 http. (They don't need https, do they? They aren't checking bills online or using email, or some other security oriented task...)
Block all udp connections.
Dns filter a blacklist of known cheating sites.
Block bullshit sites like facebook, myspace and pals too. That's just good sense.
"And it is frustrating to make equation collections and things like that."
(A) Suck it up and do the work once.
(B) Use a textbook that comes with a premade formula card for use on tests.
(C) Find a premade formula card online and distribute that for tests.
Personally, I use option (B) for my math classes. Trying to make the internet non-communicable is like making water not wet.
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
When I was in grad school, in many classes we were allowed to use the internet on tests, as well as our notes, any spreadsheets/programs/scripts we had pre-made, etc. The caveat was that the tests were structured in a way that if you didn't already know what to do, you wouldn't have enough time to look it up and still finish the test. Googling things takes time. And the test really only provided enough time to actually do what you already knew.
You can also use random variables for each test, or groupings of tests, to prevent direct copying of answers. With a time limit, cheaters would have to wait for someone else taking the test to find the correct answer, send it out, and then modify it to match their own variables. If they can do all of that in a crunch, chances are they understand it pretty well on their own, even if they are lazy.
Give them access to a copy of wikipedia on disk. If they can't find the information there, they will be unlikely to find it elsewhere on the internet, but there should not be explicit answers to test questions.
http://meta.wikimedia.org/wiki/Wikipedia_on_CD/DVD
My school wouldn't even proctor the exam, they'd just expel you if you were ever caught cheating (no ifs ands or buts) , so getting an A instead of a gentleman's C by cheating didn't seem worth it. It did happen of course, and roughly 0.1% to 0.2% of the student base would get booted every year.
So what happens when all the "original content" makers die off? If we just search the web, we'll only get old information. Let people figure out how to create their own OC by searching within and solving/exploring on their own, so that the future internet will have new information. In the meantime, grade on the curve just to keep the education process moving.
One must either accept that cheating is an archaic, irrelevant term in a post-internet society, ar just not allow internet. Your choice.
I was an exams officer for a couple of years and we used e-testing for certain subjects. The golden rule from the awarding bodies was to treat it the same way you would any exam, have invigilators watching the students. Requirements would vary between different awarding bodies (HE & FE levels). For a Poly it would depend on what you are teaching and who actually issues the qualification, but it's the exams office that would be responsible for telling you.
From a technical standpoint lock the PC to a kiosk mode and firewall access to anything other than the exam site. Also remind students that if they cheat they fail and are removed from the course.
If there are laptops involved with wireless connectivity, anyone could just start a 4g hotspot with their phone and bypass any firewalls blocking them from answers.
If you're going to allow them unlimited research, then why not let them collaborate too? Give the whole class a set of problems big enough that they need to organize and split them up to get them all done in time. And if they can find the solution already completed elsewhere, so be it, that's what a good engineer is supposed to do. The whole point of working in the real world is that your performance depends on those around you, so the only way to measure the performance of students individually is to put them in an artificial problem solving situation like a traditional exam. That's why we still have paper, closed-book exams in theory classes, and why there are an increasing number of "project classes" where the entire class grade depends on the success of a hands-on group project.
Don't waste their time with trivia that they will remember forever. Have them focus on solving real world problems with real world constraints. They will respect you. They may even be able to see school as relevant and congruent to improving their world and their lives.
Why are you measuring problem solving skills of your engineering students? Are you teaching problem solving? Or are you teaching a subject? All of them passed your schools entrance requirements. You should be able to assume they have some minimum IQ. If you have a dumb student that has mastered the material are you not going to pass the student?
Simple answer. Allow them to do whatever and then review what they visited. If there is any sign of going somewhere that might be questionable, call for a review.
Practical exercises another.
I'd say keep the exams closed book/no net, and the practicums open (you can't help but have them open). But then take 3-5 minutes per student and make sure that the practicum is at least fully understood by the student with an oral exam (TA's can handle that if too much workload).
Check your premises.
Ultimately, the cheater only hurts themselves. It shouldn't be your concern as to whether they are cheating. The only thing a lock does is keep an honest person honest. The cheaters will find a way, no matter what you do to restrict them, so the better solution is to make them take responsibility for their cheating by trusting them not to cheat.
My alma mater has an honor code, that is essentially this, on every assignment for credit, whether paper, exam, etc... you had to write a statement saying you upheld the honor code and sign it. In return, professors were hands off when we took a test, they weren't allowed to be on the same room (they had to be available if we had questions, but they were not allowed to watch us take the test. The school TRUSTED us to do the right thing, and the amazing thing is, most people did. Cheating was certainly not eliminated, but I'm willing to bet that there was far less cheating than the typical college.
I'm not sure how this would work with copyright, but it would seem to achieve your objective if you store just about every resource you can possibly find on a flash disk or similar and hand it out, or share through a network folder. No need to make sure all the material is 100% correct either, because that is a skill in itself. Start with Euclid's Elements.
If you're feeling particularly kind you could even make it available beforehand so they have time to familiarise themselves with the different data sources. I think that would come the closest to a real life experience a few months into a new job.
You could use a firewall on the machine of your outgoing connection or perhaps even setup a firewall on your router. The basic idea is that you want to filter connections so that only the hosts you want students to contact are available. On a Mac, there is ipfw. Here is a simple perl script:
#!/usr/bin/perl
$IPFW = "/sbin/ipfw -q"; # this will execute the commands we want and the -q will silence output from ipfw
`$IPFW add 700 allow tcp from any to en.wikipedia.org out`; # allow outgoing connections to wikipedia
`$IPFW add 710 allow tcp from any to upload.wikimedia.org out`; # '' ''
`$IPFW add 9000 deny tcp from any to any 80 out`; #deny outgoing connections to the web
You will of course, have to run it as root.
To turn it off use:
# ipfw -f flush
I can make the full script available if you would find it helpful.
How this helps.
I was looking for a way to cheat, thanks for pointing me in the right direction with openstudy.com........
I'm in a program at NYU-Polytechnic. My professors have been fairly successful at building exams that they can complete in half the allotted time. Many of the students in my classes take most of the available time. It's not enough time to be able to do research unless you're a wizard at google hacking. Sure, they could ask someone or have someone take it, but the alternative is proctored exams, which is a ROYAL pain.
If you allocate a *tight* amount of time for each problem, then students will find that it takes too long to cheat by googling. The downside is that you'll get complaints about your exam being too hard. In particular, students won't have time to make mistakes and correct them - they have to either know the material cold, or fail the question and move on.
Also, remember to change the questions every year.
You can forbid, but for some that only entices. Far better to create an atmosphere of merit, that is a clear sense that cheating is exactly that, cheating. Making clear that people'll be found out anyway and the obvious repercussions are only part of it. In a certain programming language related IRC channel I frequent we often see people with homework questions. We're happy to give a hint or explain things, but also make really clear that the solution must be your own. Exactly because homework is there for you to learn. That sense is what you need your students to have, so you need to work on conveying it.
Beyond that, there are numerous ways to shuffle the questions and all that, but while necessary they're window dressing.
Suppose it costs $20/hour to hire a student to help proctor a test.
Suppose students take four classes per semester, two semesters per year, four exams per class, two hours per exam. That's 64 exam hours per year per student.
Hire one proctor for each of ten students. So each group of ten students will have to pay for $20 * 64 proctor hours. That's $1280 per ten students, or $128 per student per year for exam proctoring.
Now, let them use the Internet as much as they want, and have one student-proctor monitoring each group of ten students for inappropriate behavior. That costs $128 per student per year.
Now, hire an additional set of proctor-proctors for another $128 to manage and oversee the first set of proctors. Hire students from the business school and give them half a credit of management.
With twice the estimated required number of proctors, that's still only $256 per student year to closely monitor the tests. That is not a large portion of college tuition.
This sounds like a very solvable problem -- if the institution is flexible enough to come up with interesting solutions. Seems like being able to come up with that kind of solution would also be a pretty good way of judging the quality of a university -- good PR opportunity.
Having grades align well with academic proficiency seems like a high-value line item for universities. Spending less than 10% of tuition to make exams more accurately test for subtle skills seems like a worthwhile investment to me.
Stop-Prism.org: Opt Out of Surveillance
So does the whole group get an A, if they have some rock star who knows the material cold while 4 of the other students contribute absolutely nothing, and should have normally failed the exam?
That's no different than one person doing their homework and letting their friends copy it.
Really. actually even pre 19th century - times in which where knowledge was more theory than practice.
Now, it should be practice. tests should be abolished. people should be given continuous assignments, projects and workshops, and instead learn things while doing them, as it should be - instead of memorizing stuff from a textbook and courses and to write them down when prompted.
Read radical news here
I don't want to do the work to create unique tests, so how can I keep them from getting outside information?
Oh, I also want to allow them to get outside information.
Look, there training to be engineers, you can not prevent them from access 'part' of the internet.
Don't let them access the internet at all.
THEY are there to learn that subject. As such they should figure it out. ANY engineer that has to look up everything its a crappy engineer.
The Kruger Dunning explains most post on
When I took the SCJP 6 years ago, Sun directed me to a one-room test center with maybe about 20 cubicles with PC's. The guy at the counter was the proctor, they handled tests for many different vendors, not just Sun.
Not sure they still have these, but it seemed like a decent solution at the time. They could probably serve online colleges as well as IT cert exams.
Require them to turn over/UL the recording when they hand in their test.
To make sure you have a time appropriate recording just have them visit, at the beginning and end of their test taking session, a website which displays the time and some unique identifier such as the an image of your liking (just make sure you never use the same image twice and always post it at the time the test starts).
Even if you don't ever go back and watch the recordings the students will be less likely to attempt to "cheat" if they know there is someone potentially watching. Kind of like cameras in a store to would be shoplifters.
Of course if you do have a student with the cajones to cheat and you don't catch them they will be more likely to try again the next time. But then again you will have all the recordings saved still so you can always go back and review previous test recordings if necessary.
If you're worrying about testing them on what you want to test them on, you may want to start looking at BLOOM's Taxonomy and try to write your test questions at a more difficult level if you want to go down the "test question" route.
http://en.wikipedia.org/wiki/Bloom%27s_Taxonomy
I have had to deal with test development and the difference between Knowledge and Comprehension level questions for a few years at one of my jobs. First off, Knowledge level questions for tests are fairly easy to develop. Comprehension and higher level questions are much more difficult in the long run because there is a lot more that is involved in the question development side of the house and also in the test question validation phase too.
Besides, if you are already asking these questions, you may want to look at the whole evaluation mechanism that you are using and maybe try to emulate it after something in the real world that would give you a better measurement of the Learning Objective(s) you are trying to accomplish with your students. To me, an overarching project with multiple stages to it over the time period of your course would be a much better method to accomplishing this versus doing a "test, quiz, etc." just to have some numbers, even though I'm sure your Stat-Addicts employers would love some kind of arbitrary measurement that they can use to justify buying more stuff or "why" they program is producing such wonderful students. Or even better, get them to work together in teams since that is how they do it in the real world anyway.
Unfortunately, the way testing is done nowadays does not EVEN come close what people are seeing when they work collaboratively in the real world. So I would suggest that you look at that maybe and readjust your testing mechanism to work better for your learning objectives. You'll come across as a much more objective instructor and actually get phenomenal feedback from students with something that will stick with them when they move out into the corporate world and start working together successfully in collaboration because they had the chance to do that in your class.
If you're at all worried about this, don't allow internet access. Either allow it or don't, but don't half-ass allow it. If you let them open any electronic device, you have to assume they have access to the full, unfiltered internet. Welcome to the 21st century, where we have cell phones with wireless tethering and all manner of wireless access dongles like 4G modems which are completely out of your control. I suppose if the classroom was surrounded by a Faraday cage and only wired internet to their desks was allowed you could try to filter it, but then you're putting yourself into an adversarial relationship with blackhat engineering students... not a great place to be. If you think you're smarter than them, you're probably right, but they still might try a thing or two you haven't considered.
If you're testing them individually, due to the problems mentioned above, don't allow internet access at all. Cheaters will talk amongst themselves, which means in the best case scenario you'll have a bunch of students to fail, and in the worst case, you won't even know, so they'll have artificially higher marks than everyone else.
It's better to vote for what you want and not get it than to vote for what you don't want and get it.
- E. Debs
You can setup which websites are allowed and block all others right in the Browser Options/Content. It's called Content Advisor, I've done this for me kids, works very well, and it can be password protected. Takes 5 min. to setup.
-- By all means let's be open-minded, but not so open-minded that our brains drop out.
Is that the cheaters are going to cheat, no matter what you do. Had a guy who listened to class material on his ipod in class. Wrote answers on their hands. Texted them to each other. Bluetooth micro-earpiece and mumbled questions under their breath. You can't stop them. You just have to let them know that when they get their dream job, with their fake resume and their unjust transcript, and they get fired within a month because they can't do it, that perhaps they would have been better off learning the material.
I had an algorithms professor who gave take-home mid-terms and finals. His policy was that students were not allowed to talk about the exams with anyone except him.
The problems sets were difficult to say the least. From a student's perspective, it's much easier to cram for an in-class exam than to solve difficult problem sets. Due to the nature of the given problem sets, I'm guessing that the professor could fairly easily spot attempts at cheating. The answers rarely consisted of a one-liner, and generally built upon the solutions to sub-problems. I'm also guessing that most students went to him to discuss at least the more difficult problems during office hours.
The trick, I think, is to choose the right problem sets. There are a lot of problems that you can't simply look up on internet. I'm also guessing that you could take the concept one step further and have students orally defend a random answer.
I think the best way to accomplish this is by having them solve a lot of problems in a time that would actually be too short. Then grade on a curve.
That way, only the ones that are really able to solve them will. Also, give an extra problem for them to solve.
For example, let's say 40 points is the maximum in the exam. Give them 11 problems to solve, each of them worth 4 points, in only an hour (or 2-3 depending on the subject and difficulty. Anyway, not enough time for them to solve everything).
The best students will probably solve 9 or 10 problems, just barely. Most of them will solve 5, 6 problems, and get enough points to pass.
The ones who don't know the basics will be able to solve 1 or 2 at most, through Google or cheating, because other students will not have enough time to help others.
Also, change the questions every year, and if fewer then 10% get over 36 points, grade on a curve.
Tell us what school you teach at so I'm certain my daughter doesn't apply.
This is perfectly true. Part of a skillset is how to get help. Even, you could say, getting help through blackmail or cheating.
But there's only so many people to get help from.
Sure, those helpful people can set up a chat room during the exam and have everyone pass.
Which is great until the helpful people all retire or stop bothering and none of the students can solve hard problems on their own.
My thought process: block all ports other than port 80 - not effective, see tools like google chat block port 80 + internal dns a records to make sure chat/email sites like gmail, hotmail, yahoo mail don't get resolved - still not foolproof, and a chat client that operates on LAN could get around it (engineering students are clever after all), alternately phones can sit in your pocket and be tethered and no one would know you're not on their firewalled connection. use school-provided laptops? - too expensive How about make a program that the students are required to install to take the exam, and the program screenschots at random times what they are doing and uploads it to a LAN address so you can just see what they're doing? Maybe even get a programming class to write the apps and analysis software as one of their own final projects. - is definitely an invasion of privacy though (if students currently taking an exam can claim to have such a thing) Or just make the exams so friggin hard that if they have to google every little thing, they won't get a good grade because they won't finish it. Ask for things like to sketch flowcharts that will not translate over text or chat in a meaningful way. (and if 20 students all submit exactly the same flowchart due to an email ring, it'd be easy to spot for the grader)
Or it will train them not to think when wikipedia goes down...
Some of the best instructors I had taught the concepts and not the "units", and all the notes and cribsheets in the world were useless if you didn't understand the concept. Not every instructor can be one of the top-10, so maybe we do need to handicap those profs with internet access. /s
https://www.accountkiller.com/removal-requested
Only allow them to use Google cache to be sure they don't use some chat site.
so you want people who can cram to pass but not test people with the can doing the work for real. If you want to LEARN you want it to be like a paper mcse when you can pass by just craning for the test?
don't let them access the internet.
Current student here - I know from experience using prefab tests makes it much easier for students to cheat. They can simply use a search engine for a part of the question and typically find an answer. Writing your own tests and changing them from year to year require students to do the work while still being able to use resources available to them.
Have all the computers face the same direction. Place the instructor behind the students so he has visibility of all the screens. Give the instructor a computer that has remote viewing access to all the testing computer. Post the rules on the wall at the front of the room.
For added effect (depending on the test content), have a grid that shows every screen projected to the front wall. Allow all the students to see it, but have them small enough that they can't really read anything. This shows the students that you are watching with just enough information for him to pick out his screen on the wall. It's like having a security cam with a monitor showing the person that is recorded.
Make the exams where the first few questions are the same. Then the rest of the pool has questions that look a lot a like, but vary because of 1 or 2 key words that change the answer.
Also offer students a 5% bonus if they report a student cheating and they get caught.
You could just allow blanket access, require everyone using a connection to get MAC address filtered access (so you know every device requesting access) and then log everything. Then provide stipulations that any live chat or forum use is forbidden. Anything except reputable / academic sources is forbidden. To make it extra fun, tail the log of the access point live (projector?) and grep it through a few good regex to weed out junk and find any terms associated with IRC, forums, etc etc. Allow them to ask for white listing sources, or provide your own (allow wikipedia, but not the discussions on each page which can be used to carry out conversations, etc). Or just allow all net access but restrict access to just the sites you think are of use (wikipedia, specific journals, publisher's reference information, google for unit converting on the search bar, etc).
You're never going to be able to make cheating as hard as it is on non-open tests as it is on open tests. That's an inherent problem in allowing access to outside information, particularly when you're dealing with worldwide communications.
What you can do is minimize the impact of cheating by working with the test itself: in particular, by setting a time limit based on its length. The idea here is to make it so that someone who constantly looks up outside information is highly likely to run out of time to finish the test. There's a delicate balance to be struck here, because you've said that some amount of going outside for information is not only to be expected but completely appropriate. But at the same time, you expect at least some knowledge to be "in-brain" (for lack of a better term), and so by using in-brain knowledge when it's there, a passing student will be able to finish the test quickly enough to beat the time limit. The trick is calibrating things, and I'm afraid I don't know a good solution for that.
Create A VM that only allows access to sites you have approved ahead of time, then get something to track all the traffic like a Netwitness or record it all with Wireshark (save it as a PCAP). Tell them traffic to unhallowed sites will be investigated for cheating. The idea that you are monitoring their internet traffic will be just as powerful (if not more so) than the monitoring itself.
The problem with a traditional teach-learn-test-forget-teach cycle is that students have to stuff as much of the lecture material into their brains as they can fit, pour it all out on the test and repeat the cycle. In my opinion, having tests that actually check for understanding rather than memorization capability would promote actual understanding of material instead of the repeated stuffing.
I've been out of school for a while, but I have recent anecdotal evidence -- vendor certification exams. Specifically, I took the VMWare exam recently. I passed, but it was quite difficult because I work with the product on an infrequent basis -- that is, I don't have the entire GUI memorized. More than half the questions would be easy to answer if you had the GUI in front of you and could just check the available options; the rest tested your knowledge of product architecture, limits and quite frankly trivia items. I've never done well on exams like these, because I'm just not a memorizer.
When I was in school a million years ago, with the Internet just becoming a viable research tool, some of my upper-division chemistry professors wouldn't give standard exams - we'd get "take home exams" which were actually mini-research projects that you could do pretty well if you were paying attention in class. The questions were just right in most cases...challenging enough to be a major pain to brute-force your way through, but made easier if you knew where to start looking (by knowing the material that was presented.) I'm not sure you can do this with a class of hundreds in freshman chemistry lectures, but when you have 20 or 30 students taking the class, and most are motivated to do well anyway, these are easier to do.
So the question isn't "how do I block Internet access for the test?" but more along the lines of "How do I make a challenging-enough test that can be finished in a finite amount of time, and doesn't just test student's lookup skills?"
You could always lock out IRC or any other places you don't want them to go. Also let them know that all sessions will be recorded in case any questioning of cheating comes up. Of course this is only if this takes place on classroom computer where you have control.
Ask them questions that require an application of working knowledge / theory, as opposed to vocab / rote memorization style questions.
A little less "What does HTTP stand for?" a little more "I need to do some task using HTTP, show me how to make it do what I want it to do." That'll nuke using Google for an easy look up (for an answer), and potentially make anyone who copies off of another (via texting, emailing, cellphone, whatever) liable to fail the class (plagiarism ho!). See, by making it a non-trivial answer, you destroy the use of search engines for an easy answer, and by requiring some creativity (or even a fair amount), you can more accurately gauge a student's understanding, while also ensuring (via creativity) that no two student's answers should be identical. Of course, there are potential problems here, but it does, with a little tweaking, should help you identify the group-thinkers or no-thinkers with some ease. Plus, job security, as a teacher / professor, as you get to grade everyone's exams manually (the techs know you fear the machines, you need not be shy about it); just be sure to announce at the beginning of class that your style is that of the Athenians (Greek philosophers, focusing on thinking, etc.), or something to that effect.
The key here, to berate the point, is to ensure each answer is unique. Since simple answers cannot be unique, it's impossible to ensure that cheating has not occurred. Whereas with the greater increase in complexity (but not necessarily difficulty, mind you) of the answer, the more unlikely it is that two answers can be the same without one person copying another. When complexity increases enough, you have the effect of the Mona Lisa, where if 5 people turn the same or similar enough picture in, you have an extremely good idea that they were in communication with one another. It's not mathematically impossible, of course, that they should all create the same Mona Lisa, only hideously unlikely. Hell, if the solutions are unique enough, you might even learn something from them.
I am John Hurt.
Let the students tell you where they're going for answers.
Tests are suppose to show that the students are learning, right? Then monitor the internet traffic and see where they're going for answers, that will show you if they've really learned how to find answers to questions or not. And give real life type word questions, not just "1+1 = ?", stuff like "If you have one apple and someone gives you another apple, how many apples do you have?"
If they're going to sites like Openstudy to just ask someone to think for them then block access to that next test or live depending on your lab is setup, but remember sometimes going to forums and such are the best places for answers to real-life problems so I'd be careful trying to decide what sites to block.
Also you didn't really explain how they would have access so I assumed they would be in some sort of school computer lab, not on their personal laptops, and you have access to the network traffic and can restrict access at will.
my karma will be here long after I'm gone
As you say, when they get into the workplace, if they need to know something, they're going to Google it. By limiting the resources they can use when answering exam questions, you are increasing the distance between education and the workplace - who does that help?
So set them realistic problems and let your students solve them by hook or by crook, just they way they'll solve them when they're on a payroll.
At least that way, the students who do well on the test will be the ones who will do well in industry, as opposed to the current situation, where we have engineers who can do CFD by hand, but can't tell the difference between Aluminium and Steel.
Ask questions that require the student to demonstrate synthesis of the various things you've wanted them to learn. They might be able to google individual steps to solve the larger problem, but they wouldn't be able to google the end result - they'd have to know what they're doing. Put a time limit on it.
For example, in an introductory programming class (a CS 100 level course at my university), the final for the class consisted of this:
"Take a sound file (speech.wav) consisting of several dozen words with brief pauses between them and create a program that will output a new sound file with a name of the user's choice. The new sound file will consist of the words from the original file reorganized into a random sequence. The code must be commented to explain your thinking at each step. You have 3 hours for this task."
That task would require that the student understand how to analyze a problem, break it into smaller problems, design a process to address each of those problems individually, and then combine those solutions into the finished program.
Oh, sure - the student could google "how do I resize an array in java" (which might be necessary to know to accomplish several steps in the task), but they would need to know so much more in order to even realize they needed to know that, and that stuff wouldn't be something they could just look up.
The only downside would be the ability of the students to send the question to another student who is more advanced and who would spit back an answer; to guard against that you'd need to actually have an idea of what your students are capable of individually.
Since I can't tell them apart, I treat all ACs as the same person.
So sure, in real life you can look stuff up. I'm in the computer field after going through a BSEE a gagillion years ago. I've seen desktop support folks try to step into the server admin role thinking they can get by with Google and the MS Knowledgebase. Invariably screw it all up. Plugging in a mouse, troubleshooting email on the client -- sure you can flip through menus or decide that the flat end of the mouse isn't going to fit into the round port. But for more complex things, some training and theory are necessary.
In terms of exams, you want your students, especially engineering students, to have enough theory that they can go out and design stuff: create it from scratch. If all they know is the right equation for this or that and when the most likely time to plug in a number is, then you've graduated a tech, not an engineer. Don't tell the physicists, but engineers can come up with ideas too. :-)
Followed up with lie detector test.
Assuming your .edu already has a content filtering firewall (e.g. Smoothwall + SmoothGuardian) just get a profile created specifically for exam purposes with rules to block all IM/Chat programs, blacklist cheating type sites and queries and log all non whitelisted activity.
If the hardware is .edu owned then you might also want to consider something like Deep Softwares "Activity Monitor" so you can audit the exams... or perhaps just insinuate that you have installed something along those lines in the hope / knowledge that most students won't risk it.
Seriously, that is the big issue. Come up with classes of questions/answers and then have them picked in random. Likewise, have the results checked by the computer. If somebody differs, then they are handchecked.
I prefer the "u" in honour as it seems to be missing these days.
It's fairly clear that engineering students likely have the skills to comb the Interwebs for information. Instead of giving tests that might require it, give an open book exam and make sure that all the references they need are in the materials allowed during the test.
This will test the students' ability to identify what information they need to solve the problem, without giving them access to information that could allow them to cheat. Creating a test that requires access to the Internet to gather the information required to take the test is laziness on the part of the test creator, IMHO.
No, no, you're not thinking; you're just being logical. --Niels Bohr
I would say to avoid short answer questions like multiple choice or one word answers.
Essays are probably harder to cheat on without getting caught.
Or...
Make every question 20-multiple choice w/ different sets of 20 answers out of 100 on every test. Picking correct one by collusion is more difficult. To actually force the problem solving, interspersing questions where the correct answer is not listed and "none-of-the-above" is correct makes collusion even more difficult.
Bonus points for giving a test that where all the correct answers, but 1, on a 20 question multiple-choice test are "none-of-the-above". My high-school calculus teacher did that and I really, really had to think hard about that one question (which happened to be the last one) that wasn't none-of-the-above like all the other ones...
If you design a good test, and grade it well, then there will be no way to cheat.
I would design questions that require a good understanding of the material to answer, let them do whatever they want online, and then talk individually with students that you think might be copy/pasting answers, whether from other classmates or from the internet.
A 2 or 3 minute conversation with them will tell you pretty easily whether they understand their answers or not.
If they understand their answers well enough to convince you they could have given them legitimately then it doesn't matter what actually happened, they are either learning what you are teaching, they already know it, or they have good enough grasp of the whole field of study that they can fake it.
Any of those three outcomes is sufficient for your purposes, at least in my opinion.
The hard part here, of course, is designing good questions. It's incredibly hard to design good tests, which explains how few of them you find out in the wild.
RUGBYRUGBYRUGBY
Many of my engineering classes allowed "formula sheets" or a "formula card", usually a single sheet of paper or a 4x6 index card, that the student was responsible for formulating themselves.
I used this to completely ace the exam in several of my EE classes where I otherwise would have had great difficulty. (Analog just wasn't my thing while becoming a CompE; I rocked my digital and computer classes.)
My tactic: Virtually all professors provide sets of review problems, and the answers to the review problems (along with all homework questions and mid-terms) were on file with the library. I'd go the library and make copies of those materials. I would then go back to my room and pass-through every single homework assignment, mid-term, and review question, and solve every problem to the point where the remainder of the solution was "busy-work." If, after much staring, I simply could not figure out how the professor got from point A to point B, I simply copied the entire solution to that problem (writing very small with a very sharp pencil if I was confined to a card, or just about 3 rounds of reducing on the copy machine if I wasn't) onto my formula sheet/card.
90% of the time, the problems where I had to copy the solutions wholesale onto the card ended up on the exam (with some trivial parts changed), and I was invariably one of the few people in the class to get it right, despite the fact that I had utterly no idea how the solution worked.
When I was screening new hires with a knowledge quiz, I would allow them Internet access - but only for the last third of the time, and after giving them a red pen. Sometimes it is knowing how to find an answer, not actually knowing the answer itself, that is meaningful. It was also a simultaneous stealth test of Internet search skills. The red answers, and ratio of red to black, was frequently interesting...
We've had this exact problem. Setting up a policy that is both fair and hard for engineering students to bypass, is absolutely non trivial.
Instead, tell the people watching the exam to watch for extended keyboard noise. Most people reading material will use the mouse and perhaps type a few keywords, while cheaters will be typing out full sentences.
Make the test truly collaborative. Indicate that anyone can ask for help, anyone can offer help (more like a normal work environment). But indicate that credit for help is required. Put a spot by each question for "person asked" and "help given" -- that way person A has to mark "B helped me" on his test, and person B marks "I helped A." on her test. It'd be best if you can also make the questions unique. You want the help to be more than "pssst! what's the answer to number 2?"
Of course, helping someone else will also reduce the amount of time you have to complete your OWN test, so there's a certain amount of selfish time-management that has to occur as well.
You could then give a certain number of collaboration points for the test: "Person B assisted in 6 questions, and asked for assistance on 4 questions. She gets an additional 6 points, and loses 4 points - for a net gain of 2 points."
This encourages helping others, and discourages asking for help. If the names don't match up, then someone may be cheating....
This should work against all types of cheating.
Place each student in a sealed metal box with an automatic rifle built into the wall. Also build into the wall a video camera and a sign dictating the rules for the gun to go off.
Hook up the output of video to a facial expression algorithm. If the output of the algorithm shows any kind of deviousness in the expression of the student, or the student attempts to leave the view of the camera, have it trigger the weapon. Make sure the student knows that the gun is controlled by solely by computer, so they won't be tempted to try and reason with it.
You might want adjust the logic to trigger in response to anything but a happy expression. This will prevent a student from trying to hide their cheating thoughts behind a frown or a blank stare.
If your test is about problem solving, finding a way to solve a problem, then allowing internet access seems fine. However, if the test is about are they learning specific engineering techniques and principles, then it seems letting them search the internet for a solution would be inappropriate. While it is true that in the real world, they will have all of those online tools, a classroom setting is not a real world. Get your calculations wrong in a class and you flunk a test, in real life a bridge collapses and people die.
Give them the same sort of info they'd have if they were doing the job for real.
No one cares if the engineer looks things up on the internet while working on something. It's all about results. If his solutions are correct and reliable then no one cares.
So test them with real world problems giving them no more info then they'd have outside your classroom.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
Make the test so long that you can only finish it if you know the material.
Unless you're confident you can keep the students from using other connectivity methods (have one terminal per student and disalow personal devices, faraday cage around every student), then there's no use trying. What's keeping students from setting up an adhoc network to communicate with eachother, or just use the personal WAN or bluetooth connectivity their smartphones provide, without taking them out of their pockets?
I'd have to go with the other posters who suggested crafting the test so the students each get different enough tests, and structure the questions so that without a prior working knowledge of the subject, all the internet in the world can't help you finish the test on time.
But, an acual answer to your question could be to have the students submit a list of websites ahead of time and restrict them to only those they requested, and block any other traffic. You could even force them to a single local server with a cached copy of each of the sites.
Caveat: I got my degree in mathematics. Sometimes friends of mine would TA and we would have these thought experiments. Some of the methods that were used:
1) First we would make half the test conceptual and half the test computational. For the conceptual part in a calculus test we would say stuff like:
a) In order to find the speed function of a travelling ball based on its acceleration, we compute the antiderivative. Why?
b) Calculate the anti-derivate. Here is the speed function. Explain the importance of the K constant.
2) Reword a homework problem to see if they really know what is going on. If the student does, the problem is straight forward:
Here is a speed function. We know that the initial starting point of a ball is $CONSTANT from where you are observing. What is the displacement function.
3) We look at lots of old text books and find well worded questions that provoke thought for extra credit during and after the exam. It offsets the focus on test taking skills.
4) We ask them not to cheat. Those that can wouldn't. Those that need to are only fucking themselves. Make the test short enough with enough questions based on testing concepts and those that need to cheat will by and large run out of time. Then the curve will take care of separating the wheat from the chaff.
Think about your job. What do you do that can't be looked up on the internet? The odds are pretty good that you deal with a lot of issues that are not something a good Google search will help with. Otherwise, Google would've replaced us all!
If your main concern is that one kid will find the answer and share it with all of his buddies, then there are strategies to combat that. The easiest strategy is to make the class competitive. Grade on a curve, or inflict severe grade penalties to everyone involved in copied solutions. This has many drawbacks, but fostering competitiveness is one way to reduce collaborative solutions.
Another strategy involves randomization. Add a certain amount of randomness to the work (this can be easy to automate with LaTeX for tests of large classes). If you're trying to teach them an equation with a formula of (A+B+C)/D =E*F, then ask some of them to solve for A, some for D, some for F, and so on. Better yet, ask for situations when this equation is appropriate and the limitations of this equation. If you're trying to teach them how to take derivatives, then give each student a slightly different equation to take the derivative of (and remember to make life easy on yourself by automating the solution, so you can just print out a list of what the correct answer should be for each student).
You can also use the perception of randomness or the perception of a curve to try to curtail bad student behavior. Tell them you are grading on a strict curve, and then actually use a different system (note: this can get you in trouble at some universities - but it's worth considering that students aren't likely to complain about getting a better grade than expected). Add a cover to each test so that it looks like there are 5 or so different variations, but don't actually vary the test. Construct a few clever questions with different variables that all reach the same answer at the end - they look random but all the answers will be consistent for ease of grading.
that the flat end of the mouse isn't going to fit into the round port. that is what you get from some people with a BA in CS.
Now for server that should be some kind of tech school / learn on the job training.
and desktop support is a good starting point to get into sever now where did they mess it up?
also troubleshooting email on the client is likely more about how that client works vs the theory of email
... the candidates could ask the person sitting next to them. Or maybe get a contractor to do the work for them. Or... at some point, exams have to stop being exactly like the real world. Otherwise, you need an exam that looks like and lasts as long as the career the candidate might take up on passing the exam.
Virtually serving coffee
If you have to rely on removing all of my resources to 'test me', then your test is invalid. Unless you're training fighter pilots or some other 'split second decision' job, my ability to perform has less and less to do with my ability to memorize.
PS: I don't reply to ACs.
and use the jigsaw voice.
Also don't put macgyver in there.
1: Ask them this question, grade the responses on a curve and use the best answers for your method.
2: (non-silly method) Split the class into small groups of 4-8 students. Arrange a list of subjects and have half the group form test questions based off of the list. Make note of any preferred methods, or requirements to use in the questions (this is where you can guide them into asking the right kind of questions). Have the other half solve the questions. Then have them switch sides and repeat. Grade based on how challenging the questions are, scope of the answers, and use of requirements. Keep a DB of previous questions submitted to verify originality. PROS: gives students ample opportunities for failure, makes for some very interesting and engaging tests, helps to teach how to find a problem AND how to find a solution. CONS: Makes grading the tests a lot more time consuming.
Sometimes, life itself is sarcasm...
One of our professor would allow us 15 minutes to review the relevant sections of the book. Then we would close the book and go from there. This was a decent compromise I thought.
What about putting VNC or any remote desktop software on the PCs. Warn them in advance that they are permitted to research, but being caught outright asking anyone inside or outside the classroom is an automatic 0. Randomly pop into different students screens from your PC. (the benefit of remoting vs walking towards them, is by the time you connect in, it is too late to alt F4 as they have no idea who is about to be screened, and no ability to predict where you are looking, or even know whether you are in their screen or browsing slashdot.
or maybe something that grabs a screenshot every 25 or so seconds (randomized so that the student never knows when it will take the screenshot). That way the trail of the student can be retraced. I am assuming that the goal is to prevent a student from posting the test question onto a place like Slashdot and get an answer that way, versus doing the actual research. I've seen this almost exact thing happen over at gamerswithjobs.com luckily they are usually good at giving hints, but leaving the answer up to the poster to figure out until a while after the question is posted.
I don't mean to rain on everyones' "wikipedia should be allowed"-parade, but wouldn't it be simple if students would just collaborate? One smart guy/gal solves the questions and posts them on his/her blog. Wham, everyone correct answers all the way. Heck, the smart one doesn't even need to be at the exam. Just post the questions to stackoverflow.com and wait for the results.
So what would be the point of that exercise? Wasn't the point of an exam to gauge the individual capabilities of every student?
It doesn't matter that you'd actually use wikipedia to answer all your real-life questions. Wikipedia is not on the test, you are. Who's gonna write wikipedia if everyone is just reading it? You're in the school to learn the stuff so that you know it and can then use your badass skills to write some wikipedia!
Provide the devices on which internet access is allowed. Visually record and log all activity on each device and let students know beforehand that the recording will be made.
Let them edit it throughout the semester, and then only give access to that site (hosted on a machine on a LAN, not connected to the internet). Then everyone would have the same information base, and they could search for information. You could even keep the information alive through semesters. Make sure it's read only during the test. And you can go through it quickly after writing the test to make sure no information is too helpful.
Even better, or also, you could host a stack exchange like service too, that people can record and exchange information for the projects they do during the semester. Teaching is the best way to learn, after all...
Get a few people to walk around and monitor students. It's hard to cheat if somebody is looking over your shoulder. That's what they did when we were allowed to use our own laptop on tests.
I encouraged, but did not require, collaboration.
As much as possible, I defined in advance how much accomplishment constituded C, B and A work, respectively. I was never able to make this perfectly objective, and the clarity varied according to the material and my own imagination in the project assignment.
To evaluate each student's individual accomplishment, I interacted with the students. Sometimes I did this in scheduled individual face-to-face interviews. It was OK for a student to talk about other people's work as well as her own. But I only gave credit for insight explained to me through description of work, not for work that just sat there on paper (or online). Once in many years of this practice I encountered a student who appeared for the interview and seemed unable to express himself in a clear sentence. I made a wild guess for that guy's grade, and vowed to have midterm interviews as well as final interviews as much as possible.
Later, when I had a Wiki server, I required projects to be displayed incrementally on the Wiki. I had a very difficult time convincing students not to wait until the last day and dump lots of material out of the blue (nobody who did so got a good grade). In (thankfully only) one case, a student who failed to attend class (attendance and participation in discussion was mandatory according to written posted instructions) sent me a "project" 1 hour after the absolute final deadline, and seemed shocked that I awarded a D (the only grade below C I ever had to assign in this system).
I never achieved 100% understanding of the ground rules (in spite of posting them in explicit instructions on the Web and discussing them in class). There were always students who failed to understand that only iteration could produce a good result on these sorts of projects, and a few who understood but failed to act (sometimes requesting incomplete grades at the last minute). I was working on intermediate deadlines to eliminate such behavior as much as possible, but stopped teaching before I worked it all out. One student who understood, but failed to perform, in one class signed up for another topic taught in the same way, and did quite well.
For the students who understood, and acted upon, the instructions, the results were very good. I could tell students as they went along what progress they had made toward C, B, A, and avoid the end-of-term suspense that I find counterproductive. Evaluations at the time of the classes varied a lot, from those who loved the system to those who couldn't stand it. A few students got in touch years later, and were very positive. These were all students who had done well, and it is very unusual to hear from dissatisfied students long after class, so I have insufficient information about students for whom my approach did not work.
The principle that I tried to follow was to avoid completely any requirement in which the possibility of cheating was relevant. I tried to design all assigned work so that the accomplishment required worthwhile learning while using all available resources, including fellow students and outside consultants. I believe that this principle is good for pedagogy, and not just a way to avoid worrying about cheating (although my experience in a few cases of cheating in classes with conventional assignments and exams was painful enough that I might choose to avoid the possibility even if it were not good pedagogy).
I found this principle to require a lot of rethinking, and the application to each topic, context, instructor, and type of student is different. I'm not convinced that conventional assignments and exams are better in this respect, I think that they only make it easier to avoid feeling bad about the failure by attributing it entirely to the student.
I also think that there are particular topics and contexts for which my approach is not appropriate, but I tend not to like teaching those topics in those contexts.
Mike O'Donnell http://people.cs.uchicago.edu/~odonnell/
It depends on what your course is, but if you want them to solve problems in a real world setting, you give them real world problems. Last time I checked, even with the vast amount of information that's available on the internet, there are still problems out there that need solving. Offer those up.
If you simply ask them to solve equations, that's not really solving problems in the real world. But if you ask them to design a bridge/circuit, that's something else. The problems should also be open-ended. If you have simple questions with only answer, that's easy to cheat. But having open-ended questions with multiple paths and/or multiple solutions makes it much more difficult to cheat. (And are probably better examples of real world situations.)
SHOW YOUR WORK! :)
The more steps that need to be taken, the more 'samples' you have to see if people are simply rote cheating or not. If I'm solving an equation and I need twenty steps, and someone else does it in the same twenty steps - guess what? One of us copied off of the other. Even if they are smart enough to not copy all twenty steps verbatim, that still requires some level of intelligence.
PUNISH HARD
As the likelihood of catching someone cheating decreases, the penalty should likewise increase. Just make it clear that if you're caught cheating, you get a zero on the exam, and it doesn't matter if you were the person doing the copying or being copied from.
Where the wind blows, the tumbleweed goes.
You can restrict access with a whitelist or a blacklist, as someone has already mentioned. Another option is to install software to track which websites the student accesses; you could factor their research process into the grading and penalize those who access forbidden sites. If you do this, be very explicit about which kinds of sites are OK and which aren't so you don't have too much of a battle on your hands after the exam.
a few weeks before the exam, create a variety of fake webpages containing incorrect info relating to the test topic (but with enough on there that someone who really knows the stuff can tell it's wrong). Then if you see those 'facts'/formulas on the exam, you've got direct proof of copying *and* of stupidity.
>so you want people who can cram to pass but not test people with the [can?] doing the work for real.
Well this is one of the biggest problems that students from China face. Chinese academics tend to be based on a reward system for rote memorization, and often go as far as to literally punish attempts at creative thinking. So a student who has been brought up in that culture and then transported into an American university situation has a huge benefit and an even bigger liability. Asset, is that they are typically much better at straight memorization and cold recitation than the average American student. Liability is that it typically does not even occur to them to do any sort of lateral or particularly creative thinking. Better teachers and curriculum developers understand this and anticipate it and design mitigating controls into the curriculum. Poor teachers just assume that "Asian students are smarter" (because they can usually memorize like mad) and reward this to the detriment of others.
Log all of the traffic and let everyone know you are doing so.
This problem is not amenable to technical solution. Trying to stop attackers from cheating via the Internet, by using some a filter or other form of limited access -- is as futile as trying to solve the halting problem, and enumerate the irrationals, at the same time.
The halting problem fails because it's too easy to craft countermeasures aimed deliberately at the scanner. Enumerating the irrationals fails because there is so much complexity, it's literally impossible to go throgh it all.
But just because you can't solve this problem technically doesn't mean it can't be solved. It's difficult, but I believe it might be possible. Don't bother beyond the basics. Get a computer lab set up with computers you control. Don't allow the students to bring in any USB sticks or CDs.
Then simply install tracking software on every PC. (You can also use a network sniffer to back this up.) The idea isn't that to prevent cheating technically; rather, you want to preserve the ability to tell that people have cheated, and simply punishing them under the existing rules.
You tell everyone in the class that you'll be monitoring their internet usage during the exam. Then tell everyone what you consider cheating. Have your grad students go through the logs manually; the difference should be fairly obvious.
Look at what they did on the internet as part of their solution to the problem you presented (the test).
Although my engineering degree is over 40 years old now, I still remember one exam in Fluid Mechanics where everything was open book and open notes. It didn't help one bit if you didn't understand the concepts. One of the final exam problems was a very complicated siphon system and the numbers that one needed to plug into the applicable equations to calculate flow rate were not too hard to find. However, the problem was rigged so that the highest point on the path of the siphon was more than 33 feet above the source so the equations did not apply. However, this was NOT very obvious. If you understood how a siphon worked, and took the trouble to see whether this one satisfied the conditions necessary, you got full credit for just writing down the flow is zero since the necessary conditions for flow are not met. If you tried to use the numbers that were more obvious to calculate the flow you would get a reasonable answer that was completely wrong. This was one of three very similar problems on the final exam.
Asian students also do solo work as group
I meant to say so you want people who can cram to pass but not test people if they can do the work for real.
well then degree should not be in the work place.
Let's say take 3 people all the same who is better for a IT / desktop / server job.
1 who if a lot cert tests (ones that can be passed by cramming by people who don't know what they are doing)
2 who did a lot of IT work on there then / maybe went to a tech school that has real job skills.
3 Some one with a mostly theory based CS with just about none of skills that you get at a tech school.
Well the real work place is not individual and with some things you need to test people on the job / in real work place conditions. So in some places testing individual may fit but most real work is group based with open books / open Google. Even before Google you had reference books in the work place.
If your talking about taking tests from home, it is impossible to determine who is actually taking the test. Maybe it's 5 people in the same room all helping one person or maybe the person just passed his login information to a senior who took it the year before. There is a reason online colleges are not recognized by most recruiters or hiring reps.
Set up a wifi network and force them to use your proxy server.
Explain to the class that any and all encrypted communications will be construed as cheating and then use firewall policy to limit what services they can even use.
As a High School Teacher, if you're going to open the Internet up to them, wouldn't short-term (1 or 2 week) assignments be easier? (Other than the marking of course.) It'd test the ability to problem solve and research more realistically, and you could use turnitin.com to ensure students aren't copying each other/the Internet.
On another point, at the University I did both a pure Science degree, and then an Education degree (with credited subjects). What I found interesting was that the Science degree was taught in centuries-old formal format - lectures/labs/exam, but the Education degree was taught in small-class format/group projects/presentation assignments/etc. Only towards the end of my degree did one of my old science lecturers announce that he was taking some of the Education subjects as a way to upskill his teaching methods.
The tests I give are all 'open book', which I define as allowing any books, notes, documents or other aids (e.g. little model chromosomes for genetics problems) but no electronics. Unfortunately this means no e-texts.
Just tell them all screens are being recorded during the test.
For bonus points actually do it or at least setup a bogus program that just looks like monitoring software and say they'll automatically fail if they end it.
For engineering classes, allow or even encourage the use of the FE Exam formula book in 300 - 400 level courses. It'll have the vast majority of the formulas they need, and they'll get practice leafing through it for FE exam.
In (mathematics) open-book exams I've taken, the books were essential. The highest scored solutions consisted of recognizing the problem, butchering equations for a few lines, and then pointing to the correct page in the correct book. The people who got highest grades were done in 15 minutes, though you were allowed to sit for at least 3 hours.
But, of course, to do that, you had to actually understand what it was all about, and you certainly couldn't learn that in 3 hours with all the books in the world.
disclaimer: I might be right.
Their are 2 way you could go about this.
1) Authoritarian route : State before the Exam starts that Key-logging and screen recording software has been installed on all machines, and no cell phones allowed.
With an automatic fail if key-loggger or screen capture software is disabled or caught using a cell phone (with a supervised resit later the same day with an automatic 1/3 drop in maximum score.)
2) Sneaky B'stard approach : Make the question so hard or badly worded that their is NO definitively correct answer, or that no 1 answer makes sense. Then it would be harder for "little Jimmy goggler" to find a correct answer on cheating sites, unless he's been taking notes and paying attention in class.
Either approach probably won't work but it's my suggestion ^_^
Laters Sol "Have you found the secrets of the universe? Asked Zebade "I'm sure I left them here somewhere"
" I think traditional exams are not measuring the problem-solving skills of engineering students, because in normal job you can access the internet and literature when solving problems."
No, they are measuring the problem solving skills of the students. During a test on, say, electrical engineering, we want to measure THOSE STUDENT's electrical engineering skills, not their ability to search wikipedia, google, etc., and certainly not their ability to talk to AIM or Skype with their friends who might know the answer.
It is true that in a real job they may be allowed to use the Internet for some things, but they are expected to know their field and have core expertise without looking everything up - and the tests are designed to measure those things. In elementary school math class when you are learning addition and subtraction they don't let you use calculators, because they would give you the answers to the thing you are trying to measure. When you are studying higher math, they might let you use the calculators, because the test isn't so much about the numbers anymore. The fact that you will almost certainly have access to a calculator some day at work doesn't mean that they shouldn't require you know basic addition and subtraction.
To give you a geek analogy: A computer has processor registers, processor cache, memory, maybe flash storage, and then finally a hard disk and perhaps network - all in order of decreasing speed. If the computer has something in memory, the access time might be a nano-second. If it has to go to the disk to fetch it, it might hake hundreds of times longer - Thus, we expect the most important data to be in memory, and hopefully in the processor cache. It's the same with people. We can't expect everyone to know everything about everything, but we expect them to have the core of what they need to do in their cache, so to speak. The exams are a type of test where we disconnect the test and run the program to see what it can retrieve from cache.
If you want to make this some kind of practical test including research skills, just set up an internal wiki or something and give people access to that, but not the internet.
When I was lecturing, a much older (tenured) colleague came up with a series of multiple choice tests, and made the time limit very limiting - with the explanation that it was enough time to answer the questions if you knew the answers, but not enough time to look up the responses.
They field tested it, made sure that the time limit was reasonable (and removed some questions that were just too tough for the 3rd year course) and made an exam bank that was 3x the size of the number of questions. The students got 2 attempts, and because the questions were randomized, it was fairly difficult for an individual student to get the same test twice.
They found that students would print (to pdf) the questions and review them and pass them around, so they ALSO made the test/retest window quite small for the entire class. It was well received, used for several years with about a 30-40% question rotation/replacement every year, and scrapped when the professor left on an extended leave/sabbatical - no one else in the department felt it was a good use of their time to adapt their materials to the system - they all preferred to stick with standard university "test systems". What a joke. I've since adapted the system for myself and lost touch with the prof who set it up.
Quartz Extreme and Core Image. Are there any other real reasons to spend all that money on generic hardware?
...this answer. Or (D) let the students make up their own cards, like many posters are saying.
From the sound of your question, you don't care about students having all the formulas, etc., which is good. Many (most?) of my exams have been some combination of these suggestions, or open book. I think my undergrad university might've had a generic physics equation sheet, because I tended to get sheets with equations that I didn't even recognize. At least one of my exams included several pages of common VAX assembly instructions.
My more recent degrees have either had open book exams (those were the toughest) or "make your own notes". Usually restricted to how much you could bring into class.
The toughest exam I've had in the last four years (I'm halfway into my second graduate degree right now) was open book, open notes, you could look up your notes on your computer, etc. I even think you were allowed to get onto the internet. But the questions required sufficient understanding of the processes that if I'd tried to answer it by searching online I'd have run out of time before I was halfway through.
That's no different than one person doing their homework and letting their friends copy it.
Hmm, that sounds just like one person uploading an MP3 and letting everyone else copy it. Which, as we all know, is hallowed here on /. So, who cares?
Anti-Cheat #1: If you are using university computers to do the exams, you can always use a key-logger/screen-logger - so you know exactly what information they search for. Though open information and privacy are huge concerns in every day life, try to find a way to explain the need of Internet privacy during an open Internet exam. You must of course, pre-warn them that if they access their own email or any site with passwords, that snapshots will be recorded, as will be their passwords, etc.
Anti-Cheat Backup: This should go with #1, but used separately if #1 proves too difficult or if there is somehow a privacy issue even during an exam (oh, i just thought of one - my fingers slipped 104 times in a row and sent an email to a friend with the exact question i needed answering and got a response quickly was pure coincidence and I couldn't possibly not write down that answer since that was what I was going to write anyway!). Anyway, the back-up: don't install one, but SAY it's installed (be sure to go on about the name of the software, developed by X-high-tech-company-just-for-universities, what kind of new technology that has gone into them, how it's code is updated quarterly, and be sure to emphasize that every single keystroke makes a difference and can be used as official evidence for expulsion in the case of cheating).
Ways to Cheat: Photo cryptography may go undetected in some situations, though it's difficult when you have a key-logger present in the case of two-way transmission. One way cryptography is possible, though also difficult (If the list of answers to all questions exist in photos somewhere good for them.) (two-way is not practical -> the student can't send without typing keys - unless he uses a Ceasar cipher or something that is relatively quickly broken. Any other cipher for that matter, though this is already a huge red-flag in the first place if they aren't typing an accepted language or are using obvious code wording (What does the Gangster mean daddy when he says "The duck is about to lose its feathers..."?) Another crack, if someone uses something like a wiretap (leaving a phone on or other device) so the listener can hear the question and post a page. student could write every question on the screen as well while another 'listener' is watching (which is relatively easy if there's a pre-agreed upon site or by listening in on the phone), post the site relatively hidden on another pre-agreed upon website after x-minutes - and do this in of a pre-agreed upon cipher (such as the first word of each sentence is the solution (or more obscure than this if they are hard-working enough)). I think that's a good start - and unless I'm missing something obvious, it is about as tough to cheat, if not tougher, than current exams (university computers are relatively secure, and no phones allowed during exams - oh i guess hidden phone may be another one, but that's general cheating related, not open Internet test). Someone may find another way, at which time that knowledge can be used to thwart future use of that method, or discontinue it's use if it's a devastating one.
-Ultimate Stickman Game Developer Infinite World Puzzler
Put the answers out there on the internet. But, purposely muckup the applied theories on them as well as the answers in small ways that students would only know if they actually learned the material and even do the simple stuff like forgetting to carry the negative. The students who follow the cheat test will all end up with basically the same answers as the cheat test or pretty damn close to it and they will learn not to trust google. Then, you basically give them three options, 1. retake new test right now and whatever grade you get is where your grade is resetting to today(if its higher they have learned something, if its lower, not your problem), 2. drop all classes for the semester, 3. you report them for cheating and leaving school.
The biggest way to prevent cheating on open book tests where the internet is involved is to time the test. Keep track of usual sources of test answers. If you have a proxy system in place, use it to block the usual test data access spots. You can't get them all but you'll make it harder for it to happen.
I was a secondary school teacher for many years. We used to allow open book tests. The students always found those to be the hardest. Many students wouldn't bother to study, since they could look up the answers. But looking up answers takes time, and exams are time-limited. I suspect that you will find exactly the same thing in your classes. Let them look up whatever they want, but structure the exam so that the questions are problems that require understanding of principles, not just recitation of facts, and you should be just fine.
Aah, the flaw of old style knowledge.
Supposing there is no problem that can't be solved by some juicy resources.
(Insert complicated test question.)
(Correct but suspicious answer comes back.)
"Congratulations! You have earned your B. S. in Business Management!"
"But I am an Engineer!"
"No, you clearly don't know the class material. But you have already set up a consulting business. Therefore I automatically forwarded your exam to the Business School!"
Problem Solved : )
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
Perhaps you could video-capture their screens and have your TA's review the footage?
Respondus Lockdown Browser will 'lock' the student's screen so that s/he can't go anywhere else but that test page. It doesn't minimize, alt/tab, or allow any access to new tabs/windows etc. Of course, that pre-supposes that the student doesn't have a cell or some other device to find answers with, even if it's just to text ChaCha with LOL
You don't have them use the Internet. At all. You ensure your engineers are trained to be able to still use their knowledge when the Internet goes down.
Cranky educator.
Well, if they're taking the test on school computers would it be possible to monitor which sites they visit while taking the test? Like keep a log? You won't catch them before they cheat but you'll know that they cheated afterwards.
Unfortunately, you failed to expand upon what sort of cheating, but this solution should work for most.
The same activity trackers used by companies and parents. Add a screen video capture. Add webcam capture of the student.
A program to locate potential cheats in the set of files should be fairly easily written..
If the program looks for cut & paste operations & duplicated text--as well as an SE scan to see if there is a match online, looking for email activity (make email chart & such automatic failures illegal.) In the case of suspected cheating you have a log of keystrokes & the screen, and the accused which can be synchronized as evidence,
When introducing it, point out that it is a small number of cheats which cause the problem*, but by capturing this data those cheats can be caught, not graduate as completion in society. Such incompetents out in the world, not only reflects poorly on the school, but taints all future graduates.
The problem is that while most of the students are trustworthy, there's no way short of monitoring to identify the bad eggs.
Cellphone signals can be blocked &/or recorded fairly easily.
*most of the time , there have been numerous cases where entire classes were cheating.
OP, You strike a very important note regarding the deficiencies of exam based assessment, at least within the scientific / mathematical disciplines. Yes, in the real world you can always look stuff up. I always considered the skill of finding a relevant piece of knowledge and putting it into practice in the field to be far more important than any amount of memorised formulae and example cases.
Once outside the student world, there is no concept of "cheating" to improve your performance. We draw the line at falsely claiming something is your own work, enforced through copyright / patenting, but other than that you are free to use every tool / option at your disposal to achieve your aims.
When trying to project the results of an experiment into a different environment you need to know under what conditions the experimental data remains valid. One pretty reasonable way of ensuring the validity when moving between the experiment locale and the target is to make the environmental conditions of the experiment as identical to the target's conditions as possible (eg. the surest way to know if a building you design will stay standing in an earthquake, is to build the entire thing and put it through a real earthquake, perhaps not quite as useful for construction as for education but you get the idea).
So, why re-introduce any artificial restrictions for the purpose of assessment? If the assessment is then incapable of distinguishing between people in a useful sense then you need better assessment criterion, not better exam conditions.
Set up or request a closed intranet. As in, no access to the outside internet. Perhaps have saved versions of the resources they might need. I know that digital versions of encyclopedias and textbooks do exist.
What you are asking for is very difficult to provide, even if you had the perfect AI-based proxy server of every librarian's dreams: the person simply is honest and does not cheat or access prohibited content. I have taken take-home tests where I explicitly avoided looking up the solution, but the Internet is crafty, and I eventually ended up reading a variant of the problem description that subtly provided me with hints without my knowing. When I reflected on my thinking, I realized that it was severely effected by what I found on the Internet and that some of the leaps I should have made on my own were actually provided to me. Was I cheating? Perhaps, but not on purpose. The line between 'related' and 'solution' is very blurry, and even humans have a hard time distinguishing, let alone some automated arbiter or policy.
--"You are your own God"--
I teach at a university, my course is about network protocols and IT security. I prefer to trust my students rather than use punishment as a way to influence them. My attempts to eliminate cheating are quite effective, because the results of the exams are always within my expectations, i.e. a mediocre student never got an A out of the blue.
Here's a review of my methods:
- The final grade is derived mostly from the practical assignments they get throughout the semester. In this context I get to talk to each of them and spend a lot of time interacting with everyone in my group; this is how I know what they know.
- The final grade is computed as 60% = practical assignments and 20+20% = midterm and final exam. This way, even if you cheat at the exam, it won't help you very much, unless you also worked hard during the _entire_ semester.
- Formulate questions that don't take answers that can be copy/pasted from a book, the lecture notes or the Internet. Any question must require analysis. One who thought about it in the past will easily deal with it, one who has never been exposed to the ideas of the course won't be able to construct a good answer in a reasonable amount of time.
- Give them more time than they need, to ensure that time is not a bottleneck of their performance.
When I mentioned social engineering, I relied on research by Daniel Ariely. You can influence people's behaviour in multiple ways:
- a written commitment not to cheat
- give them a moral problem to think of, before giving them the exam itself
- adjust the environment (in your case, tell them that all the Internet traffic is logged - so they know that they _can_ get caught)
For example, I used these tasks in the previous semesters:
- "write as many of the 10 commandments as you can remember" (taken "as is" from Ariely's experiment)
- "actually, there were 11 commandments, but one of them was lost. Think about it and write down a rule which is worthy of being listed as the 11th commandment"
- I once tried a written commitment too. Everyone who was in class signed it and smiled: https://www.facebook.com/photo.php?fbid=469536753019&set=a.453850808019.243204.739418019&type=3&theater Three years after that exam, people are still talking about it and are proud to be a part of that experience.
You may be interested in:
- "Predictably irrational" and "The upside of irrationality" by Daniel Ariely
- http://duke.edu/~dandan/Papers/BadApples.pdf - here's an example of a paper he wrote about cheating, there are other ones too.
You must also make sure the students care about the course and want to learn, rather than just get a passing grade. Have a look at my notes of a book about this, "Punished by rewards" by Alfie Kohn: https://www.facebook.com/media/set/?set=a.10150475760123020.375546.739418019&type=3&l=70e1f3712e
I tried to ensure my assignments are not only useful, but also interesting and fun to play with. A basic requirement is to make sure some humour is always involved, with some references to Futurama or Monty Python or some sci-fi book or movie. Here are some examples:
http://info.railean.net/index.php?title=Lab2_-_HTTP_crawler
http://info.railean.net/index.php?title=Lab1_-_simple_client/server_application
At the moment I'm in the process of devising a very short code of ethics (if it is long, no one reads it). You can read the draft: https://docs.google.com/document/pub?id=115bLhvMUisnw
The saddest poem
The only thing that can bind a man is their own actions.
So why try stopping them?
Instead copy each and every packet. Then do forensics on the packets. Those who cheat run the risk of being caught.
Either declre encryption to be verboden OR teach the valuable lesson that unless you have some control over the keys - you are subject to man in the middle and bust those people also.
One should not be permitting the use of the internet and other common approaches to how someone who is experienced and working solves problems when dealing with students. The whole point of being a student is to learn the fundamentals to prepare them for solving actual problems found in the work place. It's not just a matter of potential cheating but rather, how do you tell if what you've found on the internet is accurate or correct or even applicable?
You know, if something like SOPA ever does pass, perhaps you could at least make it work to your advantage... just copyright your questions and when one of your students posts them on a web site somewhere, sue them and garnish their wages for the next bijillion years. Or at least tell them that's what you'll do and that you have friends in the MPAA. :-)