Preventing Networked Gizmo Use During Exams?

bcrowell writes "I'm a college physics professor. My students all want to use calculators during exams, and some of them whose native language isn't English also want to use electronic dictionaries. I had a Korean student who was upset and dropped the course when I told her she couldn't use her iPod during an exam — she said she used it as a dictionary. It gets tough for me to distinguish networked devices (iPhone? iTouch?) from non-networked ones (calculator? electronic dictionary? iPod?). I give open-notes exams, so it's not memory that's an issue, it's networking. Currently our classrooms have poor wireless receptivity (no Wi-Fi, possible cell, depending on your carrier), but as of spring 2011 we will have Wi-Fi everywhere. What's the best way to handle this? I'd prefer not to make them all buy the same overpriced graphing calculator. I'm thinking of buying 30 el-cheapo four-function calculators out of my pocket, but I'm afraid that less-adaptable students will be unable to handle the switch from the calculator they know to an unfamiliar (but simpler) one."

Comments

Open Notes & Well-Designed Exams

[slifox]

First off -- I applaud your use of open-note exams. That is the ONLY real-world way to learn and demonstrate knowledge. There is almost never a situation in the professional world where one must solve a problem with absolutely no references (and it would be stupid to do so on a production system -- when solving a critical problem, why risk everything based on what you *think* is right, when you can verify against documentation; at least if something breaks, you can point to the incorrect docs...)

Some people can simply memorize anything they look at, while others struggle at this. A proper exam should be designed to test one's ability to demonstrate processes: exams should give you all the information you need, but the questions should be designed such that only someone who has invested prior effort in practice and learning will be able to solve the questions in the allotted time.

For less-concrete subjects such as the arts, I'm not so sure how this can be accomplished. However this is a trivial design decision for exams in maths, sciences, programming, and engineering.

Furthermore, I think any physics or math exam that requires a complex calculator really has a wrong approach. Assuming everyone at this level has already demonstrated their ability to perform arithmetic several times over, the calculator should only be there to free them from making mistakes on the menial number crunching (addition, subtraction, multiplication, division, squares, squareroots, proper value of e,Pi, etc...). The exam should test for core concepts: ideas where you simply must understand the knowledge through prior practice and learning.

Sadly, I think many professors fall back on rote-memorization exams just because they can't be bothered to design proper exams each semester. These types often teach straight from the textbook-provided lesson plans, and then wonder why students cheat...

But honestly -- an exam is but one facet of demonstrating proficiency in a subject. Personally, I think projects & labs the best way: sure one can cheat, but it's easy to determine who has spent time polishing a proper unique lab report. In this respect, open-ended projects are the best, as the room for creativity limits the possibility for undetectable cheating, and lets the students show their enthusiasm for the subject. If you're really worried about cheating, a lab-practical may even be a legitimate tool: it's pretty damn hard to make stuff up as you go while you've got a one-person audience of the professor.

Short answer: let them use basic scientific calculators, the textbook, their notes, and a dictionary; design your tests so that students have all the resources they need, but don't have enough time to learn-as-they-go during the exam.

"Never memorize something that you can look up." --Albert Einstein

Re:Open Notes & Well-Designed Exams

[jdong]

The design of the exam is not the problem -- if students have networking access to someone inside the exam room (or worse, outside the exam room), no matter how hard you make the exam, you are testing the brainpower of their lifelines, not them. This the 2010 version of "how do I prevent students from whispering to each other during a test", for which there is no straightforward solution short of "no electronics in the exam room".

Re:Open Notes & Well-Designed Exams

[Anonymous Coward]

if students have networking access to someone inside the exam room (or worse, outside the exam room), no matter how hard you make the exam, you are testing the brainpower of their lifelines, not them.

This.

I didn't come here to troll, but I feel obliged to point out that the grandparent poster lost himself in a forest of his own self-righteousness regarding assessment design. The problem under discussion is that two students in the room working on the same test at the same time could have IM ability through wireless networking, and so one could do the work while passing the answers wirelessly to the other, who could simply copy what he sees on IM.

Parent is almost right: there is no solution short of "no unapproved electronics." Students with poor English abilities who need dictionaries should bring their dictionaries to office hours for inspection; the invigilating TA's (if this is a large lecture) could be notified on the sign-in roster (I'm assuming a sign-in procedure for a large lecture) or be introduced personally to the student in question to be sure they recognize them. The policy would have to be on the syllabus and announced in class the first day. It's not a simple solution, but it's probably more fair to students not fluent in English than denying all electronics.

Of course, the administration could also try requiring that all course instruction and assessment (outside classical/modern languages in Arts & Sciences) be conducted in English, and that all students entering pass an unaided (no lexicon!) English proficiency exam that's not a total joke. I mean, imagine a student going to France and expecting accommodations because he doesn't know French... Of course, administrations in the US will never enforce a convenient language policy, so you're going to have to make allowances for electronic devices like dictionaries if you ban electronics from testing sites.

Re:Open Notes & Well-Designed Exams

[apoc.famine]

The GRE and MCAT aren't in the professional world. While your point is well taken, it's a bit misdirected as those are just gatekeepers to parts of the professional world. Their widespread use doesn't make them good or useful gatekeepers either. They're a way for lazy people to do a half-assed job to cheaply and poorly assess a lot of people in the least amount of time. Nothing more.

Re:Open Notes & Well-Designed Exams

[Iron Condor]

so you're going to have to make allowances for electronic devices like dictionaries if you ban electronics from testing sites.

I heard they have a totally newfangled dictionary now, that works entirely without electronics. It consists of lots of sheets of paper, with the words printed on it. Like as if you made a big printout of a site-rip of m-w.com.

Re:Open Notes & Well-Designed Exams

[arth1]

You seem to be confusing training with ability to solve problems.

Training can make a doctor better at handling the familiar, but won't help when the doctor is faced with the unfamiliar. In some cases training can even be harmful -- if you have extensive training in recognizing a particular condition, you are biased towards it, and are likely to score more false positives for the diagnosis than you otherwise would. The typical scenario is the young idealistic doctor who takes a course in [moderately rare disease], and then sees that disease behind every bush for the next few months.

An exam should measure how a student does when faced with problems they can neither memorize nor train for, but where they have to show enough understanding of what's been taught that they can apply their knowledge without having been shown exactly how.

10 years ago

[porkThreeWays]

What the hell did these students do 10 years ago? AFAIK two semesters of English and perhaps 1 semester of literature are the norm at every reputable college in the U.S. If their English is too poor for your physics exam, they probably have no hope of graduating.

thin client exam takers

[pikine]

I think he's worried that you could IM a friend during an exam to work the answers out for you, as if you're a thin client, with all that computing power over in the cloud.

SLIDE RULE

[Steve_au]

two words - SLIDE RULE

cultures AND pressures

[Lord Ender]

First remember that foreign students pay FAR more than we do to go to US schools. Compound that with the fact that many come from poor countries. The pressure to succeed is EXTREME. Furthermore, not all cultures despise cheating as much as Western culture. The results are predictable.

Personal anecdote: I was invited to the Indian CS students' "study session" once while on a group project. I was AMAZED. They had a library of homework and test questions and answers. They passed them around casually. They also begged me for graded solutions from my previous courses to add to their collection. They were all cheating their way through and thought it was normal.

They also kept asking me how I could come up with working algorithms to programming assignments on my own (without copying from something). It was as if actually being able to program was wizardry to them. I wonder why.......

Take a page from the ETS.

[wickerprints]

Look toward standardized testing practices for how to conduct tests in a rigorous and fair manner. Quite simply, the rules and expectations for the course should be clearly stated at the outset. Don't wait until the exams come around to drop the bomb. Tell them that you expect them to use a calculator that is on an approved list. No other electronic devices will be permitted. All other possessions not explicitly allowed must be placed at the front of the room, and any mobile devices must be turned OFF. No "vibrate." Watches are permitted but cannot have an alarm function. If they need translation, that's too bad; the ETS does not offer to administer mathematics tests in the language of the examinee's choosing. This is a college level course, with lectures in English. You don't provide lecture notes in twenty languages. It is the student's responsibility to become sufficiently proficient in the English language in order to continue their studies. That may put them at a disadvantage, but we don't try to equalize the playing field for someone who hasn't learned calculus.

Education necessarily requires that some students have to work harder--sometimes, much harder--than others to achieve the same proficiency level as others. That is not being unfair, that is just the way life is.

Why do the complicated expensive solution?

[sjbe]

You can make WiFi unusable, however.

Technically possible but not practical for economic reasons.

Or you could alter the classroom so RF cannot enter through the walls or ceiling.

VERY expensive. Colleges don't really have the funds to justify that, especially when just banning the offending devices is free.

I suppose convincing the university to alter the classroom in this manner could be difficult, but they could also see the value in having some exam rooms that are essentially faraday cages

Why not just take the figurative bullets out of the gun (no networked devices allowed) instead of building an expensive figurative bullet proof vest. If they don't need the networked device for the test, there is no reason to allow it in the room in the first place.

Re:Well not sure if this is the right approach but

[Mike Kristopeit]

perhaps you haven't heard of ad hoc networks.... even inside a faraday cage, as long as one student is willing to help others cheat, all students are capable of cheating.

Unnecessary complication...

[BrokenHalo]

You're all looking at the problem the wrong way around. It should be simple enough to design a test that doesn't need a calculator. The devices are great if you need numeric "answers" to a given problem, but in any kind of assessment there's no need for this. The students can leave their result as an expression, which is actually more meaningful in that it makes it clearer to the examiner as to how the student arrived at it.

The student shouldn't need to show that he can substitute values in an expression to arrive at a numeric answer. Any idiot can push a few buttons to do that, so it's just wasting time.