Slashdot Mirror
Brain Scan Can Predict Math Mistakes
itwbennett writes "Computer Science Ph.D. candidate Federico Cirett says that he can predict with 80 percent accuracy when someone is about to make a mistake on a math question. Using an EEG machine, Cirett can identify the patterns in a volunteer's thinking that are likely to result in an error 20 seconds or so before it's made. 'If we can detect when they are going to fail, maybe we can change the text or switch the question to give them another one at a different level of difficulty, but also to keep them engaged,' Cirett said. 'Brain wave data is the nearest thing we have to really know when the students are having problems.' He will present a paper on his findings at the User Modeling, Adaptation and Personalization conference in July."
The first thing I can think of to do with this is figure out how to trigger it and then proceed to get the problem correct, just to screw with everyone.
William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
A patronizing system that tells you that you've already failed before you've actually done so that gives you amateurish problems so it can see you succeed.
I hope it comes with a robot arm that tears the test paper out from under your pen, pats you on the head and give you a first grade replacement problem. Bonus for cheering with a nonenthusiastic voice whenever you pass a problem.
I am going to guess that Federico Cirett didn't go to UF.
Fascinating research, but I am not a fan of his suggested application. The last thing I want as a test taker is to have a computer dumb down the test (with presumed accompanying grade reduction) to help me relax and feel good about myself.
'Brain wave data is the nearest thing we have to really know when the students are having problems.
Most people have been raised with the notion that it's more important to appear competent than be competent. There's several college-themed cartoons out there about that express hatred for "The Question Guy"... and most people are acutely aware that asking questions on material is a great way to earn the irritation and ostracism of your peers, who feel they have better things to do than get an education and really just want to go through the motions and get out.
This is another technology that's trying to solve a social problem, and like every attempt in that regard, it will fail, be impractical, and people will try to defeat it -- because they don't see the point and they don't want to appear incompetent. In 20 years, we'll be getting coached on how to have the right brain wave patterns for getting through the airport unmolested, how to cheat on your final and not get detected by the brain wave readers, etc.
The problem is in our social values and attitudes. It's systemic and institutional. No technology can fix that, however advanced.
#fuckbeta #iamslashdot #dicemustdie
There should be a system like that for posting on the internet... "Error Code 427 - there is a 80% probability you're posting bullshit. Your post was discarded."
I wish I had something like this to wear on a wrist to check my thinking.
Years ago, back in the dark ages (80s), I was taking a thermo exam. We were given data and we had to derive an equation from said data. Anyway, after pondering it, deriving the equation, checking it once, checking it twice and seeing that it was nice, I turned in my exam.
'D' on the Final
Why?
Forgot to divide by '2' and that screwed up everything else. That ended any dreams of a science or engineering career - thermo was absolutely required and it had to be a 'C' or better.
I went to 'B' school instead, became a programmer (only job I could get. The bond traders wanted nothing to do with me.), and now I'm a long term unemployed loser.
So, what's the moral of my story?
I don't have a fucking clue. And I guess I failed at story telling too.
Wait here's something:
Kids, learn to concentrate. Learn to give 100% of your attention to the present moment. Ignore folks who want "multitaskers" and ignore the media that insists on dividing your attention - pretty much anything electronic. Video games? Not from what I've seen. Yeah it requires attention, but it does so with a lot of variation.
Anyway, never mind. I'm a loser.
Carry on.
Some testing system, like for the CPA license (in California at least) already do this. the computer system adjusts the difficulty of certain questions based on how you're doing so far. How exactly it does this is proprietary information and it doesn't dumb things down too much, but it can also make the test harder if you are doing really well. Then something magic happens inside the computer and it tells you whether you passed.
This seems like a silly application for such research though. Who is going to want to have to have electrodes hooked up to their head just to take a test? It's already stressful enough without having more stuff to distract you.
This could be useful for programmers. It may be possible to detect some programming errors while programming.
That's nothing, I had an ex-girlfriend who could predict with 100% accuracy when I was going to say or do something stupid, usually in response to her being upset.
More likely, the brain knows that it HASN'T GOT A CLUE about the answer and works harder to find one. It goes out of its normal operation to find memories, skills, techniques that it could use to perform the operation and get an answer and thus activates areas of the brain not normally activated for a question you DO know how to answer.
These idiots then suggest we should take the problem away from them at that point and feed them an easier/different question.
The brain has lots of subconscious thoughts but knowing the answer to a high-level abstract question when the conscious part doesn't is incredibly unlikely.
They are not detecting wrong answers or internal bickering between the conscious and sub-conscious. What they are detecting is confusion.
I bet you could do better than 80% just by looking at their faces.
I view this as research into how to better teach mathematics, or really how to better teach any intellectually challenging subject. I don't think they are hoping to hook up every test-taker to this thing, but rather trying to understand how the brain picks apart challenging problems. I feel such research is very useful.
At least some of the MCSE-related exams do as well, though they're adaptive in a different manner -- if you miss a question on one subject area, it asks more (harder) questions on that subject to determine if it was just a tiny mistake or if your knowledge on that subject is actually lacking. The drawback to the format is you can't go back and revise your answers before time is up since, if you could, you could pay close attention to the questions being asked and go "Oh, I must have picked the wrong answer to this one, let's try this other answer."
The net result is a much shorter test than when I took the NT Server 4.0 exam (70-067) way back when (a few months before they changed to the adaptive format.)
(Disclaimer: I never actually did anything with my (now defunct) certification, I just had a high-school level class (as a pilot program) that actually taught it and included a trip to take the actual test.)
Of course nobody reads the FAQ! If people read the FAQ, the Questions wouldn't be so Frequently Asked.
I find your stereotype of educators incorrect. This study is attempting to figure out how students learn and solve problems. Such information is useful to educators. So in your words... If a student is having difficulty solving a math problem, we identify what deficiency is holding them back, then give them a simpler math problem that remedies the deficiency. Mathematics is highly structured, and I find that many times students need to go back and practice a prior topic before attempting the current exercise.
I'm going to guess most people complaining didn't RTFA. Changing math problems in the middle of a test was an offhand comment in the last paragraph, discussing possible applications for his current research. The current research being understanding how the brain works.
It is called computer-adaptive testing. I can't speak to the CA CPA exam, but the algorithm is usually not secret. Questions are categorized as easy, medium, and hard, some tests may have more categories. Your first question is of medium difficulty. When you answer a question correctly, your next question is harder. If you get a question wrong, you are given the next easier question. You get more points for correctly answering a hard questions than an easy ones. The test taker does not know the difficulty of the current question, and you are never permitted to return to a question once it is submitted.
This is how the GRE was run when I took it. I recall that you could request a statement of how each question was scored, but it was missing the question and the choices, so it would be of little diagnostic value to most people.
I know I would not want to submit to a test that scores the question before you actually respond. I suppose it is a fun research topic, but I don't see a practical application for the work. Maybe you could add it to a game show like Who Wants to be a Millionare. There would be no need for Regis to ask "is that your final answer?"
Forget nit picking over optimizing testing strategies. The real news is you can predict where someone is likely to make an error. Imagine recording all this data while someone was writing code. Eventually there is a bug detected but where is it? Well you might want to color code the code to show sections where the person was struggling with logic. That might be a place to look first. Of course it might be in some place where it never occurred to the person they should be struggling or was just a typo. But at least automatically flagging every place where the programmer was unsure of things would help with code reviews or code refactoring.
Some drink at the fountain of knowledge. Others just gargle.
FYI, CAT (Computer Adaptive Testing) is *not* proprietary. There a lot of papers out there about how to do adaptive testing and how to do it well. That said, all of these systems, as an earlier respondent noted, are based upon actual responses rather than predicted responses. As a professional in assessment, I would not want to base any decisions about item presentation on 80% accuracy. We assess because there is uncertainty and we need evidence to model and demonstrate our best estimate of whatever it is we are measuring. The trouble with adapting before you have evidence is that you never push a examinee to their extremes. You've already artificially constrained the range of difficulties and items that a student will see. Restriction of range is already a huge problem on existing tests because of people's preconceptions of what's appropriate for certain ages or groups of examinees. It's promising technology and I intend on watching how it evolves.