Slashdot Mirror
IT Literacy Test
crumley writes "The Educational Testing Service just announced a new
test that is designed to measure information technology literacy. The test is supposed to measure the ability of students to use software to solve problems, and not just how to use particular programs. So has anyone out there taken a test like this? Did it seem to measure critical thinking and problem solving skills?"
... but a test like this seems long overdue. I can't tell you how irritated I get when some new snotnose paper-MCSE comes strollin into my office thinking they know everything. If the test is accurate, fair, and relevant, I might consider it as part of the candidate screening process when hiring new IT workers.
bash: rtfm: command not found
What a useless thing to test. If you think logically and can break problems down, anything in operating computers simply comes down to Googling, reading, and thinking logically. This is about as useless as just a plain old IQ test, SAT, or any other standardized "bubble" test in assessing future work/educational performance. In fact, I bet an IQ test would be just as effective in this situation. My guess is that it is simply knowledge based, not action based (wasn't willing to drudge through ETS's corporate "Yeah us!" language). ETS should take a hint from Cisco. Their tests are difficult and actually ("GASP!") test performance in real world situations in solving real world examples and problems using real Cisco gear.
I'm the network admin for a school. I've been doing this for seven years and have been teaching computer classes for five. I teach the 7th grade how to do simple programming in LOGO. They learn the concepts of loops, variables, functions, etc. They learn how to take a problem, break it into parts and come up with a program to solve the problem. They also learn a bit of computer history and how to count and do simple math in binary. I believe I'm the only one doing this in my area.
I deal with a number of people in my position in other schools. Without fail, the computer "literacy" classes in those schools is training in Microsoft Office. They're just training kids to use a particular version of a particular product from ONE company. They're not teaching them the concepts behind a modern word processor, they're training them how to click buttons in Word.
When I started this job, I thought education was all about teaching people how to think and solve problems. I was wrong!
I'm fortunate at this job in that I'm pretty much free to use whatever solutions get the job done. 80% of the machines here are Linux based terminals (using LTSP). I'm also fortunate that I won the old teaching concepts vs. training argument with the administration. I'm free to teach the computer literacy class however I wish.
But you know what the best practices are (or were at the time they became codified in a test). If you are mindful of what the standard way is, you can at least choose it when there is no reason not to do it that way.
Also, when the time comes to make product recommendations, you can say (for example), "Well, the industry standard is that your offline backup solution media should have at least the capacity as your online storage." (I made that up.)
People like to follow standards, but in this case if they chose a cheapo backup solution you'd have made them decide to go against the standard to do it. Never underestimate the value of C'ing Your A.
That's not exactly what I wanted to say, but you can take it from there.
sigs, as if you care.
I disagree. My college Discrete Structures course (data structures, logic, recursion, topics like that) was taught in Logo. Logo is a list-based language, and actually comes across as somewhat of a "LISP Lite" when you start using it for this stuff. After the first day of screwing around, we never used the "Turtle" part of it again -- and it was FANTASTIC for breaking the C/C++/Java mode of thinking, so we could focus on studying the algorithms and the "real" take-away knowledge.
You can certainly break LOGO programs into manageable chunks with procedures, conditionals, and the like... You just have to make yourself comfortable with the "list" aspect of programming.
Not a joke: I still have nightmares of writing a program to parse an arbitrary Boolean expression and print out a truth table... in Logo...
seven two six five
seven four six one seven
two six four two e
I found logo to be an excellent introduction to procedural programming after having my mind poisoned with basic. I just wish that someone had tried to teach me data structures when I was young and impressionable. That's what I really need to learn.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
It's easier to test for incompetency than for competency.
A company I worked for got so many liars applying for jobs that we made these rediculous little tests to give people. Here's a sample question from our C test:
Or for electrical engineers:
I thought these tests were a waste of time. I think I said something like, "If someone is breathing they'll pass it." Then I saw how many people who claimed "expert" on their resume failed the liar's test. Weeding out the liars left us with a much smaller pool of candidates.
People I know who score higher on ETS tests tend to be smarter than people I know who score lower on ETS tests.
Or they cheat better.
Or they have money to spend on preparation classes.
Or they're white.
Either way, the ETS tests can very actively tell admissions counselors which students are the the wealthiest white cheaters who are not totally stupid.
paintball
You ask: if you get 100 applicants for 10 positions, then why not use tests...?
The answer is simply that you are making assumptions about what the test is measuring. Maybe the test is filtering out the best people for the job, not the worst? There is only one way to know, and that is to validate the metric, test, against the goal, selecting the best candidates from a larger group.
To the best of my knowledge, no one has done this groundwork. Therefore, you can not know what applying the metric will actually accomplish. You have a hypothesis that it selects the best candidates from a larger group. But, as any researcher in the social sciences will tell you, doing the study to validate this hypothesis, and thus the metric, frequently yields surprising results.
While it is easy to argue that people are very subjective, and that they apply criteria other than those desired, in reality these are frequently exactly the insights necessary to identify that superior individual from the crowd. There are things one can do to protect against overly subjective evaluation by people during interviews. There is a long history of experience in this area, and for the most part it is successful.
Testing has a far shorter track record than the personal interview, and thus requires MORE care and checks rather than fewer checks. Since each test is a new metric, testing actually also requires more work to establish its validity than personal interviewing. The saddest bit is that most people not only do not perform the necessary work to validate a test's validity, they rarely even understand it's need.
The best problem solving test I ever took was while applying for a job as a student tech. The boss put me in front of a computer with WordPerfect running on it. He said "change the background of the entire page to another color." and watched what I did for a while.
What made this an interesting problem was, despite being able to go to the menu and look at, I forget exactly, but Format -> Page or something, and having some options there for changing some things, background color was not an option. So if you are me you dig around on the menu a while.
Then you finally give up and pull up the help and search. Then you find the entry that says: To change the background color, go to Format -> Page and click on the background tab. Then select a new color. You say wtf (quietly) and go back to Format -> Page. There is no background tab. You go back to the help. Yep, that's what they told you to do.
Turns out, by default, some checkbox on the first tab of the page properties dialog disables the tab you need to get to to change the background. Some seemingly completely unrelated checkbox. I got lucky, clicked around and found it. Then changed the background color.
Turns out, I was the only person to have ever successfully completed the task. The purpose of the test was not to see if you could solve the problem. The purpose was to give an unsolvable problem, and watch the proccess you use to try and solve it. I thought that was a much better way to test skills.
Lucky me, I figured out the solution, which so impressed them that I was hired, despite them having already picked someone else for the job (they just hired both of us instead).
Try it yourself, I believe it was whatever version of WordPerfect was out in '98.
Got Apathy?