Slashdot Mirror
IT Literacy Test
crumley writes "The Educational Testing Service just announced a new
test that is designed to measure information technology literacy. The test is supposed to measure the ability of students to use software to solve problems, and not just how to use particular programs. So has anyone out there taken a test like this? Did it seem to measure critical thinking and problem solving skills?"
one wonders how in two hours one can truly test someone's compentency when most projects requires at least 2hours to determine the source of the problem
I make my face look like this and concerned words come out.
While this is a good measurement of progress for our current interface and will be useful to measure the average worker. I expect it to be hopelessly interface dependent. There is more than one good and useful GUI or TUI (is there another name for a text only UI?)
Any test will necessarily miss relevant skills, and will over- or under-emphasize certain things. If you can be a paper-MCSE, there's no particular reason to believe that you can't be a paper-"IT literate".
The idea of this test is fundamentally flawed. If you are good at problem solving, you will be good at problem solving in a technology-rich environment or a technology-poor environment equally. Analytical thought and problem solving has been around a lot longer than computers, and the same people that are good at solving problems with computers were good at solving problems with other things before computers.
I've always hated technology tests, because they necessarily favor a particular way of doing things, and show a clear bias to the solutions preferred by whoever came up with the test. My techniques are rarely the "industry standard" techniques, so I often find myself on tech assessment tests choosing the answers I know the test creators think are right, even if in my experience they aren't the best or most efficient way to do things.
Many are familiar with the often frustratingly cryptic interview questions from Microsoft and others... my favorite (which I would certainly use if I were running a support department):
"You come home and flip the switch, but the lights don't come on. Describe in as much detail as you can how you would go about fixing the problem."
Most people I've asked respond with "replace the light bulb." While that might be the problem, I think the first thing you should do is try the switch again. Maybe you didn't flip the switch all the way, or you hit the wrong one.
The point is that it's a problem solving experience - and not just in technology. It doesn't require expertise in Microsoft Word. It doesn't take a techie to know this stuff. You can measure a person's aptitude for logic, problem-solving, etc. without ever testing specific examples of those skills. That's what I really like about that question. Hopefully, the ETS exam will take this perspective when developing such tests.
It's hard to know much about this test until we see it. Tests in general are measurements based on a body of knowledge. Maybe it's good, maybe it's not. Damning tests because they seem unfair is silly. Measurements are referential. Some have bad reference points and therefore can't measure very well. Some can, some tests age and become useless, some are good in specific areas. Without the knowledge of the test questions and the criteria for them, comments will likely be specious.
---- Teach Peace. It's Cheaper Than War.
Yes, people are still teaching LOGO. What's wrong with LOGO?
7th graders (as are most people) are into instant gratification. Using LOGO, I can start with the simple drawing commands. They type a command in and see the result instantly on the screeen. I start off with them having to draw simple shapes command by command. I then show them how to use LOGO's repeat command.
The instant gratification aspect allows me to get them interested in computers and programming. The difference between 1982 and now is that few people in 1982 had computers and many of those that did learned to program them. Most people today see computers as nothing more than another appliance for doing email, shopping and IM and have no interest in learning how the machines work. Most schools certainly don't encourage anything more.
My techniques are rarely the "industry standard" techniques...
Man, when I was in college, we had 8 or 9 different "Industry Standards". While most teachers were absolutely convinced that their method was the "Industry Standard", there were a few knowledgable enough to explained the whole thing to us. Mostly when people talk about "Industry Standards", it's manager-speak for "The Way We Do Things Here." So if you don't follow the "Industry Standards", you will not be working for long.
Also keep in mind that "Industry Standards" in the sense that I'm talking about has absolutely nothing to do with real ISO or QS standards. Those are actual organizations that create a set of standard rules for companies to follow, usually for the safety of workers and quality assurance of products. No, I'm just talking BS manager-speak...
Our society has gotten far to hung up on testing as a silver bullet. Tests are just attempts to measure something, exactly as a ruler measures length or a scale measures weight. But tests such as the one in the article are, due to their nature, far less accurate or precise.
I see the introduction of yet another test as a poor substitute for one on one personal interviewing of the test subject. They are looking for a quick fix, one that is not people intensive when the fact is, people are best able to evaluate these complex abilities and skills, so long as they take sufficient time in doing so.
This is just like "No Child Left Behind". Instead of investing in the people, in that case teachers to work with students, a battery of standardized tests are introduced as a substitute. Yet, there is no validation of the testing against its objective, while we entrust our decisions to those very tests.
That is the biggest issue here as well. We are attempting to replace human judgement with supposedly objective testing, when it is precisely the complexity of that human judgement that is called for.
You are obviously an employee of ETS or severely disconnected with reality.
Your points mainly address the amount of effort that goes into these tests. A lot of useless effort is just that- useless effort. An the GRE and SAT and worse than useless, since they arbitrarily make some people appear much better problem-solvers than they are.
It is fairly common knowledge that the SAT and the GRE, the two other tests for which this organization is known, have little to no correlation with the skills that they supoosedly measure. You can google some pages about the debate, but it's pretty one-sided as almost everyone with an opinion is in favor of dropping support for ETS and their tests.
I am about to take the Physics GRE and my practice scores improved from mediocre to far above average in one week of no physics studying. The secret? Don't read the questions: only read the answers. Eliminate the ones that are clearly wrong and without solving the question, you just got the correct answer. I challenge anyone to explain to me how this anything to do with real physics problem skills.