Slashdot Mirror
IT Literacy Test
crumley writes "The Educational Testing Service just announced a new
test that is designed to measure information technology literacy. The test is supposed to measure the ability of students to use software to solve problems, and not just how to use particular programs. So has anyone out there taken a test like this? Did it seem to measure critical thinking and problem solving skills?"
one wonders how in two hours one can truly test someone's compentency when most projects requires at least 2hours to determine the source of the problem
I make my face look like this and concerned words come out.
While this is a good measurement of progress for our current interface and will be useful to measure the average worker. I expect it to be hopelessly interface dependent. There is more than one good and useful GUI or TUI (is there another name for a text only UI?)
Any test will necessarily miss relevant skills, and will over- or under-emphasize certain things. If you can be a paper-MCSE, there's no particular reason to believe that you can't be a paper-"IT literate".
The idea of this test is fundamentally flawed. If you are good at problem solving, you will be good at problem solving in a technology-rich environment or a technology-poor environment equally. Analytical thought and problem solving has been around a lot longer than computers, and the same people that are good at solving problems with computers were good at solving problems with other things before computers.
I've always hated technology tests, because they necessarily favor a particular way of doing things, and show a clear bias to the solutions preferred by whoever came up with the test. My techniques are rarely the "industry standard" techniques, so I often find myself on tech assessment tests choosing the answers I know the test creators think are right, even if in my experience they aren't the best or most efficient way to do things.
If the test is accurate, fair, and relevant
Get a grip. It will weed out the 'tards, nothing more.
Many are familiar with the often frustratingly cryptic interview questions from Microsoft and others... my favorite (which I would certainly use if I were running a support department):
"You come home and flip the switch, but the lights don't come on. Describe in as much detail as you can how you would go about fixing the problem."
Most people I've asked respond with "replace the light bulb." While that might be the problem, I think the first thing you should do is try the switch again. Maybe you didn't flip the switch all the way, or you hit the wrong one.
The point is that it's a problem solving experience - and not just in technology. It doesn't require expertise in Microsoft Word. It doesn't take a techie to know this stuff. You can measure a person's aptitude for logic, problem-solving, etc. without ever testing specific examples of those skills. That's what I really like about that question. Hopefully, the ETS exam will take this perspective when developing such tests.
Part of the problem with computing these days is the lack of basic education that the users have about computing in general. There are many people that believe that the CD-ROM drive is a cup holder, that Microsoft is everything (how many users have you seen say that "Microsoft isn't working?" Incredible), don't know anything about the dangers of the Internet and properly securing their computers (i.e., installing a firewall, protecting themselves from viruses and malware by using anti-virus and anti-malware tools, and using Firefox or an alternative to Internet Exploder), don't know about files and folders, or other basic movements with a computer, and faint at the sight of having to learn anything that is more complicated than moving a mouse around and clicking an icon (command lines are a great example). I would love to see a required high-school or college test that tests on all of the basic computing ideas and tasks, plus a little more. Unfortunately, many high schools and colleges have already implemented MS-centric "computer competency" requirements that test on nothing more than how to use MS Office and Windows; they test on specific applications, not about how to use computers per sé.
Am I saying that all of these people need to be tested on writing shell scripts, C programming, configuring ipfw/iptables, and compiling a kernel? Of course not! I'm just saying that I believe that all people using a computer need to be eduacted about the responsibilities and risks of having a computer, and all of the things needed to do in order to protect yourself. We all have to take driver's education and driver's training before we even step foot into a car, because we know the responsibilities and dangers of riding a car. When we get our cars, we have to learn how to maintain it, too. So, how come most users expect that their computers are magical boxes that don't need to be maintained or taken care of? The computer is a powerful multipurpose tool that can be beneficial when used correctly, but can also be a weapon (or zombie) if used incorrectly.
It's hard to know much about this test until we see it. Tests in general are measurements based on a body of knowledge. Maybe it's good, maybe it's not. Damning tests because they seem unfair is silly. Measurements are referential. Some have bad reference points and therefore can't measure very well. Some can, some tests age and become useless, some are good in specific areas. Without the knowledge of the test questions and the criteria for them, comments will likely be specious.
---- Teach Peace. It's Cheaper Than War.
Yes, people are still teaching LOGO. What's wrong with LOGO?
7th graders (as are most people) are into instant gratification. Using LOGO, I can start with the simple drawing commands. They type a command in and see the result instantly on the screeen. I start off with them having to draw simple shapes command by command. I then show them how to use LOGO's repeat command.
The instant gratification aspect allows me to get them interested in computers and programming. The difference between 1982 and now is that few people in 1982 had computers and many of those that did learned to program them. Most people today see computers as nothing more than another appliance for doing email, shopping and IM and have no interest in learning how the machines work. Most schools certainly don't encourage anything more.
My techniques are rarely the "industry standard" techniques...
Man, when I was in college, we had 8 or 9 different "Industry Standards". While most teachers were absolutely convinced that their method was the "Industry Standard", there were a few knowledgable enough to explained the whole thing to us. Mostly when people talk about "Industry Standards", it's manager-speak for "The Way We Do Things Here." So if you don't follow the "Industry Standards", you will not be working for long.
Also keep in mind that "Industry Standards" in the sense that I'm talking about has absolutely nothing to do with real ISO or QS standards. Those are actual organizations that create a set of standard rules for companies to follow, usually for the safety of workers and quality assurance of products. No, I'm just talking BS manager-speak...
Exactly. I have the same problem at my school, I was required to take a "computer competecy" test that was really a MS Office test. I wish that these schools taught that there is more to computing than MS Office. For example, with word processing, they could have shown them the concepts of word processing in general, and showed them different approaches to it (for example, an introduction to LaTeX or troff, and/or showing them LyX would be lovely, because it shows them a completely different way of going about a task. Then the students would learn that there is more than one way of doing something). We even have a "web design" class that is nothing more than a MS Frontpage 2000 class (yuk, and no, I refused to take it; it wasn't required)! No learning how to type whole webpages using Notepad or vi using HTML, nor is there an introduction to other important stuff such as XHTML and CSS, just raw Frontpage. But, hey, it's a MS world, and sadly, school districts are teaching them how to use a specific application, but not the ideas of the application.
If you really want to know if someone can perform a task, then ask them to perform it for you. It's called a "skills based exam".
Most of the IT exams are fact based exams. The CCIE is skills based as are some portions of the RHCE. Never have I even heard of a paper-CCIE or paper-RHCE, because it's not possible to pass unless you can peform the tasks within the exam.
Fact based exams are easy to write, easy to administer, and easy to design supplemental study guides / course schedules / etc. Skills based exams require you to use the technology until you gain some level of comfort and competency.
Our society has gotten far to hung up on testing as a silver bullet. Tests are just attempts to measure something, exactly as a ruler measures length or a scale measures weight. But tests such as the one in the article are, due to their nature, far less accurate or precise.
I see the introduction of yet another test as a poor substitute for one on one personal interviewing of the test subject. They are looking for a quick fix, one that is not people intensive when the fact is, people are best able to evaluate these complex abilities and skills, so long as they take sufficient time in doing so.
This is just like "No Child Left Behind". Instead of investing in the people, in that case teachers to work with students, a battery of standardized tests are introduced as a substitute. Yet, there is no validation of the testing against its objective, while we entrust our decisions to those very tests.
That is the biggest issue here as well. We are attempting to replace human judgement with supposedly objective testing, when it is precisely the complexity of that human judgement that is called for.
I'll bet it's even worse. Our company bought a 'skills test' a couple of years ago. Very basic PC/Microsoft skills, to supposedly evaluate a new persons training needs.
One in particular I remember was "How do you create a shortcut to an application?" Of course, there are several different ways to do this in Windows. If you didn't answer in the single, exact way they wanted, you failed the question.
Our senior MS developer got a "Needs remedial training". (No, not because he's a crappy developer, or works with MS...but simply because the test was badly flawed)
I thought I was going to get an IT degree from Montana State University, but after 2 semesters of GenEd classes, I finally found out that the _entire_ CS/IT curriculum is MS-based: MS Office for "productivity software", FrontPage for web design, Access for database, etc. I've reworked my degree program so I'm only getting a general AA; I'll use that to get into a school that teaches IT versus Microsoft for my B.Sc.
--- Asking inconvenient questions for over 30 years...
I am continually amazed by the number of my fellow college students who can barely read. It has made me extremely cynical about college because a solid 2/3 of college students in all but the most selective schools are both cheating and/or functionally illiterate. I wish I was exaggerating but I am not.
This is what is making me want to jump ship and get a job instead because If I have one more group project where my group consists of people who are just in college because they are supposed to be I am going to just say fuck it and start a business which is probably what I should have done.
If 4 years of college costs about 100k on the average (including living expenses) then I think I would be a lot better off if I had just been handed 100k at 18 for a business or a property investment.
With all the bootcamps and such it doesn't take much knowhow to pass. Although there will most likely be a bootcamp for this test soon enough!
-- "Life's not fair, but the root password helps."
You are obviously an employee of ETS or severely disconnected with reality.
Your points mainly address the amount of effort that goes into these tests. A lot of useless effort is just that- useless effort. An the GRE and SAT and worse than useless, since they arbitrarily make some people appear much better problem-solvers than they are.
It is fairly common knowledge that the SAT and the GRE, the two other tests for which this organization is known, have little to no correlation with the skills that they supoosedly measure. You can google some pages about the debate, but it's pretty one-sided as almost everyone with an opinion is in favor of dropping support for ETS and their tests.
I am about to take the Physics GRE and my practice scores improved from mediocre to far above average in one week of no physics studying. The secret? Don't read the questions: only read the answers. Eliminate the ones that are clearly wrong and without solving the question, you just got the correct answer. I challenge anyone to explain to me how this anything to do with real physics problem skills.