Slashdot Mirror
Replacing SAT with LEGOs
A reader writes "The Denver Post has a story about Colorado College, in an effort to attract minority and disadvantaged students, is dumping college-admission exams in favor of a Lego-building test, but only for a handful of applicants. 'The Lego test helps identify initiative, leadership and an ability to work in groups - qualities that hours-long ACT and SAT tests never quite get at.' " The college is working on this as a pilot program, along with eight other schools. Bet I could have gotten a better scholarship if they would have let me build a space station.
Perhaps we should drop attendance requirements as well, since the children do not 'really want' to go to school everyday. You know, to attract more minorities. Perhaps not.
In an era when job competition is becoming more intense everyday, our schools should strive to emulate countries which are far ahead of us, and send our children into the work place with a firm foundation of knowledge. Or we can just baby them and continue the recession into stupidity.
Perhaps.
First, it's excellent that universities are finally, after over 100 years of this, beginning to understand standardized testing is a poor indicator of value/intelligence/leadership. It's been known for a long time that there are many varieties of intelligence - the IQ test only tests one - basically math and spatial visualization. Wuzzah. So if you know 13 different programming languages - from LISP to C++ and can pick up new ones within a few days.. well, sorry - that won't show up on the test.
It's still a nice idea - give college kids some legos and see what they build. However, if they're still bent on using tests (an ultimately doomed approach) instead of interviewing potential students, may I suggest giving potential candidates an objective (which varies from person to person) and see how they solve it? In my opinion, it's more important /how/ you solve it than /whether/ you solve it. After you give the same problem to a few hundred people you'll know what solutions are typical and be able to spot the innovative and/or unconvential people in a group. What you do with this information is up to you, as an administrator, but you all know who my money's on.
I would think that minority groups would feel insulted by having tests that have lower standards then the average white kid. Doesn't just say that they are stupid? I know I would feel insulted if someone game me a test that was not as challenging as someone else. I am willing to say that all people despite ethnic or cultural background are smart enough to handle a challenging education and tests. I am going to go off on a limb here but I would bet that these lower standards are not needed. Rather keeping all students accountable to a higher standard in the class room no matter who they are.
So Linus, what are we going to do tonight?
The same thing we do every night Tux. Try to take over the world!
i can just hear it now...
some lady in screaming because her child didnt
have legos when he was growing up because the
family was too poor. now all of a sudden, this
test is biased against the poor minorities of
the country.
let's get serious here. the ONLY people who
complain about unfair tests are the people who
didnt do well (or the family and friends of said
person).
my parents didnt even graduate high school and
were not wealthy (the two biggest predictors) and
i got a 1350 on the SAT.
there will never be a perfect test. the only
"good" alternative is to base it on face to face
interactions with the candidates (something few
colleges could afford to do)--and then there would
still be the argument that "the interviewer was biased".
bah
A year spent in artificial intelligence is enough to make one believe in God.
Lego® bricks, paper, and computers are just tools. You can't judge the quality of a test by the tools used for taking the test.
Furthermore, the woman who designed the test will be keeping track of college students who took it, to see how well their success on the test correlates with their grades and so forth.
--
"But, Mulder, the new millennium doesn't begin until January 2001."
send all spam to theotherwhitemeat@ropine.com
Based on your performance in our Lego Application Process Demonstrating Aptitudes Needed for College Entrance (LAPDANCE), [name of school] is pleased to inform you of your admission to the College of Engineering.
Due to your particularly anal-retentive use of color you were passed over by the College of Arts and Sciences, Department of English. However, this same aspect to your creation was considered a strength by admissions specialists for the School of Computer Science....
Yadda - yadda - yadda ...
"For every complex problem, there is a solution that is simple, neat, and wrong."
My office has been taken over by iPod people.
Okay, let's assume for the sake of argument that there are "multiple kinds of intelligence" (certainly not implausible), and that standardized tests are really testing only one or two of them.
Item 1: College, in general, is *AIMED* at those one or two kinds; these tests measure *ability to do well in college*, for the most part. Giving people a test of something else just sets them up to fail later.
Item 2: Why is it that these people are assuming that the minorities "can't" do well on standardized tests? Isn't that sort of like saying they think the minorities are "stupid"? Frankly, I don't know whether or not racial groups have differences in brain structure, or whatever - but if they do, we'd damn well better start facing it head on, or we're going to wreck a lot of people's lives trying to push them into something they aren't. (Admittedly, it's no better to assert that an entire group will behave in the same way.)
Item 3: Why don't they give *EVERYONE* the lego test, and see how it pans out? If you give it only to the students you think won't do well on the other test, you aren't learning much. Let's be fair; make *EVERYONE* take the lego test, have their results graded by people who don't know which color people did which projects, and find out what the lego test tells you.
Item 4: "Kinds of intelligence" is probably meaningless anyway. "intelligence" is supposed to refer to the generalized set of "kinds of intelligence". Sure, the tests don't measure them all, but the lego test doesn't *MEASURE* anything, it just gives you a platform to balance your prejudice on.
Honestly, I think it comes down to this: There uexist people who are not "disadvantaged" in any way, and who feel guilty about this, and who will seek out "disadvantaged" people, and try to "help" them, in a way that makes it absolutely clear that the people doing the helping are in charge, and the "disadvantaged" people oughta be grateful for the help. These people are just as racist as the overt racists, they've just found a better way to sublimate it. Better for them, anyway. Not sure it's any better to be talked down to than openly hated.
This is a joke. With any luck, they will realize it, and start trying to do something useful - for instance, if the "different kinds of intelligence" thing pans out, start building a real curriculum for them, not just excuses to shove them into a curriculum that doesn't match them.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
Tests are imperfect at best and misplaced at worst, but reducing admissions to the level of "job" interviews is far, far worse than even the most poorly designed and executed test.
A test has at least the possibility, and if adminstered correctly, probability of giving objective results. The criteria may be misplaced or imperfect, but the results will generally stand on their own, all imperfections aside.
An interview can never even aspire to be resonably objective. Subjective prejudices in terms of personality, appearance, gender, and culture are intrinsic to any interviewing process. The result will not be some utiopian "fair deal" for those who score poorly on tests but would have made good students anyway, but a system so completely weighted by the personal opinions of admissions interviewers that fairness of any kind will not be even remotely possible. Ultimately it will no longer even be expected, or strived for.
Standardized testing, for all of its flaws, at least eliminates the worst of the cultural and personal bias of the admissions process, by stating more or less "you are required to know a modicum of the following, if you don't, learn it and come back for another try." Far more fair, even for the disadvantaged (however one defines the term) than an interviewer commenting "You really aren't State U material, sorry kid" become some arrongant jerk doesn't like your accent, your nose ring, your hair style, or your skin color. Or worse, they've simply had a bad day and just feel peaved enough that playing God with your future makes them feel better. (If you don't believe both of these happen quite often in the private sector, I suggest working for a time in the personnel department of any large firm.)
No system is perfect, but your proposal amounts to throwing the baby out with the bathwater, then chucking most of the nursery out the window as well.
As to the notion of using legos for colleges admissions, I can only cringe at the value a college education received in the United States will have fifteen or twenty years from now. All of the arrogant posturing by Europeans with respect to the American system of Higher Education will become appallingly accurate if this silliness continues.
The Future of Human Evolution: Autonomy
Ludicrous.
Look. College is many different things; I'm not going to argue that. But its primary purpose is a place of study. If a student can't perform well on a test (which requires that one exercise a reasonable amount of study skills), that student simply isn't going to do very well in most collegiate enviornments.
The bit about testing leadership and other types of intelligence is, I suppose, a valid concern. But do it in addition to the tests, not instead of them.
Personally, I always saw racial preferences as one of the more wacko ideas. A person's race doesn't determine how well they'll do in a college environment at all. Religion might, depending on the tenets of the given religion, but that's exceedingly rare and a student whose religion might affect college life is very likely to choose a place where the two won't interfere with one another anyway. Sexual orientation doesn't affect study skills, or gender, or anything else along those lines. So why even put them on a college application at all? Statistics?
Statistics and such are of interest to statisticians, but in the end they're not really indicative of very much. Personally, as I see it, if you want to make college admissions fair across races, don't make all kinds of special treatment. Make the process race-blind (and gender-blind except in the case of single-sex colleges, and such). Completely eliminate the race question on the application form; if you feel you need the statistics then send the students a survey after the admissions decisions have already been made. I don't think anyone here will argue against the assertion that race and such has no place in the decision whether or not to admit a student. So why even ask the questions until the decisions have been made?
But I'm ranting again, so back to the subject...
A college is a place of study. I fail to see how objective merit is an invalid concern. The SAT's and ACT's are meant to be a measure of the skills a student needs to succeed in a college environment, namely study and reasoning skills. I think the ACT tends to do a better job of measuring this than the SAT's (since the ACT's test a broader range of skills), but both do well in their given fields. The rest, such as leadership, are of course very nice skills to have. But they have little to no effect on how well the student does in classes, and can even hinder the student in some cases, as they get involved in more activities than they can handle and their grades suffer as a result (I've seen this happen to far too many students).
So attacking merit isn't the way to go. Replacing onjective tests with subjective ones only adds the potential for more racism and bias than the terminally insecure claim exists now. Combining the two is actually something of a good idea, so long as you're careful about how that's done.
Oh, one last thing. Someone talked about the idea of interviewing prospective students. Most colleges already do that. I went through a bunch of interviews, as did my sisters when they went off to college, as did everyone I know who went to or is currently at college. It's an important tool. But all tools are inherently flawed in some manner or another (even a simple hammer is flawed: it can miss the nail or worse, hit your finger); that's why only rarely can any one tool be used to get the job done well.
A close reading of the article reveals that the school is not "throwing out" the SAT, but allowing kids who flunk the school's SAT benchmarks to take a battery of alternative tests, including this lego test. The battery will apparently include a traditional interview. That thought, however, is too complex for us mere mortals to grasp. Watch as our friends in the media helpfully simplify the story for us over the next few days to: "SAT thrown out for Legos!" Dan
I believe that aptitude test should be devised using an "open source" methodology. Lest you think I'm just trying to drop the phrase "open source" for karma points, I'll explain:
Aptitude/intelligence tests are typically devised behind closed doors. A small committee gets together and creates questions. The questions are then tested on students. Questions with a high miss rate are considered "hard". Questions with a low miss rate are considered "easy".
This methodology has very real flaws. First of all, the elite group which gets to create the questions is too small. They are very rarely questioned. The fact is that SAT questions have been proven to contain questions which could be misunderstood. Sometimes, the "correct" answers are just wrong. Lawsuits have been filed over this and ETS (the company who creates the SAT) has been forced to change scores. This is kept quiet, of course. We can't have people questioning the almighty SAT.
Since the elite group is too small and closed to rule out the possibility of group error, we should open up the question-writing to everyone. Let educators from around the country create the questions. We have the technology to do this right now.
In fact, it should go beyond just the educators. The process should be open to students, as well. How about a pilot program where Slashdot members devise an aptitude test? I have no doubt that the combined intelligence of this site could produce a far more informative and perceptive test than anything in use today.
-- In the future, everyone will code Perl for 15 minutes. --
The person who can build with the LEGOs would not be on par with the person with the knowledge, they would be playing a different game all together.
Maybe so but the goal is still the same. To get into college.
There is a different kind of intelligence at work when a person uses LEGOs as opposed to memorizing material. The whole point of the LEGOs is to level the playing field for people who didn't have the resources to learn the material
that someone else had to luxury of having access to or the time to learn it.
Funny I can take the material that I supposedly "memorized" and apply it in various situations (such as posting to slashdot) without just reciting the things chapter and verse.
Could someone please tell me in the late 20th century early 21st century in America how exactly can one *not* get a HS education? Maybe if they have a severe series of operations or maybe they are physically beaten up on the way to school then it might prevent this. However even the poorest student in the USA has some local school where he/she can get access to education. Now if the school sucks or dosn't teach right isn't the issue. It's wheather you can *learn* from teh HS and get a diploma. Technically if you get D-s all the way through HS there are still colleges that will take you. As far as time well why do you even do anything? People in the late 20th century seem to think that they have less and less time then people in earlier periods of history. As I recall in the early 20th century we had people who really didn't have a whole lot of time because they were working to death. Quite frankly I think such a hellish time was absolutely scary. I don't think HS students are so pressed for time (you really don't need a job at all in HS) that they can't even read a book or do some homework.
Let's face it, life isn't all peachy everywhere for everyone. People are put at a disadvatage sometimes and should not be denied opportunities for it. This is the basis of using LEGOs. It's an opportunity for certain people who are
disadvantaged to prove that they can hack it in school. You can't honestly expect ghetto kids to stop dodging bullets to learn trig or expect some kid in the swamps to study poetry or whatnot while he waits for his pops to beat his
ass because he is drunk again. The LEGO thing makes things more fair because they are blocks that can go together in a way to make something.
You know people must think that everywhere at every time that people absolutely *HAVE* to live where they are. If I live in complete and desperate poverty or are dodging bullets or what not I can always get up off my ass and actually leave. Hell if I am homeless or a bum I can always leave. People just don't realize that there are alternatives. Maybe if say a group of well armed thugs decided to take out the little ol' gang members perhaps we could actually get some peace. I mean they want to threaten others why don't we threaten them?
Basically you are saying that because johnny is too busy not getting killed or maimed in some way that he can't learn? Exactly how many of these cases are actually happening? I would like some reliable figures that indicate that the use of legos and alternative testing measures are actually an effective measure to giving these people an equal chance. If I can't do basic algebra how can I do say entry level calculus or say even good programming. The concepts of varibales and equations are basic to programming and without the ability to actually know these things I think that you can't do it well.
My point is if you are bright, but for some reason unable to learn the things that you and I have had the opportunity to learn, the LEGO test is basic enough to let a person like this demonstrate that he is smart enough to go to
college.
So colleges now become the centers of remedial education. These people will undoubtally end up taking level 0900 classes or equivelent and not starting out in freshmen level classes like the rest of the people.
Slashdot social engineering at it's finest
Actually, some of those sound almost as interesting, if not more, than the Lego test. Public speaking, in particular, strikes me as a good test, given that, in many, many surveys, people have consistently rated "speaking before a group" as their number one fear, above even death. Conflict resolution is going to be increasingly important in society as a whole, and on college campuses in particular. And other messages in this thread have suggested that personal interviews be used in place of "the Lego test"; it would seem that they're actually being used alongside it.
Some other things to note about the article:
- CC "has long been an innovator in higher education." Somebody's got to try these ideas first...
- CC is not the only college using these new tests; eight other universities are doing this, too (four other private liberal-arts schools and four state universities).
- Very few students are going to be admitted as a result of these tests, at least at first (it's a pilot program). CC and the other liberal-arts schools will be admitting four students each under this program; the four state schools will admit 20 students each.
- Researchers will be keeping track of these students; to see how well they do in relation to students admitted under "normal" policies.
- The intent of all this is to maintain racial diversity in college admissions without resorting to "affirmative action" programs (which are in the middle of a whole slew of political firestorms right now). This is important for CC, since (according to the article) "Colorado public colleges and universities are required to show continuous improvement in minority recruitment, retention and graduation."
Naturally, the "Lego test" gets the headline because it sounds outrageous. Nonetheless, the overall program sounds both interesting and worthy of investigation, and I hope that it works out. And, even if it doesn't, they'll have learned something as a result.Eric (Denver, CO)
--
"Free your code...and the rest will follow."
Be who you are...and be it in style!
Didn't read the article, huh?
Where is the evolution then?
I wasn't aware that low-scorers on the SATs were fed to the wolves.
As in "reading the article." Looking over comments, here are a couple of things most posters seem to have missed:
- The lego test is only one of a series of twelve workshop tests performed.
- The schools are looking for tests that are better predictors of college success. These workshop-based tests are an experiment to find such tests--they are only being used for a highly limited number of admissions, with the outcome carefully tracked. Maybe they'll work, maybe they won't.
As alluded to in the article, the dirty little secret of standardized tests is that they correlate better with socioeconomic background than they do with ultimate college success. And that's to be expected--kids from more affluent families went to better schools, got more help from their (usually) college-educated parents, and so forth. Most of us who have been through college know people who aced their SAT's but royally screwed up their coursework. These schools are looking for something better, somthing that measures ability to succeed as well as general knowledge.It's worth a try, in any case.
Well, as some of my posts undoubtably underscore (spelling in particular), my rather high score on the ACT was NOT due to the English portion of the exam. :-)
The standardized test (and later the net, with it's emphesis on typing and not writing by hand) also enabled me to do well despite my appalling handwriting.
Standardized tests do work pretty well, albeit imperfectly. Smart people with significant failings in other areas can and do do very well, with those weaknesses hilighted by their respective scores.
As an aside, the fact that two private firms came up with two competing, national tests (the ACT and the SAT), and their subsequent success, clearly shows there was an unfulfilled need for a national standard whereby colleges could judge the relative educational level of prospective students coming from diverse backgrounds and school systems of widely varying quality. These tests may be imperfect, but they are far more functional (and useful) than building with legos.
The only real "argument" with respect to national standards tests is not whether or not they are needed, but whether or not they should be designed and maintained by private companies or by some kind of public (government or acedemic) institution. As stubborn Americans we may steadfastly refuse to learn from our European and Asian neighbors (who have had standardized testing on the national level for a long time), but we can hardly ignore our own free market, which we hold in such high, almost religious, regard, and which has very unambiguously demonstrated a need for the same kind of national standards right here in the good ol' US of A.
The Future of Human Evolution: Autonomy
How do you intend to afford to move to a new place when you're in "complete and desperate poverty"? How do you afford to travel to your intended new neighborhood to hunt for an apartment? How do you come up with the security deposit for your new place - and the deposit for the electric service, and for the phone? How do you get your stuff (meager though it may be) to your new place?
Have you ever actually known any poor people, slashdot-terminal? I think that if you did, you'd know that moving to a better place is at the top of many poor people's dreams. But it's much easier said than done.
Tom Swiss | the infamous tms | my blog
You cannot wash away blood with blood
Armed with a number of pre-programmed lego mindstorm control modules, he not only duplicated the robot in the other room but activated it. Once activated the robot began attacking all of the other entries, destroying a large number of them before being turned off. Ken stated that he got the idea from watching Battlebots on pay-per-view as well as a recent SRL performance.
Although banned from Colorado College, Ken is not disturbed by the results of his actions. Both CalTech and M.I.T. are offering him full scholarships.
-----
No Zen is good zen
I'm all for alternative testing to get into college, but it's only a step in the process. Jusrt getting disadvantaged students into college does not guarantee their success. Traditional curricula also need to be adjusted.
Getting "disadvanted" folks into college is a laudable goal, as long as you aren't throwing more qualified applicants out to do so. If you are, then you are at best merely substitutied one unfairness for another. More likely you have chucked a reasonably fair and objective system for a very unfair one, which happens to favor whatever disadvantaged group you are wishing to promote at the expense of everyone else.
Indeed, if you substitute arbitrary standards (or worse still, subjective interviews) for reasonably objective standards, you eliminate any degree of fairness whatsoever from the system and replace it with an economy of favors and influence.
If there is a group of "disadvantaged" people who can't cut it in academia as it now stands (due to "test paralysis" or whatever), feel free to establish an alternative university with different standards and metrics designed for that group. But do not deny those of us who are capable of excelling at academics a good education to do so by dumming down our existing universities, or so slanting entrance standards to such a point that they become meaningless.
What is next? Getting rid of exams altogether (after all, if you can't pass an entrance exam what makes you think you can pass the first semester's mid-terms or finals)? Social promoting? ("It just isn't fair that someone with a BS makes a better living than a high school graduate, so everyone gets a BS!") Some other nebulous notion of achievement based on some administrator's completley subjective notion on how well a student did (and how do you document this "performance" to insure even a modicum of fairness?), as opposed to imperfect tests which at least strive to be fairly objective and which, for whatever other weaknesses they possess, can at least be referred to, reviewed, even regraded if necessary)? That might actually fly in areas with a great deal of subjectivity anyway (e.g. some of the Arts and Humanities), but in areas of hard science such an approach would be absurd in the extreme.
As for objectivity being a myth, or unattainable, as another poster suggested, that is simply hogwash. Perfect objectivity may be unattainable, just as a perfectly (i.e. 100%) effecient engine isn't possible to build, but high degrees of objectivity and fairness are achievable (just as highly effecient motors which run quite well, if not "perfectly", are). The effort to achieve objectivity is certainly not something to be discarded in favor of selection methods which are fundamentally subjective and completely unfair altogether (as the proposed "interview" approach would be), or simply so off-target as to be meaningless (as the "Lego" approach is).
The Future of Human Evolution: Autonomy
Here's the problem:
... when preparing for the SATs, high school students from solidly middle-class to upper-middle class families with white-collar, college educated parents have a HUGE advantage. Make that several:
First of all, race makes a handy metaphor for class in the U.S. since anyone talking about the "lower class" must be a dirty Commie (everyone's middle class, and all the children are above average, blahblahblah). So enough with the racism complaints (not directed at the previous poster, but at the thread in general). "Non-white" seems to be the only way to describe "dirt-poor" that will make it into public discourse.
This, of course, creates a problem. Namely, the rich non-whites are getting into college just fine, and poor kids (white and otherwise) are left to rot.
Next problem: As others on this thread have already pointed out, the SAT questions are biased towards the upper-middle-class suburban life. Someone from Appalachia or from the "concrete jungle" is going to have a serious problem with those questions. One example that's always stuck with me (not an SAT, but another standardized test for younger kids) that I saw back in my days of work-study work for an educational consultant was this:
You were playing with your friend's ball and lost it. What do you do?
1. Buy him a new one and pay for it (answer worth 2 points)
2. Look all over for it, try to find it (answer worth 1 point)
3. I'd just cry; say I'm sorry; apologize (answer worth 0 points)
Um, do we see a slight problem here? The damn test is using "throw money at the problem" as the right answer! That's NOT going to work for kids that have no money.
Speaking of throwing money at the problem
1. The parents have probably had to go through the standardized testing process at some point, and can sit down with their little wannabe college students and grill them.
2. They can also afford SAT prep courses.
3. They can also (sometimes) afford private schools, and/or to live in places where the public schools are good.
4. In some cases, there's a difference between merely getting into a specific college and getting into said college with enough money to go there. The kids who got (at most) 1200 on their SATS and come from rich families are a "level playing field" for the kids who got 1500 and are on full (or nearly full) scholarship.
I know a lot of this from personal experience. I was homeschooled from 3rd-6th grade, and every year I went to take the Iowa tests with the kids in school. The first year, I ran into a severe crisis on the math section that can best be summed up as "lots of problems really fast." All my other scores were in at least the 75th percentile (most were in the 95th plus), but this one was somewhere around 38th percentile. Mom thought something was strange about this and asked me what was going on. I explained, and she started drilling me on fast-paced timed arithmetic tests. I think I jumped up to 97th percentile on that same section the following year.
And as for the concept of the SATs et al testing "what you need to know to succeed in college," they do no such thing. Again from personal experience. I was in Johns Hopkins University's Center for Talented Youth program. A prerequisite for admission to said program was scoring at least 430 on the old Verbal and 500 on the old Math SAT at the time I participated. Some of us did fabulously well in life. Some of us who were big fish in small ponds in high school got to college and realized that we might be intelligent, we might be brilliant standardized-test takers, but we didn't know how to study! I was screwed the first time I had to write a research paper in college. Didn't know what in the H-E-double hockey sticks I was doing. I also tested out of (or nearly so) classes that I really should have taken, and the holes in my mathematical, scientific, and music theory background came back to haunt me again and again in college.
One of my friends from CTY summer camp? She got a 1500 on the SAT when she was 12 years old. She received early admission to the same college program I did, then transferred to Alfred -- and flunked out. Last time I talked to her, she was working fast-food. So much for high SAT scores predicting success, huh? She wasn't the only one in a similar situation, just the most drastic example I can think of to show how completely the SAT doesn't predict success on a damn thing other than taking standardized tests.
And yes, I know that a poor and hard-working kid can beat the odds, study on his/her own, and make it into a good college. (I went to school with quite a lot of them.) I also know that, again, the playing field isn't level. They worked a lot harder to get where they are than those of us who had money and/or parents in an educational field.
This issue is so incredibly complex that writing off any one approach as "laughable" is, itself, laughable.
"Somebody exploded a letter-bomb today