GRE Computer Science Exam Canceled For '02

An anonymous reader writes "This may be a bit dated, but the Educational Testing Service has canceled the Computer Science GRE exam for November due to the fact that students were sharing and posting exam questions. One has to wonder about the immediate effect this will have on grad school admittance, as well as the long term changes that will likely occur to the tests as a result."

Comments

sharing?

[matt4077]

that's what I call peer2peer education!

Well, at least they passed the first question

[matrim99]

Well, they at least passed the first question:
1) Can you use a computer well enough to share information?

Never happened before

[LRNG_LNX]

Exactly. While I was at school, an instructor pointed me toward the MCSE "brain dump" site. I never bothered to remember it because I am not planning on becoming MS certified. People would go there after the test and submit as many questions as they could remember from the test.

In high schools, first period Chem students would share info on the test to the later period Chem students. How much info can be retained with so little time to cache it?

Was it actual test questions? Or just pointing people in a more focused direction?

How many tests have you taken where half of the "knowledge" was "researchable material?" What year did the OSI model come to be? That was an exam question I faced on a multiple choice exam. I wrote in my own answer . . . E) go look it up because I don't need to know that to set up a network.

Re:Never happened before

[Rich0]

Keep in mind that with a test like the GRE you have a pool of only a couple hundred questions, and thousands of people taking the test. If a website were written which assigned you a random question number to memorize (just a single question), and then you reported back after taking the test, chances are you would form a library of almost the entire question pool - you'd have a number of people memorizing each question.

Yes - I realize the order of the questions - and even the selection - varies from form to form. There wouldn't be just 1 person memorizing question #2 - there would probably be 50. So you might get 10 different answers 5 times each - but the point is that you would get the questions.

I believe the questions are rather heavily recycyled - that is part of the benefit of hte tests since you can use the recycled questions to compare results across multiple editions of a test, or between years. If every test were completely unique, it would be harder to determine whether one edition was harder than another. Sure, you can look at the mean, but that assumes that student performance does not change from year to year. In fact, one of the best uses for standardized tests is to test the educational system itself by seeing if this is in fact the case.

Re:Never happened before

[Tenebrious1]

Microsoft revoked the certifications of a few people who ran those sites. Cisco sent warning letters as well. Those of us who used to post full dumps don't anymore, the risk of losing certifications is too high. When you take the exam, you agree to a NDA.

The mainstream study sites make up their own questions, which is no different from old school study guides. They can't use exact questions, and all the guides tell you they are not exact questions.

Seems silly -

[jpellino]

If the sharers are so savvy, they can no doubt move this info anywhere they want - why only cancel the local tests? Seems everyone who wants this stuff will have it in short order.

Unless the tests have regional forms in addition to the other multiple forms, this is either useless or a symbolic slap on the wrist.

scores (alone) are used in 2 main ways

[Anonymous Coward]

First, scores may be used at the low end as a first sort method. People who score below a certain cut off might be eliminated from further consideration. This is often done to limit the number of applications that must be examined.

Second, scores may be used at the high end to get considered for fellowships and other academic awards.

Most schools do not use the scores alone, but consider the whole package. Still, scores and GPA are used most everywhere in the initial sorting of applicants. If you are on the admissions committee of a CS department, and there are 3 of you, and there are 300 applications for 20 spots, you are going to want to winnow the pile in *some* way.

Keep in mind that standardized test scores do an ok job of predicting success in school at the high end, but do a very poor job of predicting failure in school at the low end.

"Education is not preparation for life, education is life itself." John Dewey

Re:scores (alone) are used in 2 main ways

[pagansage]

First, scores may be used at the low end as a first sort method. People who score below a certain cut off might be eliminated from further consideration. This is often done to limit the number of applications that must be examined.

This is not always true. In fact, I would say that this is far from the truth when it comes to the GREs. Being in Grad schol myself at a large state school I have a friend whose research advisor happens to also be the graduate advisor for the largest department on campus. This department recieves MANY applications from Asia and the GRE scores are a joke. Almost all of the applicants have a 99% percentile on ALL three sections of the general exam. And yet when they come here they can barely speak english.

My friend was asked by her advisor which applicant she would pick out of the scores and she pointed to the individual with the low score. She said that he probably earned that score. Most schools know about the score inflation caused by Asia and take that into heavy consideration.

If your going to apply to graduate school, especially for the sciences, then GRE scores should be the last of your worries. The best thing you can do is take as many classes in your field as possible. Get involved in some type of research: REUs, ERULF, etc. And when you apply show that all you want to do is research, research, research. Make sure you pick out a field (doesn't matter what really) within your discipline and show that you have some interest and knowledge in that section.

Not a tragedy

[back_pages]

I am applying to grad school for Computer Science this spring, and it turns out that many schools only require the subject test (what this news it about) if you are an international student or if you did poorly on the general test and want to show that you have potential in your area.

I'm sure not all schools use this approach, but many do. Maybe we'll see fewer international students coming in this year. That will certainly be a loss, but like I said, I'm applying and I really don't mind a temporary loss of competition. ;)

I've never understood...

[cperciva]

... the point of GREs. Undergraduate entrance exams make sense; they verify that you have the prerequisite knowledge to take (pretty standard) undergraduate courses.

At the graduate level, however, you're supposed to be doing research. How do you define what knowledge is prerequisite for doing research in computer science? You can't -- all you can do is interview the students, get a feel for what sort of projects they are interested in, and decide if those projects sound as though they would be worth a degree.

Maybe things are different in the US of A, but I don't think I would personally want to study at any institution which would admit me on the basis of how well I did on an exam.

Re:I've never understood...

[rknop]

At the graduate level, however, you're supposed to be doing research. How do you define what knowledge is prerequisite for doing research in computer science? You can't -- all you can do is interview the students, get a feel for what sort of projects they are interested in, and decide if those projects sound as though they would be worth a degree.

Things probably are different in the USA. I should disclaim that I'm in Physics&Astronomy, rather than Computer Science, but I suspect many of the same things apply.

A student who goes to grad school knows that he wants to get into research, and, yes, that's what grad school is all about. However, it's the rare student who really knows exactly what he wants to do when he goes to grad school. Many will have some idea as to what field they want to go into-- e.g., in Physics, they may know if they're interested in astro, nuclear, particle, condensed matter, or something else-- and a few students will know what profoessor or project they want to work with, but it's the rare student that knows beyond that.

Part of the purpose of grad school is so that students can learn, in an apprenticeship sort of mode, how to go about doing research. As such, judging them on the ideas for projects they have isn't really fair. Yes, it would increase the quality of grad students, as only the very top ones would ever get in. However, undergraduate education does very little to prepare students for that sort of thing, as that is part of the purpose of graduate education.

On to the GRE's. In Physics, there *is* a core set of knowledge that "any" Physics graduate student ought to have. Indeed, most schools have core courses (or core exams) which the students must take beyond that, to get the basics of the field, before they can be admitted to candidacy (at which point, yes, it is based partly on their presentation of what they will do for their research, and thereafter mostly all they are doing is working on their research). This admission to candidacy will typically come after the student's second year of graduate school. The GRE's give some vague indication (up to the general utility of standardized test, which is a whole different debate) of how well prepared a student is to survive those first two years of graduate classes.

-Rob

Re:I've never understood...

[paranoic]

At the graduate school I went to (SUNY at Stony Brook), they didn't care about your GRE in their subject, becuase they could teach you that. They cared about the Math and English, because they didn't want to teach you that.

Re:I've never understood...

[astroboy]

The GRE -- or a standardized test in general -- is a useful tool. It gives you a few very broad results for lots of people in a way that's known to be uniformly assessed, at the expense of a certain arbitrariness.

With personal interviews, you get a lot more discretion in the interviewing, and you can get a lot more depth of information, so things are a little less arbitrary -- but then you get the biases of the interviewers; someone who one interviewer might think is brilliant, another might dismiss as a hack. So you get deeper, more nuanced information, but less uniform and differently arbitrary.

Don't forget that in the US, most students coming into Ph.D. programs are coming straight out of undergraduate, so there is a real question of making sure everyone has the same background, and the students aren't launched immediately into research the way they might be in a system where you do a Master's first. Also, in a field like CS, you have people coming in from Math, CS, Physics, or Engineering background, so a `level laying field'-test is a useful tool.

The GRE provides four numbers -- a score on a subject test, and scores on three general tests -- logic problems, basic mathematics, and english skills. All of those can be useful pieces of information -- a bright person who can't read or write is unlikely to be a good TA or, for that matter, work well in groups where reports need to be written often. A person who knows tonnes about their subject but can't solve simple logic problems is unlikely to be a really creative researcher, etc.

When you have 500 people applying to your department, and you can only accept 20 or so students, you need a quick way of pruning the list to 50-100 or so, who you can then start looking at in more detail. For that, you need a broad, uniform measure, which is exactly what the GRE provides.

I don't know any department that bases its acceptances solely on the basis of the highest GRE scores, and those who did probably deserve the students they get. (A Yale and Cornell study found that GRE scores corellated well with the students coursework grades, but poorly with overall sucess as a grad student). But it is a useful piece of information, and helps sorting through huge pools of applicants. Like any standardized test, the problems come when people take the tests too seriously, usually by assuming that the tests measure something that the test doesn't even claim to test for.

Re:GRE?

[Anonymous Coward]

GRE stands for Graduate Record Exam. Its the graduate equivalent of the SAT (another test taken to get into university). The GRE is broken usually into 2 parts. A general exam similar to the SAT ( MCAT, GMAT, LSAT etc etc ) that is supposed to test general knowledge and aptitude. Then there is the course or area specific portion which tests your knowledge of the area of study to which you are applying. eg: Comp. Sci. specific.

These are very popular or fairly popular at least in the US

Re:GRE?

[t0rnt0pieces]

Could someone in the US please tell me what a "GRE" exam is?

The Graduate Record Exam is a standardized test administered by the non-profit (yeah, right) College Board (same people who do the SATs). It's usually a required test for anyone wishing to gain admittance to graduate school. In addition to the general test, which tests language skills, basic math, and analytical skills, they also offer a wide range of subject tests. Some schools just require the general, some require both the general and subject tests. The cost for the general test when I took it was a whopping $90 or so. It's a grueling test that will ruin your day. Takes about 3-4 hours. The best part is that it's on computer and you get your scores instantly.

The person responsible

[Alsee]

Considering it's a Computer Science exam, I propose that whoever hacked in to get the questions in the first place should get an automatic 100.

P.S.
Yes, I read the article. I took comedic licence with the facts. (I didn't like them, so I ignored them.)

-

no crap

[xeeno]

How much of a kickback from ETS does a university get from requiring that students take the GRE? It's a pointless test, the real culling of the herd occurs after the comprehensives - and it's not like departments don't look at the student's transcripts before sending a positive admissions letter and offer letter.
So the real issue isn't forcing quality of students, it's a way for the university and a 3rd party to scam a little cash before taking the next 2-6 years of your life. People really don't realize how poorly this test reflects ability - when I first took it, I took it cold and got a 1300 on it. I bought a cheapo study guide and then made a 2200 on it. The only thing I did was practice a little on the included online adaptive program and look at some of the TRICKS at getting quick and easy answers.

Re:So what?

[TheSync]

My quantum mechanics teacher has a very good solution to this. He had a file with 100 problems. An exam was a random draw of 3 of those.

Yeah, I had the same teacher, and it was tougher than just that.

The three questions were actually superpositions of the 100, so you wouldn't know what they were until you observed them on the test ;)

GRE really does make sense!

[Gaurang]

So the real issue isn't forcing quality of students, it's a way for the university and a 3rd party to scam a little cash before taking the next 2-6 years of your life. People really don't realize how poorly this test reflects ability - when I first took it, I took it cold and got a 1300 on it. I bought a cheapo study guide and then made a 2200 on it.

Wait. You are wrong here.

Dozens and dozens of my friends and acquaintances (and myself) have taken the GRE, and based on their results, I can assert that, given that you study for the test seriously, it reasonably estimates your general aptitude. And that a GRE score is a reasonable measure of your academic skills, which when used along with other application materials, is really useful for admission decisions.

And the GRE Subject Tests are even more useful, since they objectively measure domain-specific knowledge, and tell you how much a student really knows about his subject. (cant rely solely on GPA!)

More importantly, however, there is one very significant positive point about the GRE, which just cannot be put down. The applicants to US Graduate School include a large number of students from dozens of different countries each having its own peculiar education system. Its almost impossible to compare such students coming from schools which the US schools have never even heard of before, let alone having some knowledge about their education quality. The GRE and other standardized tests are most helpful in this area because each and every student across the world has to give the same test, and hence the scores can be used to compare students coming from very different backgrounds.

Re:CS Students from Abroad Should Be Curtailed

[Gaurang]

This is under the assumption that America has qualified students who have the potential to succeed and make significant contributions to the CS field. If there are foreign students who are more qualified, then we're dumbing down our own nation by refusing them for less qualified local students.

I am a foreigner studying in US CS Grad School right now. I must say that it is more correct to take the _best_ students from whichever country they are, since they will add value to the department by doing better quality research, and also enhance the classroom experience. And the US Schools, in fact, really do that today - they take the best students.

I just feel that as we import more students, educate them, and then subsequently export them back to their country we have just devalued our own country a little bit more by shifting knowledge and skills to foreign lands.

Thats clearly happening. Lots of skills, knowledge, and technology is being transferred from the US to outside. But for a developing country like from where I am, this is important and very useful, and helps our country develop. Even otherwise, its good for society, since all countries should develop equally, right?

Also, many times, the foreign students do not go back to their country, so the USA benefits by having a high-skilled person added to the community...

So the present situation turns out to be beneficial for the US as well as other countries.

Re:CS Students from Abroad Should Be Curtailed

[davechen]

In my experience (I was born here, my parents are from Taiwan), Americans are just not as interested in graduate school. Asians seem to value higher degrees (especially Ph.D.'s) much more than Americans.

To be honest, unless you're interested in academia and research, there isn't much reason to get a CS Ph.D. From a job/financial standpoint, you might as well start working straight from college.

Heck, when I was in college it seemed like a lot of the CS majors were just there for the job prospects. Those aren't the type of people who go to grad school. You have to have a real passion for the subject to want to slog through N years of grad school. Or you have to really want the letters after your name. Otherwise it ain't worth it.

how admission actually works

[dwallach]

I'm an assistant professor at Rice University, in the CS department, and I've been part of the committee that admits grad students. Since there seem to be a lot of questions here, I'll try to answer them as best I can.

Admissions to a research program, like ours, is based on a bunch of different things. A good way to think of it is that your statement of purpose and your letters of recommendation are the things that build your case and everything else is an opportunity for you to shoot yourself in the foot.

First, on behalf of every professor inundated with e-mails, let me say that there's no benefit, whatsoever, in mass e-mailing every professor on the planet to see if they're interested in you. However, if you have a particular research interest that matches a particular professor (say, security of peer-to-peer systems or of mobile code systems, in my case), then do send that one professor a personal e-mail.

So, then, what makes for a good statement of purpose or good letters of recommendation? I always look for evidence that you've got technical interests beyond what you've done in class. I also look for evidence that you didn't just go to "how to get into grad school" school and follow their instructions blindly. Some of these letters just stink like they came from an insert-your-name-here template. Don't tell me "I have lots of energy!" Describe all the work that your energy has produced. Likewise, in many countries, it's customary for the student to write letters of recommendation on behalf of their advisors, who just sign them. Make your advisors write personal letters.

So, what value are the GRE scores? For the General GRE exam, there's some minimal value if the applicant is from the U.S. or Europe, but absolutely zero value if the applicant is from India or China. I don't know what they do over there, but every student seems to ace the exams, probably because they study so hard for them. The exams, thus, aren't measuring anything more than rote ability.

The CS subject test used to actually be useful and a strong score there would catch my attention (and a weak score was a huge red flag). Now, without that, we'll probably end up looking more at transcripts.

Still, let me emphasize, the best way to impress somebody like me is not with good grades or test scores, it's with research and technical experience beyond your class assignments. If you've worked with a professor on a research project, or if your code has found its way into the Linux kernel, that will get my attention (and I'll go look at the source to make sure you're telling the truth).

Obligatory plug: I'm looking for good security-minded students with strong backgrounds in systems and/or programming languages. If that's you, contact me.

Re:how admission actually works

[bruthasj]

> I don't know what they do over there, but
> every student seems to ace the exams, probably
> because they study so hard for them. The
> exams, thus, aren't measuring anything more
> than rote ability.

You're a very upbeat guy, Dan! I mean, optimistic. I'll tell you how it works -- they cheat.

I live in Taiwan and have a sister-in-law who is studying to take the GRE. The advantage in Asia (India?? -- I think this is a Rumor!) is that they have "Teachers" that take the GRE test every few months on the computer. While they are taking the test, they memorize in their heads each question. Once they're done they come back out and teach 2 month long classes for big $$$ to the students.

I don't know what the history is as far as what cancellation came first, but the GRE people have cancelled the Computer-based General test for the express purpose of reducing continuing cheating that happens in Asia.

Everything in Asia is now written and its going to stay that way for a long time. This is why previously any GRE scores coming from Asia has zero value!

Re:CS Students from Abroad Should Be Curtailed

[donutello]

I'm shocked that such a blatantly racist statement can get modded up on this board.

I'm sorry you were rejected from a whole slew of grad schools. I'm sorry your abilities, your knowledge and your intelligence were deemed inferior to a whole bunch of people inspite of the natural advantage you held as far as the language and accessibility of technology is concerned.

I'm sorry, universities are not run as a charity system. Most scholarships are paid for by industrial sponsors who want to see research results from their money.

I went to grad school at a top-10 school for CS. There is a whole slew of scholarships and opportunities that are available for US citizens and residents which are just not there for foreign students. As a result, many US students were admitted over much better international students. The few foreign students that were there were the cream of the class - and outdid most of the local students in terms of what they achieved for their professors and for the prestige of the university.

Deal

[Convergence]

Deal..

I applied for grad schools about 2.5 years ago, and got 5 rejection letters...

Did I hate 'abroad students'? No.. I did research and grad classes for 2 years, and tried again, last winter, and had a choice between two top shcools, in a much tougher year for grad schools.. I still got 3 rejection letters, but I got in.

Another thing, if you're applying to top schools, try setting your sights a little lower. There are enough spectacular faculty that *any* top-40 school will have them.

Finally, remember, a lot of the application process is chance and brownian motion. If a faculty member is interested in your particular skills&background, spectacular, if that member isn't on the admissions committee this year, better luck next year.

Re:They are

[elflord]

When I was applying to grad schools, I lost count of the number of things that "required US citizenship" (in particular, financial support). Most of the decent schools accept mostly American students, and only a small number of overseas students (who are usually much stronger). Some of the weaker universities accept a lot of foreign students.

As for selecting honest people, that's a noble sentiment, but it's difficult to determine honesty on an application, and it's not practical to interview all candidates.

Test ability ... not memory ...

[ProfMoriarty]

This is a very interesting story, with many interesting thoughts.

Please note, that I have never even seen these tests, nor know anyone who has ... but I'll still pipe in :)

Instead of keeping the questions and answers "secret" ... create something like the amateur radio community has ... a large question pool (which is published), with a very limited number of questions actually asked.

The reasoning behind it, is for the sake of easily administering tests. Everybody knows the questions involved, and what you need to study is the theory behind it, not the answers ...

Now for something like this, you would have to have a HUGE question pool for each section, just to make sure that nobody knows WHICH question will be asked ... just make sure that you know the theory behind the question.

Obviously, you'll run into problems with programmable calculators being brought in, so a way to fix that would be to supply calculators (if necessary) that have all of the capabilities required.

Anyway ... just my thoughts on the process ...