Slashdot Mirror
Clemson Staffer Outlines College Rankings Manipulation
xzvf writes "A disgruntled Clemson University staffer shows how US News and World Report college rankings are manipulated. Techniques include bad-mouthing other schools, filling out applications from highly qualified students that never intended to apply, and lying about class size and professor salaries." The school, naturally, denies that anything unethical went on. The New York Times has a more detailed article, which links to this first-person account of the presentation.
Raise your hand if you are surprised that this is going on.
Seriously, with all the incentive to attract and hold onto students and the funds they bring. Who would have thought that this is all above board and regulated?
http://www.google.com/search?hl=en&q=college+rankings+corruption+&aq=f&oq=&aqi=
It's not like this is new.
) Human Kind Vs Human Creation
) It'd be interesting to see how many humans would survive to serve us.
water is wet!
Of course they cheat. They have to. If they don't know how to cheat then how can they catch the students when they cheat so they can cheat better and better so they can cheat into a job where others learn to cheat from them! /cheat cheat
Any time an important ranking system is devised, those being judged will figure out how to cheat the system. Given how important these rankings are perceived to be, this should be no surprise to anyone. I am more surprised this is a surprise.
20 characters max for the password? How will I use my favorite poems as passwords?
How is this surprising? It's difficult to fact-check a lot of this stuff, simply because there is no uniform way to measure it. It's like contrast ratio and response time for LCDs. Does anyone actually base their college choice on these rankings, anyway?
I am SHOCKED that anything unethical would go on in academia, especially with regards to admissions and maintaining image.
Surely this is all bullshit and academia is focused on teaching students, not patting themselves on the back and striving for U-peen and the subsequent moneys.
If there is a way to monkey with the rankings, schools will do it. USNews rankings are taken seriously enough where they should really improve their methodology so that it is at least more difficult to cheat.
Simply comes back down to schools are a business, even the ones that get funding from their State. Higher rankings means more attending students, and the ability to raise their prices and get more money. Plus there's the application processing fees, registration fees, and all the other fun BS. Who wouldn't expect them to bullshit their information to get more people to apply?
I've always wondered about these rankings. If you get into Harvard, aren't you pretty much going to go there, regardless of whether it is 1st, 2nd or 15th on the school rankings. And doesn't the same go with most Ivy League schools, as well as schools like Stanford, MIT, and a few others.
And by extension, it seems likely that these schools get the lions share of the best applicants... again, regardless of their rankings that year. And on top of that, aren't all the best professors trying to get jobs at these schools?
Maybe these rankings are more helpful to deciding where to go to school once you get past the top 20 schools, but based on this news I doubt it.
Back in the late 80s, Georgia Tech would have any incoming freshmen with lower high school GPAs start in the Summer quarter. This was under the auspices of giving those who were struggling, a bit more time to adjust to college curriculum before the incoming fall crush.
The interesting "side effect" was that the GPA of incoming Fall freshmen was thus higher, and the university had no trouble repeating that fact.
+----------+
.\|.||/..
| FIX YOUR |
| FUCKIN' |
| CODE |
+----------+
| |
| |
One alternative is to bow out http://web.reed.edu/apply/news_and_articles/college_rankings.html of the rankings game and take a principled stand as Reed College has done. One way of thinking about attending a fine school like this is that you "want to go to a school that isn't interested in selling out its education." Perhaps not surprisingly, US News didn't actually remove Reed from the rankings, they just ranked Reed (lower) with an incomplete data set. The other alternative could be called 'open source' ranking. The University and College Accountibility Network http://www.ucan-network.org/ ranks colleges in a common format, has useful information, and best of all, you don't have to buy a copy of US News to get the rankings!
About 20 years ago Playboy Magazine picked MIT as one of the top ten party schools. Rumor was that Playboy called some random dude on campus who listed out all the parties happening that year, making it sound like they were all happening that weekend.
I feel badly for all those kids who chose MIT because of its top-ten Playboy ranking, only to go and find a bunch of nerds, forever regretting not going to Clemson instead.
This brings to mind an article I read way back in Inc magazine where the author talks about how employees will figure out how game any system that rewards them.
http://www.inc.com/magazine/20081001/how-hard-could-it-be-sins-of-commissions.html
Clemson is just gaming the system, I imagine other schools that change quickly change their ranking probably are doing the same. Even if US News and World Report changes their ranking methodology, I guarantee that schools will simply change their tactics to beat the system agian.
bleuchez!
I've been working in academia for years, and gaming of the USN&WR rankings is hardly news to us. Talk to any college administrator off the record, and he or she can rattle off the names of peer institutions that are almost certainly fudging the numbers.
The USN&WR numbers are self-reported by each university, with no verification by the USN&WR staff. With so much funding and prestige riding on the rankings, who is surprised that some schools play fast and loose with the facts?
What is unfortunate is that USN&WR has manipulated itself into the position of being the arbiter of school "quality", through no other action than being the first to create the poll. A news magazine shouldn't have that kind of influence over the entire U.S. educational system, especially when it can't even be bothered to check the numbers that it publishes.
Mostly, it doesn't sound to me like they did anything wrong.
They raised admissions standards. They lowered the student-to-faculty ratio from 16 to 14. They raised faculty salaries (and also changed the definition of salaries to fold in benefits, which apparently is allowed by U.S. News, so it was simply a mistake not to do so previously). These are all things that you would absolutely expect a school to do if they wanted to improve their academic reputation.
They seem to have good results to show from their efforts. "[...]the retention rate of freshmen has climbed to 89 from 82 percent and the graduation rate to 78 from 72 percent."
They did increase the number of large classes at the same time that they increased the number of small classes. TFA claims this was done in a cynical effort to match up their numbers with the exact criteria used by U.S. News. Maybe so, but it's also just an ordinary thing that big state schools have been doing for a long time. When I took freshman chemistry at UC Berkeley, they had 300 students in the room, but a lot of my other classes only had 20 or 30. That's just a normal way to make the school's money go further. The fact that the over-all faculty to student ratio did improve shows that they weren't just cooking the books.
This one seems bad. However, she hasn't provided any evidence for her claim that it was widespread.
What really seems to upset a lot of the people quoted in the article is that they perceive Clemson as getting more elitist, which they thing is not appropriate for a land-grant university. Well, "elitist" is what you become when you join the elite.
Find free books.
This proves rankings such as these are complete garbage.
My College was always top on a list of Colleges that the highest percentage of alumni donating to the college after graduating. The rankings would score a college or university based on what percentage of alumni donated back to the school the first year after graduating.
My College found the simplest way to manipulate that index. Just have every single student who graduates donate one dollar back to the school and then find one or two students with extremely wealthy parents (this was not hard at my school) and have them donate thousands and thousands of dollars. This way the school would report absurd figures like "90 percent of students donated back to our school within the first year of graduating from our undergraduate program" and it would make the school look good and it would make the degree you just got look a little more prestigious. They never told the index that we only donated a dollar and were instructed to by some of administration.
And with the few giant donations from one or two individuals, the school could artificially say that the average donation was way higher than typical, while hiding the fact that it was offset by just one or two massive donations.
Other ways to cheat is hiring adjunct professors or part time professors under different titles like 'technician' or 'consultant'. This makes the percentage of full time faculty and professors look way higher than it actually is because the school hides its adjuncts under different titles. Another way they cheated the system was renaming classrooms as different titles. One of the rankings is how many classrooms on campuses have TVs/projectors/computers and if you hide the classrooms without those your percentages increase in your 'technology' score as well.
If I think of any more I'll them but these were the ones that came to mind immediately.
GO COCKS!
Not to spread doom and gloom but academia has been like this for a very long time.
Colleges and universities are struggling internally. On the one hand schools have to generate revenue which requires advertisement, marketing and "looking" better than other competing schools. On the other hand the primary roles of universities and colleges in society are to increase societies overall intellect and be a lightening rod for research, learning, and understanding.
The internet offers free access to knowledge and is stealing thunder from individual academic institutions. For example, I can communicate almost instantaneously with authors, researchers and professors and get an answer in most cases. I can view lectures and get materials on most subjects. Most educators/professors have blogs and some have tweets. We are not as dependent on academia to facilitate intellectual communication as we once were.
I have compared academia in the US, especially the Ivy League schools, to the luxury car industry. The information rumored in the original post enforces the legitimacy of my comparison. I recently read an small article on luxury vs performance that kinda applies to this topic.
Luxury is about appearing better. Performance is about being better.
The way things have worked in the past and probably still do is the first step is having numerous associations that rate colleges.In some rating systems a dead cat could look better than Harvard.
Another nifty trick is an accredited university that has some departments that are not accredited at all. Graduate engineers have been shocked to find out that their credentials carried no weight as they had graduated from an unaccredited engineering college that was part of an accredited university.
Suspect tactics are placing weight on the SAT scores of entering students and grade averages of entering students as opposed to looking at college boards for years other than the freshman year. Placing undue weight on the quality of the school library or the investment in lab machinery and supplies is also part of the corrupt rating association game. The most likely to play such games are small private colleges with big reputations that charge a lot of money to students.
There maybe a few good, honest educators. But overall, it's just a business.
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
Even if you're not going this far.... the business school at Wake Forest University a few years ago suddenly became a lot more selective and shrunk the number of people it would accept. The idea, I believe, was to increase the standings in various rankings. Of course, there were side effects of this, such as the economics department being flooded with people who didn't make it.... and it's not really good for the university as a whole, either... or "education" in the abstract.... It's going to look real good on someone's resume, though.
Typical principal-agent problem at work.
(Then there's the "omg new logo" debacle... gaak... and you guys wonder why I don't give you a $5/yr pittance to "improve your ranking" in the alumni-willing-to-give category)...
The World Wide Web is dying. Soon, we shall have only the Internet.
At no point in any of the three articles did I see anyone accused of "lying" about class sizes or professor salaries. The number of classes less than 20 people actually did increase--at least partly by bogus 'load balancing'. And the professor salaries increased, both by raising them in reality and because the old reported numbers didn't include benefits (as they should have).
I also couldn't find the source for the claim about filling out fraudulent applications, though it's possible I missed it.
None of this is to defend the ranking gaming, but the summary gives an extremely different picture than I got from the source material, which mostly is in the category of 'administering to the test'
So, dues this constitute criminal fraud by Clemson? It sounds like it was used to like to students and their parents regarding what their tuition was getting them.
I'm at a University in the UK. There are many students here on the MSc Computer Science scheme that can not program. Any language. At all. One of the group-work programming modules has been altered this year, so that rather than programming a solution, students can use Access / Excel / Word to produce the prototype of their 'system'. And as students might not find that easy, rather than do a presentation demo-ing their work, they can instead videotape the demo, allowing for smoke and mirrors tricks.
We have students with 80%+ plagiarism according to TurnItIn, and being let off with a slap on the wrist due to "cultural differences".
We have a Professors of Multimedia who can only code a web page when using Word 2003, and requires help opening Visual Studio solution files. We have lecturers who write down which files need to be moved to which folders, because they have yet to master drag and drop in Windows...
On the other hand, we have lecturers who are experts in their field. We have some young, highly knowledgeable and enthusiastic lecturers who know their subject inside out, yet don't for a second come across as arrogant. Who continue to be told "If we achieve less than 80% pass-rate on your module, it's your fault for not teaching the subject properly" despite the powers-that-be allowing students onto the degree who clearly have no skill with computers, whose only contribution to the School is 3x the normal annual fee.
The good lecturers get more work put upon them. The crap lecturers have told the-powers-that-be to sod off... so they are not given extra work any more. This gives them more time for leisure, recreation, and outside pursuits.
Sorry to rant... but for all the crap, I do love my University. I just wish the Executive, the Dean, the Associate Deans, and the Senior Lecturers cared enough to do something about it.
They don't.
This seems to be what happens when you introduce greed into a system. If education was free and universities were more specialised it may reduce this, still, the greed factor will always affect the system.
Maybe I'm too altruistic and this clouds my judgment of others, but I'd like to think that if there was equality of education there'd be less chance of greed in the system.
If education where free? You do know that there is no such thing as a free lunch? You have to pay teachers, administrators salaries and benefits, and that money has to come from somewhere. In the case of people how advocate for 'free' education, this inevitably leads to the government providing the education, and the government has to get that money from somewhere, and that somewhere is called taxation. Which again, does not make it free, it just appears to be that way.
If the US News & World Report model actually captures good things about a university, then what's wrong with attempting to match that model?
That a university tries to match what it considers a good model shouldn't be surprising. The validity of the model may be questioned. The methods to match the institution to that model may be questioned. But I don't see how attempting to get better under some model they consider good (by whatever criteria they pick) is bad.
I don't know enough about it to know if the USNews model is any good - maybe, maybe not. But I know that institutions I'm generally familiar with land about where I might expect in the rankings. Ivy leagues on top, small underfunded state colleges much lower.
Now, the claim that Clemson administrators purposefully rank other universities lower, that's a different matter. That is the most troubling claim to me in the whole bit. That action is highly unethical and I would be sorry to find out that it is true of Clemson, or anywhere for that matter.
Cheating is in their blood...
(I kid, I kid... maybe :) )
I just remembered this one; rumour has it that the WFU Business School hired anyone who couldn't get a job last year so they could put out some BS about how all their graduates got jobs even in These Turbulent Economic Times (tm).
The World Wide Web is dying. Soon, we shall have only the Internet.
Just out of curiosity, I have to ask:
Did the 'Disgruntled Staffer" happen to work in the mailroom?
Knowing Google's lust for data collection, the Soviet Union is still alive and well inside the psyche of Sergey Brin....
Yup, I get you. I completely understand why my generalizations would annoy some people (a lot of people). I guess you could call me cynic, but I just feel like in most cases, people care an awful lot about things like pedigree and prestige. So in real world terms, I feel like I made a pretty fair generalization.
In a perfect world, college applicants (and their parents) would think about what the right learning environment is for the student. They would look at which institution will care for and nurture intellectual and personal growth.
And we haven't even started talking about tuition costs. Ivy League is supposed to be need-blind-admission, but I know of a lot of middle class kids who get in but don't qualify for financial assistance. Those kids are in a very tricky middle ground of being neither rich enough nor poor enough to afford the school.
Mel Elfin pretty much let the cat out of the bag. When asked how he knew that the U. S. News and World Report rankings were sound, he answeredthat he knew it because Harvard, Yale, and Princeton always landed on top.
In other words, the rankings are simply a way to give the trappings of science and objectivity to a system whose purpose is merely to reaffirm the conventional wisdom.
"How to Do Nothing," kids activities, back in print!
So the students can go to the game on Saturday, go hunting on Sunday, and pick up trash along the highway on work days.
Of course, everybody cheats. But what university trained the geniuses that managed to be caught cheating?
CLEMSON!
http://en.wikipedia.org/wiki/Wellesley_College
Wellesley College is a women's liberal arts college, in Wellesley, Massachusetts, that opened in 1875.
One of the reasons Reed College, of Portland, Oregon, decided to stop participating the various ranking systems available. I'm surprised more student bodies don't protest at their own school to shame em into quitting this arbitrary popularity system.
I went to the school that has been at the top of the list for ~9 years now.
Everything was swept under the table. Not a single drinking or drug incident made it to the local news, magically.
There was a guy my freshmen class that got caught dealing. Not only did nothing happen to him, nothing ever made it to the papers. I don't even remember hearing of any on campus discipline. There were never any parties broken up by police, and the one or two that were, they also never made it to the paper.
When I transferred to a Big 10 school, that was in the same state (That's also >10x larger). There was none of that. Freshmen getting kicked out of school first week for possession. It was so integrated into the town so the town cops had their hand all the underage drinking and PIs. They all made it to the paper. They're consistently in the top 10, but not #1. (Of a different list).
*Funny thing that when I started to interview. I got asked if I went to a technical college for 2 years. Just because you're #1 in Engineers minds doesn't mean you're #1 in HR's. As soon as I dropped them from my Resume I actually started getting callbacks.
As a humanities major, I may be off on the math, but if you increased the number of $1 donations, then you would need increasingly large donations to increase the average since the $1 donations would drag it down (assuming they are using "average" to mean "mean" and not "median", and also you'd need a relatively small graduating class size). So the only shady thing that has been done is the $1 donations, but they should be congratulated for the increase in donations at the high end.
Every post I make begins with the assumption P=~P.
College ranking is as meaningless as measuring an average temperature of patients in a hospital. You can rank swimmers, chess players, runners. But giving some abstract value to an educational institution ?!
OutputLogic
after reading the "first-person account", i'm really lost.
i imagine sitting through the talk and then hearing the question and answer session that followed.
just going by that first person account, it seems like during the talk we hear what doug lederman puts as: ...where those more questionable things seem to, according to doug's rendition, include rolling the benefits into the salary.
"Watt continued, as she described what she called the "more questionable aspects of what weâ(TM)ve done.""
it sounds like in the questions at the close of the talk people were pretty interested in these questionable things. according to doug, it seems that when the presenter was pressed for details "Watt said that the university had added benefits to its faculty salary reporting to U.S. News after previously having failed to do so, as the magazine requires."
so at what point does reporting the salary the way you are supposed to report it enter the box of "questionable practices"?
were there other things she was alluding to that failed to be highlighted? seems like those, if they exist, were the important points. ...so is watt the moron for presenting one thing and backsliding in the questions or is doug the moron for giving us a summary of this backsliding talk?
are we morons for wasting time reading this crap?
or is it just me... ?
am i the moron for thinking this is all garbage?
so university looks at the U.S. news model and conforms to that model. that model is what describes a top school. (they say) if your boss gives you a model for a top employee, you might conform to the model. you want that raise don't you?
Here's some shit law schools have done:
Last year, Berkeley (#6) sent fee waivers to a ton of underqualified students. Students who would have never applied to Berkeley because applications cost money to submit. (Hence the fee waiver.) Underqualified students apply (because why not? it's free) and get rejected. Berkeley artificially deflates their acceptance rate, which helps their ranking score. This is likely done by a ton of schools. I just know of Berkeley doing it.
Another factor that affects LS rankings is the offer acceptance rate (basically, how many students who get accepted elect to attend that instutition). Schools will frequently reject obviously overqualified candidates because "they'll decide against going here and attend Yale, Harvard, Stanford, Chicago, Columbia, NYU, etc. instead." Thus, qualified students are rejected for being "too qualified."
Finally, schools like Georgetown (GULC, #14) used to admit a ton of transfer students and part-time students. Neither transfer students nor part-time students affected the LS rankings. Thus, GULC basically could accept many less qualified people, extract $100K from each of them over the next two years, use these extra millions of dollars to entice very qualified candidates to attend with generous scholarship packages (full rides and the like). Because these transfer and part-time students didn't affect the rankings, GULC was effectively using a money-generating machine to attract very qualified candidates who may otherwise have attended a more highly ranked school like Chicago. However, this year, the USNWR started including part-time students in the rankings. Transfers still aren't included.
Of course, the question remains: Does this matter all that much? When a law school like Yale or Harvard has so much money and prestige to leverage to attract the best students even if the students won't get a better classroom education there, aren't other schools equally entitled to game rankings that, at the end of the day, are pretty much bullshit anyway?
Look, I attended a top law school, but I'm willing to acknowledge that the rankings are almost completely meaningless outside of job prospects. The rankings do create some sort of "job prospect tiers." But aside from that USNWR rankings are crap (at least in law, I don't know about other fields).
One fallout from the rankings is that lots of people, myself included, get badass scholarship deals for high PSAT scores.
In short: a high PSAT score can earn you the title of "National Merit Scholar", and the number of NMSes attending the school factors into the college rankings. Schools looking to make a serious entrance into the rankings, therefore, offer really nice scholarship packages NMSes.
I'm not sure how I feel about it. The rankings might get gamed a little, but...it ended with me getting over half my college education (including room and board) paid for. My sister's going to get all of her college expenses paid for.
For being "academic" institutions, there really aren't very many academic scholarships to college anymore, comparatively. Pretty much everything has some percentage of need-based judging. The idea is good, but in reality there are a lot of us whose families can survive alright but really can't afford college for multiple kids, and we just don't qualify for need-based scholarships even though we need the scholarships. The PSAT-based scholarships made it possible for my sister and I to get bachelor's degrees...so I'm not really sure how I feel about it.
http://gamecocksonline.cstv.com/genrel/092205aaa.html
There is a roll of Clemson diplomas down in the men's room.
If Slashdot were chemistry it would look like this:Cadaverine
Boston University does this with their College of General Studies. CGS is a two-year program (basically an Associates Degree) and when you finish you go straight into the regular university. Essentially, incoming students with poor high school grades are sent to CGS, and this college is conveniently left out of ranking calculations. It's a huge cash-cow for them, as well, since most CGS students aren't receiving financial aid.
There is a company Academic Analytics http://www.academicanalytics.com/ that, for a price, will search the web and provide rankings to "help" Deans and Boards of Trustees to evaluate their graduate programs. The "better" programs have all ready been accused of search engine optimization. I guess it could also be seen as an opportunity.
At least with US&WR we all know how much they know about quality.
The other gripe I have is that they tell everyone "2 years of GERs to figure out what you want to do, then major in anything you want". That is unfortunately a straight up lie. I talked to a music professor and he said that to be a music major a person would have had to declare freshman year, and would have had to play since late middleschool. He said it would be completely impossible for any sophmore to get a music degree. Also, other majors like CS, and Chemistry are either very linear, or only offer a few courses each term, so it takes at least 3 years to complete the major.
There are 11 types of people, those who know unary and those who don't.
I have enough trouble with the 400 and 500-level courses!
I listen to both RIAA and non-RIAA stuff if I like the music, tangential business/politics nonwithstanding.
On class sizes, the way Clemson "manipulated" the data was by... um, actually changing their actual class sizes. They made their smaller classes smaller and let their bigger classes get bigger, because US News uses thresholds of 50 in evaluating class size. Sure that helps their numbers... but it's also not a bad thing from a pedagogical point of view. With a discussion-oriented seminar, reducing below 20 makes a real difference. And with a big lecture, 55 versus 100 is not that much of a difference. So they might have actually improved their delivery of education.
I have first-hand knowledge of this as I taught in a very large department at Clemson during grad school in recent years. In several courses that were taken by almost all majors, the department capped the individual class sections taught by grad students at 19 and created a lot of them (those teachers are cheap!) and then the ones taught by a prof were allowed to be huge. We aren't talking about different courses here, we're talking about sections of the same course in the same semester.
The measure of "percentage of classes with fewer than 20 students" is not a good measure. Instead they should look at "average class size", which is harder to manipulate.
Stanford University has been "gaming the system" for years. The following excerpt was published almost a decade ago:
Despite its high US News and World Report ranking, Stanford has significant problems. An article in the Stanford Report by Ray Delgado (published May 19, 2004) admitted that Stanford's faculty members are apathetic towards undergraduates:
Five years later from that report, nothing seems to have changed:
Complaints by students have been published: