Slashdot Mirror

← Back to Stories (view on slashdot.org)

Mathematicians Deconstruct US News College Rankings

Posted by timothy on from the deep-springs-college dept.

An anonymous reader writes "US News makes a mint off its college rankings every year, but do they really give meaningful information? A pair of mathematicians argues that the data the magazine uses is all likely to be at least somewhat relevant, but that the way the magazine weights the different statistics is pretty arbitrary. After all, different people may have different priorities. So they developed a method to compute the rankings based on any possible set of priorities. To do it, they had to reverse-engineer some of US News's data. What they found was that some colleges come out on top pretty much regardless of the prioritization, but others move around quite a lot. And the top-ranked university can vary tremendously. Penn State, which is #48 using US News's methodology, could be the best university in the country, by other standards."

8 of 161 comments (clear)

  1. Reputation by Reality+Master+201 · · Score: 5, Insightful

    A college degree is an education, and that should be of paramount concern. It's also nice to be in a place you'd enjoy living, etc.

    But then there's reputation. You might get the same education at CMU and MIT, but if you're looking for jobs, all other things being equal, someone's gonna pick the MIT grad because it'd a bigger name. I realize it's variable across fields and with individuals, but names mean something to a lot of people, particularly when they're not really qualified to judge on merits.

    1. Re:Reputation by Ethanol-fueled · · Score: 5, Funny

      George W. Bush graduated from Yale.

    2. Re:Reputation by the_weasel · · Score: 5, Insightful

      And he became president. That's a pretty clear success for him. :-)

      --
      - sarcasm is just one more service we offer -
    3. Re:Reputation by Abreu · · Score: 5, Interesting

      Something very weird happens here in Mexico.

      According to several international studies*, The National Automonous University of Mexico (UNAM), and the National Polytechnics Institute (IPN), the two largest public universities in the country, are the best institutions of higher learning in the country.

      Yet it is very common to see "UNAM, IPN, graduates need not apply" in job listings. Why?

      Because employers seem to believe that the networking and prestige of the exclusive private schools are worth more than being a graduate of the two institutions that generate 90% of the scientific research in the country!

      Sources:
      http://www.topuniversities.com/worlduniversityrankings/results/2007/overall_rankings/top_400_universities/
      http://www.arwu.org/rank2008/ARWU2008_TopAmer(EN).htm

      --
      No sig for the moment.
  2. Playing the numbers by timholman · · Score: 5, Interesting

    Several of the metrics that U.S. News uses do seem to be arbitrarily weighted, leading to some bizarre contortions on the part of the various schools to enhance their ratings. Most of the data is self-reported by the universities, which clearly provides a powerful motivation to spin or "enhance" the numbers to one's advantage. I have no doubt that several colleges fudge the numbers to raise their rankings, leading to a lot of frustration at other schools that are playing by the rules but feel that they're being cheated in the rankings.

    And some of the metrics make little sense. For example, engineering schools can raise their rankings by several places just by having one or more faculty members in the National Academy of Engineering. Yet NAE membership is essentially meaningless in terms of research and teaching, and hardly more prestigious than having faculty members who are Fellows in other established engineering societies. Yet U.S. News ignores the number of Fellows in IEEE, ASCE, ASME, etc., and focuses on NAE membership. So why the emphasis on NAE? Probably because the NAE told U.S. News that they were the most important engineering society, and U.S. News never questioned it, when in fact the NAE has almost negligible impact on higher education.

  3. link to original paper by Boghog · · Score: 5, Informative

    Selecting universities: personal preference and rankings

  4. Rankings by Anonymous Coward · · Score: 5, Insightful

    I've actually played with ranking data quite extensively, and usually for reasonable weighthings of the parameters the movement in position is in the order of plus/minus 5 places. Sure, Penn State would be number one if all one cared about is retention rates, but really nobody does. Instead we can define a range of reasonable weights for retention rates (say between 7% and 35% of the total weight) and test all possible combinations in that space, suddenly Penn State place goes up and down a fairly small amount.

    A bigger concern is what is the value of selecting a school based on the ranking as a whole, without paying attention to the your likely area of major. Say, Yale is a great school but in CS is a non-entity. If you are positive CS is your thing, MIT, Stanford, Harvard and Princeton are far better choices.

  5. That's actually pretty sad... by DeadDecoy · · Score: 5, Interesting

    That's actually a pretty sad indication of how much a college name matters over what you do. I had a friend who did his undergrad at MIT, and when applying for jobs, he was insta-accepted to various tech jobs. No interview. No background checks. Just an open door. Given that, he refused those jobs because that easy entry gave him some indication as to who the companies hired and on what criteria. On the other extreme, were a couple of people who had to work twice as hard because they had to sell the college they attended. It's a little sad, but it's the reality.