A Look At Competitive Ranking Systems
Christopher Allen writes "Competitive ranking is used in sports, chess, and other games, and by online services, such as the new TrueSkill system used on Microsoft's Xbox 360 Live service. Rankings establish who is best, create fair competitive matches between players, and handicap players with differing skill levels. An article at 'Life with Alacrity' discusses a number of approaches to competitive ranking on the internet, each with different issues and advantages."
Screwed up that last link. It should say:
Note: A related problem is Judging Systems, where players rate in-game works of art. We've tried a number of algorithms there, and just recently have come up with one that seems to work.
Yeah, I known, "preview button", blah.