Slashdot Mirror
Quality Assurance In The Games Industry
Thanks to NTSC-uk for their opinion piece discussing the perceived lack of 'quality assurance' in the videogame industry. Amid oft-repeated claims that "many games fall short of the mark" on overall quality, there are some more interesting arguments that QA testing "rarely promotes the criticism and fine-tuning of the most important aspect of design - gameplay." The author even goes on to suggest that hardware manufacturers should again get more involved in the quality of games on their machine: "Nintendo demonstrated during the 80s and early 90s how the power of the manufacturer can be used... to ensure that the design of new games, and particularly good gameplay, was top of the agenda - hence Nintendo's 'Seal of Quality'."
There are still a lot of console games released while suffering from major clipping and control issues, so some QA from console manufacturers would definitely be A Good Thing. Gameplay and content, on the other hand, are very subjective things, and things that console makers might be better off not dabbling in. Remember the santised RPG translations that appeared under the Seal of Quality in the NES days? I would hate to fire up FF:Crystal Chronicles and find Holy replaced with White, or bars replaced with cafes, as per the bad old days.
Vino, gyno, and techno -Bruce Sterling
Console and PC development tracks are entirely different. Console manufacturers have stringent controls on what they allow to be published on their platforms. Obviously you can publish anything you want on the PC as you can publish it yourself.
Fatal bugs (ie: crashes) are a lot less common on consoles than PCs. That's because they often must undergo a "burn in" process. For example, Xbox games have to stay running for 7 days - being played - without crashing before Microsoft will send the game to duplication. Similar thing for Sony. This is why there's no Star Wars Galaxies or Planetside on the PS2. The PC versions crash. A lot. Even after patches.
As to the quality of the gameplay itself - there are more people working on today's games than ever before. Teams are larger, movie-scale assembly-line production techniques are being used that simply do not work well for high quality videogames. With many members of the team only focused on one small area of the game, it's far easier to produce a great seething mess of grotesque compromises. If it's just two guys working in a garage - as it often was in the old days of games - they would have a very clear vision.
I believe that great games are formed around great ideas - and keeping those great ideas in sight as the game is built. If a broken idea makes it into production, the game is probably doomed from the start. No amount of QA shine can change poop into gold.
AHHH but Nintendo DOES this (or at least they used to). I remember having a game in submission for final approval by Nintendo and they'd come back to us with a rejection for reasons such as "If you hit the reset button 30 times in a row the game locks up".
Nintendo used to (and I assume still do) pretty rigorous hardware testing, code analysis (since there's certain things such as certain cpu registers you shouldn't access) etc.
It's the bad gameplay that often gets ignored.
For awhile there, console games were relatively bug free. On the Super Nintendo, Genesis, and to some degree the PlayStation, there were very few crippling bugs that every single person that played the game had to watch out for while playing. Lately, this has changed. Enter the Matrix was riddled tons of different bugs. Tomb Raider: The Angel of Darkness was filled with situations where Lara would get stuck in a jump and you would have to reset. Jak & Daxter had a bug that randomly occurred in the middle of the game that kept you from fully completing it. Knights of the Old Republic's bugs are just as infamous (though not as numerous) as Enter the Matrix's. Star Ocean: Til The End of Time, a best-selling Japanese console RPG, had crippling bugs. Yu-Gi-Oh! World Wide Edition is filled with bugs toward the end of the game, including a relentless crash bug that causes the game to crash during almost any battle with the final boss.
These are some of the best-selling games in their respective countries and consoles, but they're riddled with software bugs and glitches that, in some cases, ruin really great gameplay ideas. And these are just the ones that are popular! Play any of the less popular GBA titles, such as Megaman Battle Network 3 or the GBA port of Super Puzzle Fighter 2 Turbo, and you'll find many more of those GBA crash bugs.
We've actually gotten to the point where games made by veteran game developers like Capcom, Shiny, and Konami that have been certified by NINTENDO (of all companies!) are riddled with crash bugs, so I think gameplay is the least of our worries at this point. If you can't even play the damn game, then the gameplay doesn't really matter much.
(And as a brief side note, some of the practices that the article mentions have already been standard at Sony for years. Sony Computer Entertainment America has wielded its broad monopoly in the United States to keep what it sees as "below average" Japanese PlayStation and PS2 games from entering the US. Some notable victims are The King of Fighters 2000, a Metal Slug title or two, a Persona game, and Goemon.)
The Nintendo Seal of Quality wasn't worth the paper it was printed on. It was nothing more than a Nintendo Fee, and promise to promote the game in Nintendo Power. Ever read a Nintendo Power review? Ever hear them trash a game? Were all of the games good? Nope. Nintendo Power was just propaganda. There were plenty of great games that came sans Seals.
But feel free to step up now, i hear SCO is selling Seals of Quality for all the major Linux distros.
As someone who works in software QA, all I can say is thus:
If QA has the ability to block the release of something due to defects then this is an almost absolute way to ensure quality (other factors notwithstanding).
If QA doesn't have to give their seal of approval before something goes out the door, then things will be released with defects (some known, others not).
I was fortunate enough to work for a boss who stated to development and the project managers that he would not sign off on releases simply to meet deadlines. If the powers that be wanted something shoved out the door simply to meet customer expecations, they'd do so without QAs consent - and that we'd not take the blame.
Cruising the internet on my TI-99/4A @ a whopping 300 baud!
Anecdotal evidence doesn't count for much, but it can be entertaining, so here is a the "description" from a genuine bug report from a major US publisher, for a game I was working on a while back (not published):
This was submitted as a "class A", "In-Game GUI" bug.
It's a cut and paste. No typos introduced in the retelling.
Thad