Slashdot Mirror
Overcomplicated MMO Betas
Heartless writes "On the heels of Vanguard's beta 1 announcement, Heartless Gamer blog has an article looking into why MMO beta processes are overly involved and detracting from the game they are meant to improve. From the article: 'But why even have such a process in the first place? If they honestly think they are going to get any sort of actual *testing* (I use the term loosely) from an over-hyped MMORPG community... they obviously failed basic MMORPG sociology. I could link hundreds of beta leaks and broken NDA contracts, but what would be the point? What you need to know is the fact that betas are infiltrated by those that want sneak peaks at the game. Definitely not by those that truly wish to test the product. Internal testers and paid testers have proved for years to be able to produce very finished products in the single player market.'"
Wow, now we are having bloggers post links to their own blogs to drive up traffic...and to make it worst the thing is hosted on blogspot. I mean seriously, at least get your own DNS and route it or buy some shared spaced.
Now onto the topic at hand, this just shows that 90% of bloggers are talking out of their ass. In this case he really misses the point of beta testing in an MMO. There are several very good reasons and they are reasons that are necessary to test. Later stages in almost all MMO beta test are load tests. They could care less at that point who reports bugs, they just want to make sure the servers don't go kaput when the game launches.
Another problem is you need a large number of people to beta test any MMO. There are literally hundreds of possibilities when a game is in the beta mode (if not thousands) for character development. Look at World of Warcraft. You have to test every race with every classes. You have to test quests at multiple levels, you need to test raids and dungeons. There is a lot more to beta in a MMO then in a single player or even regular multi-player game.
Not to mention the fact that the Beta taste gives you the chance to hook all those players and then start charging them. It also gives you the chance to get free word of mouth advertising. Open betas are a bit more of a joke on other games, but in those situations companies release "demos" to perform essentially the same task without calling it a beta. I have beta tested a regular game and an MMO. Let me say it is much bet to use a community of people in the tens or hundreds of thousands to test your game then to use your few hundred employees with an MMO. Like I said before, the sheer scale of MMOs makes fully closed Beta's an unreliable and ineffective means to test the game.
"Some days you just can't get rid of a bomb."
yup what he said, you shouldnt even do an open beta test if your looking for bugs, you need to have your show stopper bugs nailed down before you ever let the public see your game.
This has been the bane of many MMO's that have come and gone. Pushed to get the game out too soon, rushed into a buggy beta test that reveals nothing but how premature the
game really is, it ruined Earth and Beyond for example, beta'd too early, too buggy, too unbalanced and for too damn long (linke 6 months or more of open beta, ugh).
The reason a MMO game should beta is to test for scalability and load problems with the service as a whole. Stress test the whole system, test the patcher, test the web stuff, test the community pieces (like the forums), test the game servers, test the login servers, test the whole infrastructure that supports the game.
checked for things like memory usage, cpu usage, are you getting the players per CPU that you expected, is it holding up under the load.
its icing on the cake if some players actually report a bug you missed (and you should scold your testers for missing it!).
You also can test out your content delivery system under load (ie new updates, background downloading or whatever your doing).
also it tests your network and game monitoring systems under load, are they reporting what they should, can you tell whats going on with the game, where people are bunching up etc.
It can also help a little with game balancing, people using min/max the hell out of you game even in beta and find the best combinations of whatever is available. something youll need to fix before launch.
you can also use betas to help drive some subscriptions when the game launches with things like (you get a discount after launch, or you get to keep what you have done already) this is useful especially if the beta was nice and short so they arent yet done playing your game already.
Beta is an important step, but it should be used for the above purposes and not as a primary way to debug your game.