Slashdot Mirror
Overcomplicated MMO Betas
Heartless writes "On the heels of Vanguard's beta 1 announcement, Heartless Gamer blog has an article looking into why MMO beta processes are overly involved and detracting from the game they are meant to improve. From the article: 'But why even have such a process in the first place? If they honestly think they are going to get any sort of actual *testing* (I use the term loosely) from an over-hyped MMORPG community... they obviously failed basic MMORPG sociology. I could link hundreds of beta leaks and broken NDA contracts, but what would be the point? What you need to know is the fact that betas are infiltrated by those that want sneak peaks at the game. Definitely not by those that truly wish to test the product. Internal testers and paid testers have proved for years to be able to produce very finished products in the single player market.'"
In that order. You don't get many useful bug reports from the large betas (you do from the smaller stages), but you don't know how it will handle release type number of users until release, unless you beta it.
I still have more fans than freaks. WTF is wrong with you people?
I understand load testing... hence why I pointed out that is what the MMORPG developers need to do. Skip all these baby step "only a few in at a time" elitist testing phases. Drop 1,000 players in a day and keep going until the servers don't work anymore. The whole process of selecting a few select beta testers only wastes time and resources which could be better placed into the back end bug reporting systems or fixing the actual bugs themselves. Darniaq pointed out something on his blog that I totally agree with about the actual bug reporting features.
It's like giving out free maryjo samples.....a dealer is bound to get some instant new business. Heck, they could get a commercial on every TV show for free and it wouldn't be near the value of 1000 potential customers getting a taste. And not a taste with 1 million others but a very tight availability taste.
Also, as a programmer, I can say you can unit test till you are blue in the face, but it only takes a user 5 minutes to find a bug you'd have never tested for.
If only one major bug is caught that's huge icing on the marketting and catching customers angles.
But I think there is one factor we overlooked...ego. If you'd spent years developing a new toy, you didn't do it to keep it locked up for yourself. You did it for someone to play with. I'm betting the rush of first players digging into the new toys you rolled out is huge! and heck, if it sucks, at least you only had to deal with the a small amount of laughs the first day. Ha!
Be swell
As someone who has worked QA on a variety of SOE games and have had to read through player-submitted bug reports, there are FAR fewer than 1% of the player base submitting bugs in the first place, and the ones who bother cannot be considered "decent QA testers." Someone else mentioned it earlier... the beta phase is merely to hype the game. It's incredibly rare that players, even those with legitimate bugs, submit reports that are useful for QA purposes.
Actually some MMO players generally UNDERESTIMATE what hours they play, when and how powerful/weak their PC rigs are. You have to consider the users who stay logged on 24/7 'because they can' or because they run AFK player-run shops or whatever. If the servers stay up 24/7, theres someone out there who will stay online 24/7.
How much work is it to review countless beta applications? I have no solid numbers, but there is no way they can convince me that it doesn't take away from the game development.
Plug data into an Excel worksheet, randomly pick X number out and thats Group 1. Sort the data by 'average time spent playing games' and thats Group 2. Etc, etc, etc. Any programmer who can write a MMO game can write a program that can automate this.
The idea of NDAs is also hard for me to understand. World of Warcraft had no problem without one.
Actually, WoW didn't have a NDA because that wasn't a beta. It was free marketing. And the 'closed beta' prior to it DID have a NDA. Given how poorly the servers did at launch day, the fact that they SERIOUSLY did not take that last 'beta' seriously is clear.
WoW beta only suffered from too much interest, but Blizzard did a remarkable job of eventually getting 500,000 testers online.
Bolding by me.
Sigil will be balancing this game as any other MMORPG... over time!
Yeah other MMO games like WoW and FFXI tried before and nearly killed themselves. WoW's player run economy is a joke since anyone who can use a keyboard could craft and all but the most extreme basic materials were sold by NPCs. On top of that people were hitting the level 50 limit within weeks of the games launch and the best equipment is all obtained from time consuming, pro-hardcore instances with drop rates on par with winning the lottery. Its the exact opposite with FFXI. Leveling up in FFXI is considered to be the worst 'grind' in out of every other MMO out there, crafting is a near impossibility without learning economics 101 and accounting 101 not to mention the Chinese 'gilfarmers' screwing around with inflation. The best equipment either costs more money than your day job's wages or is dropped from an impossible to solo monster and probably has a bad drop rate.
The general rule of thumb for MMOs is that time does not cure all. Give WoW a year and people will be bitching about lack of things to do after hitting level 60 on every class and race combination. Give EQ2 a badly designed economy and one year later SOE will start acting desperately... oh oops, that already happened.