Slashdot Mirror
On World of Warcraft's Network Issues
alphaneutrino writes to mention a C|Net article discussing some of the recent problems the World of Warcraft playerbase has experienced. From the article: "'Being a system administrator myself, I have some understanding of what goes on in a corporate data center,' said Evgeny Krevets, a sometimes-frustrated WoW player. 'I don't know Blizzard's system setup. What I do know is that if I kept performing 'urgent maintenance' and taking the service down without warning for eight-hour periods, I would be out of a job.' Blizzard blames some of the problems--such as the disconnection, for several hours on Friday, of players linked to several servers--on AT&T, its network provider. (AT&T did not respond to a request for comment.) "
if I kept performing 'urgent maintenance' and taking the service down without warning for eight-hour periods, I would be out of a job
The difference is that Blizzard sees itself as already having it's customer's money. Therefore, there's no reason to spend any more for service. Your boss needs the network up just to make money.
Free, not Pay per month, and as long as I have played it, only 2 spots of down time in 6 months. I guess WoW has many things one upped on GW, but still.
"God of Rock, thank you for this chance to kick ass. "
Oh yeah, that monthly fee is totally going towards maintenance costs, just like they said. That much is apparent.
Seriously, I still can't believe how easily people took to paying monthly subscription fees to play games that already cost $60 and, without paying the fee, are completely useless. It's kinda like giving cold, hard cash to a charity. You have no idea where that money is going, and you sure as hell can't trust Blizzard's PR department to give you the whole truth.
I stand fast in my assertion that I will not pay a monthly subscription to play any game except under one of two circumstances: 1) the game must have an equally fun single player mode (and it better be damn good), or 2) the game itself is free, and the monthly subscription is the only cost.
Call me anal, but it's bad enough when I pissed half my college years away playing Diablo II online for free. I don't see the point in having to pay for the privilege to waste my time.
"You will pay for your lack of vision..." - Emperor Palpatine to Ray Charles
I'm not a WoW player but if it's true that these systems regularly go dark for 8 hours at a time I have to wonder if they're not racing through some software patch. In other words, I don't know an architecture out there that can't be rebooted in 8 hours so a straight-up crash seems unlikely. I would assume they've taken care of scalability problems by now so system load / tablespace, etc, ought to not be an issue.
... and some succeed, requiring a quick patch to the code base? I wouldn't doubt that they have monitoring mechanisms in play which detect unreasonable changes in a character's level / gold, etc.
Could it be that WoW suffers constant attempts at subverting the framework of play
CommentBot 0.7a running with args "-module irritate,disagree -target random"
Just like ISPs are and held to accounts for it. When I played WOW they were good about refunding for extended downtime. Yet at no time do any of these pay to play games make any guarantee of service availabilty.
As far as their continuing stability and growth issues.
STOP SELLING THE DAMN GAME.
Sheesh, how hard is that to understand? If you cannot provide a stable set of servers and servers where people can play WHENEVER they want to then stop selling new copies until otherwise.
Hopefully with the number of professionals playing the game one of them will get annoyed enough to sue them in court, either to force a change by ruling or just having their named dragged into the mud.
* Winners compare their achievements to their goals, losers compare theirs to that of others.
Our game had its server problems and we were in "learning mode" to deal with some major outages, major gameplay renovations, major strife from jerks, and major socio-legal issues behind the scenes such as player-to-player harassment and real-life stalking. EA/Origin's Ultima Online started later and had some of the same issues in an almost predictable order and timing. Then EverQuest repeated our mistakes, and so on.
I would think that as an industry, as a set of geeks, we MMORPG server managers would learn from each others' mistakes, but apparently, we do not. It is also a problem in that the management in *product* companies think it is easy to become a world-class *service* company, where the service is being sold to thousands to millions of *household* mass market customers.
[
15x6million is not how much blizzard is getting in revenue.
remember each region pays a different rate.
AND most importantly, Vivendi Universal gets a MASSIVE cut of this figure. Why? Because they footed the bill for Blizzard to finish the game during the last few years of development and as part of that agreement they dictated they get a tremendous amount of the subscription revenue (upwards of 70%, I've heard.)
So then it becomes a question of, who actually is responsible to maintain the servers? Blizzard of VU? Also remember, that's not all going to be spent on WoW. Blizzard has other games in development (Starcraft 2, Starcraft MMO, Diablo 3, WoW Expansion). These people need to be paid something. Not everyone is happy to work purely for recognition. Some of us have these things called bills that we have to pay every once in a while.
As an example, I came home from holiday (I'm in the UK) on Sunday evening & I immediately noticed my ADSL connection was down. So I phoned my ISP to report the fault, only to be told that they knew about the problem - a faulty server had been down for 48 hours!!! And when the tech support person could not tell me when the service would be restored, she seemed totally bemused as to why I was angry about the duration of downtime & demanded to speak to her manager.
The manager was even worse... polite and courteous but did not have a clue as to the cause of the problem or when the ADSL service would be back up. He even admitted that they'd been making some network changes to accomodate a recent merger with another company and that they had no backup server to put in place to at least give some degree of restricted service.
I may pay (the equivalent of) $30 a month for my ADSL service but am I the only person who expects good service from any company I deal with, whether I spend £3 or £30,000 with that company? I accept that sometimes there are service outages, I'd even view an 8-hour outage a few days a year as being understandable. But 48 hours???
I've been in the telecoms/computer industry now for about 20 years now and I've seen the whole perception of what is and isn't good customer service change over that time - it seems now that customers are forced to accept worse service because every company has reduced the level of service they give.
And when it comes to poor Joe Public "peons" like ourselves, who only spend a small amount each month with these companies, we're expected to endure countless menu selections, long delays in call-centre queues and lengthy outages as a matter of course.
It would be good to see a lot more people complain more and cancel their services with some of these providers - I'm sure this is the only way that they will be forced to offer better service to us.
Gentoo Linux - another day, another USE flag.
Generally it's a source management problem. Improper config management and source labeling leads to promotion of untested or incorrect code files so chagnes not intended to go get packed up with the ones that are intended. That's probably most of it. The rest is just not doing correct impact analysis. For example changing something in a base class and not realizing that 400 things inherit from it instead of just the one you're trying to fix. If people did better interface managemnt and impact analysis and they did proper source and config management many of these patch side effects would vanish.
Because they have to pay developers, bandwidth fees, datacenter fees, customer service people, billing people, web designers, janitors, office supplies, and basically everything else it takes to run a business. $35 million / month with probably 15-20 million a month in overhead.
Yes they are making money (businesses are allowed to do this, remember?) Re-architecting a massively distributed game like this takes time *and* money. They underbuilt their infrastructure to begin with, which is where they really went wrong. They are supposedly trying to remedy that, but by the time you have re-architected the system it has grown to the point where you have to do it again.
Also, they're pulling so much bandwidth from so many disparate places that when a link close to them goes down, all the other links have to compensate and there's not necessarily enough fat pipes close to their datacenters to allow everyone on. I would be curious to see what percentage of traffic flowing over certain core routers can be attributed to World of Warcraft; I am betting it is non-trivial.
It strikes me as odd that we constantly compare MMOGs to other games or services regardless of the validity of the comparison.
MMOGs are Entertainment. There are very few other services that one may purchase for "only" $15 per month that will provide the volume and quality (yes, quality) of entertainment that a MMOG will.
One night at the movies - easily $20 for ~2 hours. A night out drinking/dancing >$40? for 4 hours? Any concert >$40 for a few hours. A date? (I know this is /. just trust me, they are expensive).
My point is that it's not a waste of time. It's entertainment. We choose to play them. We choose not to watch TV. MMOGs are actually social behavior (we chat and make friends). If you play MMOGs instead of watching cable/direcTV/TiVo you are paying considerably less per month and interacting with more people while you are doing it.
I consider myself a casual gamer (maybe an hour or two every-other day, more on the weekends) and per-hour I pay about 20 cents/hour to play WoW. If I was hard-core, it would be considerably less.
Relax, and let the silent majority have their fun.
-A
For programmers, networking is a nightmare. Forget the difficult part of making a complex game. Networking is a nightmare all by itself. There are simply game design limitations attributed to that.
Being aware of these limitations, Blizzard should have known better. On my webpage criticizing World of Warcraft, http://www.redrival.com/hateown/ I postulated that Blizzard's design team is to blame for network instability. In a graph on that website I've shown my view of Blizzard's design team constantly increasing massive interaction throughout their worlds. I suspect this is the sole reason why they are unable to cope with volume of customers. As more and more people fight a single super-monster, the networking takes a beating. If the monster hits someone in a particular dungeon (aka instance), aside from the obvious the server has to:
- cycle through 40 players to see if anyone else got hit by splash damage
- notify each of the 40 players that someone or more people got hit
furthermore, when a single player whacks at the monster with their sword (or what have you), the server has to notify every other client about it.
In their design, Blizzard made use of a synchronized clock "aka tick" that is used for synchronizing actions. Good thinking, but it has a breaking point. I believe they are now reaching that point by causing more and more and more players to "chunk". Now, they are dumping the task of making everything better on network admins and scapegoating network issues as opposed to content design issues. It is too late now to take back multiple instances added through patches.
See website for more complaints or to add your 2c:
http://www.redrival.com/hateown/