Slashdot Mirror
On World of Warcraft's Network Issues
alphaneutrino writes to mention a C|Net article discussing some of the recent problems the World of Warcraft playerbase has experienced. From the article: "'Being a system administrator myself, I have some understanding of what goes on in a corporate data center,' said Evgeny Krevets, a sometimes-frustrated WoW player. 'I don't know Blizzard's system setup. What I do know is that if I kept performing 'urgent maintenance' and taking the service down without warning for eight-hour periods, I would be out of a job.' Blizzard blames some of the problems--such as the disconnection, for several hours on Friday, of players linked to several servers--on AT&T, its network provider. (AT&T did not respond to a request for comment.) "
Sunday: The day the server stood still
Monday: *gasp*, playable (until 11pm)
Tuesday: Weekly Maintenance Day. Nothing else EVER needs to be said about this day.
Wednesday: Playable (until 11pm), good chance maintenance aftermath.
Thursday: The 10 second instant-casts day for MC & BWL.
Yeah, it goes on. Our server reliably bites the dust around 11pm every night for 6 hours, not to mention the constant plague of login issues and 30-minute loading screens during peak hours. Funny how this is all on a low-medium population server.
...so THAT'S how Blizzard is combatting server lag.
.. What's that come to again?
And _why_ are there any problems whatsoever?
Blizzard, I can guarantee this: if you spend $35 million per month on refactoring, hardware and bandwidth, all your problems go away. Guaranteed. I promise.
Maybe it's the Blizzard guys' moms that come in and say "Enough of those stupid games already, go to bed!"? ;)
Or are they too cool to be running the servers out of their parents' basements like the rest of us?
"Well, at least I have chicken!"
Don't take the above poster too seriously. He doesn't.
if I kept performing 'urgent maintenance' and taking the service down without warning for eight-hour periods, I would be out of a job
The difference is that Blizzard sees itself as already having it's customer's money. Therefore, there's no reason to spend any more for service. Your boss needs the network up just to make money.
I tend to agree, today's net users are so spoiled. Back in 1993 AOL went down every Tuesday and Thursday at 11PM for six hours- at least. And AOL isn't even a recreational service as WoW is, it is vital things like email and stock tickers.
The above poster needs to learn to become tough like valient AOL users did, and who were also the tip of the spear that tamed the net for WoW players to come a decade later.
Those communists over at Blizzard want to use Edward Whitacre's pipes for free!
wow...And I thought SOE was bad at maintaing MMORPGs.... I'm sure glad I'm playing Saga of Ryzom. 6_6
Ive barely even seen any issues since patch 1.10. I think patch day the servers were down all day, but thats to be expected.
Server preformance varies from realm to realm. I hadn't really had any issues until the last week or two when my server decided to drop 40 minutes into our 45 minute baron run, and then again in the BG's later on.
As someone else mentioned, I think they are still a victim of their own success. Sure it's been over a year since launch, but they were expecting 250,000 subscribers and got 6,000,000.
Free, not Pay per month, and as long as I have played it, only 2 spots of down time in 6 months. I guess WoW has many things one upped on GW, but still.
"God of Rock, thank you for this chance to kick ass. "
I've noticed that since starting to play WoW last year around June that over time the performance issues and network performance have just gone in the toilet. Game patches result in difficulties too numerous to enumerate here. Login queue times have skyrocketed over the last four months, and I keep sending in complaints about how $150 a year should get me better performance than this. I'd love to see their setup and critique it.
Oh yeah, that monthly fee is totally going towards maintenance costs, just like they said. That much is apparent.
Seriously, I still can't believe how easily people took to paying monthly subscription fees to play games that already cost $60 and, without paying the fee, are completely useless. It's kinda like giving cold, hard cash to a charity. You have no idea where that money is going, and you sure as hell can't trust Blizzard's PR department to give you the whole truth.
I stand fast in my assertion that I will not pay a monthly subscription to play any game except under one of two circumstances: 1) the game must have an equally fun single player mode (and it better be damn good), or 2) the game itself is free, and the monthly subscription is the only cost.
Call me anal, but it's bad enough when I pissed half my college years away playing Diablo II online for free. I don't see the point in having to pay for the privilege to waste my time.
"You will pay for your lack of vision..." - Emperor Palpatine to Ray Charles
No one except the 6 Million users that play the game.
I'm not a WoW player but if it's true that these systems regularly go dark for 8 hours at a time I have to wonder if they're not racing through some software patch. In other words, I don't know an architecture out there that can't be rebooted in 8 hours so a straight-up crash seems unlikely. I would assume they've taken care of scalability problems by now so system load / tablespace, etc, ought to not be an issue.
... and some succeed, requiring a quick patch to the code base? I wouldn't doubt that they have monitoring mechanisms in play which detect unreasonable changes in a character's level / gold, etc.
Could it be that WoW suffers constant attempts at subverting the framework of play
CommentBot 0.7a running with args "-module irritate,disagree -target random"
Just like ISPs are and held to accounts for it. When I played WOW they were good about refunding for extended downtime. Yet at no time do any of these pay to play games make any guarantee of service availabilty.
As far as their continuing stability and growth issues.
STOP SELLING THE DAMN GAME.
Sheesh, how hard is that to understand? If you cannot provide a stable set of servers and servers where people can play WHENEVER they want to then stop selling new copies until otherwise.
Hopefully with the number of professionals playing the game one of them will get annoyed enough to sue them in court, either to force a change by ruling or just having their named dragged into the mud.
* Winners compare their achievements to their goals, losers compare theirs to that of others.
Our game had its server problems and we were in "learning mode" to deal with some major outages, major gameplay renovations, major strife from jerks, and major socio-legal issues behind the scenes such as player-to-player harassment and real-life stalking. EA/Origin's Ultima Online started later and had some of the same issues in an almost predictable order and timing. Then EverQuest repeated our mistakes, and so on.
I would think that as an industry, as a set of geeks, we MMORPG server managers would learn from each others' mistakes, but apparently, we do not. It is also a problem in that the management in *product* companies think it is easy to become a world-class *service* company, where the service is being sold to thousands to millions of *household* mass market customers.
[
The reason they're having so much trouble is because the integration with the AT&T to government monitoring station upgrades are taking too long.
AT&T: Keeping terrorists off WoW!
- Just my $0.02, take with a grain of salt, your mileage may vary.
I think patch day the servers were down all day, but thats to be expected.
It's funny what people get used to. In the original EQ, patches were just a few hours in the morning, one day a week, unless something went wrong (which generally didn't happen, despite what the boards say).
In Horizons, another MMORPG, database lag was so bad that you could pick up an item and not see it in inventory for 10 minutes. You could run through an area full of monsters and not see one by the time you were through, because the client couldn't build them fast enough.
If you're shovelling out $15/month for a service, about what you pay for cheap telephone service, you'd expect it to work when you wanted it to work. Like I said, it's funny what people lower their expectations to.
No wonder Bush is still president (snap!)
Sounds like WoW has a house of cards network with single point of failure architecture problems.
And that AT&T is exploiting them, marketing a new "premium service/support" contract by letting them go down.
I can't wait until WoW has to pay AT&T (and its handful of competitors, if they get rid of the SPF) the extra "premium tier" routing fees, once the telcos market their "nonneutral" Internet. Because a world of angry Warcraft players jonesing for their fix will be a nice gift for telco suits just trying to make it home from work.
--
make install -not war
It's hard for AT&T to cater to so many millions of users *AND* filter/direct all of their customer data illegally and directly to the NSA.
As an example, I came home from holiday (I'm in the UK) on Sunday evening & I immediately noticed my ADSL connection was down. So I phoned my ISP to report the fault, only to be told that they knew about the problem - a faulty server had been down for 48 hours!!! And when the tech support person could not tell me when the service would be restored, she seemed totally bemused as to why I was angry about the duration of downtime & demanded to speak to her manager.
The manager was even worse... polite and courteous but did not have a clue as to the cause of the problem or when the ADSL service would be back up. He even admitted that they'd been making some network changes to accomodate a recent merger with another company and that they had no backup server to put in place to at least give some degree of restricted service.
I may pay (the equivalent of) $30 a month for my ADSL service but am I the only person who expects good service from any company I deal with, whether I spend £3 or £30,000 with that company? I accept that sometimes there are service outages, I'd even view an 8-hour outage a few days a year as being understandable. But 48 hours???
I've been in the telecoms/computer industry now for about 20 years now and I've seen the whole perception of what is and isn't good customer service change over that time - it seems now that customers are forced to accept worse service because every company has reduced the level of service they give.
And when it comes to poor Joe Public "peons" like ourselves, who only spend a small amount each month with these companies, we're expected to endure countless menu selections, long delays in call-centre queues and lengthy outages as a matter of course.
It would be good to see a lot more people complain more and cancel their services with some of these providers - I'm sure this is the only way that they will be forced to offer better service to us.
Gentoo Linux - another day, another USE flag.
The problem really is visible when you are adventuring in difficult to beat places. You depend on having your team perform to their best ability. It is then so frustrating to be constantly dealing with part of you team getting disconnected or being lagged to the point of ineffectiveness.
My guild is doing MC BWL, ZG and AQ20 right now. It is a regular occurence right now to wait 20 minutes to start a fight because of disconnected people, only to then lose that battle because you lost two priests to a disconnect during it.
The anger may not be at the threshold point yet Blizzard, but it most definitely building fast. The thing about angry customers is that there is a point of no return when they are forever lost. Blizzard has a lot of customers right now, but they would lose them fast if somebody else stepped up with a great game and more reliable game play.
Blizzard, you executed very very well on game content by effectively removing much of the grind that other games are plagued with, but you have failed with customer interaction. Some of your representatives treat your customers with borderline contempt (Tseric) and you fail miserably at explaining properly the multitude of changes you make to the game.
Blizzard, your six million customers are waiting; it's your move, take too much time and you could lose them. Start with being public about your server improvement plans, telling people what you're doing and why and how its going to make things better. Not knowing when things are going to get better is really making people angry.
Do not spread "09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0" over the internet, thank you.
I think there's a common failure of even technologically inclined individuals, including system administrators, to understand exactly what is required of the servers that allow us (usually) to play World of Warcraft. For every player connected to the server, the server has to recieve a packet explaining what the player's current attempted actions are, and send back relevant information regarding the actions of every other player and object in the immediate area. This is a constant process for each and every player!
The common system that is administered in a corporate environment does not have thousands of users connected at once all requiring huge amounts of bandwidth and processing time 24/7. That is not to say that systems with large wear and tear don't exist, or that systems with such huge numbers of users don't exist, or even both. What I am saying is that Blizzard has to administer two or more dozen server clusters being continually accessed 24/7 by resource intensive users (save for a usually brief repose on Tuesdays). Unless you work for Google (and even then) there's no comparison.
This isn't to say that we shouldn't raise our eyebrows at the pervasiveness of the problems WoW has. However, I keep seeing the same arguments thrown around about how Blizzard gets $15 x 6million every month (not entirely true because A) there are less pricey payment options and B) something around 1/3 of those players play in China on Asian servers whos subscription plans would hardly be purchasable were they $15 a month), how at 'my company' things work this way, etc. etc.
Blizzard should be called on to answer for why their servers haven't been made right as rain after a year or more after release, but it should be in the context of legitimate complaint and not any of this throwing around of overused and hardly consequential arguments.
Thunderclone: ONE MAN ENTERS! TWO MEN LEAVE! ONE MAN ENTERS! TWO MEN LEAVE!
... and I don't plan on subscribing again until I can see evidence that they have fixed this. Like many people at the moment I have expressed my discontent in the only manner that Blizzard will hear and voted with my check book.
I know it is tough for Blizzard, but as a customer I have been the one paying the price for that so far, from now on that cost is Blizzards again. At least for the time being.
A large part of the problem is that Blizzard's communication with the player base sucks, to speak frankly. The login server for their forums seems to be one and the same as the login server for the game itself, so when that goes down the forums tend to shut down as well. There is a "Realm Status" page which purports to show the real-time status of the various servers, but which is frequently unreachable. There is a "Realm Status" forum which *might* contain some acknowledgement of a problem while the problem is still ongoing, but usually doesn't. When you start up the game client, Blizzard can stick up a 'News' window on your screen but, again, the appearance of any news often lags the problem, even severe problems, by a matter of hours. And, of course, Blizzard's chief form of communication with players is Community Managers on the forums, who themselves tend to be given dick in the way of information, are extremely controlled in what they can and cannot say, and who are (honestly, I'm not joking), tasked with yelling at users for stuff posting subject headers that contain excessive capitalization; what an obscene waste of resources.
Seriously, a little timely information goes a long way. Yes, I agree that the downtime they have is absurd; consider that *every Tuesday* the game goes offline for *six hours* of maintenance. That's *planned, scheduled* downtime, folks, so that *alone* means they aren't even attempting to have greater than 96.4% uptime, and I can't think of another commercial service for which you pay a monthly fee where that would be even remotely acceptable; if your cable or your phone just plain didn't work for 6 hours every Tuesday, heads would roll. Then things just get asinine when you factor in all the spontaneous, freewheeling, unplanned downtime as well.
But know what? I'd feel a lot better about it if, when something shits the bed, or goes tits-up, or whatever colorful metaphor you'd use to describe a server-killing technical problem, Blizzard would tell us, promptly, as they receive the information themselves:
1. We know there's a problem.
2. We know what the proglem is.
3. Here's what we're doing to fix it.
4. Here's when we expect it to be fixed.
5. Update as old information is obsolete.
They don't do this. A few hours after something happens, you might get some of the above information. Or you might not. Usually, it's the latter.
I don't play any online games but I thought the whole idea of them was that you subscribe to that service for it to be available just about 24x7 whenever you feel like jumping in. Sure, occasional outages are to be expected but if it gets to the stage where the game is frequently slow or unavailable, the common sense solution would be to cancel your subscription until Blizzard (or whomever) improves the service they deliver you. If enough people did this, they'd have to do something about it...
I'm sorry but I think far too many people have become "slaves" to marketing by truly believing that they simply cannot do without a lot of the products & services that they pay good money for - to the point where they "need" those items so much that they're afraid of complaining in case they're denied those things completely.
Gentoo Linux - another day, another USE flag.
Yeah, we can do other things; but not all of us have a lot of free time to play the game. We have jobs, family, other interests, etc... It's very frustrating when we set aside time to play the game we pay for and the servers are down.
WoW is hardly the first online service to be hit by network and server problems. Over the years, services like eBay, Amazon.com and E*Trade have all dealt with various forms of outages.
Yeah, but the difference with WoW is the money. When eBay, Amazon.com, and E*Trade have outages they are losing money. When WoW has an outage they don't lose a dime. Only thing they lose is the 1 or 2 players who get frustrated and abandon all the 'work' they put into their characters and cancel their accounts. Blizzard still collects your $15 every month, outage or not. No $$$ incentive for them to provide good service. I thought they were a good company, but my opinion of them is changing drastically. Diablo 2 and Warcraft 3 are both Blizzard games you can play online for free, they are more stable then WoW. Something is wrong here!
WOW server downtime is saving my marriage.
tidokoro
what turns a man's karma neutral? lust for gold? power? or just a heart born full of neutrality?
Generally it's a source management problem. Improper config management and source labeling leads to promotion of untested or incorrect code files so chagnes not intended to go get packed up with the ones that are intended. That's probably most of it. The rest is just not doing correct impact analysis. For example changing something in a base class and not realizing that 400 things inherit from it instead of just the one you're trying to fix. If people did better interface managemnt and impact analysis and they did proper source and config management many of these patch side effects would vanish.
To be honest, what bothers me more than the lag itself is the distinct lack of interest their support staff give. Tickets submitted to the in-game helpers, Game Masters, result in them telling you to go post on the tech support forum. The tech support forum tell you first of all to uninstall all the addons and to phone your ISP, despite the fact that the problem is occuring to everyone on the server, and then they tell you to contact a Game Master on your server... An in-game friend of mine recently called Blizzard directly to speak to the tech support staff there. After informing him that we were currently in the middle of an Ahn'Qiraj raid where all forty of us were experiencing lag of over 800ms the friendly staff member told him that "Well it may be your ISP". Why yes, we have members located from Britain to Hungary to Russia but we are all having ISP problems at once. If they'd just admit that the servers are over-populated, open more servers and allow migration then this would help alleviate the problems. I'm on the EU-Arathor server which has 12k players, the highest in the EU, but has yet to be offered a migration option. It's a poor show.
If these problem are really related to AT&T, then why do we Germans experience exact the same problem? Over here T-Online is the bad guy. To solve the problem, Blizzard even suggested to alter you MTU-rate for your dsl to 1400. I don't know how many people ever heard of a thing called MTU ever. (the common people, not the nerds here ;-) )
Blizzard should ask themself why the whole IT ifrastructure are haveing problems with there product and if it is really the isp's fault.
Actually, that's how software maintenance happens in the real world.
Real code is complex, and generally written as a massive matrix of inter-related side-effects causing things to happen*. When it gets written, the entire matrix is designed, intended, documented, and understood. Two years later the guys working on the code have no clue about the matrix of side-effect driven code, no clue about the complex set of business factors driving the technical aspects of the code (and by business factors, in a MMORPG I mean things like class X has bad faction with everybody making it more difficult for him to start out, but in return for overcoming that challenge has more powerful magic later in life - stuff like that) and when they are making a change they go in, find the one line of code that looks like what needs to be fixed and just change it without knowing all the places that change will ripple back to, invisibly, via the side-effect matrix.
A technical phrase to understand here is 'globally scoped variables' - and another one is 'design intent' - and as the current set of hacks don't understand the ramifications or scope of either, this is what happens.
Footnotes
* I didn't say it was a good idea. I just said it happens.
Glonoinha the MebiByte Slayer
As someone else mentioned, I think they are still a victim of their own success. Sure it's been over a year since launch, but they were expecting 250,000 subscribers and got 6,000,000.
The controlling factor for their server performance should not be the total number of subscribers, but the number of subscribers per realm, and Blizzard has complete control over that number, because they can mark a realm as "full" and disallow logins/signups. IOW, as you know, those 6,000,000 people are not all playing in the same game at the same time.
It should be possible to make the realms completely independent, so that this just becomes a matter of horizontal scaling, and having hardware/systems monkeys roll out new realms via some standard operating procedure.
Unfortunately, based on the rumors I have heard, Blizzard has chosen to tie a bunch of stuff together. For instance, the common web forums use the characters from all the realms (the web forums know about your level 23 mage), they have a single set of auth servers, it's not clear that the item databases are not shared between realms, and so on. This is sort of sad, because it's not like Blizzard are the first people to roll out an MMORPG.
Now, some might argue that tying some of this stuff together makes for a better user experience. However, when this entanglement leads to downtimes, one could make the argument that it's not worth it.
Anyway, my point is not to bash on Blizzard; I'm sure they've made some difficult design decisions correctly, and some difficult ones incorrectly. My point is that "we have lots of users" is not a good excuse when you have a service that lets you divide those users into sub-populations, and that there are probably architectural improvements they could make to improve their scalability. The real question is whether they have competent and experienced systems engineers to help them make those improvements, and whether management is committed to supporting them.
Anyway, so much for pre-coffee ramblings....
I don't have time to look up the post from the forums. But a while back one of the CM's let slip that Blizzard does not manage the WoW datacenter. They actually have contracted every bit of that out to a third party. I sincerely think this is a core part of the problem with connectivity. Of course Blizzard has a time lag between something going wrong and them finding out about it. First a tech at the third-party has to notice it. Then they forward to their superiors and so forth and so on until eventually someone at Blizzard might find out something went wrong. It also doesn't surprise me that the third party providing datacenter hosting is not multi-homed (isn't that the correct term?).
I think the largest lesson to learn here is that you shouldn't contract out the entire core aspect of your business model.
...while you're not an idiot, I can understand where they could end up with one supplier for bandwidth.
:-)
1) You need a SLA with each ISP you pull backbone level feed from. You can use InterNAP and hook into the peering points in the US and a few other places, but it's got it's own issues- and if you just use them, you're still with only one ISP; if they fail, you're still up a creek without a paddle.
2) You'd need to frame the servers into one massive data center with a HUGE honking data-pipe from each ISP with BGP routing on the inbound routers from the ISPs to your DMZ to establish one IP address range for the front-facing servers
OR
Come up with some sort of nasty DNS trick to hopefully make the server front-ends transparent to the clients and spread them across multiple IP blocks (Which is what epicRealm did to make their CDN actually completely transparent to client and customer- and to be able to handle dynamic HTTP content...)- but be prepared, because in order for this to work right, you either need to trust the client's state, share state across server pools on different IP blocks, be stateless, or somesuch like the previous.
There's a bunch more, but those above two and the first item will hopefully show you why someone (a bean counter, most likely...) will make the decision to just simply hold the ISP or Tier-1 host (Which is the most likely case here- they're very probably colocated at an AT&T Tier-1 facility...) to the SLA they promised- because it's cheaper and waaay simpler if everything goes right and they're "not to blame" if things go wrong. If you went an alternate route and had a mishap that wasn't server related, then you'd be to blame and have nobody to point fingers at when it all broke (And you just KNOW it will at some point- it always does...
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
As someone else mentioned, I think they are still a victim of their own success. Sure it's been over a year since launch, but they were expecting 250,000 subscribers and got 6,000,000.
Then Blizzard should not have distributed 6 million copies of the game. They've brought this upon themselves. Open beta consisted of a very small user base (relative to what it is now). So the kind of resource pressures they face now were never realistically tested pre-release. So really, the first (and it seems, current) adopters of the game became part of a second beta test of sorts. And I'll grant Blizzard that they were caught off guard but they've had almost 2 years to get things right and they still haven't managed it. And regardless of their known problems they continued to distribute more and more copies of the game. They should have shown a bit more restraint after initial release but they went all-in and now we're stuck with what we have.
The big problem is that customers, especially those who've been around since release, will be reluctant to leave even if the service drops further. Reasons being addiction and the loss of investment on the time spent playing the game.
I think WoW could go completely offline for a week and come back with a very minimal loss of its user-base. They don't have the motivation to do better.
It strikes me as odd that we constantly compare MMOGs to other games or services regardless of the validity of the comparison.
MMOGs are Entertainment. There are very few other services that one may purchase for "only" $15 per month that will provide the volume and quality (yes, quality) of entertainment that a MMOG will.
One night at the movies - easily $20 for ~2 hours. A night out drinking/dancing >$40? for 4 hours? Any concert >$40 for a few hours. A date? (I know this is /. just trust me, they are expensive).
My point is that it's not a waste of time. It's entertainment. We choose to play them. We choose not to watch TV. MMOGs are actually social behavior (we chat and make friends). If you play MMOGs instead of watching cable/direcTV/TiVo you are paying considerably less per month and interacting with more people while you are doing it.
I consider myself a casual gamer (maybe an hour or two every-other day, more on the weekends) and per-hour I pay about 20 cents/hour to play WoW. If I was hard-core, it would be considerably less.
Relax, and let the silent majority have their fun.
-A
However, I get a little sick and tired of people treating certain companies as being almost "beyond criticism" on Slashdot - Apple seems to be one of those, Blizzard appears to be another. (BTW, I'm an avid player of Warcraft 2 & Starcraft so I actually rate those particular products, and therefore my personal experiences of Blizzard, very highly).
The fact is that if you don't feel you get value for money, whether it's from Apple, Microsoft, Blizzard, Dell, etc. etc. then *DO SOMETHING ABOUT IT* rather than just sitting back & accepting it. Maybe I'm just middle-aged & cynical but I don't consider *ANY* product or service as being that important to me that I wouldn't throw it straight back at whoever sold it to me if it didn't meet my expectations.
The problem is that clever marketing has made certain products "cool" and/or made some people fearful of making any criticisms just in case they "stand out from the rest of the crowd". What I'm saying is that *ALL* that matters is whether or not *YOU* feel you get value for money - if you do, then enjoy it & let those that feel they don't raise a stink about it.
There is *NO* such thing as a "nice" company or corporation - they *JUST* exist to make money from you and it's the quality of what they deliver to you that is the only important thing.
Gentoo Linux - another day, another USE flag.
"My crack pipe...My crack pipe!....suck...suck....It's not working right!"
This is a trick that all service industries (Sat. Radio, directv, etc.) use for subscriber count. The subscriber count is the number of people, over the lifetime of the product, that have subscribed. Not the same as the current number of subscribers. Blizzard is *NOT* collection 6 million * $15 a month in subscription costs!
For programmers, networking is a nightmare. Forget the difficult part of making a complex game. Networking is a nightmare all by itself. There are simply game design limitations attributed to that.
Being aware of these limitations, Blizzard should have known better. On my webpage criticizing World of Warcraft, http://www.redrival.com/hateown/ I postulated that Blizzard's design team is to blame for network instability. In a graph on that website I've shown my view of Blizzard's design team constantly increasing massive interaction throughout their worlds. I suspect this is the sole reason why they are unable to cope with volume of customers. As more and more people fight a single super-monster, the networking takes a beating. If the monster hits someone in a particular dungeon (aka instance), aside from the obvious the server has to:
- cycle through 40 players to see if anyone else got hit by splash damage
- notify each of the 40 players that someone or more people got hit
furthermore, when a single player whacks at the monster with their sword (or what have you), the server has to notify every other client about it.
In their design, Blizzard made use of a synchronized clock "aka tick" that is used for synchronizing actions. Good thinking, but it has a breaking point. I believe they are now reaching that point by causing more and more and more players to "chunk". Now, they are dumping the task of making everything better on network admins and scapegoating network issues as opposed to content design issues. It is too late now to take back multiple instances added through patches.
See website for more complaints or to add your 2c:
http://www.redrival.com/hateown/
With proper pipelining, you CAN get one baby after an initial nine-month delay, then a baby a month of throughput until your cache is depleted.
When I first joined the industry, telcos (and I speak more of business/PBX providers rather than PSTN service providers) were able to make a lot of their money through custom hardware & the maintenance thereof - now, of course, most of the hardware is standard (Intel) server platforms so it's the software only that generates the income. And, yes, the cost of bandwidth has reduced many times over but I think much of that is as a result of technology enhancements such that you can get "more bits down a piece of wire" now than you could do 20 years ago.
Likewise, a lot more service can be carried out using remote connectivity now than could be done 20 years ago - in 1986, it was virtually unheard of to do a major software upgrade over a modem or VPN link but now it's commonplace. That means less bodies "on the street" and therefore much lower staffing costs as a result.
However, this has resulted in customers "feeling" that they are getting lower quality service because they no longer have direct face-to-face contact with the same familiar engineer they're used to dealing with. (There was a time when I was working in the field where I couldn't possibly drink the number of bottles of whisky given to me by my customers at Christmas - those *were* the days!)
In summary, add that to the fact that just about anyone dealing with a service organisation today *expects* to be queued in a call centre across the other side of the world plus the outsourcing of programmers and a technical person who might not understand your specific needs in the country where you are who is "monitored" by a statistical system that cares only about fault numbers, times to clear, etc. and, you get, in my view, a worse quality of service than was delivered 20 years ago.
It's simply the fact that *every* organisation now does it this way that the poor customer has no real choice in the matter...
Gentoo Linux - another day, another USE flag.