Slashdot Mirror
World of Warcraft AQ Gates Open!
Tayman writes "Wow...who didn't see this one coming? The players on the World of Warcraft Medivh server opened the gates to AQ. What happened next? The server crashed repeatedly. Why create content the servers can't handle?
The very first time I read about this patch, I knew the servers would crash. The more people who open the gates, the more angry customers Blizzard will have in my opinion. With 5million+ subscribers, you would think Blizzard would have the best servers/connection money can buy. Although, I'm sure it's more complicated than simply plugging in a few ram chips and faster processors though.
Most of the people involved in the raid are having a great time though. Could this be the most epic battle ever introduced to the mmorpg market? All signs point to yes. Let's see how long the mobs will respawn. Hopefully, the people of the Medivh server haven't seen anything yet.
Either way, I would hate to be a network admin for Blizzard atm. ^_^
Here are some pics of the event. Thanks go out to all of those who took these pics.
World of Warcraft AQ Pics Check out MMORPG Veteran to keep up with the events as they unfold." Update: 01/23 13:44 GMT by Z : Additionally, brandor wrote in with a link to some video of the event.
What AQ is supposed to be (for those that don't play WoW).
An expansion? Just a new dungeon? What's so special about it that it causes such server overload?
retrorocket.o not found, launch anyway?
I work for blue chip companies setting up websites that are the busiest in their respective industries, including full connection through to back-end systems.
When the systems die at peak trading, it's 10s of millions in revenue lost. An hour.
My current company provides video downloads off our main sites. We service several hundred retail outlets. We offer very complex product search capabilities, and obviously we permit those products to be purchased. We're dealing with exceedingly large bandwidth, CPU and memory use. We have IBM mainframes, more Sun kit than you could fit in your house and more Wintel boxes than I'd like.
All of this is being provided for less than Blizzard's monthly subscription revenues. Far less. In fact, 3-4 months of WoW subscription revenue in Europe alone would cover the IT costs of our entire business.
So for Blizzard to be unable to handle the loads involved is frankly astonishing. Their systems architecture clearly isn't adequate. Their bandwidth isn't reliable. Heck, they can't even keep their website up and running at peak times - quite a simple website, at that.
This is despite being live for well over a year now. They know how much bandwidth each user needs. They know how many users they have. They know what the capacity on each server is. They already have logon queues at times of peak load, to control the numbers of logged in players.
I have no sympathy for Blizzard on this matter, because they've had plenty of time to get this sorted, and consistently fail to deal with it. This isn't rocket science, you don't need to steal Google's employees to find a solution, just get someone competent in and fund the necessary infrastructure.
These games are the biggest regret of my life. Seriously. I once spent over 1400 hours in one year playing one (back in the text days). That's seventy days, if you are counting.
I could have been getting good grades, chasing chicks, and figuring out what the "#$# to do with my life. I seriously messed up all three. Instead, I just had the coolest equipment in some worthless game. A couple people I know failed out of school entirely because of these games.
You can do better.
WoW crashes when it gets only a few hundred people doing anything meaningful inside a zone. In total it handles a few thousand people connected to any given server at any given time, with some of those a sizeable portion of those people off-loaded onto other servers via instances most of the time.
In addition, they're well aware of their scaling problems, and have added things in to prevent the type of occurances that caused crashes(City Raids), as well as scaling back the max server populations(hence queues). So not having prepped properly for a World Event of this magnitude, especially given their revenues is inexcusable.
My guess is the situation at blizzard is the following:
Most of the core devs went onto other projects
Like most MMOs their network code is laughable.
Their code doesn't parallellize well, so they can't just toss more hardware at the problem
They can't fix the above without a drastic redesign, and by the time they did that it would probably cease to matter.
And yes, it's doable, I've seen MUDs/MUSHs written as hobbies that handled several hundred concurrent users on hardware from the mid-90s. An MMO doesn't push *that* much more traffic than a text-game that saturates a 56k connection(as many did), and it certainly doesn't do many more back-end calculations. Considering how much hardware has scaled and how much further we've gotten in various areas, there just isn't any excuse for several hundred people in a single world-segment causing the server(not the client) to go OMG and crap out.
No web-based business craps out under that kind of traffic. How to cope with it is well-known at this point. I mean shit, this is the type of crap DIKU's massive list of socket descriptors did under load, and that was written over 10 years ago!
Imagine a phone switch doing this! That's tech from the 70s that handles waaayyyy more traffic than one of blizzard's servers. Google easily copes with orders of magnitude more traffic every moment of every day, and holds up like a champ. Stop being an apologist for a drastic lack of planning and poor engineering.
The key to the enjoyment of pop music is to replace any instance of "love" with "C.H.U.D."
Actually, I'm in a guild that runs ZG/MC weekly. Many people who "Know how to play" think ZG has harder bosses than MC. Nobody ever runs a pick up group through ZG. There is a simple line between UBRS and ZG, where as the casual player can run through UBRS with strangers, you need a guild, vent/teamspeak, etc to do ZG and above. UBRS typically is run with 20 people over an hour or two, so I'm not sure where you get your '3 man in 20 minutes' idea other than the typical blizzard fanboy crap.
The war effort isn't 'cool' either. I played world of warcraft for the WAR part, not to team up with the other faction. They could have made it a competitive effort on the pvp servers. Of course, they haven't cared about pvp since last June.
If they continue to release carebear crap like this, and endgame super dungeons for the ubernerds, they will lose a good deal of their playerbase. Personally, I'm getting sick of having to do lame ass raids just to hang in the battlegrounds.
Or, you could have spent those 70 days doing something equally stupid.
The only problem I have with your logic -- or anyone that heavily criticizes people for spending too much time on any one activity -- is the assumption that if they did other activities, they would inherently have more value.
I know people that spend hours a day, pretty much all of their leisure time, watching sports on TV. Is that really any better or worse than playing WoW for an equivalent amount of time? I don't think so (especially given that ESPN costs more).
I'm willing to bet that most people who are on WoW, if Blizzard went under tomorrow, would find something equally useless to do in their spare time. This idea that people who play games are all going take up triathlon training or feed the homeless in their spare time, if games weren't available, is dumb. In all likelihood they'd just watch TV.
I'm not arguing that too much of anything can't really mess up your life -- when people stop going to class or work to play games (or watch TV, or whatever), it's a real problem. However I'm not sure that games are much worse in this regard than any of several "time wasters" that I can think of, it's just that you don't hear about the other ones.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
None of what you mention is nearly as dynamic and complex as a high-traffic MMORPG server back-end.
Also serving static web content is trivial compared to tracking the state of 5 million clients and letting them see each others in real time is so far beyond web hosting that it is laughable.
I worked at video game companies (Turbine) and I worked at some of IBM's large server farms (Poughkeepsie, Southbury) doing performance balancing. As far as software goes I have to say video games server technology makes web content delivery look like the stone age. The only thing that even compares in complexity is when IBM hosted the Olympic coverage. Trying to compare simple web content to a system where clients are all making updates to each others environments in real time is impossible.
I hate it when the Wow server's crash, but I have had my ego battered by what the guys at Blizzard have managed to do. They have done some great work and I am curious to see other game companies surpass the work Blizzard has done.
Nothing here is trivial. If it was it would have been done right the first time.