An Inside Look At Warhammer Online's Server Setup

An article at Gamasutra provides some details on the hardware Mythic uses to power Warhammer Online, courtesy of Chief Technical Officer Matt Shaw and Online Technical Director Andrew Mann. Quoting: "At any given time, approximately 2,000 servers are in operation, supporting the gameplay in WAR. Matt Shaw commented, 'What we call a server to the user, that main server is actually a cluster of a number of machines. Our Server Farm in Virginia, for example,' Mann said, 'has about 60 Dell Blade chassis running Warhammer Online — each hosting up to 16 servers. All in all, we have about 700 servers in operation at this location.' ... 'We use blade architecture heavily for Warhammer Online,' Mann noted. 'Almost every server that we deploy is a blade system. We don't use virtualization; our software is somewhat virtualized itself. We've always had the technology to run our game world across several pieces of hardware. It's application-layer clustering at a process level. Virtualization wouldn't gain us much because we already run very close to peak CPU usage on these systems.' ... The normalized server configuration — in use across all of the Mythic-managed facilities — features dual Quad-Core Intel Xeon processors running at 3 GHz with 8 GB of RAM."

Comments

70 * 16 ? Are they kidding me?

[Anonymous Coward]

They have like eight servers (worlds) up, tops. They closed all the other ones down. There's no way they have 1120 servers running.

Re:70 * 16 ? Are they kidding me?

[SCPRedMage]

That's PHYSICAL servers. A single game "server" is made of multiple PHYSICAL servers.

Re:At least gamasutra labels their paid placements

[_Sprocket_]

From the beginning of the article:

[In this Intel-sponsored feature, part of the Visual Computing section, the technical experts behind Mythic and EA's Warhammer Online discuss the mechanics of keeping the MMO running across multiple servers and data centers.]

So yeah - if you start picking up on the glossy brochure language... it's because it is, in fact, marketingspeek. But burried in all the "gosh golly isn't this product swell" blather is some interesting little tidbits about the environment; wheat from chaff.

Article summary

[chanrobi]

Blah blah blah, intel cpus are the best, blah blah blah. Reads like one giant intel advertisement

Virtualization

[Fractal Dice]

I've always felt virtualization was pushed too heavily as a concept, making up for failings of the operating systems and how they are used. Many admins have become so used to consolidating existing legacy servers together that we start thinking every app should be wrapped in its own private copy of an OS - as if the whole point of the concepts of users and groups and limits have been forgotten.

Re:Virtualization

[Anonymous Coward]

It is also great when you have an old server that has been running for a long time on piece of crap hardware that is about to die. You can just do P2V and not worry about trying to transfer data over, especially those ones that have been in production for years and reinstalling software is not feasible.

Re:Virtualization

[dkf]

Basically, enabling legacy applications to survive by giving them a slice of a real machine and running them that way is a great crutch. But not more. It would be more efficient to revamp the system and bring it up to contemporary code, but often that's not possible. I blame closed source and companies that wrote it going out of business, but that's me... I could ramble about shortsighted management decisions and putting the life of a company on the line and dependent on the existance of another company, but ... I won't.

The open/closed status of the program code has got nothing to do with it. We use virtualization with plenty of code where we have the source (either open source or written in-house) because that lets us greatly improve the utilization of hardware. Why take up a whole rack of servers to do what one modern blade can cope with? (Most servers aren't CPU-bound.) What's more, it can do this without you having to figure out how to get all those silly deployments to work together nicely.

The other good thing about virtualization is that it lets many people have control over their own machines without needing lots of "servers" under desks. That means you can do things like ensuring that everything that the business really depends on has UPS power and sane networking. (I know. Critical stuff shouldn't be put under someone's desk. Virtualization makes it easier to bring reality closer to that ideal.)

Virtualization isn't perfect at all, but it does cure a bunch of problems that crop up in reality and at far less cost than "doing it properly". (For one thing, it's not cheap to build a new datacenter. Even fitting out a new server room isn't something that you want to have to spend on every day.)

Re:Virtualization

[TheRealFixer]

6: Ability to run on future hardware. Say everyone ditches x86 and amd64 and decides to go to IBM's POWER architecture and emulate legacy stuff. The stuff in the VM won't care that is is actually isn't running on a different CPU.

This is not true. Hardware virtualization is not emulation, which is what you're talking about here. Processes in the VM are run directly on the host processor, they're just managed by the hypervisor. There's no emulation layer, since that would make performance pretty atrocious. So, the stuff in the VM absolutely will care about what processor you've moved to, especially if you've suddenly changed instruction sets. Binaries compiled for x86 won't magically run on PowerPC just because it's running on a VM.

Dell ad?

[Degro]

This game failed in big part because of their extremely poor server performance. Who cares how they did it?

Re:Dell ad?

[Vrallis]

This game failed in big part because of their extremely poor server performance. Who cares how they did it?

Well, both poor server performance (Fortress battles were completely unplayable when I quit) plus they opened WAY too many servers at the start. If they'd started with 1/3 as many servers the game would probably be in far better shape today. The server transfers that they opened up as a last-ditch effort prior to merging servers was the straw that broke the camel's back for me, destroying my server's population.

WTB a fantasy (okay, I'll say it...WoW-like) MMO modeled after EVE's economy, industry, PVP, territory control, etc.

Re:WTF is Warhammer Online?

[jhoegl]

It is impossible for Mac users to nerd rage.
Mac users would have to be nerds for that statement to qualify.

Re:Irrelevant

[Anonymous Coward]

Sure you do, if you're using archaic server software from the days where DAoC was born.

Re:WTF is Warhammer Online?

[glwtta]

It is impossible for Mac users to nerd rage. Mac users would have to be nerds for that statement to qualify.

So... douche-rage?