Building Scaleable Middleware for MMORPGs

CowboyRobot writes "ACM Queue has an article exploring the challenges of developing a reliable platform for an MMORPG, specifically looking at Wish by Mutable Realms. From the article: 'A common scalability problem for distributed multiplayer games relates to managing distributed sets of objects... A player may not be a member of more than one guild, or a guild may have at most one level-5 mage (magician). In computing terms, implementing such behavior boils down to performing membership tests on sets of distributed objects.'"

Comments

Scalability and joining guilds

[Space cowboy]

Surely this is a classic example of the Manager pattern. You have a bunch of objects [Avatar] (all alike, at least programmatically :-) who want to perform operations on other objects. If the system has a [GuildManager] class, then access to this distributed network of avatars can be forced through the choke-point of 'can this avatar join this guild'.

The trade-off in terms of scalability is in frequency versus computation. If the operation is commonplace (such as moving around), then a distributed system has a problem. If the operation is not commonplace (such as joining a guild!) then it's painless to use the 'choke' of a manager class to resolve any issues.

Even in the commonplace situation, I would have thought it useful to use overseer-objects whose job it is to remove the extra (unnecessary) information from the problem before trying to solve it... There's no need to care about the avatar in sector (-1000,-1000) if we're currently in sector (0,0)...

It's a cliche, but the rule is 'divide and conquer'. Screaming and leaping is a satisfactory, but usually fatal approach to problem solving, unless you're Kzin.

Simon

Re:Scalability and joining guilds

[ackthpt]

Surely this is a classic example of the Manager pattern.

I dunno, I've seen groups where there are 5 managers and one peon. Managers seem to accumulate.

You have a bunch of objects [Avatar] (all alike, at least programmatically :-) who want to perform operations on other objects. If the system has a [GuildManager] class, then access to this distributed network of avatars can be forced through the choke-point of 'can this avatar join this guild'.

Uh. By the time you're an Avatar you shouldn't be joining guilds. You should be heading them. Unless, and I may be mistaken here, Avatar is now an inflated title for someone who's really the assistant to the vice grunt.

Re:Scalability and joining guilds

[Anonymous Coward]

This problem is in no way limited to MMORPG, the problem of authenicating and managing objects across multiple servers/clients is central to all online games. As a hobby games developer with a pretty good understanding of this I suggest you read Policing Online Games and then compare the conceptual pitch to issues in digital cash and online money transfers etc.
These ideas also overlap with the much hated and draconian 'trusted computing' models.

Enforcing a set of rules across a network of untrusted hosts is a fascinating problem. For example Gnunet and Freenet forgo a centralised trust agent and allow trust to emerge from the interaction, and recorded past behaviour, of individual nodes.

Digital 'trust' is sure to remain a huge area of interest. However it will also continue to be an area dominated by soothsayers, witchdoctors and charlatans because it contains a numer of fundamental logical problems which are not solved in the traditional human way of appeal to authourity.

Re:Scalability and joining guilds

[RollingThunder]

Avatar doesn't mean any particular "agent of god" powers in this message. He's using Avatar as the generic word for "the persona that a player plays as".

Re:Scalability and joining guilds

[t0qer]

Even in the commonplace situation, I would have thought it useful to use overseer-objects whose job it is to remove the extra (unnecessary) information from the problem before trying to solve it... There's no need to care about the avatar in sector (-1000,-1000) if we're currently in sector (0,0)...

Been there, seen that. Alternate Reality had a creature in place for doing just the thing you were describing above. Taken from the AR Faq.

The problem with this was that in the AR system each object was unique (except commodities like food, money, gems) and had a structure with attributes (like "spells" which were small intrepreted programs embedded in objects). Those data structures took up memory (16 bytes to about 64 bytes) and on an 8-bit 64K (or 48K) computer we had to limit the amount of items somehow. The way we came up with that was least artificial was to introduce a creature that would eat up objects when the object queue was getting full.

Re:Scalability and joining guilds

[bentfork]

Surely this is a classic example of the Manager pattern.

I dunno, I've seen groups where there are 5 managers and one peon. Managers seem to accumulate.

Looks like some people out there need to review what design patterns are.

from first result:

Patterns and Pattern Languages are ways to describe best practices, good designs, and capture experience in a way that it is possible for others to reuse this experience. The Hillside Group takes pleasure in sponsoring many different PLoP conferences that are provided for the betterment of the pattern community.

Re:Scalability and joining guilds

[humankind]

Even in the commonplace situation, I would have thought it useful to use overseer-objects

I know this is done to some degree in Everquest. There are NPCs in each zone that exist to augment existing zone-related, PC and NPC situations.

For example, in each zone in EQ, there's an invisible NPC called, "pain and suffering" which appears to inflict damage on a player in certain situations (falling or bleeding to death). I would imagine that similar objects exist to control the weather, which in many cases might signal the client to narrow a player's depth of view and receive less information on objects in the vicinity.

Re:Scalability and joining guilds

[FuzzyBad-Mofo]

*PLOP*

Summs up my opinion of "design patterns" nicely, yes it does.

Re:Scalability and joining guilds

[MMaestro]

Indeed, digital 'trust' is something that cannot be easily handed to, let alone established. Not counting real life relationships, anybody who meets another person online immediately (should) distrust that person. In a sense, this is destroying the foundation of the internet (the sharing of information openly and freely) but it not only works, it is also necessary (sadly). Why? Simple.

When you play a game online, would you rather trust "HarryGoatDeezNtz" who has an absolutely offensive name but you've seenen play online and play well for over 3 months, "UnnamedNewbie(6)" who you've never seenen before and is asking how to play in the chat, or "KingofSpades" who is an absolute asshole with no skills but has been playing the game longer than anyone other than the developers?

The same thing is true with businesses trying to estabish themselves online.

Would you trust Microsoft's Windows which is virus/bug/hack/etc prone, Linux which would require you to hire a full time IT staff just to keep your servers and computers working, or Macs and have your staff constantly ask 'wheres the right mouse button?'

So the question remains? Do you trust the 'veteran' of the three? The 'pro' or take a chance and give the 'newbie' a try?

Re:Scalability and joining guilds

[Anonymous Coward]

1. Acquire any USB mouse with two (or more) buttons and/or scroll wheel.
2. Plug into Mac.
3. Right click and scroll AS MUCH AS YOU BLOODY WANT.

MMORPGs need better real-time characteristics

[Bryan Ischo]

I've tried a few MMORPGs and have found them all to be lacking in the same key area: one's control over one's character is not real-time. This is a generic description of a problem which surfaces in many ways in MMORPGs, most notably in the combat system. I haven't found one yet that allows real-time combat; it's always "click on the guy you want to fight and press the 'attack' button", then sit back and watch. Typically can do things like cast a spell or use a buff or otherwise make strategic changes to the way that your character is fighting, but you can't aim, run around, swing at the monster, etc, as you can with first person games.

The game that comes closest to the combat system I would want is Jedi Academy, in which the multiplayer mode works just like the first-person real-time perspective of the single player game. You do have to aim, you do have to run around and avoid shots, you do have to swing your light saber yourself. I find this to be infinitely more enjoyable than the MUD-like "you hit the spider for 10 points, it hit you for 5 points" back-and-forth that is common on all of the MMORPGs that I have played.

One gets the feeling in playing these MMORPGs that your client view of the world only loosely approximates what is happening on the server. You can make your character run from here to there and find that other people are "sliding" by or popping in and out as you get only sporadic notification from the server of what's really happening. It all gives a very disconnected feel that I really find unappealing about MMORPGs.

There must be some kind of scaleability limitation though because Jedi Academy only supports about 30 players or so at a time in an area that is far smaller than a play area in an MMORPG. I think that if someone could design an MMORPG that played like an FPS, but had all of the depth and breadth of one of these not-so-real-time MMORPGs, it would be ideal.

As an aside, has anyone beta tested Worlds of Warcraft? It like an excellent execution of the MMORPG genre, but I have yet to read any comments from beta testers on whether or not the fighting is real-time or "faked" like other MMORPGs is ...

Re:MMORPGs need better real-time characteristics

[Dodger73]

This burns down to the scalability problems the article is mentioning. Real-time characteristics always mean more frequent transfer of potentially larger data packages, and the more frequent processing of those packages. While you may be able to run Jedi Academy with 30 players on a cable connection, the same is not necessarily true for 300 players.
There are ways to at least make bandwidth and processing requirements scale less than linearly with the numbers of players, but the actual problem persists. The more players, the more data. The more data, the more bandwidth requirements and the more latency. The more latency and bandwidth requirements, the more the realtime characteristics suffer. Needing halfways reliable security (read: hack protection) methods doesn't make it any easier.

It is not only the reason why MMOs aren't realtime like an FPS, but also why FPSs aren't MMP like MMOs ;)

Re:MMORPGs need better real-time characteristics

[MaineCoon]

The issue here is you are looking at MMORPGS. RPGs are traditionally turn based combat. I haven't seen many successful RPGs that required one to actually swing or fire their weapon manually.

There are two massively multiplayer games that feature realtime combat

* Planetside, which is an MMOFPS with RPG characteristics (levelling, improving your character by gaining extra implant slots and additional simultaneous skill sets)

* Neocron, which is an MMORPG/FPS (I may be wrong on this one, it was a while ago and I only played the offline trainer, which was supposed to simulate online play)

One of the biggest issues is lag; to reduce lag, which would get horrendous when there are many people in close promixity doing things, the client-side visual representation and simulation, and the server side simulation are never in sync with each other. The server is the final arbiter, but the client tries to the best of it's ability and available information to provide a visual representation of what is going on.

Planetside (and I assume Neocron) solves the lag issue by moving combat resolution for attacks to the attacker's client, and trusting the client's integrity. As a result, you can easily die 3-4 seconds after running behind cover; likewise you can run through an intersection, be in the clear on your end, yet be shot and killed 5 seconds later as someone sees you go through the gap 3 seconds earlier and shoots you.

- MaineCoon

Re:MMORPGs need better real-time characteristics

[Surlyboi]

There must be some kind of scaleability limitation though because Jedi Academy only supports about 30 players or so at a time in an area that is far smaller than a play area in an MMORPG.

You hit the nail on the head with the scalability issue. Unless you're playing a game like Planetside where there's no significant penalty for dying, (other than just having to respawn and grab more gear) you're going to have a lot of unhappy players who get 0wned by the LPB twitch freaks.

I think that if someone could design an MMORPG that played like an FPS, but had all of the depth and breadth of one of these not-so-real-time MMORPGs, it would be ideal.

I agree, it would be spectacular. But as it is, there're a ton of people playing SWG who'll just spam damage on players as they load into new zones. Unless everyone in the world is on the same footing connection-wise and the ganeworlds are seamless; a real-time implementation of a combat system would only compound this kind of grief play.

Re:MMORPGs need better real-time characteristics

[fewnorms]

Dunno, but I've been playing this Eve Online stuff and it seems pretty realtime to me. And it supports 7500 people at the same time with no lag... And I must say it's pretty fun to play too, if you like space rpg's ;)

Re:MMORPGs need better real-time characteristics

[AuMatar]

I've played damn near every MMO out there. if one came out that made me aim and run around dodging, I wouldn't play it. Its not what I want from the genre. I've done twitch combat. I've done it for 20 years. It bores me. So you can hit the left arrow button faster than me. I don't give a damn. Currently MMOs test if you can use your skills and tactics to out think me. That is fun. And luckily, it seem the genre will stay like this for some time to come.

Re:MMORPGs need better real-time characteristics

[Bryan Ischo]

Well you are lucky that there are DOZENS of MMORPGs that satisfy your gameplay requirements. I am just looking for ONE which satisfies mine.

For what it's worth, the MMORPGs that I have played are pretty weak in the strategy area anyway. Really there is no reason for a fight to last more than 1 second anyway. It might as well work like this: you click on the spider, the server pre-calculates how much damage it would do to you and you would do to it, and the server does the damage and it's done. There is no reason to have to sit and wait while your avatar hack smindlessly at the spider at a pre-determined rate and the spider does the same to your avatar. If there is no skill involved in the actual fight, then just skip it and go to the results!

Yes, it is true that you can cast spells and such, or switch to a different weapon normally. But I've found that it just leads to a formula which you use over and over again when fighting. You click on the spider, you say attack, when it hits you you heal, you watch the attack while you want for your mana to recharge so you can heal again, etc. You might as well just code all that up into a script that you run whenever you want to attack a spider.

I like real time fighting because it brings a fun arcade-y aspect to the game. It also makes it feel like you're more "in" the world and actually controlling your avatar, instead of just sitting back and watching what could have been a MUD anyway if it weren't for the 3d graphics.

Re:MMORPGs need better real-time characteristics

[cbreaker]

Exactly. A 20 person UT match requires a surprising amount of bandwidth alone to make it as "realtime" as it can get, and a lot of CPU usage. If they were to try and accomplish the same thing on a scale of 5,000 players per server cluster at any given time, it would require too much bandwidth and resources to be profitable.

I see this becoming better in the future as CPU power and bandwidth get more and more available, and the prices of these games get higher and higher.

Re:MMORPGs need better real-time characteristics

[harlows_monkeys]

I've tried a few MMORPGs and have found them all to be lacking in the same key area: one's control over one's character is not real-time....Typically can do things like cast a spell or use a buff or otherwise make strategic changes to the way that your character is fighting, but you can't aim, run around, swing at the monster, etc, as you can with first person games.

That's because of the RPG in MMORPG. In an FPS game, it is supposed to be a contest of your skill and reflexes vs. mine. In an RPG, on the other hand, if I'm a 20th level Fighter and you are a 10th level Fighter, I should be able to always beat you on physical skill. The only way you should be able to win is if I do something strategically wrong. Hence, the lack of detailed control over the physical aspects of combat.

Re:MMORPGs need better real-time characteristics

[WorkEmail]

Have you played the game Planetside? I found that to be a nice blend of MMORPG and FPS. It is massivly online multiplayer. I did not find it "shooter" enough for my tastes. With the big worlds and many many players,, they were not able to focus enough on weapon physics and accuracy. I too felt slightly detatched and delayed from what was going on.

I have also played The Sims Online, and that game I really liked. I played it constantly for about 2 months, maxed all of my skills, had a huge house, etc, and then quit.

Another huge problem for MMORPG's is to keep people wanting to play. I have friends who have been playing DAOC (Dark Age of Camelot) for a few years now and have had their skills maxed, and have been basically running around with the same group doing the same thing for 2 years. MMORPG's also need to focus on adding new items, new objects, and new quests, etc, more often. I like the Sims Online idea of only offering certain items for purchase for certain time perionds, so there is only so many of them floating around. It adds to an items appeal when new items are added, and old ones cease production, makes it feel more specialized, and real feeling.

Re:MMORPGs need better real-time characteristics

[SnappleMaster]

Have you tried Dark Age of Camelot? Combat is "turn-based" like most MMORPGs but because of the way the combat system is designed, competitive player-versus-player combat is really more like FPS than "traditional" MMORPG.

It may not be quite what you're looking for, but combat in DAoC is certainly not "click attack and wait for someone to die".

Re:MMORPGs need better real-time characteristics

[Bryan Ischo]

I did, but I didn't get very far. I found it boring to have to run around the initial little town performing mindless tasks (take this letter to that guy over there, only we won't tell you where he is so you have to run around all over the place looking for him), and then when I did get out of the town and had to fight a bunch of hyenas or whatever, the combat was just point-and-click-and-wait like the other MMORPGs I have played, and then when they died they dropped *bags* that were supposed to represent hyena teeth. I just found it all very unappealing.

But I'm probably far too critical of these games ...

Re:MMORPGs need better real-time characteristics

[Fourier]

I haven't found one yet that allows real-time combat; it's always "click on the guy you want to fight and press the 'attack' button", then sit back and watch.

Yeah, that's always bothered me too. Why do you have to click to attack? Surely the computer can take care of the attacking all by itself. I'd rather just sit and watch, or maybe go grab a pizza and sit in front of the TV while my Half-Halfling Organist decapitates some Hair Elementals in the Killing Fields. Good times.

Re:MMORPGs need better real-time characteristics

[EvilTwinSkippy]

My answer is write a game that plays itself. Rather than be a contest between who can keep the most number of balls in the air (ala RTS), we should instead write a system where you code your own AI's, and let the computers to the dirty work.

It would be like a stock-market simulator. You send your moves in, and you can check your porfolio, but the game goes on no matter how much (or little) attention you pay.

My thought is that you set the game in a universe where it's only possible to send robot fleets everywhere. As the Lord High Stick In the Mud, you control the fleet for your planet. Inhabited worlds are off limits. (So if you space off and your fleet and colony are destroyed, you can always start over.) It will really be about who can devise the best automated strategy, because everyone's got to sleep sometime.

Anyone interested, my address is out there. TTFN.

Re:MMORPGs need better real-time characteristics

[Anonymous Coward]

Right. Rather than by physical skill or some other merit, the combat is decided based on which fat guy spends more of his free time sitting in front of his computer, levelling up his character.

Re:MMORPGs need better real-time characteristics

[dwpro]

I too wish that there was more strategy in the fighting for most mmorpgs...I play final fantasy XI, and there are elements of strategy(you have specific abilities that can be used in succession with other characters that do more damage, heal life, ect.) but for the most part it lacks what you are looking for. Also, such a heavy reliance on other party memebers is particularly frustrating sometimes, but that is another story altogether.

Re:MMORPGs need better real-time characteristics

[Eskarel]

Well the biggest problem with this idea is that, regardless of whether they could, very few companies would choose to design an rpg, let alone an MMORPG in the style which you suggest. It has been done to an extent(Morrowind is really the only example I can think of off hand), but one of the primary aspects of most rpgs of any sort seems to be the idea that your ability is determined more by your characters skill than by your own(even in morrowind all you can really do is hack and slash, blocking and armor protection are entirely controlled by your skill level).

Another problem is of couse the fact that a large percentage of RPG's take place in at least a pseudo mystical/medieval world(I know there are exceptions), and pretty much without fail every single FPS which has taken on this sort of combat style has been mediocre at best, melee combat does not translate very well to the world of the first person shooter.

In essence the problem is not so much the fact that it couldn't be done(it could, especially if you were willing to place relatively high system requirements on the product and take the financial losses from limiting your audience), but that the demand is not there for this particular genre.

In a somewhat side note, I too am somewhat looking forward to World of Warcraft(though I won't be able to get it until I can find steady employment which isn't all that easy in this world), but more because they have been selling the idea that the game is playable in lower amounts of time. Many games, especialy those where you cannot restrict who you play with, suffer because a casual player simply cannot really survive amongst those who have more time to spend.

Re:MMORPGs need better real-time characteristics

[Anonymous Coward]

*cough*Planetside*cough*

Re:MMORPGs need better real-time characteristics

[cbreaker]

I was thinking of Planetside, but I know nothing about the game so I couldn't really say anything about it.

Do you play it? Is it good, as far as FPS goes?

Re:MMORPGs need better real-time characteristics

[RollingThunder]

World War 2 Online has sort-of beat that. It's a MMOG FPS, in essence.

Basically, there is a 64 unit "visibility limit". You're only ever told about 64 units max (and sometimes, due to oddities, less) player entities around you, prioritized among several criteria (distance, threat, minimum friend/foe allocations), etc.

It works fairly well, and the structure of the game is such that you have dozens of 30-60 player battles going on at all times, and can move anywhere around the map as you choose, realtime, either by slogging it on foot, driving, flying, or steering your ship. You can also jump from place to place but leave your equipment behind.

Best estimates put the peak server load at about 3-4000 players, with 500-1000 during the low tide, but the game runs 24/7 on a single arena.

The developers aren't swimming in money, but they're in the black and have recently turned up the data update rates to make it more smooth, so there's evidently some room in the budget for bits.

Disclaimer: I'm a day one player, from June 6 2001 on, aka Krenn, of the 1/16 Panzerdivision "Windhund".

Re:MMORPGs need better real-time characteristics

[L7_]

The more that I've been reading /. lately, the more it seems that Game Myth aka Risk Your Life has the right way to go about it (at least in terms of the various games that I've been looking at). Although the English engine is currently in development, it seems to be the solution that a lot of pvp type MMORPG players are looking for. It is a persistent world with character development and learned skills and such, but the combat (spell, ranged, and melee) is all FPS like. Meaning you have to pick and choose which abilties you want to learn as you 'level'. It really is a neat-o system.
Its too bad that

Re:MMORPGs need better real-time characteristics

[cbreaker]

It sounds interesting.

I think you can download Planetside. I've always been a fan of FPS's, and this looks like a pretty neat game if it's done well. From what you're saying, it sounds pretty good (at least no worse then your average server running single-map online games like Quake.)

Re:MMORPGs need better real-time characteristics

[Moraelin]

No offense intended, but I'm thinking you might have the wrong idea.

Yes, you are right: the current crop of MMO games are dead boring and involve no strategy. They're a boring, repetitive affair in which you just click on the monster, then wait and see when you have to heal. And, yes, it's such mindless repetition that you could just write a simple script that does it.

(And some people on MUDs do. It's a banning offense on many MUDs.)

Worse yet, there is no story, no plot and no justification at all. Why am I beating these rats with a stick? What have they done to me? Does my beating up rats save the world or anything? Nope, it's just for a level-up. At which point, you'll be allowed to wield a slightly bigger stick against slightly bigger rats.

If the sheer mindless repetition didn't suck all the fun out, the realization that you're doing something pointless, useless and unneeded will. You're asked to do something as pointless as moving a pile of sand from here to there, and then moving it back to where it was. Repeatedly. For no other reason than that you might eventually get a promotion for it. (I.e. level-up.)

That's not gaming, that's _work_.

Worse yet, most of these companies are greedy. They want you to keep p(l)aying for as many months as possible. This is done not by adding more content, but by quickly increasing the time you need to invest for a level-up or other reward. So that promotion which is the only reward or justification, comes less and less often. It gets quickly to the all work and no reward in sight point. Where you have to hack monsters for 6 months for your next promotion.

So not only it's turned into work, it's also turned into being less rewarding than your day job.

IMHO most of the complaints about MMO games stem from these problems.

However, I really don't think that making it a twitch game would really be the answer.

At the end of the day it would still be both repetitive _and_ pointless. Real time or not, you'd still be beating the same rats with the same stick. And still having to beat another thousand of them for your next reward.

Exactly what's the improvement?

Basically what I'm saying is that what needs a total rethink is the very idea of these games. Just tweaking the combat code will at most change the tip of the iceberg.

Re:MMORPGs need better real-time characteristics

[Anonymous Coward]

I'm actually working on something like this. One of the criteria for my project was low (server) bandwidth, so executing scripts server-side makes a certain amount of sense. Another one of the requirements was to get rid of cheating, basically by making 'cheating' (robots) part of the gameplay. Hey, if it's in the rules, it's not cheating, right? :)

Re:MMORPGs need better real-time characteristics

[Psychochild]

There are games that work close to what you want; however, as other people have pointed out there are problems with latency in heavily "twitch" based games. Appropriate "twitch" mechanics require fast reactions, faster than the time it takes for the client to communicate with the server in most cases.

Another issue you run into is cheating. Think about the aimbots for most FPS games. When you're paying a monthly fee, there's the expectation that there won't be cheating in the game. Therefore, most online RPGs focus on mechanics that can be verified and can't be cheated.

However, there are some games that have compensated for this and offer some level of twitch gameplay. Planetside, World War II Online, and Neocron are two FPS-type games. Asheron's Call had some twitch-based elements in that you could "dodge" projectile weapons. My own game Meridian 59 has some twitch aspects to it since the game encourages a lot of movement, especially in Player vs. Player (PvP) combat. Knowing where to move in order to outmanuever your enemies is a key skill in M59's PvP. The lower system requirements of the game also means that you have pretty low latency which lends itself better to twitch style.

There's options out there if you care to look beyond the highly advertised games.

Re:MMORPGs need better real-time characteristics

[Surlyboi]

I dig it. It's a fairly comprehensive FPS experience with a lot of nice extras. The only real problem I have with PS is how often the nerf bat comes flying out every time one empire or another thinks their prime exclusive weapon or vehicle is underpowered compared to the other empire's weapon or vehicle. And yes, the whining flies fast and furious on the PS forums.

Re:MMORPGs need better real-time characteristics

[Anonymous Coward]

Like "Codewars" or "Redcode" in the 80s?

Re:MMORPGs need better real-time characteristics

[EvilTwinSkippy]

Heck, "cheating" should be part of the strategy. The whole science of espionage is knowing what you rightfully shouldn't, and hitting your enemy where they least expect it. If you have the skills to intercept the communications between the enemy's ships, that's perfectly fair. I'm even thinking it would be cool to have a facility that allows you to 0#N38 someone else's ships.

There should also be a race of omnipotent droids (think The Day the Earth Stood Still) that will clean up anyone who tries to control too much of space. It's not the UN. It's the disinterested ass-kicking squad bent on making sure no side gets too much of an advantage to disrupt the game. If you grow to be a constructive power, no problem. But if you start using your size to push other people out of the game, they will happily destroy every ship and space station you own, forcing you to start over.

Like the greeks, periodically every player in the game gets to vote for the Ostra. The "winner" of the election gets his/her entire empire obliterated by the Gort. (Home planets are left alone.) To be Ostracised should be considered an honor. It means you were powerful enough to be a threat.

Even if your entire fleet is destroyed or captured, you always have your home planet to fall back on. Trying to occupy a world of billions of people is counterproductive, and the rules of interstellar war prohibit it. Attacking an inhabited world bring upon the instant retribution of the Gort.

Re:MMORPGs need better real-time characteristics

[ScuzzMonkey]

Another point worth mentioning (and one of the best kept secrets of the game, IMHO) is a fairly revolutionary update system which virtually eliminates lag differences due to bandwidth constraints. I've played this on 56K and DSL, and there is absolutely no difference between the two. Although there is still lag, of course, it's not dependent on your own connection speed, but rather the more even latency rates across the Net. It's a huge equalizer for HPBs who are typically pwned in most online FPS contests.

Re:MMORPGs need better real-time characteristics

[cbreaker]

hah yea I know how it is.

I've played EQ for a long time (until the last few months) and people love to bitch.

Sometimes it's legitimate. Most of the time it's not. And it could always be handled a little better then "you sux i h8 druids!!11"

It's all part of the Internet, unfortunately.

Re:MMORPGs need better real-time characteristics

[nothings]

RPGs are traditionally turn-based combat.

Try the grand-daddy of first-person PC games, Ultima Underworld. You interactively swung your sword for every blow.

It was released a few months before Wolfenstein 3D, and is widely credited for spawning the first-person CRPG.

The lag problem is in no way unique to MMOGs; the description you give for Planetside is exactly how Valve has described Half-Life's multiplayer working.

Re:MMORPGs need better real-time characteristics

[BlackHawk-666]

Ultima Underworld was not a network game like a MMORPG is, so it didn't have to deal with networklag and the like.

Re:MMORPGs need better real-time characteristics

[Kaa]

Sure, no problem.

http://www.phial.com/angborg/

Article linked without all the crap

[(1337) God]

That page is annoying. For anyone on low-bandwidth or Lynx connections, here is the printer/human-friendly page.

Re:Article linked without all the crap

[Michi]

You can find a PDF version of the full article (exactly as printed in the magazine) here.

Cheers,

Michi.

It's the Economy, Stipud

[ackthpt]

As my Econ professor used to say, "Economics is the study of scarcity." So they need a scalable economy engine, right? Compatibility of guilds is another matter.

Here are some initial beta screenshots

[(1337) God]

- Dragon
- Cute guy with his shirt off
- A bunch of players chatting
- A troll

If you're really bored, there's a whole index.

Re:Here are some initial beta screenshots

[bartash]

Doesn't look any better than EQ :-)

Re:Here are some initial beta screenshots

[morcheeba]

Cute guy with his shirt off

The cute one has his skin off, too.

Re:Here are some initial beta screenshots

[geekpuppySEA]

Oy, what have we come to when Mutable Realms counts as pr0n for this poor guy?

Baby, you need to get out more. After a few drinks, they all look like the avatars of your dreams. :)

"Authors thought of everything".

[SharpFang]

Lots and lots of modern cRPG, online or not, have way less features than games like Nethack. Just think what you can do about a towel - wrap it around your head, dip it in a fountain, wipe your face. Lots and lots of options. Incredible richness. Seems it can be done after all...

Are there any online rogue/nethack clones out there?

Re:"Authors thought of everything".

[Tofino]

Amazing what you can do with an open-source project with over a decade of development behind it. If only there were an operating system that worked under this principle!

Re:"Authors thought of everything".

[Anonymous Coward]

I might be wrong, but isn't Nethack text based? It's a lot easier to write a description of an action being carried out, than to actually simulate the action in a virtual world (with physics etc). Animating a towel being dipped into a fountain (an animation which must be carried out by all sorts of different player types and models), is hardly comparable to just writing "you dip the towel in the fountain" in some data file.

Re:"Authors thought of everything".

[Anonymous Coward]

Certainly, but that's an engineering problem. Ideally, your systems would evolve to become sophisticated enough that dipping towels in a fountain requires as much coding effort as 'you dip the towel in the fountain'. Hey, if you had to write Nethack in binary, it'd be hard, too. :)

Who needs 10,000 people in a zone?

[bartash]

They say:

Wish is the first Ultra Massive Multiplayer Online Role Playing Game (UMMORPG(TM)). "Ultra" means that Wish supports more than 10,000 simultanous players in a single, seamless world, without any zones or "shards".

In EQ you can have an effect on other characters in your zone (say a hundred people) but you can talk with all the other people on your server (thousands, maybe tens of thousands of people). This is a limitation, but in practical terms it works OK. I don't actually need to interact with more than a few people at a time.

Re:Who needs 10,000 people in a zone?

[WarderDot]

Will it be a Super Duper Ultra Massive Multiplayer Online Role Playing Game (SDUMMORPG) when a single application server supports tens of millions of users?

Re:Who needs 10,000 people in a zone?

[chad9023]

Having played DAoC and SWG in the past, I know it can be a pain to try and coordinate you and all your friends playing on the same "shard". By having everyone playing in the same world, you eleminate that issue, which would certainly be appetizing for those who enjoy playing with friends.

Scalability problem?

[Anonymous Coward]

I don't see a scaling problem in the example.

Why not just keep a variable on each character associated with each guilds he's in. And then, each guild keeps a list of members.

Thus, a query on the data is always fast and local.

Re:Scalability problem?

[Anonymous Coward]

Um, you most definitely do NOT want that data to be too local; that is, stored on client. That's asking for trouble. In no time hacky players will find a way to modify it locally, and get access to things they are not supposed to.

As to having it in multiple places (local caching); sure, but problem then is cache coherency. It's nothing too specific to such games, of course... it's rather common problem on any distributed system.

Re:Scalability problem?

[OzJeep]

There is a design problem with that. You have the same information in two places. Even if both pieces of information are stored in the same place, all the code that affects it must know to update both places. This can lead to the subtle "random" bugs that make us developers drink so much. :)

They should focus on a game that has shipped.

[Anonymous Coward]

Everybody has their own MMO now, but unless they've actually shipped one they really probably don't understand the true challenges in building one.

Wish sacrifices

[Rexz]

You make major one major, major sacrifice for so many simultaenously players in Wish. Movement is point and click. It feels like you're playing a strategy game rather than living in a real world. Those of you complaining that you can't joust and dodge in today's MMOs will hate the stilted movement mechanism of Wish, where the path you take is left to a pathing routine.

Re:Wish sacrifices

[ScooterBill]

I agree this kind of movement (point and click) is non-immersive. The same goes for third person view. The best I've seen for movement, viewpoint and most important, content, is Everquest.

This is not an easy thing to solve. I would say that a MMORPG is not mature until it's been online for at least a couple of years. You can't reach a stable environment until you let the system find it's equilibrium. After this, it becomes tweaking and expansions. But let's face it, the average player's bandwidth and computer is not going to give you FPS action in a 10,000+ player world.

Re:Wish sacrifices

[Anonymous Coward]

Everquest, where you can't even see anything around you? Fight a monster melee and you interact with it by clicking on its floating name, because it's out of your field of view? And totally impossible to get the tactical overview that comes with an overhead perspective. Everquest style is the equivalent of wearing blinders and having no neck, which isn't immersive at all, since you spend your time wondering where the hell that thing that's attacking you is.

If you want a decent interface look at Neverwinter Nights. The overhead point and click system is infinitely superior for tactics and strategy, moving through clautrophobic areas, and dealing with objects around you. But you can seamlessly switch to keyboard based third person control for immersive perspective and dodging in combat.

Re:Wish sacrifices

[ScooterBill]

It's much more subjective than that. There are those who prefer to know all the details of their hit points, mana, etc., have maps, radar, multiple viewpoints, alerts, etc. I've found that all these things tend to interfere with the feel of the game. If something hits me from behind, I should probably turn around to see what it is. I think the current crop of interfaces aren't good enough and so the designers give us things like 3rd person view to make the game easier. That's ok as long as it isn't the only way of playing.

I guess it becomes a tradeoff between ease of play and the reality of what the character would need to do in real life. Everquest used to be much more demanding of the player. There were no on-screen maps, no compass(unless you bought one). You got around by remembering the scenery. They've dumbed it down IMHO, the same as the other new MMORPGs.

I haven't tried NWN. Perhaps I should.

Lacking confidence here...

[Fiz Ocelot]

I don't really get a warm fuzzy feeling of confidence after seeing things like this:

"At ZeroC we used Java because some of our development staff had little prior C++ experience..."

"...however, a few of us had previously built a commercial object request broker..."

"...designing and implementing middleware is difficult and costly, even with many years of experience. If you are looking for middleware, chances are that you will be better off buying it than building it."

Frankly, I'd feel rather uncomfortable using ICE 1.0 as middleware for my new MMORPG. Yes they could succeed and do a nice job, but that rarely happens especially in the world of MMOs where nearly all games are released way too early in beta form.

Re:Lacking confidence here...

[Michi]

> "At ZeroC we used Java because some of our
> development staff had little prior C++
> experience..."

Actually, this was something the copy editor messed up, unfortunately. At ZeroC, there is absolutely no shortage of C++ skills :-) The sentence was meant to read "At Mutable Realms...", where some (but not all) developers have C++ experience.

In terms of whether you want to use Ice (or another middleware product) for your game, that's up to you, of course. But taking on all the networking chores during game development (as well as all the other complex things) is a lot of work, and middleware certainly can reduce development time by a lot.

Cheers,

Michi.

Re:Lacking confidence here...

[alien_blueprint]

Actually, this was something the copy editor messed up, unfortunately. At ZeroC, there is absolutely no shortage of C++ skills :-) The sentence was meant to read "At Mutable Realms...", where some (but not all) developers have C++ experience.

Thanks for clearing that up - I wondered about that. It jumped out as probably the most untrue statement regarding the skill-set of an organization that I have ever seen! ;)

Its not fake, its a dice game

[rufusdufus]

The RP in MMPORG means Role Playing ala Dungeons and Dragons. Thus, the focus is on character development, not first person shooter style twitch. RPGs are based on a dice game, and is really about mathematics. The people with powerful characters are the ones who can do math, not the once with a cable modem in the same town as the server.

Re:Its not fake, its a dice game

[Pxtl]

No, they're the ones with no lives that have been logged on and fragging critters since time immemorial. I prefer games that don't preclude a social life, thankyou.

Nothing To See - Move Along

[Anonymous Coward]

Wake me up when you have something written by someone who actually has written and shipped a commercial MMORPG.

Do some of the work client-side...

[mooman]

Given their monstrous system requirements:
* P4 2.0GHz; P4 2.8GHz recommended (or Athlon equivalent).
* 512MB RAM; 1GB recommended.
* 64MB DX 9.0 Video Card (GeForce 3/4 Ti; ATI 8500+); 128 MB GeForce FX or Radeon 9600+ recommended.
* 16bit Sound Card; 24bit recommended.
* 8 GB free disk space; 7200+ RPM recommended.
* Connection to the Internet; 33 Kbps modem minimum; broadband recommended.

...maybe they should find a way to send datasets to the client machines and let them do their own manipulations.. Needing 8 Gig of disk on a 2+ GHz machine has to imply that the server doesn't handle all the real-time work... They are prime candidates for middleware that does some distributed computing and let all the customers' beast machines do the grunt work...

Re:Do some of the work client-side...

[frenetic3]

wtf?

The seti-at-home-type approach doesn't work here for several reasons:

First of all, they aren't implying that they're having scalability problems.

Second, on a MMORPG server, many of these calculations have to 1) take place instantaneously (latency of communication over the 'net precludes this) and securely 2) and operate on enormous datasets (i.e. databases containing world/player/object/etc information) that can't exactly be sent down some DSL line due to size and security reasons.

Third, you can never ever trust the client to make decisions that can adversely affect other clients or provide undue advantage.

-fren

Re:Do some of the work client-side...

[Peredur]

The problem with this is you cannot trust the end user. They will find a way to catch this information and use it to cheat. Everquest was plagued by this. Encryption is the only way to go if you are going to do this. However, you may find that you are spending more time finding a _easy_ encryption that is _hard_ to break.

Re:Do some of the work client-side...

[Dodger73]

That is one of the things to be done to improve on scalability issues. However, to prevent hacking, most of the data has to be at least verified by the game servers. Otherwise you end up with players using speed hacks to outrun you at 150MPH, fight four times as fast, or cast fifteen devastating uber-magic-spells at a time. So, again, you end up with the game server having to know everything, even if the frequency of processing here doesn't have to be quite as high.

Another problem, and this is the major issue I see with Wish's MSRQs, is, the target market. To date, and I don't see that one changing anytime soon, Asia is by far the largest market for MMORPGs, Korea being the largest in the bunch, with some 15 million people being involved in one or more MMORPGs (on a side note, interestingly enough, the 2D MMO Lineage being the most successful with supposedly over 3 million (!) subscribers. Let that one sit in your head for a while. Then imagine the continuous revenue stream at, say, $4 a month per subscriber. Scream. Read on once your antidepressants have kicked in).

The majority of asian MMO-players play in gaming cafes. Machines like, say, a P3/800 with a GeForce1 or Radeon, or equivalent systems, is the predominant amount of gaming horsepower you'll find in Asia. This is one of the big problems a developer of an MMO has to deal with, and it is one of the big reasons, why there are so many MMOs that either are not even released for the western market, or simply don't make it. The asian gaming landscape is quite different from the US and european ones. With its current requirements, Wish has little of a chance of wide spread adoption in Asia (unless everyone there suddenly decides to perform major upgrades on all of their machines). With accumulated development and infrastructure cost for a big MMORPG at launch time exceeding $8 Mio. more often than not, it is quite a gamble to not give the asian market major consideration in both gameplay and requirements.

Re:Do some of the work client-side...

[SnappleMaster]

Encryption in a MMORPG client is an inconvenience at best. The hax0r can read bytes right out of the client process after the data stream has been encoded.

Sure it sounds like a lot of work, but people have done it before for several games. Don't underestimate the amount of work some idiot will do just so he can cheat in an online game.

Re:Do some of the work client-side...

[Chester K]

...maybe they should find a way to send datasets to the client machines and let them do their own manipulations.. Needing 8 Gig of disk on a 2+ GHz machine has to imply that the server doesn't handle all the real-time work... They are prime candidates for middleware that does some distributed computing and let all the customers' beast machines do the grunt work...

Rule #1 of online games: Never trust the client.

Re:Do some of the work client-side...

[Thomas Charron]

The data will be found. Exactly as your example. Everquest. It still has the same old issues, if not worse, in that now it can all be wrapped up in the easy to use 'Macroquest', which, ironically enough, doesn't sniff packets, it attaches and reads the memory..

Re:Do some of the work client-side...

[Stray7Xi]

I disagree... I know everyone will say "Never Trust the Client" but it shouldn't be considered a rule, but taken as very sound advice. If it was to be considered a strict rule multiplayer FPS and MMORPG's just wouldn't be viable. In the mudding community there's been a few games that have tried to move to graphical versoins using a dumb client, they always suffer awfully from lag, very tilebased movement or slow movement.

There are certain things that are still appropriate to be done on client, in fact some are pretty much required. Most (all?) mmorpg's do movement prediction and collision detection (if it's in the game) on client side. This is where distribution could help the most.

Most games players CAN speed hack, it's possible and most of the security mechanisms are on client (so the client turns you in as a cheater). The GM's take a reactive approach, banning anyone getting caught doing it (and it's quite easy to detect a speed hacker)

So how can this be distributed? Make player's clients police eachother, every client observes the movements of X (3?) other nearby players. If they go over the speedlimit or break collision rules (walk through trees? other players?), flag them for the server. Get enough flags and the server will do calculations. If there's false flaggings (hacked client reporting others as cheaters) coming from someone, take less heed from their reportings (or investigate them).

This doesn't need to be instantaneous (because this is reactive not proactive, just like how mmorpg's are done now). This will work on huge datasets since it's being broken down into small problems (the bandwidth requirements may be a bit higher, but should be negligible). Finally, it puts less trust then the current model.

The problem with this is it still allows collusion (not to be confused with collision) of players with hacked clients. But they can't cheat near other players at the cahnce they'll be policed (which they pretty much have to do with current model anyways). So it'd work well in adversial (pvp/pk) games but maybe not in EQ-type where 3 people can go to a corner of a world and cheat out of sight (all 3 with hacked clients).

Distributed clients would be perfect for integrity checking of others. Since these games require you to trust the client to some extent. Furthermore it may not be game related, but its about time these patch servers took a BT approach.

Re:Do some of the work client-side...

[Anonymous Coward]

Actually, considering that network bandwidth is the biggest obstacle in MMP games, CPU time is relatively abundant. Performing encryption on every packet isn't a big cost; there are numerous algorithms (DES not being one of them) that are designed for efficient software implementation.

The real issue is that the encryption only protects the data communication from snooping; at some point or another, the client has to perform computations on the unencrypted data (as several other posters pointed out). Unless you can somehow figure out a way to do general, secure computations on encrypted data (and if you can, there's probably a Field Medal and early retirement in your near future), encryption doesn't help defeat client-side hacks.

Encryption is still necessary on some level (some information does need to be protected from snooping by third parties), but the user (second party) still can't be trusted.

This isn't a problem restricted to MMORPGs, by the way. Cheating in distributed projects like SETI@Home is a big problem, where you don't know whether the data you get back is right or not (as compared to projects like the old Distributed RC5).

One solution is to farm out the work to multiple (presumably independent) clients, and then use a majoritarian approach to decide if anybody's cheating. This would probably work in practice, but at the moment it's restricted to applications like SETI@Home that don't care about latency. You'll probably have to wait until we all have ubiquitous, 24/7 multimegabit wireless Internet connections and always-on computing to make that level of distributed computing a reality.

Re:I *SO* want this game...

[ScooterBill]

and here's the solution:

- Don't buy the game (it's cheaper that way)
- Don't play the game (you can't get spammed/cheated if you're not a player)
- Don't install the game (you'll save lots of resources this way)
- Dont' wait for it to come out (you can spend the time with your loved ones or go on a hike)

Shalom

Re:I *SO* want this game...

[Anonymous Coward]

- Resource intensive. Wish requires 512MB of RAM and recommends 1GB of RAM (yes, 1 BILLION bytes of RAM space).

Guess that's what you get when there aren't even enough coders in your software shop that know C++. You get huge applications that gobble up tons of ram just for a VM. Back in the day, 640KB was plenty...

Terazona

[meehawl]

one's control over one's character is not real-time.

If you've attended GDC then you may have played ZonaBattle, a real-time mechanized battlecar demo game for the Terazona MMOG system. Disclaimer: I work for Shanda Zona, the developer of this MMOG architecture, and my views may not represent those of the company, etcetera.

The purpose of the ZonaBattle technology demo is to illustrate that MMOGs do not have rely on sluggish, pseudo-turn-based gameplay. Using the right architecture produces excellent results.

ZonaBattle is not as fluid as some FPS games, but it is peppy and, unlike peer-based FPS games with~64-138 players, Terazona's client-server design enables you to scale the playfield to several tens of thousands of players and those players will experience no increased lag or message bottleneck.

Of course, you can also use Terazona to build "classic" seamless MMOGs. Terazona games do not have to have zones or "shards" and feature a heuristic, autoconfiguring grid system for game servers with dynamic region ownership, environmental simulation, and load balancing. You want more performance to support more players or more complex environment? Just slap in a few more commodity servers, or racks. The game will integrate them automatically and immediately begin dispersing Players and Entities among them.

Players can also exchange state with other local or non-local Entities using various bandwidth- and set-based configurable channels. This is not as easy as it might first appear.

Finally, the entire Server-side system is Java-based, for maximum flexibility and cross-platform support.

Re:Terazona

[Bryan Ischo]

Sounds cool. You lost me at the "java-based" though :)

Question: how does the game world map get distributed amongst the grid servers? Does each one have to have a copy of the entire world, or do they split it up somehow?

Re:Terazona

[meehawl]

how does the game world map get distributed amongst the grid servers

Dynamic ownership, distributed object-view model. Very similar to the system described in Queue. You would never maintain a complete unitary in-memory representation of a world - that sucks up too server juice.

I come from a CORBA background as well - what you see with all this kind of MMOG Middleware (Butterfly, Quazal, There.com, BigWorld) is a classic example of evolutionary convergent adaptation.

I forgot to add a standard /. angle - we support MySQL for persistence, among others. In fact, any RDBMS with JDBC should work. We also support Entity creation, modification, bandwidth optimization using an XML-based schema editor (written in Java, of course). So game programmers don't have to fiddle forever making lots of structs and trying to optimize dirty bitmasks for message delta optimization. This lets you get past the tedious stuff quickly and get to the game logic.

Re:Terazona

[Bryan Ischo]

I probably wasn't very clear when I asked my question. Or maybe my question didn't make any sense.

Maybe I could illustrate the problem I am wondering how Terazona solves, and you can tell me how you solve it (or at least give me a general idea) ...

The world includes models of objects which the player interacts with. Let's say a player is in a certain location and wants to walk north. But they can't because a tree is in the way. Whatever server is keeping track of the player's movements and doing collision detection or whatever, has to know that there was a tree there. So the server has to have a map of at least the part of the world that the player is currently in, to know what obstacles there are for the player to walk into.

What I'm wondering is, does every server have this entire map, so that it can, for any set of players distributed anywhere around the game world, keep track of what obstacles they are walking into? Or does each server only deal with a part of the map, and only keep track of the players that are in the part of the game world represented by the server's map?

The way that Terazona was described, I got the feeling that the servers divide the players up pretty much randomly, in which case each server would have to have a map of the whole world to be able to represent the area that any arbitrary player that they are managing is in.

Is that what it does? Or does each server keep track of a part of the map, and only those players that are in that part? If so, what does it do when there are many many people in its segment of the game world? And, how does the map get broken up by the servers? Automatically or does the game designer have to break the world up on behalf of the servers and assign servers to different quadrants of the world map?

Am I making any sense at al?!?

Re:Terazona

[killmenow]

Dynamic ownership, distributed object-view model.

I'm obviously not the parent poster and I'm not familiar with Terazona, but I'll take a stab at your question based on this sentence.

It sounds to me like what the guy is saying is this: The map is divided up (distributed object-view) among servers and no one server ever has the entire map in-memory. If everybody in the game moves to one part of the world, you've got servers sitting there doing nothing...so they switch (dynamic ownership) to the part where everybody is to balance the load. As players move out of that area to other areas, some of the servers would "move" with them so to speak.

Of course, I could be totally wrong on my understanding of what Terazona does because I'm basing it solely on what I've read about it here on /.

Re:Terazona

[mr_death]

What zona.net has developed sounds somewhat like the RTIME networking system. You might want to be careful here -- RTIME was issued a substantial patent covering their technology, and now that Sony owns the patent and the technology, zona might be getting a nasty letter sometime.

That sounds complicated

[PowerBert]

Lets all go play Savage on Linux instead.

It's the best multiplayer FPS/RTS game I know. Infact it's the only game that combines the two. It's great!!

Re:That sounds complicated

[Great_Jehovah]

Lets all go play Savage on Linux instead.

Is it actually available for Linux? The site seems to imply that it is currently Windows Only.

Re:That sounds complicated

[The Analog Kid]

No Savage runs on Linux, infact it comes with the binaries right on CD. The map editor also works on Linux.

Re:That sounds complicated

[Pxtl]

You missed BattleZone 1 and 2, then? Good games.

Re:Is it me...

[kewsh]

i was thinking the same thing. i suppose they are the nerd's game of choice whilst people like us like FPS and RTS

Real-time Roguelikes? Try Crossfire

[OgdEnigmaX]

Yep!

Stop wanking over nethack.

[Anonymous Coward]

It's a shitty game. There, I said it. It is, frankly, unplayable for all but the most "hardcore" and "1337" UNIX morons. It's shit. Yes, it is. Stop holding it up as the gold standard by which everything else should be compared. Wow, you can use a fucking towel in it. Big fucking whoop. Now gimme a player character with a fucking face.

Truly a Dilbert moment.

[psoriac]

I came across this quote:
At ZeroC we used Java because some of our development staff had little prior C++ experience.

and immediately though of that Dilbert strip (sorry, no link) mocking the "if all you have is a hammer, every problem looks like a nail" saying. That strip was particularly memmorable to me because the last panel featured a porcupine saying "we must stick them with quills! it's the only way!"

Re:Truly a Dilbert moment.

[marclaukien]

This is a typo in the article. It should say that some of the Mutable Realms staff don't have a lot of experience with C++. The authors of Ice have many years of experience with both C++ and Java.

Also, the philosophy of Ice is the exact opposite of the "golden hammer" anti-pattern. We fully recognize that there is no one-size-fits-all with respect to programming languages. That's why Ice works with C++, Java, PHP, and soon also with C#.

For Wish, C++, Java, and PHP are used. The client is pure C++, for performance reasons. The performance-critical parts of the server are also written in C++, the non performance-critical parts (or those there the performance bottleneck is the database, not the code) are written in Java. In additition to that, there are PHP scripts for account management and other administrative tools.

Re:Truly a Dilbert moment.

[humankind]

I agree with you and this is what really, really disturbs me about programmers and development projects these days.

In the classic sense, you select the tools/language best suited for the job. When you're dealing with huge amounts of data and lots of simultaneous usage, performance is going to be the difference between success and failure.

Yet, what do these people do? They choose a set of tools because development would be less problemmatic? Catering to the narrow specialties of their development team is a priority over the long-term performance and stability of the application? Or better yet, let's offload the responsibility for the game being "playable" upon continued development of new technology, which at some point might actually provide the resources necessary to make our badly-designed application perform adequately.

I understand the importance of selecting tools that are well-supported and easy to use, but some applications, such as MMORPGs really require hard-core performance. As a programmer, I've always felt it was a cop-out to force the companies I develop software for to throw more hardware at the application until it stops burping.

On the other hand, Slashdot is using Perl in real time, so maybe I should just shut up.

it's all about design & bandwidth

[humankind]

I really think computing power is less significant than the overall game development design when it comes to MMORPGs. After design, bandwidth becomes a factor, and only then is computing power a factor. The only exception I can think of would be requiring power for encryption/decryption.

The notion of parsing datasets for something like guild membership is really trivial. If you want to design a solid MMORPG, it's going to come down to how the world, objects and players are represented.

I continue to be in awe of the capabilities of games like Everquest and SWG. SOE has really created a very robust MMORPG technology -- it's hard for any other game developer to really say they have anything comparable when they can't demonstrate superior performance under the same conditions due to no other MMORPG having anywhere near the quantity of simultaneous players (as Everquest).

IMO, the client side of EQ is pretty straightforward. What makes the game special is the server side and how they manage to manipulate so many players and objects in real time. People complain that too many objects/players per "zone" can lag things down, and that is true, but I have yet to see a better implementation than Everquest. SWG has done away with the concept of "zones" to some degree, but basically, they seem to have implemented some client-side intelligence to indicate at which point additional graphics and information on objects in the distance should be loaded or reported. There are still "zones" in all these games. Some of them implement noticeable loading lags, and others don't.

My outside impression of the technical layout of Everquest is something like this... and I'd love anyone with more info/insight to correct me or elaborate further. I ASSume their system is made up of racks of servers, running Solaris I think. The have some low-level, propietary engine that manages the objects in the world, probably to a back-end database like Oracle. The reason for zones in EQ is that when you enter a new zone, you may actually be switching from one physical server to another. Not only do they have different servers for different shards/worlds, but different servers for different zones. When I see a system message such as, "North Karana, Velketors and Plane of Mischief going down for a brief update", I think that perhaps that's one server they're rebooting, which runs those particular zones. I suspect they stagger high-traffic zones with low-traffic zones on servers, and occasionally when the X number of zones managed by a single server have an unusually high amount of traffic/visitors, you get lag.

What's interesting about MMORPG game design is the balance between handling as much client-side as possible without creating security issues. If the server keeps track of players, NPCs and objects, it's much more difficult for someone to hack, or at least, logs are available to identify issues. The more client-side processing done, the more likely the game can be inappropriately manipulated.

When you take into account the amount of real-time data that goes back and forth, EQ (and SWG) are quite impressive. I don't think database/dataset issues are really the problem as being able to efficiently encapsulate, protect and send/receive the large amounts of data in the real-time world.

Re:it's all about design & bandwidth

[jerky42]

The EQ "zones" computers were running on NT 4 when it first came out, and for about the first 2 years. I don't know what it has switched to now.
The application basically only used the TCP/IP stack of NT, but yes, Virgina, it was on NT4.

I can't "prove" it, but I got that straight from a friend of mine, that was one of the original EQ team (and you would instantly recognize his Avatar name).

Re:it's all about design & bandwidth

[Stray7Xi]

You're right in that zones are on different servers but the real reason for the "Loading .... " is to load stuff client side. Remember when EQ first came out computers had a lot less memory, so they couldn't keep all those textures/models in memory.

The transfer from server to server shouldn't take long at all, the time to write the character file to database and load it again.

Everquest puts real variety in textures/models between zones, For example, a zone as a sunny green pasture with animals is adjacent to a desert, a city, a dark dreary forest. For a zoneless crossing, the computer would have to load the resources from all zones you can see.

This works in some of the other games (like DAOC) because one zone is quite similar to the next. the transitions from snow textures to grass textures takes a while. Plus with the higher memory machines, DAOC can keep many of these textures loaded all the time and just unload/load the zone specific stuff as you approach. That way you only get the hiccup crossing zonelines as it transfers between servers.

Re:it's all about design & bandwidth

[Anonymous Coward]

In terms of gee-whiz factor, I doubt EQ is all that sophisticated. It's benefited from evolution over time into quite a robust and refined product, but remember that it's a product. As a commercial entity, the focus at SOE has to be on profitability, not esoteric computer science research. I imagine SOE focuses on using well-proven techniques of distributed computing over whizbang pancaea solutions. Remember, they can afford to invest gajillions in hardware and bandwidth, while smaller upstarts have to make do with less.

Also, part of the original gameplay design behind EQ was to avoid a lot of the bottlenecks of its predecessors, like Meridian 59, which took a more ambitious approach to things like combat. Again, a clean, elegant design beats any number of research papers, when it comes to commercialization. Not that the research papers aren't cool, of course, but a company isn't going to bet the farm on unproven technologies on something as critical as scalability.

Java Server-Side, Clients... whatever

[meehawl]

You lost me at the "java-based" though

And oh yeah, only the Servers run Java only. The Client-side API is language-agnostic and platform-agnostic. So you can write Clients in C++ or Java and compile them to Win32, XBox, PS2, GameCube. The Servers don't care which Client belongs to which platform.

The analogy I like to use is NTSC. In the early days of TV without NTSC you had no guarantee that your GE TVs would be able to pick up Motorola format broadcasts. TVs competed within closed markets and featured lock-in. Creating a common broadcast standard enabled all TVs to pick up all broadcasts. TVs could compete on quality anf fucntionality, and broadcasters could compete using content. Using a platform-agnostic MMOG Middleware lets you enjoy economies of scale because your Servers communicate with all kinds of Clients. Client experiences vary, of course, according to display resolution and frame rate ability.

Re:I *SO* want this game...

[Anonymous Coward]

Wow, you've got an MBA, run your own company, and can't afford ~$120 (the price for 1GB PC3200 DDR)? That's really pathetic.

China Larger Than Korea

[meehawl]

Korea being the largest in the bunch

China (PRC) is now the largest MMOG market. Also, China is now the largest installed base of DSL in the world.

Re:China Larger Than Korea

[Anonymous Coward]

China also has more peasant farmers than any country in the world. It's somewhat disingenuous to emphasize the size of a market of a billion people. The scary thing about the Korean gaming market is that it involves such a large percentage of the population, at quite obsessive levels, with the associated profit potential. There's a reason why RTS developers have been pandering to the crowd raised on Starcraft, for example.

UT Design Is Not MMOG

[meehawl]

A 20 person UT match requires a surprising amount of bandwidth alone to make it as "realtime" as it can get

The large quantity of bandwidth exchanged in a UT (or similar peer-based FPS game) is an artifact of a design as single object view game with no distributed Server-side processing. Instead of waiting for bandwidth and CPU nirvana, there are smarter ways to maximize Server-side entity state updates while optimizing Server-Client bandwidth and delivering only environmentally-relevent data. Also, using multiple, distributed Servers enables you to multiplex Server-Client entity state updates using multiple pipes so you don't get a blocks or racing on a single message broker.

Re:UT Design Is Not MMOG

[cbreaker]

Cool, program me one and let's see how it works!

Last I heard, computing everyone's 'environmentally relevant' data uses a lot more server processing for a game like UT. Some games have tried this (I know of a couple mods for UT that did this) but it ended up being very difficult for them to keep track of where everyone was looking and what they should see. I turned this feature off on my TacOps servers because it used WAY too much CPU cycles. There were a lot of problems with it. A perfect implimentation would work well but boy does it add complexity.

MMORPG's use this type of thing now, however, like EQ. Updates to MOBs are only sent when you're in range. They can do this effectively since they offset the cpu usage with the fact that updates are 1/20th of a game like UT.

Math is not the point, though

[Shoggoth of Maul]

"RPGs are based on a dice game, and is really about mathematics. The people with powerful characters are the ones who can do math, not the once with a cable modem in the same town as the server."

Basically true (about D&D and other RPGs, I mean), although it's not the ideal. The Holy Grail of both P&P (Pencil & Paper) and MMO- RPGs is a system that conforms to common sense, so that math enters your gameplay only a little more than it enters the processes by which you live your everyday life.

That way you can really focus on *CHARACTER* development, rather than *STAT* development.

MOD PARENT UP!

[bartash]

Link is informative and much better than grandparent link.

use a REAL RDBMS from the beginning

[SethJohnson]

Whatever you do in this model, for christ's sake start with a real RDBMS. I know a couple people who work at an Austin-area MMORPG studio (not EA's Ultima-- that Austin office was shut down last week) and their developers have decided to go with flat files for storing all the data. This is just out of half-assed design specs from day one. Now, the game is several years released and it'd be too much work to re-write code to pull all their flat-file rewrites and have it do proper DB transfers.

Can you possibly imagine the race conditions that exist in this model? Scalability? Forget it.

Off the top of my head, I'd say go Oracle if it's a real commercial product. If not, PostgreSQL. mySQL isn't advanced enough yet to be trusted with storing a seventh-level mage's cloak of invisibility.

Re:use a REAL RDBMS from the beginning

[Michi]

Flat files? Wow! I admire those people's courage :-)

For what it's worth, Ice and Wish use Berkeley DB (http://www.sleepycat.com). It's a DB with a small footprint, high performance, and good transaction support. Nice licensing conditions too -- you pay only if you want to keep your source code secret.

Cheers,

Michi.

Re:use a REAL RDBMS from the beginning

[SnappleMaster]

I don't think "courage" is the word you are looking for. For that matter I don't think you meant to say "admire" either!

Re:use a REAL RDBMS from the beginning

[Anonymous Coward]

I'm not convinced a RDBMS is the be-all-end-all for data storage. What does the "R" stand for, anyway? "Relational". In a game you don't tend to be doing complex relational queries on the data in real-time.

Re:use a REAL RDBMS from the beginning

[Anonymous Coward]

Berkeley DB is sorta like flatfiles-on-steroids. OK, not really (it's a true database with all those databasey integrity and performance features), but I've noticed Berkeley DB tends to be used by applications that might otherwise have used a flat file, except for the reliability problems involved (not to mention issues of storing everything in core once your DB really starts to grow). This probably has to do with the procedural focus of the key-value pair data model, as opposed to the more flexible operations supported by RDBMS (not to mention connecting to your average SQL DB generally involves a socket somewhere).

As far as I know, Berkeley DB doesn't have any sort of network interface, so I suppose you either have to write your own, or forego that particular feature. NFS shares can be done, if a little bit tricky. Anyway, it's one of the main reasons why I'm using an RDBMS backend for my own project (although I'm actually taking a database-agnostic approach, with the idea of using a souped-up flat file on the low end, and a RDBMS on the higher end).

Re:use a REAL RDBMS from the beginning

[Moraelin]

Or in other words, you're another case of the "if you only know how to use a hammer, everything looks like a nail" syndrome, huh?

"Can you imagine race conditions?" Well, can you imagine that there may be _none_?

There are millions of servers out there -- HTTP, mail, whatever -- which work directly on files. And they have _no_ race conditions. Why? Because the operating system itself is built to allow multi-threaded access to those files.

Do those files need random read/write access? I've coded on MUDs which needed no such thing. The majority of files (e.g., items and rooms) can be, for all practical reasons, read-only. Only idiots edit those files all the time, while the game is running. From commercial game I'd expect a more responsible attitude, with a test server and release cycles.

Better yet, when you want to serve hundreds or thousands of players at the same time on a server, you'll want to keep most of that stuff in memory anyway. You do _not_ want to perform an SQL query for each item used, in every single round.

Or at the very least you'll want a cache, which provides enough of a synchronization point. So you have no race conditions. And, no, if the locking is competently done, no scalability problem either.

Does it have to be _relational_? All the MUDs or off-line games I've seen do _not_ need or want relational joins to get their data. They actually benefit far more from the old DB way of storing whole objects together. E.g., storing a player and their inventory together, instead of having 100 inserts in 10 different tables. It's actually a lot faster and more scalable.

Do you actually need stuff like "select all players belonging to Guild X"? Well, that's what manager objects are for. You can get that from data in memory much faster than from an SQL query sprawled across 10 neatly normalized tables.

In fact, you'll probably have all players already registered to their manager objects and viceversa upon loading them, so that select is a list you _already_ _have_. E.g., the least overhead way of broadcasting a chat to all players listening to a channel, is to have the channel object already have a list of players registered to it.

What's the benefit of pulling that list from a relational database, instead of using the one you have in memory? No, seriously. I want to know.

It's more work, yes. But it's probably a lot faster, and if competently done, it doesn't have race conditions or scalability issues either.

So briefly: so far you're telling me that said team actually did some design work, whereas you might be a cargo cultist.

"Cargo cult" refers to what happened when airplanes and ships dropped food and other supplies to savage tribes. When the big metal birds stopped droping crates of food, the tribesmen started carving statues of them, to please the airplane gods and get crates of food again.

Think it's laughable and primitive? Well, that's what happens in the programming industry every day. People who don't understand what a factory or a singleton is, or why do you really need one, build factories of singletons or singletons of factories for everything. Just because they saw it in some tutorial. Or they see some example using XML, so they absolutely must use XML too, in the most idiotic and inapropriate places.

It's the digital equivalent of carving airplane statues. It's carving a digital statue of a singleton, to bring forth the blessing of the singleton gods.

And the same goes for blindly sticking a RDBMS into everything, whether it's needed or not. Congrats. Welcome to the 21'st century digital cargo cult world.

Re:use a REAL RDBMS from the beginning

[SethJohnson]

Moraelin,

I think you know a lot about this stuff. I think you know more about it than I do from the programming perspective. Everything you've said here is well-thought out and sounds based in your own experience with programming. I haven't programmed a big multi-player game, so in many ways, I'm talking out of my ass.

Like you said, I'm the guy with the hammer who wants everything to be a nail. I know databases. I also know architecture. It seems to me, though, that file locking could get very messy when there are several thousand potential clients. Let's say, for example, there's a guild of some kind who is trying to screw with the game. If there's a file lock occurring on the file storing character data everytime a client finishes their session, this guild could have its members repeatedly connect and disconnect to tie up the server so other people can't connect or disconnect. Then as the system gets bogged down in terms of accessing this file, as other players finish and want to quit the game, their clients have to wait for the congestion on that file to die down before their client is allowed to quit. And if they force a quit, changes to their character might go un-saved and then they might be able to reconnect and get a read on the old version of the file.

Obviously, programmers can do their best to manage scenarios like this with a queue and caches stored in memory. But the thing that I think it really comes down to is when you said, "It's more work, yes." These game studios aren't the best-funded operations in the software world. They can't afford "more work." On the plus side, some of the best programmers in the world are attracted to work at game companies, so they can perform wonders for little money and low headcount on their teams. So yes, you can use managed objects, etc. But it ends up being a lot of overhead for the developers, I think. And when a new guy comes onboard, there's a lot for that person to figure out when the game data is stored in a roll-your-own system.

I also think you get a lot of mileage from having a real RDBMS when it comes to backups. With Oracle ( I don't know much about Berkeley DB), you can run hot backups. Try that with a flat-file system and a game you can't afford to shut down even for 10 minutes. If you're starting to develop a game that you want to pitch to a publisher like EA, they're going to take you very seriously if you've got this kind of stuff in your game engine.

I don't know how content is added or modified in a real MMORPG, but I'd think a real easy way is to give your production (content) team an easy web-based interface where they could type in item descriptions and have pull-down menus for the attributes or whatever. They could also upload graphics, etc. This scales very well for a large team adding thousands of items, descriptions, etc. Obviously, this web-based interface would easily tie into a DB and then the game would leverage those items. Maybe you could do all this using XML and some CGI scripts. I don't know, though, because I'm the guy with the hammer!

I would love to tell you the name of the game my friends are working on, but they're all good guys and I don't want their company to get ticked-off that I'm badmouthing their architecture. That's the problem of using a slashdot nick that's your real name.

Region Ownership

[meehawl]

I can't give you specifics because you have not signed an NDA. I can talk in generalities, but I can tell you that TZ's load balancing is not random. Much of the information about how to load-balance such systems can be obtained from reading "Distributing Object State" which ran in GamaSutra a while back. The key is dynamic ownership, not static ownership. As you rightly fear, static ownership leads to slowdown and player hangs.

The best way to maintain ownership would be dynamically, using some sticky heuristics to predictively anticipate where a player will "be" following a move, and alert Servers within some defined "neighborhood" or "ZOC" to update their state. This is non-trivial, because you may be dealing with non-Euclidean geometries, distance metrics, or set/guild membership. Therefore, each distributed Server can update its affected Clients on-demand, without those annoying lags you get with some systems when you can "feel" the Client loading the data from a new Server.

Alerting Servers that currently "own" those possible Regions to prepare to update relevent Entities with info is also required. If no Server owns that Region, then you should have a whole other set of heuristics to determine which game server should own that Region. It may, or it may not, be the Server that "owns" the Entity that is moving into that Region. You probably need to do cost-benefit calculations for assigning/re-assigning Region ownership. You can run Monte Carlo simulations to see how best to describe possible Entity "walks" within the topology.

Similarly, because of the expense of instantiation, you need some pretty tricky finagling to figure out when to relinquish ownership and purge any "ghost" copies of the Entity State that have been following the main Entity "around" within the topology. Of course, the nice thing is that Server-Server entity state exchanges will take place along a fat pipe backbone.

Interestingly, such systems end up looking a little like a Kohonen n-tier feedforward neural network.

Has anyone used JavaSpaces for this?

[tcopeland]

JavaSpaces is more or less Sun's Jini take on distributed processing.

From a programmer point of view, you start up a "space", and then you can write objects in, take them out, and read them. And that's all. So there are a very few simple operations, and you structure your app around those.

Anyhow, it seems like a couple of JavaSpaces on a rack of servers might serve as a good way to distribute processing/notification/etc. Of course, you're limited to Java and to moving around Serializable objects....

Re:Has anyone used JavaSpaces for this?

[taweili]

JavaSpace isn't anything new. It was based on LINDA developed in Yale in early 80s. The problem with it is that it has a very easy abstraction but it's intended as a synchronization system for distributed system rather then a data moving middleware. It's hard to "tune" it.

Re:Has anyone used JavaSpaces for this?

[tcopeland]

> isn't anything new.

Right on, it's not a new idea, it's an implementation of an old idea.

> a synchronization system for distributed
> system rather then a data moving middleware

Yup. I think I misunderstood the original question - I thought we were trying to figure out how to run a server farm for the game, but I see most folks are discussed client/server comms. So I probably missed the point.

> It's hard to "tune" it

Hm. Probably.... you could do some Externalizable stuff, and maybe rewrite some things using NIO, but, you're right, three public operations doesn't leave much room...

Unreal Warfare Engine

[Anonymous Coward]

Both Lineage 2 and Ultima X are MMORPGs that will run on Epic's Unreal Warfare Engine.

Having given Lineage 2 a run, it is about the smartest thing any MMORPG has ever done. The game is beautiful and upgrading engines will go in tandem with the Unreal series of games for years to come.

There is no reason to do a custom engine unless you're just a pompus megacorp.

not really.

[jon_c]

Its only the printer friendly page for page 1, you need to go back to the original article, click next page, then printer friendly page to get to the next page.

not very convenient.

Re:not really.

[Michi]

See this link for a properly printable version.

Other types of middleware

[contradyction]

I find their choice of RPC as the middleware layer surprising. I would imagine that a vast majority of events in a MMP game need to be passed on to a fairly large subset of players, for example, anytime someone moves, attacks, casts a spell, etc. the people visual or audio range of that player need to be informed. RPC is better suited to one-to-one interactions, not the one-to-many interactions we get in MMP games. It seems to me that a distributed publish/subscribe system like Elvin, SIENA or even Mercury (a pub/sub system specifically designed for MMP games!), or a LINDA-style shared memory system would be much more appropriate.

Re:Other types of middleware

[Michi]

The problem with games is that they often need both styles of comms, synchronous twoway RPC as well as event-style pub/sub comms. Any middleware that supports one but not the other is likely to be hard to use. We designed Ice such that it can do both well. (You can look at Chapters 18 & 25 of the doc for details.)

Basically, Ice supports the usual twoway, synchronous RPC, as well as oneway calls (which don't require a reply from the server), and UDP. The protocol is designed such that it is possible to build very efficient message switches for event distribution.

BTW, I used to work at DSTC with the guys who built Elvin -- it's a fine piece of work and well woth a closer look.

Cheers,

Michi.

This project is going to fail

[jon_c]

This project is going to fail, or at least be severely delayed. Why? Well most of the article talks about how they rewrote COM/COMBRA/KOM (etc..), creating their own IDL, their own protocol etc.. However the article is not about rolling your own ORB, it's about designing a MMORPG middleware, which can have little to nothing to do with an object broker.

They even start of the article with some nice pie in the sky requirements. Like a truly scalable world capable of "tens of thousands" of people in the same world realtime load balancing, dynamic world installation. And a bunch of other stuff that probably sounded nice in some brainstorming meeting, though I suspect has little to do with any actual game design.

In my experience this usually happens when the people involved don't really understand the problem they are trying to solve, so they end up writing general purpose architecture to solve whatever problem they eventually do understand, typically dooming the project in the meantime.

-Jon

Re:This project is going to fail

[Michi]

However the article is not about rolling your own ORB, it's about designing a MMORPG middleware, which can have little to nothing to do with an object broker.

Hmmm... To build a game on that scale, we needed a distribution platform of some kind. Having looked at what we needed, and having considered CORBA (which some of us knew a lot about, having previously built a commercial ORB), we knew that CORBA wasn't going to meet our requirements. So, we ended up building a new middleware.

It's quite surprising how much the middleware and its object model end up influencing game design and performance. For example, I don't think Wish could have been built without object migration, without some of the protocol features we came up with for efficient event distribution, or any number of other things. The game requirements influenced the design of the middleware, and vice-versa, so I would say that the two actually have a lot to do with each other.

In my experience this usually happens when the people involved don't really understand the problem they are trying to solve

You might want to have a look at my home page -- I think I can honestly state that I know quite a bit about middleware and the problems we are trying to solve :-)

Cheers,

Michi.

Re:This project is going to fail

[lordpi]

I had the exact same thoughts when I read this article from a link on gamedev.net.

The whole time I read it, I was like: why does the author constantly mention CORBA? Perhaps I'm not familiar enough with the Java perspective, but I assumed CORBA/COM (etc) were basically dead (or dying) when it came to modern internet programming.

Basically, I'm curious as to what the benefits of a distributed-object approach are over a web-service approach (especially in an MMOG)?

My best guess is:
PROS (of distributed-objects):
1. The author and company were only familiar with CORBA.
2. Distributed-objects are an older, more established technology. [ie proven]
3. There are a lot of existing systems, algorithms, ideas, programs, etc. to build a distributed-object system.

CONS: 1. It masks the network from the API (yes, this should be a CON).
2. Engineering effort is required to make sure clients can access the model with any langauge, and also to allow designers flexibility in the object model. Most of this should be cheaper with a web-services model.
3. You don't get to throw around as many buzzwords.
4. There is duplicated code on both the client and server.

Does this sound right? Or am I missing something?

Re:This project is going to fail

[Anonymous Coward]

Web services were never designed to solve this sort of problem. They were never focused on the needs of a performance-critical, real time system like a MMORPG, but rather on doing things like swapping data between corporations and driving Web sites. It's probably more worthwhile to compare with things like MPI (used in supercomputing applications) and CORBA than XML-RPC or SOAP layered over HTTP. I think it'd take a lot of hubris to decide that the latest buzzword of the day (Web Services) is going to magically solve all the problems of the "old" models (CORBA, application-specific protocols).

Oh, and not to reopen that old debate, but Web services is basically CORBA ten years ago, with a different implementation (XML, SOAP, etc.) that should hopefully work better than CORBA's baroque design-by-committee industry standards. But behind all the buzzwords, we're still talking about the same basic sorts of distributed technology.

I don't imagine you want to suggest to John Carmack that he implement the Doom 3 network code using Python and XML-RPC, do you? :) MMORPG middleware is probably one of the few applications where it still makes sense to have a lot of C/C++ code.

Re:This project is going to fail

[Michi]

why does the author constantly mention CORBA? Perhaps I'm not familiar enough with the Java perspective, but I assumed CORBA/COM (etc) were basically dead (or dying) when it came to modern internet programming.

That's a fair question. Looking at the requirements, we needed middleware that is language- and platform-independent. That immediately rules out things such as COM, DCOM, RMI, and .NET. CORBA really was the only candidate. Also, all of us (the people who wrote Ice) had a lot of CORBA experience, and Ice borrows a lot of ideas from CORBA. (No, not everything in CORBA is bad ;-) So, I ended up contrasting things to CORBA because Ice is both similar and different to CORBA.

CORBA isn't exactly dead--people are still building industrial-strength systems with it. And, to be fair, CORBA is the most widely-used distribution technology to date. It's just that CORBA is ageing and that, after more than ten years of experience, we now know how to do better and avoid CORBA's mistakes.

The author and company were only familiar with CORBA.

No, not really. I have experience with Sun RPC, DCE, COM, DCOM, Web Services, and CORBA.

Distributed-objects are an older, more established technology. [ie proven]

There are a lot of existing systems, algorithms, ideas, programs, etc. to build a distributed-object system.

Yes, to are large extent. There is a lot of experience we have gained over the past ten years or so on how to build distributed systems that work.

It masks the network from the API (yes, this should be a CON).

That's an old topic that comes up periodically. The most-quoted paper on that topic is probably A Note on Distributed Computing. Personally, I disagree with many of the conclusions in that paper. True, I can't afford to forget that there is a network underneath my API because the semantics are truly different from the non-distributed case. But that does not strike me as a compelling reason to make a remote invocation substantially different from a local one.

Engineering effort is required to make sure clients can access the model with any langauge, and also to allow designers flexibility in the object model. Most of this should be cheaper with a web-services model.

I don't see where web services would be cheaper than any other language-agnostic platform. In fact, everything I have seen so far about web services is more complex than Ice (or CORBA). And web services are horribly inefficient. If you are interested, look for my postings on the topic in comp.object.corba with Deja.

The reason we have web services (or, more accurately, might have them some day) is a political one rather than a technical one: back in the hey-days of the distributed computing war between CORBA and DCOM, Microsoft were about to lose the battle. They weren't going to simply cede the battlefield to the OMG so, rather than fight a war they couldn't win, they simply created a new battlefield. Sadly, web services is about re-inventing the distributed object wheel all over again and, even more sadly, with even more mistakes and corners on the wheel than CORBA.

You don't get to throw around as many buzzwords.

Unfortunately, in some circles, that seems to be an important criterion in the decision making. Personally, I prefer to use things that work and have substance, instead of chasing after silver bullets :-)

There is duplicated code on both the client and server.

Huh? You haven't been listening to my friend Don Box by any chance? ;-) If that is what you are referring to, it's simply wrong (or certainly, it is wrong as Don presented it in his interview). Neither CORBA nor Ice require duplicated code on client and server--they are pure client-s

Re:This project is going to fail

[Michi]

Web services were never designed to solve this sort of problem. They were never focused on the needs of a performance-critical, real time system like a MMORPG, but rather on doing things like swapping data between corporations and driving Web sites.

The bandwidth and CPU cycles consumed by SOAP are nothing short of staggering. (Look at the archives in comp.object.corba for some calculations.) SOAP easily eats 10-100 times the bandwidth of IIOP (and IIOP isn't particularly good at conserving bandwidth either--Ice does a much better job of that). With SOAP's bandwidth requirements, there is no way we could have made Wish work over an ordinary 56kB modem.

Web services is basically CORBA ten years ago, with a different implementation (XML, SOAP, etc.) that should hopefully work better than CORBA's baroque design-by-committee industry standards. But behind all the buzzwords, we're still talking about the same basic sorts of distributed technology.

I agree that WS is basically CORBA all over again, but I very much doubt that it will work better than CORBA. True, CORBA has indeed suffered badly from design-by-committee, but WS seems to be headed for the same fate. And I watch people make the same mistakes all over again with Web Services. It's disheartening really: instead of learning from the painful lessons of the past, we forge ahead and make the same mistakes again. Partly, that's because, by the time a new technology gains enough momentum to challenge the previous technology, the veterans (who have the experience with the old technology to really work out how to do it better this time) have moved on; and the new people now working on the new technology don't have all the experience of the veterans and then promptly fall into the same old traps...

Cheers,

Michi.

Easy

[insanely_mad]

god, a real problem finally. I've been doing so much html and asp i miss something more substantial like this. i could solve this, i know it. The real issue here is data -- how do you access it, and how do you store it. The fact is that you could get an in memory object oriented database to keep track of it all. 4gb memory would be enough to support a ton of players. What you do is keep a "DNA strand" for each player with a bunch of binary toggles to control state. Forget "manager" and all that crap that overcomplicates the solution.

Re:Easy

[Anonymous Coward]

Eh, I think you're oversimplifying the problem. Sure, you can shove 4 GB of data on to one machine, but how are you going to process that 4 GB of data? You can't even shove that much information down the bus in an entire second, and your average MMORPG is going to have to crunch the data set a lot more often than that. Furthermore, keeping everything in core isn't the best way to maintain reliability. :)

Once you accept that you're going to need more than one machine, you get into all the messy issues of distributed computing which still occupy researchers to this day. I think all that HTML and ASP has rotted your brain; time to work on more substantial things, methinks. :)

Very complicated project

[WarderDot]

This project is extremely complex. What I would like to see is that either the big guns in the industry have a shot at it, or make the project open source so that the best talents can work on it. It would be quite ideal to have a robust and efficient distributed platform that all MMORPGs can build upon. This will save tremendously on development time. Thus, game developers can focus more of their attention on improving gameplay and graphics, rather than solving common problems related to distributed systems.

SCE-RT

[meehawl]

I think you're talking about what is now known as the SCE-RT. This is the patent.

Re:SCE-RT

[mr_death]

Yes. I'd hate to see you guys work hard and then get bushwacked by a patent lawyer.

cool, a new career!

[samantha]

I do or have done distributed, persistent middleware with tons of threads, caching, dynamic persistence, multiple-databases, multiple-languages and so on middleware for enterprises or those who wish to sell to them many times in the last couple of decades. It didn't occur to me that most of the same talents are essential to games and various virtual spaces.

Thanks! :-)

Re:cool, a new career!

[Anonymous Coward]

Yeah, but just try getting into the game industry without having some game-related experience. Still, you have a lot of experience, so provided you can find an interesting, solvent company with job openings, all the more power to you. :)

Sex and MMORPG

[Anonymous Coward]

Ok, now that I have your attention, would anybody think feasible to write a Python wrapper for NWN with NWNX??

That would make a real difference to many people who are currently enjoying RPG instead of counting the number of wizards ona guild..

If it's possible to write such a wrapper, guess I'm going to do it.

We are missing the issues.. again

[dilvish_the_damned]

What you think the game developers are stupid?

Based upon the previous posts I would think that we are all mostly clueless with regard to what the technical problems that surround multi-player games really are.

But I wont let that stop me from ranting.

DOAC recently extended the laps time in order to account for satellite users. Please understand that these users typically have no lack of bandwidth, its the latency that kills them being some 600ms round trip time due to physics and stuff. Its hard to trump physics yes?
It only makes sense to level the playing field by making it turn based, just like every other MMPORG out there. After all, there is a really good reason they did it in the first place.
I am assuming that by turn based they mean that your have something akin to time slices within your area of infuence.
If it is not turn based (within time slices) then it becomes freely timing based. I got OC48... what you got? DRAW! BANG! I win.

Just kidding, I dont really have OC48....

ZODB

[Hylander]

Seems like a perfect application for the Zope Object Database. It does all these things for you out of the box. And you get to write it all in lovely lovely python. aaaah.

Reinventing the Jini and JavaSpaces wheel

[Patrick May]

This would have been much easier if they had used Jini and JavaSpaces technology. There is even a commercial implementation that supports incremental evolution of the distributed model.

Jini and JavaSpaces are being used in a variety of organizations to build large, distributed, reliable, scalable systems that integrate a wide variety of existing systems, including those written in languages other than Java. The technology seems a good match for this problem.

Patrick

Re:Reinventing the Jini and JavaSpaces wheel

[taweili]

You can't be serious! JavaSpaces is NOT a general distributed system middlewware. The concept of Space is developed for LINDA in 1982! and it's intended to be a synchronization machinism for distributed system. JINI is a service discovery framework. It doesn't specify the object in the system.

Why do people keep thinking about Java and related technologies as innovation? Do you think the entire IT industries and reasearches only started after Java was invented in 1994?

Re:Reinventing the Jini and JavaSpaces wheel

[Anonymous Coward]

He never said it was new, just mentioned that it existed. Why get your panties in a knot over it? Sounds like your got burned by that technology a while ago and are still hurting over it.

obvious solution: DRM

[Anonymous Coward]

a guild may have at most one level-5 mage (magician).

Obviously they need DRM for level-5 mages.

I mean, it's a proven technology, right?

No , thats just to run Windows

[Viol8]

The client itself probably takes up only a small fraction of that requirement!

Erlang

[fnc]

It seems a possible application for Erlang. From the FAQ (http://www.erlang.org/faq/t1.html): "Erlang is a general-purpose programming language and runtime environment. Erlang has built-in support for concurrency, distribution and fault tolerance. Erlang is used in several large telecommunication systems from Ericsson. The most popular implementation of Erlang is available as open source from the open source erlang site." Included in the libraries, there is also a soft real-time database called Mnesia (like JavaSpaces but extended with relational operations). There is some articles about using it in games here: http://www.erlang-projects.org/Public/projects/gam es/

SimNet

[meehawl]

I suggest you look at the history and publications relating to SIMNET, an old MMOG military simulation project from the 1980s DARPA funded. The classic paper is from 1993: "The SIMNET Virtual World Architecture", Calvin & Dicken, Proceedings of the IEEE Virtual Reality Annual International Symposium 1993, pp: 450-455.

Did you read the recent games post?

[PurplePhase]

http://games.slashdot.org/games/04/02/29/1642223.s html

There are several people who added comments about how they had to do some very complicated interactions to overcome others - http://games.slashdot.org/comments.pl?sid=98732&ci d=8424235 about Planetside. The one I liked the most was this one http://games.slashdot.org/comments.pl?sid=98732&ci d=8426037 about an early UO player's play-by-play of conquering some ambushers.

So at some point the dexterity/skill-players can find a decent game or two... IIRC, a recent ad for MS's Mythica says it will have real-time combat.

No affiliation with any of the above, I just enjoyed the post and comments.

8-PP