Slashdot Mirror
Valve Shares Performance Numbers On Port of Left4Dead
New submitter nschubach writes in with an update on Valve's progress porting one of their games to GNU/Linux. From the article: "One factor in creating a good gaming experience is throughput. This post discusses some of what we've learned about the performance of our games running on Linux. ... After this work, Left 4 Dead 2 is running at 315 FPS on Linux. That the Linux version runs faster than the Windows version (270.6) seems a little counter-intuitive, given the greater amount of time we have spent on the Windows version. However, it does speak to the underlying efficiency of the kernel and OpenGL. Interestingly, in the process of working with hardware vendors we also sped up the OpenGL implementation on Windows. Left 4 Dead 2 is now running at 303.4 FPS with that configuration."
nschubach adds "It seems there are good things coming out of this for both Operating Systems!"
LINUX DESKTOP
To be fair, it was their porting to to Open GL that improved the windows Open GL performance. What I find more interesting, to be honest, is that Open GL is (slightly) outperforming Direct 3D on a windows/nvidia box.
What I found interesting was how much an improvement this is from their initial port.
Their very first version ran at a full six frames per second (167ms/frame). They've now gotten it up to 315 fps (3.17ms/frame).
That's some pretty impressive work. Pity the article is so light on the details of how they did it (I'll spare you reading the article: they found places where it ran slow due to the kernel, they found places where it ran slow making OpenGL calls, and they found places in the driver itself that ran slowly - that's about as much detail as the actual article gives you).
Yea, I noticed several years ago, back when I used to play WoW and my computer could barely handle, that it would run faster in Linux on Wine with OpenGL than it would on Windows XP. I mean I'm talking ~5fps on windows to ~15fps with better graphics on Linux -- not really playable on Windows, barely playable on Linux.
Can we bring those improvements to Mac OS X?
They weren't expecting it to run more efficiently *given the level of work they invested*. They've sunk years of work into making the Windows version run quickly. Getting a Linux port to run faster, only months after getting their initial port running (the first running Linux version ran at 6 fps under the same test), is impressive.
See for yourself. Download the Heaven DX11 Benchmark. Run it in DX11 API mode and then OpenGL mode and see the difference. There is a difference, but it's fairly minor. Usually DX11 or OpenGL will be the first to support a new feature, then the other supports it in the next release. The end result is that they normally produce a similar graphics quality.
Does the openGL driver/port have the same graphic detail as the directX version?
You may well be right, I'd be cautious in taking anything away from this particular article though, as Valve have made it quite clear now that they have a vested interest in seeing Windows fall as they see Windows 8 as a genuine threat to their existence if people start buying games directly from the Windows Marketplace rather than Steam. When they referring to Windows 8 as a general "Catastrophe" which is probably a bit of a stretch, even if it maybe is for them, then it's hard to see them as objective on this issue.
Not that this is likely to be an unpopular move here, nor is it necessarily a bad thing if a company like Valve is helping Windows fall a peg or two, but now Valve has a clear political motivation to attacking windows, it's hard to see anything anti-Windows they mention as necessarily objective. It's also quite possible that Valve's Windows engine actually just isn't well optimised, and that now that they're moving it to another platform it's given them chance to rewrite components that were long overdue for a rewrite. I believe at least some of the foundations and design of the Source engine actually stem all the way back to the Quake 1 codebase for example.
OpenGL sits over the top of DirectX on Windows
No, it doesn't. That was the plan when they developed Vista, but it was scrapped after an outcry from half the industry. The OpenGL driver is just as low level as the DirectX driver on Windows.
Er, no, it actually doesn't.
Microsoft had planned to do so in Vista (they actually wanted to run old versions of DirectX on top of DX10, then the newest), but they scrapped that plan well before release after half the game industry, most of the professional graphics industry, and the graphics card companies themselves rose up in arms (Nvidia was actually planning on circumventing it, offering direct OpenGL).
So on any actually-released version of Windows, OpenGL is as low-level as DirectX.
One of the advantages of OpenGL vs DirectX is that it doesn't force the underlying hardware to comply as strictly in areas such as memory management, command batching, shader assembly, etc. This allows implementers more freedom to optimize and usually results in much higher performance. Even if a full backwards compatible OpenGL context is huge.
This approach was proven again very succesful with mobile hardware, where vendors such as Qualcomm, PowerVR or Tegra or ARM (Mali) produce graphics chips that comply with OpenGL but at the same time use the higher level abstaction of the API to their advantage, by supplying very different backends each (Immediate Rendering, Deferred and Tile Based Deferred) as means to improve performance (per watt and silicon space) to levels much higher than the desktop counterparts.
Added to that, programming games under Linux is a joy for those used to it, as the tools are fantastic (command line scripting, gdb with hardware watchpoints, valgrind, strace, etc) and the fact the OS manages the heavy load of games much better. Many companies I worked with, and even big ones such as Naughty Dog (makers of Uncharted) develop their games primarily under Linux, even if the final versions are released for Windows, Mac and Consoles.
Much of the engine design dates back to the Quake 2 engine, but none of the code does.
History time!
First came Quake, which was coded completely from scratch. Then came Quake ][, which was a significant overhaul and massive rewrite. Some of the code was saved, but the engine design itself was changed quite a bit. Both of those engines were written by id Software.
Valve licensed the Quake ][ engine, and improved on it, adding a new renderer (a DirectX one, IIRC, but they kept the OpenGL and software renderers) and several other nice features. They used this for Half-Life, Team Fortress Classic, Counter-Strike 1.6, etc., and also offered it for relicensing under the name GoldSrc.
Over the years and years it took to make Half-Life 2, they rewrote literally the entire engine. Not a single line of code remains from Quake ][. They rewrote the renderer (several times), added all kinds of animation goodies, integrated Havok physics, and so on. But they kept the same basic client/local server/server design, the same general layout. It's much like how GNU made the basic Linux toolset - they copied the design of UNIX, but did not use any code from it.
Valve has continued to use and upgrade this engine, calling it Source to confuse everyone. They've offered it for license, and at one point were seen as a decent competitor, but they've really fallen behind in the post-UnrealEngine 3 world. I half-suspect they'll be either doing a total engine redesign, or giving up and licensing someone else's engine.
% wine Steam.exe
Yeah, look at them all.
Very few gamers can even notice a difference between 60-100 let alone over 100.
They will, however, notice the difference when a sudden spike in activity causes the fps to drop to half or a quarter of that. Rendering speeds are highly dependant on what is being rendered.
A game running at 240fps can happily suffer a 75% drop without any noticeable impact, while a game which just barely makes 60 fps will not.
This is actually a mundane sort of thing in cross platform development. Your other "ports" aren't just a money pit. They allow you to stress your code in ways you might not have thought of. Typically the bugs you find in secondary platforms improve your product on it's primary platform.
A Pirate and a Puritan look the same on a balance sheet.
This was 300fps on an *extremely* powerful system. The GeForce 680 is the most powerful single-GPU card on the market, and the 3930K is one of Intel's top consumer chips. I myself have a 660 and a 3610, weaker versions of the ones they used, and I can max out every game I have.
Getting 300fps on that means that, on a system a tenth as powerful, you get 30fps without dropping the graphics settings. Valve just doesn't chase the extreme high-end hardware - they don't bother adding more particles and such that make it look better only on a fraction of a percent of systems. Valve has perhaps the best knowledge of what real-world users are gaming on, thanks to the Steam Hardware Survey. So they can make an informed decision as to whether it is worth it to have the artists come in and add yet another layer of detail that will only be seen on the newest and most expensive computers.
That said, 120hz monitors seem to be rising in popularity, making rendering at 120fps a worthwhile goal. And it's often good to have a buffer of 10-20fps or so, because the amount of stuff you have to render isn't constant. In combat (with all the particle effects, explosions, flying debris, etc.), it often drops by 10%-20%, which can put you below 60fps if you're running at 70-80 normally.
Render rates far above 60fps means that I don't have to treat my machine like an xbox just because I decide to do some gaming with it.
It also means that I can likely get away with a much less powerful system and GPU and still have acceptable performance.
I can use less machine to get the job done.
A Pirate and a Puritan look the same on a balance sheet.
So I don't know why anyone should be surprised at this, Linux has always been OpenGL, Windows DirectX, so having Linux do OpenGL better isn't surprising, anymore than Windows doing DirectX, its just what the platform is made for is all.
As a counterpoint, TFA said that their OpenGL port of L4D2 on Windows ran faster than the DirectX version.
That's a great game. When did Valve release it?
It'd be highly amusing to me if, in a few years time, Windows users are keeping a copy of Linux around because "I need it for the games" :)
So.. it has come to this
Clearly not. They give a bare number that doesn't indicate whether it is a maximum FPS or an average FPS. They provide neither the test setup (screen resolution, detail settings, etc), nor meaningful analysis of overall performance. For example, if the average FPS is lower on one platform, but the variability is also lower, the actual user-perceived performance will be better. No meaningful details are provided, just some ePeen number which is abstract of context. The statistician in me weeps.
Its what blizzard did, of course I no longer consider them a game developer after diablo 3.
I'm not familiar with that game; did you mean Auction House Tycoon?
That only shows that they fail at communicating the results. Not that the result itself is bogus...
I was wondering why the difficulty level increased with each version.
How can I believe you when you tell me what I don't want to hear?
The test setup is provided on their blog: http://blogs.valvesoftware.com/linux/
Oh derp, you were referring to the options. And the blog was linked in TFA. Guess I'm the idiot.
It's not "clear" at all. Just because they didn't provide such details doesn't mean they don't exist. Absence of evidence is not evidence of absence.
Valve is an exceptionally competent developer of high performance, low-level graphics software. The article says "The data is generated from an internal test case". It's reasonable to assume this internal test case is specifically designed for, oh I dunno, TESTING. Which means it's designed to put the different rendering pipelines through the exact same gauntlet (or as close as can be managed).
Why would you think that them running the test at inconsistent resolution/detail settings is even a remote possibility? Do such obvious test practices REALLY need to be spelled out for you?
Yes, they reduced the test data to a simple number (I thought it was pretty obvious this represented "average" FPS). It's an easier discussion point. What of it? It's a blog post, not a scientific paper.
"Mind, as manifested by the capacity to make choices, is to some extent present in every electron." -Freeman Dyson