Slashdot Mirror
Valve Shares Performance Numbers On Port of Left4Dead
New submitter nschubach writes in with an update on Valve's progress porting one of their games to GNU/Linux. From the article: "One factor in creating a good gaming experience is throughput. This post discusses some of what we've learned about the performance of our games running on Linux. ... After this work, Left 4 Dead 2 is running at 315 FPS on Linux. That the Linux version runs faster than the Windows version (270.6) seems a little counter-intuitive, given the greater amount of time we have spent on the Windows version. However, it does speak to the underlying efficiency of the kernel and OpenGL. Interestingly, in the process of working with hardware vendors we also sped up the OpenGL implementation on Windows. Left 4 Dead 2 is now running at 303.4 FPS with that configuration."
nschubach adds "It seems there are good things coming out of this for both Operating Systems!"
LINUX DESKTOP
To be fair, it was their porting to to Open GL that improved the windows Open GL performance. What I find more interesting, to be honest, is that Open GL is (slightly) outperforming Direct 3D on a windows/nvidia box.
What I found interesting was how much an improvement this is from their initial port.
Their very first version ran at a full six frames per second (167ms/frame). They've now gotten it up to 315 fps (3.17ms/frame).
That's some pretty impressive work. Pity the article is so light on the details of how they did it (I'll spare you reading the article: they found places where it ran slow due to the kernel, they found places where it ran slow making OpenGL calls, and they found places in the driver itself that ran slowly - that's about as much detail as the actual article gives you).
Takeaway: the linux kernel and OpenGL are arguably more efficient than their Windows-based counterparts. I think a lot of people have thought this to be true for years, but it's always nice to see solid comparisons. It's a good time to have a linux box!
Probably that, when dealing with GPU-limited things like framerates in a moderately intensive OpenGL application, a substantial portion of your performance is going to come down to the togetherness between your application and the GPU vendor's drivers, so working with said vendors might help...
Deal with it, menial M$ users...
I'm just curious; how does the visual quality stand up I wonder? I always thought OpenGL didn't have the same amount of features that Direct3D does.. Can someone here speak to that?
Save the Earth, use Linux!
You don't care about the environment? You know, that place you live in? The only one available with no alternatives?
Okay then.
Save your wallet, use Linux!
Can we bring those improvements to Mac OS X?
Last I checked FPS over 60 generally doesn't mean much for most people. Very few gamers can even notice a difference between 60-100 let alone over 100.
So there is no real point in posting huge FPS numbers in Windows or Linux as it really doesn't matter.
Could we get back to DOS 6.22 and benchmark it?
Does the openGL driver/port have the same graphic detail as the directX version?
OpenGL sits over the top of DirectX on Windows so just using it would incur a penalty. I'm fairly certain that anyone porting a DirectX game to Linux using winelib or some commercial derivative would incur a penalty in the other direction.
OpenGL sits over the top of DirectX on Windows
No, it doesn't. That was the plan when they developed Vista, but it was scrapped after an outcry from half the industry. The OpenGL driver is just as low level as the DirectX driver on Windows.
Nice numbers, but where is Episode 3?
[citation needed]
One of the advantages of OpenGL vs DirectX is that it doesn't force the underlying hardware to comply as strictly in areas such as memory management, command batching, shader assembly, etc. This allows implementers more freedom to optimize and usually results in much higher performance. Even if a full backwards compatible OpenGL context is huge.
This approach was proven again very succesful with mobile hardware, where vendors such as Qualcomm, PowerVR or Tegra or ARM (Mali) produce graphics chips that comply with OpenGL but at the same time use the higher level abstaction of the API to their advantage, by supplying very different backends each (Immediate Rendering, Deferred and Tile Based Deferred) as means to improve performance (per watt and silicon space) to levels much higher than the desktop counterparts.
Added to that, programming games under Linux is a joy for those used to it, as the tools are fantastic (command line scripting, gdb with hardware watchpoints, valgrind, strace, etc) and the fact the OS manages the heavy load of games much better. Many companies I worked with, and even big ones such as Naughty Dog (makers of Uncharted) develop their games primarily under Linux, even if the final versions are released for Windows, Mac and Consoles.
One key factor omitted is which version of OpenGL they are using. I certainly find the advantage highly interesting, but the comparison may unfortunately not be completely fair if they used OpenGL 4.x. Bear in mind the source engine only supports D3D9, which is 10 years old in a few months.
Er, no, it actually doesn't.
Microsoft had planned to do so in Vista (they actually wanted to run old versions of DirectX on top of DX10, then the newest), but they scrapped that plan well before release after half the game industry, most of the professional graphics industry, and the graphics card companies themselves rose up in arms (Nvidia was actually planning on circumventing it, offering direct OpenGL).
So on any actually-released version of Windows, OpenGL is as low-level as DirectX.
One of the advantages of OpenGL vs DirectX is that it doesn't force the underlying hardware to comply as strictly in areas such as memory management, command batching, shader assembly, etc. This allows implementers more freedom to optimize and usually results in much higher performance. Even if a full backwards compatible OpenGL context is huge.
This approach was proven again very succesful with mobile hardware, where vendors such as Qualcomm, PowerVR or Tegra or ARM (Mali) produce graphics chips that comply with OpenGL but at the same time use the higher level abstaction of the API to their advantage, by supplying very different backends each (Immediate Rendering, Deferred and Tile Based Deferred) as means to improve performance (per watt and silicon space) to levels much higher than the desktop counterparts.
Added to that, programming games under Linux is a joy for those used to it, as the tools are fantastic (command line scripting, gdb with hardware watchpoints, valgrind, strace, etc) and the fact the OS manages the heavy load of games much better. Many companies I worked with, and even big ones such as Naughty Dog (makers of Uncharted) develop their games primarily under Linux, even if the final versions are released for Windows, Mac and Consoles.
There's no mention of graphic detail in their article, so I'd presume (might be a big one) that they're running both at equivalent resolution, level of detail, and shader effects
This is not true - OpenGL is still a native driver. Microsoft wanted to cripple OpenGL in this way, but the CAD industry pushed back through Khronos and Microsoft agreed to retain native driver access.
Microsoft still hasn't updated the Windows OpenGL library bindings in something like a decade, so OpenGL developers all have to use the extension mechanism to access all of the new features.
OpenGL sits over the top of DirectX on Windows so just using it would incur a penalty. I'm fairly certain that anyone porting a DirectX game to Linux using winelib or some commercial derivative would incur a penalty in the other direction.
Games running on Wine/WineX/Cedega used to run faster than natively on Windows in a lot of cases, but I suspect that has to do with many functions not being implemented or just quick hacks to work rather than render correctly, not some magical performance increase while doing the same operations on the same hardware.
Back in the day, many games used to have a choice of rendering systems, D3D or OpenGL, (and or software even) and everyone knew different features were implemented on each. There's no reason to assume Valve's OpenGL port is feature complete.
Rather then going for broke and getting as much FPS as possible, why don't game developers focus on optimizing the experience for a SOLID 60 FPS, that is, instead of peaking at 300 FPS in one scene, and then dropping to 45 FPS in another, strive for a constant frame rate.
If 60 fps does not tax a rendering system, then focus on MORE content in the scene, such as more particles or physics effects which enhance the gaming experience.
Maybe this is just some raw development figure, but there is absolutely no point in dumping as many frames as possible to a screen which only refreshes a given number of times per second.
I mean if someone came out with a car that outputs 300hp when driving 60 mph, I wouldn't be impressed, so why do I care how many frames are rendered between screen refreshes.
I haven't thought of anything clever to put here, but then again most of you haven't either.
The summary mentions 'Windows'. Therefore it has no place on /.
You're starting to slip!
The this means only one thing for me. Time to build out a full native Linux install again. I switched to Windows only and with Sun( yes sun not Oracle, well it Oracle now, but I miss Sun) Virtual Box Linux VM's both Servers and Desktops for my Linux needs. But, I can build out a new BOX with some new kick ass hardware and start tweaking out a desktop. Hmm, things are looking up for Linux on the desktop. Sweet!
Eric
% wine Steam.exe
Yeah, look at them all.
% wine Office.exe
Hey, look, MS has released Office for Linux! Let's give them a round of applause!
This is actually a mundane sort of thing in cross platform development. Your other "ports" aren't just a money pit. They allow you to stress your code in ways you might not have thought of. Typically the bugs you find in secondary platforms improve your product on it's primary platform.
A Pirate and a Puritan look the same on a balance sheet.
DirectX10+11 improved performance a lot. This study don't talk about it.
Frankly it shouldn't be surprising, as OpenGL frankly isn't the focus on Windows so why should the Windows team optimize it? It doesn't gain anything for MS to have OpenGL work any better, DirectX works great and most cards frankly have a lot better DirectX than OpenGL support, and Windows is going to GPU accelerated everything while Linux can be stripped down like Win98 used to be so you can rebuild it and make a hell of a "bare metal" system for a specialized task...say gaming? Even if you don't strip it Linux distros don't usually have a bunch of GPU acceleration, nothing like Aero or Win 8 where every frame is GPU accelerated. Naturally using the GPU for more tasks is gonna lose a few frames for gaming especially when Joe Average don't know how to turn that stuff off.
So I don't know why anyone should be surprised at this, Linux has always been OpenGL, Windows DirectX, so having Linux do OpenGL better isn't surprising, anymore than Windows doing DirectX, its just what the platform is made for is all.
ACs don't waste your time replying, your posts are never seen by me.
So I don't know why anyone should be surprised at this, Linux has always been OpenGL, Windows DirectX, so having Linux do OpenGL better isn't surprising, anymore than Windows doing DirectX, its just what the platform is made for is all.
As a counterpoint, TFA said that their OpenGL port of L4D2 on Windows ran faster than the DirectX version.
So I don't know why anyone should be surprised at this, Linux has always been OpenGL, Windows DirectX, so having Linux do OpenGL better isn't surprising, anymore than Windows doing DirectX, its just what the platform is made for is all.
Assuming I understand you correctly, I should perhaps have rephrased this:
What I find more interesting, to be honest, is that Open GL is (slightly) outperforming Direct 3D on a windows/nvidia box.
To:
What I find more interesting, to be honest, is that Open GL on a windows/nvidia box is (slightly) outperforming Direct 3D on the same box.
That's a great game. When did Valve release it?
Does the openGL driver/port have the same graphic detail as the directX version?
You think they don't know how to benchmark their own game?
No sig today...
I've noticed the different in speed of OpenGL versus DirectX in the past when the first versions of Google Earth came out. You could choose between OpenGL and DX to run the program. OpenGL was very smooth whereas DX's FPS could be counted on your fingers.
It'd be highly amusing to me if, in a few years time, Windows users are keeping a copy of Linux around because "I need it for the games" :)
So.. it has come to this
Clearly not. They give a bare number that doesn't indicate whether it is a maximum FPS or an average FPS. They provide neither the test setup (screen resolution, detail settings, etc), nor meaningful analysis of overall performance. For example, if the average FPS is lower on one platform, but the variability is also lower, the actual user-perceived performance will be better. No meaningful details are provided, just some ePeen number which is abstract of context. The statistician in me weeps.
When you port to several systems you find, even if they have the same userland and syscalls, that your code becomes better and more robust by your efforts to get it to compile cleanly on all platforms.
Which makes more sense?
Yellow paint looks better than black on a house.
or?
Yellow paint on a house looks better than black paint on a house.
The "on a house" part is orthogonal to the comparison of yellow and black paint. No need to be redundant.
"What I find more interesting, to be honest, is that Open GL is (slightly) outperforming Direct 3D on a windows/nvidia box."
Why would you find that interesting? It's been rather well-known OGL is superior to DX, just by the very nature of being extensible without having to wait for a new update to the OGL spec. If you want a new feature and the card has the power to do it, you can implement it directly into your engine and send the calls direct to the GPU.
Using DX, you're stuck with what you're given, and with the additional abstraction layers in DX, performance drops.
GOL has less overhead. No wonder it beats out DX, which is still partially CPU limited (moreso than OGL.)
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
That only shows that they fail at communicating the results. Not that the result itself is bogus...
I was wondering why the difficulty level increased with each version.
How can I believe you when you tell me what I don't want to hear?
The test setup is provided on their blog: http://blogs.valvesoftware.com/linux/
Oh derp, you were referring to the options. And the blog was linked in TFA. Guess I'm the idiot.
This depends on the operation before, but one of the big areas I noticed improvements seemed to be in loading times due to disk access. As it the same game on the same machine/disk (though different partitions), I'd put that down to filesystem efficiency.
Was it a laptop?
I found that often where the windows driver hadn't been updated in awhile, the Linux driver was actually more fully featured at the time.
My old HP laptop ran some games (I believe it was Battlefield 2) that wouldn't work at all in windows due to driver issues.
It's outperforming DX9. DX11 is a different animal.
The price is always right if someone else is paying.
Silly question from someone who doesn't keep up with cutting-edge 3D stuff: is there any reason for the industry not to standardize on OpenGL ES 2.0? That already seems to be the de facto standard on portable devices. What features does it lack that make it unsuitable for desktop and console gaming?
It's not "clear" at all. Just because they didn't provide such details doesn't mean they don't exist. Absence of evidence is not evidence of absence.
Valve is an exceptionally competent developer of high performance, low-level graphics software. The article says "The data is generated from an internal test case". It's reasonable to assume this internal test case is specifically designed for, oh I dunno, TESTING. Which means it's designed to put the different rendering pipelines through the exact same gauntlet (or as close as can be managed).
Why would you think that them running the test at inconsistent resolution/detail settings is even a remote possibility? Do such obvious test practices REALLY need to be spelled out for you?
Yes, they reduced the test data to a simple number (I thought it was pretty obvious this represented "average" FPS). It's an easier discussion point. What of it? It's a blog post, not a scientific paper.
"Mind, as manifested by the capacity to make choices, is to some extent present in every electron." -Freeman Dyson
It isn't surprising for anyone who knows DX and OpenGL well, but it is surprising for game developers and other graphic intensive users since the "everybody knows" that graphics on Linux sucks.
But... the future refused to change.
One of those people who says... whats the point of 32 GiB of RAM.... so short sighted. because 4GB will always be enough for EVER. It wont run 300FPS on YOUR computer. The point is to improve efficiency for all computers, even the low end crud you own so that you get 60+ FPS instead of 2
Hardware OpenGL has always been faster than DirectX (there were a lot of companies and talented engineers pooling ideas for OpenGL, plus it has been around a while and worked out a lo of kinks). This shouldn't be a surprise. There was a hiatus where DirectX was ahead on features (we'll ignore the OpenGL extensions, and consider Core only). Now OpenGL has feature parity wih DirectX and works on a lot more platforms (which you could take as OpenGL actually having more features) - including OpenGL's superior support for Windows XP.
Yes, it's mentioned in one of the replies in the comments.
Mada mada dane.
Again this is surprising...why exactly? Valve has ALWAYS been more OpenGL focused, going back to HL1 where you would get better visuals and framerate if you chose OpenGL over D3D so why is it surprising that a game made by them runs better on OpenGL than DirectX?
This is like saying "Sony games run better on the PS3 than the PC" which of course would get a bug DUH! from everyone because it should be obvious, same here. Valve and ID have always been more OpenGL than DirectX, been that way since the beginning, doubt its gonna change barring OpenGL going the way of Glide. I'm sure the Valve guys know OpenGL like the back of their hands, know all the tricks to squeeze performance out of it, so not a shock when that is what we see in the FPS.
ACs don't waste your time replying, your posts are never seen by me.
As a counterpoint, TFA said that their OpenGL port of L4D2 on Windows ran faster than the DirectX version.
Which makes me wonder where Valve is hiding the OpenGL for Windows version... as far as I'm aware, all of Valve's Source games for Windows require DirectX 9.0c.
GLaDOS for President 2016! "Well here we are again. It's always such a pleasure." -- GLaDOS, 2011
The statistician may weep, but the writer would rejoice. A single number is the correct way to deliver information given the context of the statement. The information you are looking for would be appropriate in a full review. This statement was from a quick blog post.
32 GB of RAM, i don't know any others off the top of my head. read better articles
Don't get me started on the "absence of evidence" line. It's a BS saying by those who want to support a particular claim without having any way to show others they're not just making stuff up. This isn't even absence of evidence, though. It's poorly-stated evidence, or possibly even misleadingly-stated evidence.
When a company that is "an exceptionally competent developer of high performance, low-level graphics software" fails to communicate test results in a fashion which can be meaningfully compared by people who would be interested in such a developer's blog, it either calls into question their competence or the content of their communication. A competent group that realizes "hey, that's really cool, we got a higher average FPS" and also sees "that's weird, there are a lot of sections of benchmarking where the actual FPS drops to 15-20", might choose to communicate the first result and hide the second in order to drum up support. Simply reporting the average FPS does nothing to assure us that the performance is better for an end-user (other considerations include micro-stutter, periods of low playability,
Additionally, such "obvious test practices" do really need to be spelled out, and conformity results reported, for a reader to even infer a meaningful comparison. With the system they described, if they were pumping this out to a 1920x1080 display, the test results mean something completely different than if they were driving 1, 2, or 3 2560x1600 displays. I (and others in the field or even just interested in the field), would like to know this information so that it is meaningful, and not merely an "ePeen number".
Perhaps it's due to [H]ard|OCP that I've come to expect better benchmark reporting, but what is shown here disappoints. A test case can be designed to test a number of factors, so simply stating it's for "testing" indicates you're unaware of testing in the graphics area. Those factors include, but are not limited to, conformance, pixel throughput, vertex throughput, fill rate, average fps, memory utilization, GPU utilization, CPU utilization, inter-frame jitter, and average dropped fps. Not all of these always need to be reported, but some of them are linked, and should be reported together.
So yeah, one factor is marginally faster and was useful for finding a previously-missed bit of overhead. Cool, great work, etc. But don't report a single number which may or may not be misleading (and will obviously be touted by some a place to claim platform or API set x is better than y), without giving meaningful context to the results. As you said, Wraithlyn, they're experts. How about they demonstrate that in a meaningful fashion for the other experts who might gain some insight from their blog?
I still can't believe they've not made a simple WINE/DX API wrapper built natively into Linux. I remember GLide wrappers that worked very well under Linux when 3Dfx support SUCKED.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
This is not correct. The OpenGL backend was abandoned with Source engine, and removed completely, while Direct3D9 back-end was integrated more closely with the actual engine. This was more than 10 years ago, so given the changes in hardware and driver infrastructure, all relevant expertise at Valve is Direct3D-centric.
The OS X OpenGL port piggy backs off the Direct3D9 back-end and uses an emulation layer, a custom written library which presents a Direct3D9-like interface to the program while emitting OpenGL calls. This is evident from the backtraces of OpenGL related crashes of Source engine games on OS X which have been posted elsewhere. It's unclear what the scope of this library is - i suspect it encapsulates the state management and shader translation.
On the other hand, NVidia has always said: if your application is geometry centric, if it makes a lot of draw calls and is light on actual GPU load, you SHOULD use OpenGL because this will yuild you up to 30% more performance on NVidia driver stack. Thankfully, Microsoft has improved things for Direct3D starting with Vista, but not enough to close the gap.
-Siana Gearz,
Singularity Viewer