Slashdot Mirror
Valve Sponsors Work To Greatly Speed-Up Linux OpenGL Game Load Times
An anonymous reader writes "Valve Software has sponsored some interesting improvements developed by LunarG for the Mesa OpenGL library on Linux for deferred and threaded GLSL shader compilation. What these changes mean for users of the open-source Linux graphics drivers when running their favorite games is that OpenGL games now load a lot faster. As an example, the time from starting Dota 2 until the time actually being within the game is reduced by about 20 seconds on an Intel system. While Direct3D has offered similar functionality for a while, OpenGL has not, which has given it a bad reputation with regard to game load times until all shaders are compiled and cached — fortunately it's now addressed for OpenGL if using the Mesa Linux graphics drivers on a supported game."
OpenGL probably need to move to an IR for shaders. For thost that don't know, shaders are compiled by the driver today. Many games generate shaders with a combinatoric explosion which creates a lot of work for the driver. You can cache the result, which is often done, but you'll end up recompiling when the user change settings, etc.
The solution is off-line compilation to an IR, but of course everyone need to be aboard. The downside is that an IR spec will probably add 'DRM' considerations whereas today OpenGL shaders are shipped in source form.
This won't affect Tux Racer's FPS. This will affect the load time on DotA 2, as well as other games which compile many shaders during load time.
Caching of compiled shaders is supported by the OpenCL spec, and I presume by GLSL as well (I've not looked, but they generally use more or less the same code paths). The application is responsible for asking the driver for the cacheable version and then loading it again later. The problem is that, on first load, the game is effectively doing ahead-of-time compilation of all of its shaders and, previously, these were all done in a single thread. The multithreading part is a bit odd, because most DRI GPU drivers use LLVM on the back end and LLVM has supported multithreaded compilation for a few years.
I am TheRaven on Soylent News
So Linux sucks because anyone can improve it in the areas where they feel it is needed? Yeah, that does suck. Booo, Linux.
Pretty good is actually pretty bad.
I think the argument is that nobody does so for several years.
If nobody is willing to develop a certain feature, then maybe there isn't a real demand for it.
Pretty good is actually pretty bad.
I think you miss something:
On Windows, OpenGL already competes with DirectX. Especially with the indie-game revolution of the last 10 years.
On Linux, OpenGL wins 100% complete, hands-down. Because DirectX can't even run, let alone come close on performance.
On Mac, OpenGL wins 100% complete, hands-down. Unless you count boot-camp, which is really just Windows.
You can try to paint a different picture all you like - fact is that OpenGL is not only "the same" as DirectX when you're on Windows, but also runs in a ton of other places. That fact that it has slightly less performance than the ideal scenario on one of those (it has to be said) more obscure platforms is pretty inconsequential (and now fixed). I haven't seen anyone complain about the OpenGL performance on those millions of smartphones that run it. I haven't heard much about DirectX on smartphones, however.
Cherry-picking the battlefield for a comparison is no worse than cheating because you know you're going to lose.
except that no one does.
"anyone" sounds like a lot of people, but Linux isn't just for coder enthusiasts with the know-how to fix their own problems. If linux is going to really take off on the desktop those things simply need to be already taken care of. With microsoft floundering around with windows 8 and tablets taking off, if someone wanted to really get market share away from microsoft dumping money into Linux like valve is doing here is a good start. Especially if it can be done in such a way that major game studios can easily make their games multiplatform. Games are what keep a lot of dedicated enthusiasts of all ages away from Linux. So are things like photoshop or microsoft office, or etc. A lot of the core products that people need just don't work well or at all. You can carry on about alternatives, but people don't want alternatives for those kinds of things. The OS, which is mostly background to a lot of people is easy to persuade them on, you can make it look and feel like windows. but gimp will never feel and look like photoshop.
Even on windows, OpenGL is not that much slower, on the order of 10% for 'normal' 3D game code. With a modern video card that difference almost doesnt matter.
Drivers to be renamed "Black Mesa OpenGL Library"?
I doubt people are using Windows because of the deferred and threaded GLSL shader compilation. I think it has more to do with the fact games are barely available for Linux at all. And that also has little do with how shaders are precompiled I think.
This feature just shaves off a few seconds during load time. That's great of course, but by no means a killer feature that previously has been a real problem for anyone. That's why the feature is late to the party and it is a gaming company who comes up with the patch, as they want their games to load faster. Makes sense, right? In no way I see how this makes Linux look bad.
Pretty good is actually pretty bad.
How is it Linux fault that Microsoft doesn't provide Office or Adobe doesn't provide Photoshop for it?
Pretty good is actually pretty bad.
And that's just the desktop. On phones, OpenGL wins all. On tablets, OpenGL wins all. On consoles, DirectX is only relevant for the Xbox, the rest runs OpenGL.
The sphere of influence Microsoft has on the computing industry as a whole has been shrinking with each passing year for at least a decade now. The diminishing relevance of DirectX is yet another proof of this.
Pretty good is actually pretty bad.
Lack of games is a factor, but the other factor is games just aren't as good. Load times is a factor in this. And while it may not seem important, 20 seconds is a lot! On consoles, load times are a reason to fail compliance testing.
For games devs, the choice of API to use really depends on platform targetted.
The Direct3D API isn't *quite* the same as the one used on the XBox consoles, but it's very close. That makes porting a much easier, cheaper prospect. I don't know what API Playstation games require.
So hurray to Valve for fixing this and hurray to Linux for letting them, right?
Pretty good is actually pretty bad.
And that also has little do with how shaders are precompiled I think.
As far as I know, you cannot precompile shaders anyway because the compiled code is hardware-dependent. The shader processors are different among architectures and manufacturers, and do not have a common baseline like "x86-64" to target, like we have on the CPU side.
I meant precompiling in the sense of compiling them during loading, so they can be used during gameplay.
Pretty good is actually pretty bad.
Well, there's no demand for this because there's a relatively convenient alternative. In this case, the alternative is Windows and DirectX. It doesn't really say a lot for Linux if this is what people are doing.
Yes, exactly right. 3D gaming isn't some half-dead community with no revenue stream on other platforms. Hell, other platforms were created for gaming due to demand (From Atari to PS4), which has been going on for decades.
In the meantime, the Linux community sat on the sidelines and assumed what everyone really wanted in any new distro...a new version of GNOME or KDE to keep other more "important" debates alive.
While I can understand an efficiency within demand, it's rather odd that Linux still looks at 3D gaming like it's a 20-megapixel camera in a cell phone, when we in fact have 20-megapixel cameras in cell phones these days...
Kinda, yeah. What we're seeing now is the breaking of the chicken-and-egg problem of gaming on Linux. Up until very recently, virtually no developers bothered developing games for Linux, because no-one does gaming in Linux. No-one ran Linux for gaming, because there were very few games for Linux (and the drivers were a pain).
Up until recently, Linux had merely taken over the world when it came to servers and mobile (Android). Now it's being given a real shot at gaming.
Actually the hardware support is quite good already. I agree with your other points though: Linux world is a mess with parts missing, lots of bugs, and lacking quality assurance.
https://en.wikipedia.org/wiki/...
"Valve created the critically acclaimed Half-Life (released in 1998) and Portal series (released in 2007), as well as the software distribution platform Steam (released in 2002) "
btw as of last year Steam has a native Linux version
Any person using FTFY or editing my postings agrees to a US$50.00 charge
As I understand it, compiled shaders aren't very big, so precompiling for a few dozen chipsets isn't much of an issue storage-wise.
How many different chipsets are there to support?
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
The problem with OpenGL isn't speed or load times. Microsoft made directx so 3D programming in windows would be easier. It's just like C# or VBA. They made it much easier to use, and therefore much cheaper to hire for. If you're hiring people to write a new 3D engine using OpenGL you need people at the top of their field... If you're hiring for DirectX there are dozens of local tech colleges filled with mediocre talent that will fit the bill. Now, you could argue that you should be getting top talent anyways, but that's why you don't run a major game producers.
When I was working on this sort of thing a few years ago, we were experimenting with a RISC machine with a fairly deep pipeline for shader operations.
It worked pretty well, and wouldn't be surprised if the big boys used a similar idea but optimisation for these is very architecture dependent and can be an NP complete problem. Even register allocation can be quite slow and they'll have to deal with that.
...both have functionality for accessing (and saving) a compiled shader so that it can be loaded and used instantly on next run.
Not by GLSL as such, but it is supported in GLES 2.0.
Because linux isn't a cohesive platform. That's the problem. As I was googling around one of the staff at adobe mentioned last year that Linux lacked standardized APIs on a forum thread regarding photoshop on Linux.
There is a perception that Linux is a bit like the wild west and in this day and age when you have stable mature platforms like Mac and Windows available, that's risky for developers. Even for big companies.
The intrinsic connection they have is market share and having already been the platform for this programs for a long time. Linux needs to really step up and say "Hey we're ready look at us" but they haven't had that moment yet.
Ubuntu is a step in the right direction. If a company with real money can get behind it and drive it to some kind of consumer ready level like Windows or Mac is, enthusiasts can still sit there and fork and tweak and do as they like, but getting a real ready version there that gets people's attention and wants to make people use it and develop for it is what will drive Linux's success.
It might not be directly Linux's fault that Microsoft doesn't make office for Linux, but they just got office for IOS not that long ago. Who knows what kind of wrangling that took. But if I was someone like Canonical I'd see just how much money it would take to convince Microsoft to make it for linux and make that happen. I'd do the same with programs like Photoshop, and other major programs that have major user bases that are seen as core apps. Valve already seems like they're moving in the direction of taking care of games so I'd make sure I was meeting with them and getting everyone on the same page. They don't have to arrange all the programs. If they do a few core programs that reach a large percentage of the user base, the other programs will start to get ported to linux as user base picks up. For example if they paid to get photoshop and office ported and linux went from the low single digits its sitting around now on the desktop up to 20% or a little higher I think you'd see companies start to take notice and start to focus a little more on it.
OpenGL. Just like WiiU, Android, iOS, and every other platform that isn't Microsoft.
"[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz
Im also hoping they deprecate the libraries in Team Fortress 2 that are patent encumbered before someone hauls them into a texas court. W have perfectly reasonable alternatives to S3TC.
Installation and posix portability are also on my wishlist but thats sort of offtopic and not likely to happen without a bit of ecosystem backlash..
Good people go to bed earlier.
In most cases OpenGL is 10 to 20% faster on Windows, just shader compile times were 5 to 10% slower on average.
but once things are running DX11 isn't very competitive, and that's also because it's hard to make proper drivers for it (see Nvidia's late optimizations that made them reach near AMD Mantle performance, that took almost 2 years to reach)
Because linux isn't a cohesive platform. That's the problem. As I was googling around one of the staff at adobe mentioned last year that Linux lacked standardized APIs on a forum thread regarding photoshop on Linux.
So how come Autodesk is able to ship Maya for Linux? Or MathWorks has no trouble releasing Matlab for Linux?
You're talking like it's impossible to create software for Linux. Clearly this is not the case, as there are numerous applications available for Linux and have been for years, in all sorts of forms and business models.
The lack of Photoshop and MS Office is not really because of any technical reason. The reality is that Adobe and Microsoft don't want their products available on Linux.
Pretty good is actually pretty bad.
wearing handcuffs and a muzzle.
Go ooooon...?
systemd is Roko's Basilisk.
Yes, there are some dickheads in charge of major Linux projects that refuse to do things users want. There are also dickheads in charge of major Microsoft proejct that refuse to do things users want. Same with Apple, Adobe, Oracle, and many other companies.
and no, it's not only used by cultists. It's used by smartphones, GPS, DVRs, servers, supercomputers, and other places. The desktop is more of an exception than a rule, but the desktop is the place where the OS is more visible.
This is my signature. There are many like it, but this one is mine.
Because being free isn't a pass on criticism. I'm not sure when that happened. Yes all the people who have devoted time and effort and money on Linux deserve credit for doing so. But for the end consumer there is no real difference between an OS you got for free and one you paid $100 for. They still need to work and they still need to do what you need. Otherwise free doesn't mean anything. In the end they're still providing a product regardless of the cost and the consumer is going to form an opinion on it and give feedback. It's simply just not there yet and the only thing that's really going to get it there, in my opinion, at this point is a huge infusion of cash in exactly the right direction.
On Windows, if you use Visual Studio (which most studios do), DirectX is easier.
The documentation is fully integrated, and you don't need to use a wrapper or the hideous OpenGL extensions mechanism to use any of the remotely modern features. You can use GLEW for the latter but you really shouldn't have to rely on a third party library just to use it.
You are smoking crack. The reason is because non-open drivers have had this implemented since word "go". That's what people wanted to use. Hence, the whole supply/demand thing kicking in. That someone is doing it in the open-source drivers means that they aren't getting the love they expected from the third party, and suddenly there is a business interest in having better support in the open driver.
To draw a parallel, would you use the default drivers that come "out of box" on a fresh install on Microsoft Windows whatever, or would you actually go to the vendor's website and download their specific drivers? I think we're done here.
While a valid excuse, that's another reason why Linux is still "not there" yet. The fact that there were only a couple legit games for it proves it's still in it's fetal stages of becoming a consumer OS. It's got a long way to go. Valve was a huge shot in the arm though. They are the best thing to happen to Linux in awhile.
If I am not mistaken, Win8 - and WinPhone8 - support OpenGL as the Direct3D/etc APIs were too power hungry....
Truth is like the sun. You can shut it out for a time, but it ain't goin' away. - Elvis Presley (source: imdb.com)
Mod parent up. Atleast Valve found that Left 4 Dead 2 run faster after porting OpenGL on Windows (and even faster on Linux).
http://blogs.valvesoftware.com...
FWIW, the Nvidia proprietary drivers have had a shader cache on Linux since the 290.03 release in late 2011 (search for GLShaderDiskCache). It probably helps Mass Effect 2 under Wine somewhat (here's a bug report from before the cache was added to the driver: http://bugs.winehq.org/show_bu... )
It certainly doesn't help that development and distribution on Linux sucks ass, so porting to Linux is a huge time-sink compared to porting to OS X. Unless you pick a specific distro and *only* support that, in which case the Linux fans who yesterday were begging you to port the software are now bitching that you built it for the "wrong" distro.
Then your support costs run up the roof, because even in the "correct" distro people can install the "wrong" windowing system or sound subsystem or what-not, which becomes a support nightmare. So you pay more to port, then you pay more to support the port, then...
when you're done, nobody buys the thing anyway because "information wants to be free" and, shockingly, a community based primarily around a free-of-cost OS and making their own free clones of commercial products doesn't like paying money for things.
This stalemate isn't likely to change unless there is a major cultural shift in the Linux community.
Comment of the year
Programming in OpenGL ES is still very similar to programming in normal OpenGL.
And mostly it's been ignored on Linux because it was nearly impossible to get specs on how to program the video cards. On Windows most everything went with DirectX so that there wasn't even the portability that you might get via OpenGL (ie, there are very few realtime 3D games on macs either, except via Wine for older games).
And that's just the desktop. On phones, OpenGL wins all. On tablets, OpenGL wins all. On consoles, DirectX is only relevant for the Xbox, the rest runs OpenGL.
The sphere of influence Microsoft has on the computing industry as a whole has been shrinking with each passing year for at least a decade now. The diminishing relevance of DirectX is yet another proof of this.
Speaking as a game dev, I don't really burn a lot of brain cycles on the DirectX vs OpenGL debate that seems to enthrall some people. Any well-written game engine has abstraction layers for the graphics renderer anyhow, and if you're porting to multiple platforms, you're going to need platforms-specific renderers.
On Windows and Xbox, developers use DirectX. Some developers support OpenGL on Windows as well, but very few do so exclusively. For everything else, developers use OpenGL, or perhaps a proprietary version of OpenGL (like on the consoles), which is optimized for the platform, because that's what's available. DirectX is not becoming any less relevant for the Windows and Xbox platforms. I'm not quite sure how you can make that claim. Microsoft's sphere of influence has actually grown in the last 13 years or so, with the successful launch of three gaming consoles. They're still completely dominant in the desktop OS market, despite minor inroads by other platforms. They're not a significant player in the mobile market, so that may be what you're referring to.
Believe it or not, at least in my experience, most game developers I know tend to be somewhat agnostic about platforms. That is, they're interested in making games, and simply choose the platforms that are the best supported, along with the libraries / APIs best suited to those platforms. DirectX is a stable, well-supported, well-documented choice for Windows, which comprises about 90% of the desktop market share. As such, it makes sense that most games would support it. If the devs want to release multi-platform, then they'll probably just add an OpenGL renderer in addition to the DirectX one.
Irony: Agile development has too much intertia to be abandoned now.