Slashdot Mirror
RAYA: Real-time Audio Engine Simulation In Quake
New submitter bziolko writes: RAYA is a realtime game audio engine that utilizes beamtracing to provide user with realistic audio auralization. All audio effects are computed based on the actual geometry of a given game level (video) as well as its acoustic properties (acoustic materials, air attenuation). The sound changes dynamically along with movement of the game character and sound sources, so the listener can feel as if they were right there — in the game.
This is what happens when you DON'T open source your games. Your game doesn't make the news when researchers DON'T use your games for research.
https://github.com/ioquake/ioq3/
Updated 14 hours ago. The updates never ended.
I liked what I heard, but I really like to have a demo of it to check out.
Be seeing you...
Quake audio consists mostly of footsteps and bangs. This might be fun for, say, GTA IV/V, where the NPCs have conversations to which you can listen if you're close enough.
Somehow this will cause someone to puke.
There was a company back in 1997 that had a fantastic (series of) cards that did all this 3d transformation, reflection, deflection and occlusion of audio in hardware. The company was Aureal, and their A3D system was fantastic, doing everything that this demo showed. The competitor, Creative's EAX, instead used the entirely dumb method of "turn on reverb in a room". Creative sued Aureal, thinking that they had a leg up on 3D audio. Aureal countersued, and won, but the legal costs drove them into bankruptcy. Creative then bought Aureal's assets, and buried the company, and all it's technology, never to be seen again. In fact, EAX is still the stupid-simple (and very broken) "turn on reverb" (though now it also has "Adjust reverb"). And, as Creative have shown before (With the whole "Carmack's Reverse" fiasco) They're more than willing to use legal means to muscle their way.
For those who play in headphones, not with 5.1 or 7.1 surround audio, a system that tracks head rotation and tilting (similar to what they have for airplane sims, where you wear hat with markers and a webcam tracks your head position... and view in displays is changed accordingly) is needed. I haven't seen any of those at the market yet. Maybe you've heard about such things?
It is good to give devs the option of realistic audio, but for games in medium - big settings, the relative slowness of sound propagation is a problem. Getting a headshot and later hearing the sound is counter intuitive, at least for the hollywood generations. I guess that realistic effects with no delay in sound propagation is the way to go.
---- MISSING MISCELLANEOUS DATA SEGMENT --- [sigdash] trolololol
Guild Wars 2 implemented a system like this to dynamically calculate both occlusion settings and reverberation and echo in real-time.
Irony: Agile development has too much intertia to be abandoned now.
Remember when the SoundBlaster Live! came out and Creative Labs were telling you that it had as much processing power as a Pentium 166MHz MMX, dedicated entirely to sound processing? Well, it turns out that now you can have far more CPU power than that dedicated entirely to sound processing without custom hardware...
I am TheRaven on Soylent News
How is this at all different from GSound: http://www.carlschissler.com/g...
I even have a basic working implementation of it modded into Arma 3...
*BLORPH*
Oh man! Right in your lap! Sorry about that dude!
I'll try to aim someplace else next time...
*BLORPH*
Well...your face...at least it wasn't your lap this time...
Chas - The one, the only.
THANK GOD!!!
10% hit on relatively low poly collisions and very few sounds.. :/
Using the video as a basis, its clear the framerate halfs when RAYA is active.
I'd love to see some "real" benchmarks, but obviously the performance impact is too big at this stage. Which worries me, considering their using iD Tech 3 engine for this demo.
Theres alot of raytracing and calculations going on.
In essence, theres just as much (if not more) processing going on here, than a full game's logic loop.
The only way this would work in real world applications is to ensure this processing is done on its own thread.
Interesting, but it clearly needs alot more work on optimisations.
Would like to see the API and the reference documentation.
This (audio raytracing) was done in the late 90s by a company called Aureal.
Their 3D audio cards were UNBELIEVEABLE. I played the original HL using one - and played CS using them - and they were a game-changer. If you had one, you were 10x better off than someone who didn't. You could tell how the battle outside was going on, by hearing how the people firing were changing position - if your team (you knew which direction they were entering combat from) were firing and moving forward, then they were winning.
One of the demos was a helicopter, circling the players head. You tracked it with your eyes and mind as it went round - it actually R E A L L Y sounded like a true, physical helicoptor circlng your head.
The Creative sued them into failure.
I've never forgiven Creative for this. I've never and will never buy any of their products.
A3D v2.0 demo on Youtube. I find it much more impressive than RAYA, possibly due to the HRTF in addition to the wavetracing. I had such Aureal Vortex2 card in the day. It was amazing how good the 3D positioning was, even with two pc speakers next to the monitor. Creative ruined it. For me, that alone is more than enough reason to boycott Creative to this day, and beyond.
Similar to what Aureal was doing with A3D back in the 90s, but obviously not tied to a specific piece of hardware like back then.
I enjoyed the Quake 3 demo, but it while it works decently well with just the player in the level, it sort of falls apart during the deathmatch. I think that's probably because the stock Q3 sounds have a bit of reverb baked in. I would love to hear what it would sound like with a complete set of reverb- and echo-less sound effects, so the RAYA can handle everything by itself, instead of working in top of the baked-in reverb.
Eat the rich.
The thing is stuff like this and raytracing graphics are best served by parallelization, so it's not so much an issue of cranking out more performance as it is just finding a way to put it on something like a GPU's worth of stream processors.
A bullet may have your name on it but splash damage is addressed "To whom it may concern."
Realistic sound has been around, as people point out, since the Aureal days. Now, to be honest, it should be baked into every engine and tied to your textures (soft textures absorb sound, shiny textures reflect sound, etc.).
The fact that it isn't means a couple of things - it's too expensive (which I can't believe nowadays), it adds too much cost to development time (but surely modifying those sounds for echo etc. is more costly than just putting in a pure sound and letting the engine modify it as necessary),, people just don't notice that much, or the patent field is too heavy.
Take things like TF2, HL, CS, etc. They are all same-engine. They are all 3D open environments. It is vital to know where shots etc. are coming from in order to play properly. But we don't see such audio tricks. That, to me, suggests they aren't necessary or certainly not the right value to waste time on.
And, to be honest, I watched "ray-traced quake" over, what? Ten years ago? That tech still isn't used in modern games because of the above reasons. It's do-able but expensive, the development time is costly, the effect isn't that much different from pure cheating on the 3D drawing, and it's not in any of the major game engines. This is suggestive of the value of such things being minimal.
And, to be honest, the realistic-"ness"of a game is the first few minutes of unboxing and then that's it. What destroys your immersion from then on is crappy plot, unrealistic capabilities, and AI that still - to this day - sucks. Fire gun, run around corner, wait for the idiots to pile round. The "better" ones might well throw a grenade but once you know that, you take account of that, and that's the AI beaten. To "win" the AI has to have reactions infinitely better than yours and outnumber/outgun you. Think about the average FPS game - there are several THOUSAND bad guys. And you. And though you might get stuck occasionally, you will win. You can use first-aid kits, they can't. You can lure them into traps, they can't (unless scripted). You can sit and wait them out. You can guess where they will walk next, they forget about you one second after they stop seeing you. It's ludicrous.
Please stop wasting our game industry by reinventing tech we've had for decades and could put in any game, given time. Let's try and make a game with one, single, scary opponent (and maybe some NPC's to fill in the gaps). A Matrix-like game, for example. Agents are few and far between, maybe one per real player. There is only one that's a real threat. And there's you. And a world that you can both use to your advantage.
When humans play humans you HAVE to have the same numbers on both sides. When humans play AI, you HAVE to be vastly outnumbered.
I'd much rather Half-Life 3 had intelligent enemies who will choose to camp the chokepoints and not be lured out, than some fancy water effect or proper audio reflections or whatever.
You're not telling me that with the CPU/GPU available nowadays, we couldn't make a Quake 1 opponent that - with the same programmed reaction times, capabilities, and facilities available to them as a human player - couldn't be a serious threat. I'd rather play that than yet-another "look how shiny" kind of game.
Never mind making Quake/QuakeII/Quakex give audio cues that match the environment more precisely. When do I get a holosuite? I'd very much like the sound to match the image there, especially for some of the more, er, interesting holosuite programs.
Actually it was originally Aureal's project back in the 1990s.
https://en.wikipedia.org/wiki/...
The problem has always been that in games audio (sound tracks withstanding) is seen basically as a gimmick. A few games do it well, but for most, it's an afterthought.
The selling point has been, and always will be, graphics. Some reasons: Humans are predominantly visual, magazine based reviews can't demonstrate audio (this is changing due to youtube and other video reviews), lack of audio hardware that WORKS properly (IE; a sound card that processes EAX/positional audio, speakers to take advantage of it, and non-shitty drivers.. Fuck you Creative.)
This will not change until things like the Occulus rift become more mainstream; and shitty, non directional audio is the new bottleneck for immersion.
there's really very little difference between optimizing audio and video. back-culling polygons and all that magic to increase framerate by lowering processing overhead. Same thing with audio. It's just that it hasn't really been taken very seriously in the past.
When Marathon came out, it had "ambient sounds" that changed as you moved in relation to their source. They were also in stereo. (these were new, no other fps had it) Sound effects from map features, weapons, and ordinance were adapted based on distance from you and were also in stereo. Sadly, lery little has changed since then.
I work for the Department of Redundancy Department.
I was never fortunate enough to actually own my own Aureal card.
But I really really can't understand why having bankrupted them, and taken all of their technology, creative didn't do the sensible thing and USE IT.
Even now A3D is still vastly superior to the latest EAX FIFTEEN YEARS LATER.
To me, this demo is serious uncanny valley territory.
When I was composing MOD music on my Amiga back in the late 80's, I was very much aware of the problem of playing the same instrument on the left and right channels at the same time, especially when doing pitch slides. You got all kinds of weird interference problems, or the audio version of moire effects, if you will. If you were good composer, it could be used to good effect in music in a lot of cases, but most of the time it was a real pain, especially with sound effects in games.
I hear plenty of that in this demo, and it's far different and more annoying than actual reverb. As it is, the sound is just too "off" for me to consider it an improvement, and just like 3D sound, I'd have this feature turned off.
For whatever reason, it isn't something there's much interest in, but it does exist. I am aware of three options:
1) The HeaDSPeaker. The cheapest option. A little device from a not very well known company called VLSI Solutions. It handles the head tracking and HRTF, you provide the headphones. Runs about 340 Euro ($450). It can take input either as a Dolby Digital stream, or directly as USB from the computer.
2) The Beyerdynamic Headzone. This is an all-in-one solution from Beyerdynamic. Has a decoder, HRTF calculations, headphone amp, head tracking, and a pair of DT 880s. Costs about $1700. Requires DTS or DD input for multi-channel input.
3) Then the grand champion, the Smyth Research Realiser A8. This thing takes measurements of your headphones, ears, speakers, and room and so accurately recreates the sound it is more or less impossible to tell it apart. The unit handles measurement, decoding, HRTF, head tracking and so on. However it costs $2900 for the unit alone, $3700 with the Stax headphones and amp they recommend for it. Oh and you need a good surround system to measure, so you either need to own one or book time on one. Needs either multi-channel analogue or HDMI input.
So it is out there... but you pay a ton for it. That's all I know of at the moment, it is a topic I keep track of because I have a lot of interest in it.
This is the equivalent of ray tracing in graphics - nice effect, but very heavy on the computation.
With graphics, rasterization is faster, and the reason is that it can be characterized as "a bunch of cheats that happen to look good". Can we identify some similar cheats for sound?
Yes, I think so. Here's a paper I wrote 16 years ago outlining one possible, very simple, basis for soundscape generation.
https://ece.uwaterloo.ca/~vrml...
Unfortunately, I didn't get to progress with it as VRML faded out pretty quickly after 1998.
Sean Ellis
Follow OfQuack's antics on Twitter.