Slashdot Mirror

← Back to Stories (view on slashdot.org)

Realtime GPU Audio

Posted by Soulskill on from the see-with-your-ears dept.

CowboyRobot writes "Two researchers at San Francisco State University has successfully implemented hardware acceleration for realtime audio using graphics processing units (GPUs). 'Suppose you are simulating a metallic plate to generate gong or cymbal-like sounds. By changing the surface area for the same object, you can generate sound corresponding to cymbals or gongs of different sizes. Using the same model, you may also vary the way in which you excite the metallic plate — to generate sounds that result from hitting the plate with a soft mallet, a hard drumstick, or from bowing. By changing these parameters, you may even simulate nonexistent materials or physically impossible geometries or excitation methods. There are various approaches to physical modeling sound synthesis. One such approach, studied extensively by Stefan Bilbao, uses the finite difference approximation to simulate the vibrations of plates and membranes. The finite difference simulation produces realistic and dynamic sounds (examples can be found here). Realtime finite difference-based simulations of large models are typically too computationally-intensive to run on CPUs. In our work, we have implemented finite difference simulations in realtime on GPUs.'"

8 of 157 comments (clear)

  1. Finally by Richy_T · · Score: 3, Insightful

    Something to do with all those GPUs when ASIC mining of Bitcoin takes over. It's going to get noisy.

  2. Yawn by sunderland56 · · Score: 1, Insightful

    Yeah, you can do computationally heavy things in a GPU. We've done that for years. All this is saying is that some audio signal processing tasks are computationally heavy.

    1. Re:Yawn by AlphaWolf_HK · · Score: 4, Insightful

      I think what's most important is now we have the mathematical models in place that allow us to simulate convincing sounds rather than "sample and include". For the creative types, this will save a ton of effort and money. It also has implications for games, e.g. with the given environment model, be able to produce convincing sounds in real-time rather than taking sound samples mixing them with reverb, attenuation, positioning, etc.

      --
      Careful with names containing L slashdot.org/~AiphaWolf_HK slashdot.org/~AlphaWoif_HK slashdot.org/~AiphaWoif_HK
    2. Re:Yawn by Instine · · Score: 4, Insightful

      I'm also excited by this. Especially as to what this could mean for Text to Speech. Generating more organically modeled TTS could really push it out of the uncanny valley. Currently if you ask a tts engine to say a word or phoneme, it is identical to the last time it was made. What if it were generated in realtime with the same variances as a human voice.

      --
      Because you can - or because you should?
    3. Re:Yawn by asliarun · · Score: 3, Insightful

      I think what's most important is now we have the mathematical models in place that allow us to simulate convincing sounds rather than "sample and include". For the creative types, this will save a ton of effort and money. It also has implications for games, e.g. with the given environment model, be able to produce convincing sounds in real-time rather than taking sound samples mixing them with reverb, attenuation, positioning, etc.

      Yes, absolutely! I see it as analogous to vector graphics vs bitmapped graphics. Vector audio is THE holy grail of accurate sound reproduction.

      If these guys can pull this off, it will be the literal (digital) equivalent of having your own live performance - every time! You will have software based models of various instruments that will play music for you by playing their respective instruments for you real-time. The possibilities of this are actually astounding. You would record or store music not as digital samples (lossy, lossless, notwithstanding) but in terms of *how* each instrument is played. You have now turned the problem on its head - you are constrained by the accuracy of your software/mathematical model of each instrument, and by how well you are able to control it to become more nuanced. At a hardware level, if you assume infinite processing power, the challenge would be to accurately play these software instruments. You could again take a completely different approach - you could for example have an array of speakers where each speaker is dedicated to playing a specific instrument, and all the speakers are fed separate audio signals.

      Contrast this to the currently audio setup - which would be a 2.0 or 2.1 or 5.1 or 7.1 stereo/HT setup - where each speaker tries (and fails) to accurately reproduce the entire audible frequency spectrum, or you have a mish-mash setup where different speakers divvy up the frequency spectrum between themselves (think sub-woofer and satellite speakers) so they can do a marginally better half-assed job.

      If you look at the entire chain in a traditional setup, you have the speaker driver's mechanicals, the speaker crossover electronics, the speaker wire, the power amp, the pre-amp, the DAC, the player, the source audio signal (mp3, flac, redbook CD etc.), the recording mike, and the recording room - all of these links in the chain distort the music in their own way.

      What I mentioned above is only my interpretation of how this technique can be used -there are a huge number of other possibilities - software defined objects, such as in games, can now have their own (genuine) sound, and that will sound different depending on how you interact with them. You could also have virtual instruments, unconstrained by the laws of physics, define their own physics and their own unique sound. You could even program room acoustics and have the instruments play sounds as if it was being played in open space, a large hall, a studio, on a beach etc.

      Sigh.

  3. People use GPUs for numerical simulation by loufoque · · Score: 3, Insightful

    News at 11.

  4. Re: Impossible geometries? by JWW · · Score: 5, Insightful

    Imagine a metal cymbal shaped as a sphere with no holes in it floating free in the air. Now hit that cymbal with a mallet that is longer than the diameter than the cymbal. But hit the cymbal on the inside of the sphere. Oh and the interior of the sphere is a vacuum.

    There you go, there are a few impossible geometries (and other things) in that scenario.

  5. It's pretty great actually by Anonymous Coward · · Score: 2, Insightful

    One of the fundamental problems with computer based music production is that we're still, unless we're working with synthesized music, limited to pre-recorded samples.

    Vienna Symphonic Library, for example, is well over several hundred gigabytes in size, many of those samples covering various articulations (playing techniques) of the same instrument.

    One set of violins playing legato. One set of violins playing pizzicato. Marcato samples etc. etc. With virtual instruments that is no longer necessary. We can just "tell" the virtual musician where to place his fingers and how to do that and cconfigure a bunch of presets for the composer to use as he wishes.

    One fundamental problem at the moment is dealing with smooth transitions from one note to another in sequence. For example the violas of a real orchestra playing a transition from one note to another would slide between them smoothly. This CAN be simulated by changing the note pitch digitally, but that loses authenticity.

    VSL solves this by pre-recording the musicians playing the most common note transitions. This is a huge undertaking and takes up a lot of space as you have to record C to C#, C to D, C to D# etc. etc.

    If we can simulate the instruments without relying on samples we can do away with all that and create some truly amazing things for musicians. I suspect that in the future we will have synthesized orchestration packages so that we can do away with samples entirely.