Researchers Synthesize Real-Time Fracture Sounds

ChippedTeapot writes "Researchers at Cornell University have devised an algorithm for synthesizing sounds associated with brittle fracture simulations. Computers can now automatically generate synchronized sound, motion, and graphics for physically based fracture events, such as in future interactive virtual environments. The results will be presented at ACM SIGGRAPH 2010 in Los Angeles July 25-29. Check out the smashing results on YouTube."

Comments

Uses?

[Reilaos]

From first thoughts and glancing at the article, it seems the first use that comes to mind is for sound effects in movies and the like. While it's great that you can not waste a perfectly good pot on a second-long sound effect, I'm wondering if there are any other implications.

Perhaps we can use this sort of things in reverse? Like "the audio from this emergency call indicates a lot of expensive china breaking! The china sounds like it is from midway through the Ming Dynasty! There's only one place in town that sell that! BS CSI TO THE RESCUE!"

Re:Uses?

[TamCaP]

Depending on computational complexity, I guess glass breaking in computer games that is (almost?) indistinguishable from the real thing.Maybe also large-scale special effects, when the physical objects are too large to be used as props. But I didn't know it was such a big issue in the first place...

Re:Uses?

[Kepesk]

Car wrecks in video games: now with hyper-realistic glass shattering sounds! Coming to a store near you.

Re:Uses?

[pinkj]

As a post-audio SFX editor/designer, I would LOVE to have a virtual room full of junk to throw around, bang, break, bend, etc. Having convincing sounds is very nice, but even sounds that are not 100% accurate can still be used for very creative purposes. I love the work these guys are doing. This is another video of theirs I saved last year from an article submitted here but didn't make it to the main page:

Harmonic Shells

Re:Uses?

[wagnerrp]

From first thoughts and glancing at the article, it seems the first use that comes to mind is for sound effects in movies and the like.

Generating realistic sounds of rigid objects is fairly straight forward. You know the material properties. You know the size and shape from your physics engine. You run it through some relatively simple set of equations to get the resonance frequency of the shard. Add in amplitude from the impact, natural damping over time, and you're done. Oh, but you have to do it for hundreds or thousands of objects, and each of those is impacting or fragmenting several times per second.

Producing realistic sound like that isn't very impressive. What is noteworthy is the optimization, generalization, and statistical modeling that they used to simplify that massive amount of computation into something that could be run in real-time on commodity hardware, rather than having to be backed by a supercomputer. Movie and special effects studios have those supercomputers and render farms at their disposal already. This work is intended to go hand and hand with the realistic and complex physics engines that games have started getting in the past few years.

Re:Uses?

[chichilalescu]

for me, as a physicist, it's fascinating that they were able to do it. the human brain is a great tool to test how "natural" various simulations are, and when you succeed in fooling it, it means your model is very close to reality in a certain way.
in the study of complex phenomena, any ability to model is an important insight, because we don't have the math to understand what's happening just by using first principles.

as a sidenote, the problem of turbulence was well-known before the revolution of quantum and relativistic physics; still, Kolmogorov's scaling law (one of the most important results in turbulence) came in the 1940s. these problems are hard, and I will take my hat off to anyone who is able to cheaply reproduce aspects of nonlinear phenomena.

Re:Uses?

[nebaz]

I think it depends on what people are accustomed to. Do silencers really work on guns as silently as shown in the movies? Or do punches really have the big "phwak" sound that you hear there? People are only good at determining "natural" sounds compared to what they are used to. They may not be realistic though.

Re:Uses?

[davester666]

Next up...fingernails on chalkboards!

Because we care.

Re:Uses?

[Monkeedude1212]

If there is any one lesson I've learned in game development, it's that you never have to actually produce what you want the scenario to be. Much like a movie or TV set, the only things that have to be right are what the user sees.

As computers have gotten more and more complex, its required more and more complex thinking to reproduce even the things that we would assume to be so simple. I remember listening to a talk on Epic Games and how much effort they put into Particle Effects in their engine. The reason being that while particles themselves are usually not much more than a sprite or a model, that look unrealistic, its how you use them to produce an effect. With the same entity - you can create a convincing firefly, or a spark from ricochet, depending on how you move it and what lighting you use.

It's all quite impressive.

Re:Uses?

[Vintermann]

Thanks for that link!

I love this stuff too. I've wanted rendered sound for years. It always irked me that programmers would use so much effort to render graphics, and so little to render realistic sound. People will gladly run a raytracer for 8 hours to get a nice picture of a car, but I've never found anyone who run a physics simulation for 8 hours to get the sound of a piano right - even though I've never heard a decent synthesized piano sound in my life. What happened, why did sound get so little attention compared to graphics?

Anyway, these videos are evidence that maybe things are changing.

Re:Uses?

[Vintermann]

Generating realistic sounds of rigid objects is fairly straight forward. You know the material properties. You know the size and shape from your physics engine. You run it through some relatively simple set of equations to get the resonance frequency of the shard. Add in amplitude from the impact, natural damping over time, and you're done. [...] Producing realistic sound like that isn't very impressive.

I beg to differ. Where is the software that can do this accurately, instead of trying to do it realtime? (realtime any half-decent synth tries to, and fails...)

I've searched many times for sound rendering software. Something that could take, say, a midi file, work on it for 8 hours, and give me a piano sound that doesn't sound like crap. I haven't found anything yet. If you could show me some, I'd be genuinely grateful (it's even something I would pay money for. The software I mean, not you telling me :p )

Re:Uses?

[Vintermann]

Or do punches really have the big "phwak" sound that you hear there?

I've heard rumors that the common punching sound is actually a wrench smashing a cabbage.

Re:Uses?

[Eil]

Probably because sound is already a lot easier generate, sample, cut, filter, mangle, and in general work with. Anyone who wants an electronic version of a high-end piano just needs to buy a CD with a sample (or few) of each key being struck and bingo. Sure, you'll always miss out on some flexibility and fidelity compared to the real thing but with today's technology, not too much.

Re:Uses?

[Waccoon]

Most people these days are perfectly aware that there's no air in space, but watching a space war in silence is pretty boring. So what if a punch from Indiana Jones sounds like a splintering telephone pole? It's obviously not real, but it sure is a lot more satisfying.

Even if people act stupid, they generally aren't. They know that a car can't bust through a brick wall and land on the roof of another building in one piece. Art isn't so much fooling the audience into thinking it's real, as it is making it bigger than life while avoiding the uncanny valley. Some people, especially geeks, are just not as forgiving as others.

Re:Uses?

[Anonymous Coward]

http://www.pianoteq.com/ Try this.

A demo http://www.youtube.com/watch?v=jTbXnbfymdc

And it works in real time.

Re:Uses?

[jack2000]

Sound in space is so jarring. I'd much rather have music ( opera choirs, BSG style battle drums etc ) then cheap sounding zooms and booms.

Re:Uses?

[Anonymous Coward]

I'm accustomed to humans making the Wilhelm scream any time they are shot and fall or tumble or fall off a ledge myself.

Re:Uses?

[Anonymous Coward]

Unless your MIDI file was performed by a good pianist, no synth will give you a good sound. Computers can't interpret music.

Re:Uses?

[Vintermann]

The midi files in question are performed by good pianists, they are midi conversions of recorded piano rolls. I'm concerned with good instrument sound.

Re:Uses?

[Vintermann]

Sure, you'll always miss out on some flexibility and fidelity compared to the real thing but with today's technology, not too much.

This appears to be the attitude of most synth designers, unfortunately. The best digital pianos I've heard emulated sympathetic resonance for a couple of strings, but not nearly enough to give a credible sostenuto effect.

Re:Uses?

[Vintermann]

I did another search after writing this, and came over Pianoteq3.

!!!

OK, now I've heard a better digital piano! And I think I may actually be tempted to buy a trialware app for Linux for the first time in my life :-)

Re:Uses?

[CrazeeCracker]

Having convincing sounds is very nice, but even sounds that are not 100% accurate can still be used for very creative purposes.

You're obviously the expert here, but the way I understood it I thought physical accuracy was not really a huge priority in the business. I read an interesting article once (at work, so I'm not gonna searchf or it now), were it was explained that the sound that subjectively fits best to the action is often made by something wildly different than what appears to make it on screen. E.g., the whiplash foley from Ben Hur was apparently a raw steak slapping against an exposed thigh.

I'm not disputing anything you're saying, it sounds like an awesome application for the idea, just adding my .02€.

Just Use Samples

[Beardydog]

If I can be endlessly entertained by only to "guy jumping" sounds, surely my cracking concrete doesn't need to be synthesized in real-time.

Hooh! HYUHHH! Hoo! Hoo! Hoo! ... HYUHHH! HYUHHH!

Re:Just Use Samples

[Tejin]

Samples are terrible. It's like saying "Just use rectangles" to display objects in the game.

In old video games where the graphics and physics were also the equivalent of samples, it was okay to have sampled sound, but so much work has been poured into graphics and physics these days and everyone has ignored sound. I'm sure a lot of the work that went into making more realistic physics can be used in making better sounds, since material properties, velocities and angle of impact are important to both sound and physics.

It's a terrible immersion-breaker when you hear the same sample every time you hit something in game, regardless of whether it should have sounded like that.

Re:Just Use Samples

[Kenoli]

It's like saying "Just use rectangles" to display objects in the game.

Pixels are rectangles amirite

Re:Just Use Samples

[pushing-robot]

No, they fixed that.

I couldn't hear any difference

[Iamthecheese]

Some sounds were more flat, as if generated by heavier and less hard objects but the fracturing sounds all seemed equally realistic. What were they comparing?

But do they get you high?

[PolygamousRanchKid]

The Oklahoma Bureau of Narcotics will claim that they are a bigger threat than binaural beats: http://idle.slashdot.org/story/10/07/15/166220/Sound-As-the-New-Illegal-Narcotic

Movies?

[garyisabusyguy]

Not so much, there are entire ranks of Foleys who work under union rules to deliver most movie sound effects (although they probably have impressive Sound Engineer titles now).

This is much more likely aimed at environments like video games and battle-ground simulations, where any number of events can occur and the presence of a tightly aligned sound track is needed to produce a more realistic effect.

The sequences of events in movies are pretty tightly controlled, so I see this as having less use there

Re:Movies?

[Monkeedude1212]

Not to mention that games are designed to be flexible in what can happen with them now. Force Unleashed would be an example where glass or wood can shatter differently almost everytime, depending on even the slightest nuances. For something like this - it's difficult to asssing premade sound effects - you can record various ones for big, medum, and small fracturing, but you run the risk of exploding your game filesize by adding more sound effects, or you are leaving the sounds a little bland as they will get repeated many times.

So for games, and other simulations, this kind of thing works well.

Re:Movies?

[Nadaka]

For games you can also use recorded sounds run through a random or fractal filter that changes them somewhat for different events.

Re:Movies?

[pinkj]

I've never heard of a fractal filter and googling it didn't bring up any results. And randomizing a single sound to make it sound different doesn't make much sense either as impact sounds are linear. Maybe you mean granular synthesis, but even then, it will completely change the sound into something very different and probably unrecognizable from the source.

Re:Movies?

[Anonymous Coward]

It's the same fractal filter they use for "zoom and enhance" algorithms, didn't you know?

Re:Movies?

[FiloEleven]

Pure granular synthesis would destroy the sound, yes. I could see some aspects of granular synthesis used to vary sounds instead of going full-on. Have a look at the "graintable synthesis" used in the Malstrom synth. It involves a lot of preprocessing of the sounds used, but an intelligent algorithm could do the trick.

Be that as it may, I think the gains of either of these synthesis methods over simply having 4 or more alternate sounds which play at random are minimal. Space is dirt cheap nowadays while CPU cycles aren't, and a large filesize is much more acceptable than a sluggish game. Something like graintables will get you more variation yet the production work is intensive. It's very easy to break some junk and record the sound. This new synthesis has its place but it won't be in a video game anytime soon.

Cool demo...

[RevWaldo]

but they left out the cricket-bat-vs-femur sound comparison. This will be used in video games after all.

.

Re:Cool demo...

[Tisha_AH]

Or the cricket bat vs lemur.

Re:Cool demo...

[mattack2]

Or lemur vs. cricket.

Though I can predict that one: CRRRUNCH.

Re:Cool demo...

[Vintermann]

Squeak!!

Re:Troll

[Anonymous Coward]

First post!

sadly, not the last douche

this can be one of two things

[nimbius]

researchers at cornell, or "researchers at cornell" meaning burned out grad students under the cracking whip. if the sounds vaguely resemble someone squishing a bag of taytos....

Wait for it....

[Itninja]

Are these scientists on crack?

...I'll show myself out

Missed opportunity

[Locke2005]

Shouldn't this have been titled Breaking News?

Re:Missed opportunity

[Mastadex]

Regardless, it will be a smashing success.

Pr0n sounds?

[Anonymous Coward]

If they can do crashing, I'm sure they can do slapping.

SIGGRAPH?

[camperdave]

SIGGRAPH? Shouldn't they have shown this at SIGAUD?

Ob Nick Lowe

[kindbud]

I Love the Sound of Breaking Glass" from the album Jesus of Cool (1978).

Re:Ob Nick Lowe

[ErkDemon]

Damn, you beat me to it.

Mmmmm

[Anonymous Coward]

I'd love to see this tried with human gibs.

for the best sound

[e**(i pi)-1]

for the best sound, we do everything!

Uh-oh. Patent infringement.

[OnB]

Ke$ha already did it.

Missing something.

[MaWeiTao]

Very neat, but the generated sounds seemed a bit flat to me compared to the real thing. It's like they were missing some amount of reverberation at the end, like the sound was cut short. It is, nevertheless, impressive especially coupled with visuals that look very convincing. Modeling this stuff might be nothing new, but I'm always a sucker for this sort of thing.

Re:Missing something.

[Vintermann]

You do realize roughly half the clips in that video were of actual things breaking?

Condition monitoring and nondestructive testing

[dbIII]

One method of nondestructive testing is to listen to things as they crack. While that initially sounds silly remember that there are modes of failure such as fatigue where there will be thousands of cracking events before the crack breaks all of the way through, so there can be time to do something about the cracking. Sound propagates very well through metal so it wouldn't take very many sensors to cover an entire Boeing 747.