Precomputed Wave Simulation for Real-time Sound Propagation of Dynamic Sources in Complex Scenes

[@ronen_khazin]

My ears are in love. Seriously, this would seriously help with creating a realistic environment.

[@hughJ]

Gunshots and other such noises are a really bad test case for trying to produce a realistic simulation of sound. It's sort of like trying to apply a newtonian physics model to an FPS that uses hitscan weapons. These sorts of games have a very compressed volume level for the sake of user experience (quiet things are made louder so you can recognize them, very loud things are made quiet so you don't damage your hearing.) This creates a problem when you apply them to a physically correct acoustic environment as the stylistically loud reloading and shell ejection sounds end up possessing almost the same energy as a gunshot, which results in a very 'wrong' sounding reverb. The voices, bottle breaks and other such audio sources sound pretty good in this video though. I think if we were to want to include very loud noises like gunshots, car engines, explosions, etc, that there would need to be something akin to HDR tone mapping where the volume curve is dynamically adjusted based on relative levels, and perhaps clipped/distorted to give a sense of being 'too loud'.

[@guyboy625]

That's a great point. Explains why the half-life 2 example wasn't entirely satisfactory, even though this is a highly impressive simulation. What worries me is that it's 6 years later, just now are people starting to add this to games, and it's still not as good yet.

[@MarterElectronics]

i used some plugin called SteamAudio for Unity and other engines. Applying real physics would make the sound low volume as fuck and didnt reach anywhere before the atmosphere absorbs the sound. So, i was thinking it could improve it when i increase the dB of the sound to a clipping level. It was not clipping from a far distance and sound acted like how a gun would fire. Even with the dropping casings. Then i added some limitter with a slight distortion to make it less loud but still noticable when it is like really really loud with out damaging your ears in real life. Its all worth it. Even the echo and reverb seem more accurate.

[@kysz1]

I've been thinking of similar sound engine 7-8 years ago. The philosophy was - if we can use shader to simuate light propagation (global illumintion, reflections, scattering) which is pretty much waves bouncing around and interacting with surfaces, then we can use similar technology for sound waves. And just like physically based rendering, each surface could change the sound. My idea was extended with 2 additional features. First was speed of sound - processed samples would be added to the buffer at further point to be played later, depending on total distance travelled. This would get complex if observer started moving and in the end you would have to resort to raytracing from observer into all directions, gathering sound samples from all around, rather than processing them starting from sound source. While raytracing, you would move sound sources and their samples back in time to the point where the collision with ray occurred. This would pretty much require 4-dimensional raytracing - with time as 4th dimension ;) The second additional feature borrowed from graphics would be the concept of HDR rendering. Sound would be rendered in real world values but the final output buffer would be scaled by algoritm similar to tone mapping in graphics. This way you would hear whispers in a quiet place, but a nearby explosion would change the final sound scale. One difference between graphics HDR and audio HDR would be that in graphics it's ok to allow sudden overbright until tone mapper adapts. With sound you would need to immediately decrease "master volume" in case of sudden loud sound. For example - an explosion that is 20x louder than your current range would immediately increase your range 10 times and therefore it would be only 2x louder than your current speaker range. From this point it would continue to adapt normally, just like HDR in graphics. So now imagine you see a truck standing idle right in front of you with engine running. You can hear the engine sound and there's a big explosion in the distance. Nothing happens with the sound - you can still hear the truck. After a while you hear a lound bang and you no longer hear the truck (or barely hear it), but gradually the sound of truck engine returns. This is what it would feel like. And it could get even better with one more idea - the voices could be recorded in different variants or realtime processed if possible, so that people would start to talk louder or even shout if there are in a loud spot. This means you would still hear them even if the sound "HDR" would rescale the volme - at least up to a certain point :)

[@youwhatmadeidk]

Yeah I noticed the shotgun shells being too loud. I can’t wait until games start using technology like this and start to use less audio compression - my high-end audio system will really show them off.

[@JohnnyNatrium]

What you're thinking of is multiple Kernels where the response is measured and reproduced for a number of separate layers of source volume. They've been doing this with Impulse-Response "convolution" effects, but by far most convolution reverbs still use just a single kernel, which means the loudness of the sound source doesn't come into play as far as the added harmonics per sonic partial are concerned. And these people from, for instance, Altiverb, act like it's the most amazing true to life thing ever; like you're really in this room. I was actually laughing out loud when a promo vid for the latest Altiverb showed them demonstrating the similarity to a recording made in the actual auditorium where you could hear the sound source, vs using the Plugin software on that same dry sound source (of a cello performance); the thing is, when you saw the recording in the actual auditorium, they didn't have a cellist playing, they had a huge speaker playing back the same recording that they used the plugin on, and it had a reduced dynamic range. It sounds the same as the software impulse response alright, but it certainly wouldn't sound the same as an actual cellist playing dynamically in the room. And yet the tech is there. Nebula have been using it for 10 years. Up to about 10 kernels per 'effect' or convolution I believe, each based on increasing levels of loudness of the sound source. It just costs a lot of CPU power to do live convolution that flows around between 10 kernels, per soundsource.

[@Swaggless]

Any way to get this impressive mod and implement it into HL2?

[@bigcoffinhunter5500]

Song is Mass Attack - Angel

[@Vermilicious]

I was looking around at binaural recording and sound, and bumped into the paper "Efficient and Accurate Numerical Simulation of Sound Propagation in 3D using Adaptive Rectangular Decomposition". I'm guessing this one's a continuation of that work. Now, I got the impression this kind of thing, emulating real sound, haven't really been used in any games, but here you are showing it! The effect is just amazing! Oh, I'd love to see game developers adapt this. Great stuff!

[@nikibastian]

Just insane! Congrats for your work! Best of luck!

[@Vermilicious]

It's been a while since I watched this video. It still amazes me, but what amazes me more is that no one is doing this. Is the Half-Life 2 mod that was made, available somewhere?

[@kalewintermute28]

God knows why this isnt used within games by now, because it improves sound immensely.

[@NikoxD93]

this still required some fine tuning, but the proof is there. this has so much potential, if you combine this technology with HRTF sound, this need to happen and people need to know

[@jamesfreakincurtis]

Just imagine how using that engine to fire a magnum in an air duct would sound.

[@svyatovrat4071]

*trigger click* [ tinnitus ]

[@jobiewan8764]

I feel like I can echo locate now with the parameters you have made me take notice of. thank you!!!!

[@TrinityAlex]

Really awesome... but I am pretty much certain that it uses a lot of CPU. I am curious if it can be executed only by the soundcard (Creative X-Fi Titanium).

[@moonasha]

would have been nice if the guy didn't use a potato mic

[@mrburns366]

Instead of a potato mic, he should've used a banana phone

[@eyescreamcake]

Nice realistic bookshelf

[@decreate952]

imagine if in the future games would be able to create object friction breaking and etc sounds real time so they match up perfectly

[@miguelpereira9859]

That's probably going to happen some time in the future

[@FranKirara]

Valve really needs to see this.

[@phuturemusic]

Should set dry/mix value at 25~50%

[@SkeletonGuts]

I love the Half-Life 2 props.

[@ElliFong]

Wow :o. Can I download and use this sound system for my half-life 2s?

[@sethbettwieser]

How can I learn to make a sound simulation like this?

[@sullivandeffinger4970]

Anyone know if theres a sort of sandbox where you could mess w this type of stuff?

[@WDShorty]

I suggest someone making this for Unity but making it more like Valhallaroom or Vahalla shimmer haha and give it some properties so people can adjust it to sound more like from a human perspective

[@ItsNotAProblem]

Awesome! I would love to be able to make that too!

[@mrkenan1]

The sound feels exaggerated in the half life video, there's too much eco

[@SunShinesBlack]

+TB well you probably stood in an empty living room before, think of what itd be like in an almost empty hall of this size, i dont think its too far off its just not what you are used to in your listening experience

[@phxcppdvlazi]

I thought they were doing that to highlight the difference their sound engine makes.

[@miguelpereira9859]

This is old as shit but. In real life there really would be that amount of echo

[@JoseSerro]

I'm wondering if the public can access the necesesary libraries for this? I'm looking forward to compile a module to integrate this system and interact with it in Garry's Mod.

[@MagnusTV1990]

That's very good and sounds natural, but... How much CPU does this thing use for computing sound physics?

[@williamnd]

Thats nice but can become a problem... i you are in a Fight situation, the sound can become very loud. Creating a "massive confused sound"

[@totty2524]

Wow, this is fantastic!

[@KorteZZ88]

Awesome! Really interesting!

[@dentistguba]

Should be possible as it's mostly precomputed, Maybe just a bit more than ordinary convolution which is a breeze for any soundcard (i use it a lot for music with just an ordinary realtek multimedia sound card).

[@francisconascimento7447]

Is this used in videogames?

[@JoexLegacy]

How is this 14 years old? Any updates?

[@jimmyrotella]

HELLO! sorry to bother you but i need your precious advice: im architect, and what i found in your channel is just amazing for me. i'd like to know some details of what you do. im very interested in crowd simulators, and in sound wave simulators. i'd ask you to tell me: 1) WHICH software did you use for crowd simulation? and which for sound wave simulation? 2) HOW are the 3d scenes generated? if i use the softwares you use, can i import a 3d scene from an external 3d software? thanx a lot!

[@racerdicethecreator4373]

I know that this is late but the scenes they demonstrated their sound engine in were created using source engine. A realistic game engine developed by Valve software to create famous games such as Half Life 2 which has paved the way for the modern games of today. The source engine was chosen for the demonstration because of its realism. the game engine included the most realistic physics and photo realistic graphics that used pre-computed lighting and shaders. The 3d scenes were created in the released Hammer World Editor used by people that own source engine games, the scenes are called "maps" in source terminology and are created with a measuring scale of each "hammer unit" being a 16th of an inch. If by crowd simulation you meant the human characters and AI Half Life 2 was made with a remarkable animation system with at least 100 bones on characters faces, Maps have preset volumes that trigger an event when a player enters one, such as a civilian speaking to you when you get near him/her. Information on the sound engine has been displayed in the video and in the paper linked in the description below. You can import scenes to the game engine if you happen to own one of the source games (A free one is Team Fortress 2, believe it or not the game has been very popular since 2007 and is growing in players 50 million have played it last time I checked). That is all if you still need this info here you go!

[@racerdicethecreator4373]

I do not know much about the sound engine and I have just discovered this 10 minutes ago. I hope my info was some help though.

[@Dinho675]

what are the songs that were used?

[@tomclavmaster]

This is great, has anyone tried to contact you for a game?

[@menteencoma]

this is pretty awesome

[@DessertMonkey]

So... you going to release that HL2 mod or not? Pretty please? O.O Okay okay I understand that you fear someone will plagiarizer your work but seriously... people are missing out on this.

[@alecb441]

at what stage is this technology right now?

[@IDrinkLava]

Look up Steam Audio news updates. The technology has come really far since 2010, and will probably go even further when the PS5 comes out with its dedicated 3D audio chip. With GPU acceleration, it's possible to increase raytraced sound propagation performance by 150 times. The real problem now is finding a universal raytracing standard that is hardware agnostic. There's no guarantee GPUs 5 years from now will support the same standards as in 2019, so performance will likely stay CPU-based.

[@alecb441]

@@IDrinkLava Yeah I've been looking in to this sort of thing. I wonder why they used raytracing technology for light only when it could be used for sound aswell. Like gloss maps that reflect light but another map that would simulate sound reflection/absorption. This would take a huuge workload of the shoulders of sound designers.

[@josephzxu]

This is really cool, but the sounds in HL2 are not pristine natural recordings, they were designed to sound good in their engine. So the optimal effect was probably not achieved using these samples. For the famous shootout scene in the movie Heat, Michael Mann insisted on using the sound recorded during filming because he felt it was impossible to recreate the sound of gunfire bouncing around LA skyscrapers in a studio.

[@xalener]

has this been picked up yet?

[@123reivaj]

Hey, Very Cool! :D

[@xalener]

I wish more games had this.

[@Pr3DaToR15]

Impressive!

[@maximilianmayer7561]

amazing!!!!

[@Vermilicious]

@dessertmonkeyjk Indeed, it would be awesome if we got to play around with it!

[@lolsato]

aaaahhhhh i want this mod ! How long must i wait for proper 3d audio in my games? Industry only cares about graphics.

[@xalener]

Shows incredible sound technology never uses.

[@MAGNUSWAKERS]

Greg Wasdyke Blame Creative for buying the patent and never utilizing it fully. OpenAL Soft may show promise.

[@NahuelNada]

Im gonna code something like that on lua for Gmod

[@zoltanzoloziak7631]

+Nahuel Nada 9 months passed, you hae something done?

[@NahuelNada]

Nope, i tried it all, but no, lua is way too limited when it comes sound wise. I got some results but it is not as good as this.

[@gaelcoral]

Wow hl2 sounds

[@theobserver4214]

This is some EAX4-5 stuff here.

[@Fwuzeem]

It's good, but it's ages ago

[@johnsherfey3675]

now it's nearly 10 years ago.

[@Henry_Walker]

Wery well , but it mast be not in old games, because it simply empty for furniture, and sounds ridiculous.

[@ImUpsetThatYouStoleMyUsername]

PLEASE FIND A WAY TO PUT THIS IN MY GAMES AND I WILL THROW MONEY AT YOU

[@deuone]

please tell me you've been hired by valve ;_;

[@DessertMonkey]

[@OktoberStorm]

This is sheer porn!

[@thelasthallow]

2010 glad this shit hasnt been used to my knowledge because its horrible.

[@udalix]

It seems like a good idea, but these examples are a tad over the top, heavy on the reverb and delay, not very realistic.

[@thelasthallow]

yeah i mean. people could just save up $100 or around that and actually buy a set of real 5.1 headphones.

[@dunedinsound]

thelasthallow You've clearly completely misunderstood what the technology demonstrated in the video actually is. It's not trying to simulate surround sound.

[@thelasthallow]

actually it is. derr. they are trying to create a fake surround sound using stereo headphones/speakers.

[@dunedinsound]

thelasthallow No they're not, they're trying to simulate environments. This has nothing to do with fake surround sound.

[@slushdrinker]

this is way too cool