Inverse-Foley Animation: Synchronizing rigid-body motions to sound

[@MikeGipsonUSAF]

This is absolutely brilliant. 3D animations are going to become way more accurate. I feel as though in the future, we will be able to turn nothing but audio recordings into visualizations that are accurate of what was seen during the recording. Maybe a blind person's hearing can then translate to simulated visual input.... hmmm

[@hjalmarnyman1313]

If a tree sounds in the forest, does it make a motion if no one saw it fall?

[@Fbalino]

Incredible. Great research. This will be very practical for many developers and animators.

[@AirDyran]

If they made a grass growth generator or a paint drying simulator, I'd watch it.

[@CognitiveOverdose]

This is superbly amazing, another step towards animations future.

[@craigmiller8315]

Awesome - love the funky dice!

[@SuperAngryPacman]

This is absolutely amazing. The detail this technique can give to animations is inspiring. Splitting hairs though, the notation shown as the music played was inaccurate. The notation was in C major whereas the audio was F minor. Nice try, but I'm calling you out. Although it does raise an idea for something to be used in conjunction with this: a program that reads notation and provides a MIDI sound (or even a real-sound format) for the animation to be based on. Then you could straight-up synchronize your objects from just sheet music. Crazy.

[@PanoPapadatos]

That's really awesome.

[@TarmacStuff]

This is brilliant. Very impressed.

[@Cloud_Stratus]

Magic, got it.

[@TheAGCteam]

Impressive work!

[@cd-zw2tt]

Is this possible to do with any sort of liquid?

[@TimothyLanglois]

We've only explored rigid-bodies so far. Fluids would be an interesting future project!

[@Eugensson]

Charlemagne Adelaar might require a lot of convolution operations

[@dgking80]

I like this guys voice I wish he would read me a bedtime story

[@Redbikemaster]

That would be neat to one day see this integrated into a video game engine. Especially one like the one that runs BeamNG.drive. That, combined with realistic sound would be very impressive, but would require a very high-powered gaming rig to run.

[@1ManArmy4Lyfe]

JUST ONE MORE GOOOOO!!!!!!!!!! 0_0

[@cedrigo]

I'm a 3D animator. I find this great and... scary XD

[@Sam-mo2mp]

Can you post a link to the two videos at the end in the description? Annotations don't seem to be working.

[@TimothyLanglois]

Sorry about that, I've added annotations and links in the description.

[@Chumtoad_Deluxe]

very very cool

[@TheMacey117]

Nerd3!!!!

[@Isoma19]

Dropping the bass will never sound so good

[@eunhyoukshin7777]

THIS IS CRAZY!!!

[@TheDimz14]

bollocks to the universe!

[@gajbooks]

Now if only the physics looked real in this video.

[@Mrbananadude123]

Could you try this with a spinning coin that slows down to when it falls over and stops?

[@TimothyLanglois]

Yep, that would be possible. It's similar to the beer can in the video, although that tends to spoll instead of rolling then falling over. The difficult part is matching the continuous sounds. Our blending formulation works between discrete events (i.e., bouncing). With continuous contact events we need to be lucky enough to sample that type of motion in the database. We also do a crude job of comparing continuous contact with the input sound. That could be improved in the future.

[@partymayker]

Genius.

[@anonharingenamn]

So wait, these animations are GENERATED from the audio?

[@Slothery]

I threw it on the GROOOUUUNNNNDDD

[@LeadingNowhere]

Amazing :o

[@Ciaran.OFlynn]

for some reason the sounds of the objects colliding with the surface is really satisfying

[@ArduinoTronic]

Great! - I loved seeing your video. But that on my phone the sound seemed a bit muffled - could be me though! I am sure we have the same sort of idea and I like your approach u have. When you get some time please examine the similar videos on my channel (see playlists) and tell me what you see. All comments gratefully recieved! Live Long and Prosper!

[@supamaorifella195]

2:24. . . around this time I adjusted myself to feel more Comfortable in the seating position. What i didn't know was that in doing so, My phone Fell Out my Pocket and hit the Ground Hard. . . I thought it was the Pink Thingy, bouncing, to its inevitable Halt. But no, Nope. . . Nopedy Nopey Nope. it was My Phone and Now I'm looking at the Fucking Cracked Screen, thinking to myself. Was it my Fault? Was it my Pants Fault? Was it Gravity? Or was it the Fucking Phones FAULT!! Either way this Conversation is Irrelevant to the Subject of the Video. . . Sounds Awesome, Thanks For Furthering Our Cinematic Experience. Cheers. P.S I have Headset So People won't Hear the Shit I watch. Peace

[@NewToneProducer]

The future.

[@ericliveson]

i'm about to go to school for this, god this looks complicated

[@aydoooo]

whenever i put something on my subwoofer this happens

[@anthonyparks505]

3:59 is spicy af

[@Jack-vw6bc]

I buy the same milk!

[@enchibla]

now do it with a fighting scene from transformers ;)

[@RockyLewisFilms]

ELI5

[@STANNco]

so let me get this? all the animation was generated on the computer based on the sound only? SHUT UP AND TAKE MY MONEY!

[@black_candles1212]

the credit card that they dropped was fake ;( 4321-0000-0000