3D Gaussian Splatting from Hollywood Films!

Turning movie shots into 3D scenes using 3D Gaussian Splatting. We found movie clips online, converted scenes to image sequences, trained their GSPLATS, and brought them to UNREAL ENGINE 5.
3D Gaussian Splatting for Real-Time Radiance Field Rendering Paper:
repo-sam.inria...
Join our discord server:
/ discord
If you wanna see us to do cool things follow us here too:
Instagram: / badxstudio
Twitter: / badxstudio
TikTok: / badxstudio
LinkedIn: / badxstudio
Bad Decisions Podcast 🎙️:
podcasters.spo...
Our personal handles: (if you wanna stalk us)
/ farhad_sh
/ farazshababs
/ farhads__
/ farazshababi
/ farhadshababi
/ farazshababi
#unrealengine5 #3dgaussiansplatting #3dscan #gaussiansplatting #3d #3drender #nerf #unrealengine #blender3d #blender #drone #ai #photogrammetry #vfx #cgi #film

[@shelfrelated]

THIS IS WAYYY TOO INTERESTING TO MISS OUT. And Faraz casually being confidently wrong in 1980 being 33 years ago (when it's actually 43) is another highlight.

[@tribaltheadventurer]

1980 is 43 years ago not 33, good video as usual guys

[@badxstudio]

Hahahah yeah we realised we fked up 😂😂

[@tribaltheadventurer]

@@badxstudio 😂😂😂

[@deniss.6205]

nah, those guys are editing some years themselves😉

[@der_alte_z0cker339]

2:18 it is even 43 years old. 😉 Wow! Great video as always guys!

[@koberko]

I wish It was "just" 33 years ago :-D I born in 1981, so I wouldn't mind 10y shaved off :-D

[@willbrand77]

Not the best math 😅

[@badxstudio]

hahahha

[@badxstudio]

Oooppsss ..Quick Math

[@foxy2348]

haha The math was amazing!

[@pavelf2002]

Imagine how cool it could be with 4DGaussians? It is a recent paper about gaussian splats, but with motion. Yes, you can recreate 3d dynamic scenes from videos with this technique!

[@pva5142]

yea code is already out!

[@irfanadamm5819]

so CGI will look more realistic now?

[@pva5142]

@@irfanadamm5819 its basically better for filmmakers in post. once the fidelity gets really good, you’re eliminating a lot of the DP’s job outside of lighting. If you’re able to change the focal length, camera movement and composition in post

[@badxstudio]

We saw the paper... and we are going to try it for sure, we just have 2 more experiments with 3DGS, VR and Video Games

[@MrGhostfaceLives]

@@pva5142 Why would they even want to do that? It's completely inorganic to do that, and the "fix it in post" mentality is gradually being forced out, thankfully. Also "outside of lighting"... so only the most important element of the job.

[@Sleezy.Design]

You mentioned that the video was super sharp despite being so old. That's because those movies were shot on film! If you can get your hands on the actual film of a movie you can basically remaster it in 4k or larger with perfect quality. That's because film is an analog medium, you don't have pixels in film. Those movies were played on a huge movie screen back then, the only reason why we connect those old movies with low resolution is that there was no need to render those movies for old TVs that were not capable of displaying HD. That's why you can now find old movies, music videos etc. with super crispy 4K quality. In my opinion they look even better than modern digital movies, there's just something about film that modern cameras can't replicate. And by the way, amazing use of this technology! I knew about the existence of this but I haven't though about using it to digitize movie scenes. That's genius!

[@badxstudio]

Thanks so much for the wonderful explanation! Loved it ❤️ And yeah turning films to 3D was a truly magical moment for us to experience!

[@eddiej.l.christian6754]

Digital films are caught at a higher resolution and color range then old film. None of what you said matches the actual facts.

[@JaapvanderVelde]

You can tell Gaussian Splatting is simple to do because even obnoxious loud people who can't spell 'Shining' can do it.

[@Ariesblack90]

I just cannot process what i have just seen .. this is insane especially that you got it from movies like this will change everything

[@badxstudio]

This is just the beginning bro ...Imagine what is going to happen next year

[@ORTyOW]

2:33 guys are living in the 2013😅go grab that bitcoin asap

[@badxstudio]

hahahha I wish bro

[@Krappenschittz]

There are no limits for G.Splatting, I guess.

[@badxstudio]

Dude its just the beginning ... why do you think we've made 6 videos about it already? This is just going to get better from here

[@OlliHuttunen78]

Great idea to test gaussian on movies. Funny and educational! Well done!

[@badxstudio]

Yeah dude it had to be done we were so curious!!! Also remember when u said try it in VR? We jus did yesterday … it was soo good! Still needs work but definitely exciting!

[@zsigmondforianszabo4698]

You guys are soooooo underrated.. the videos are entertaining and educational. Good job!

[@RicardoGonzalez-gu5vv]

They are the next corridor crew for the era of AI 3D vfx

[@wakopaco]

content is good, but their style is very annoying. cant stand all the forced wooooooows

[@badxstudio]

Thanks so much buddy! Glad you enjoy them

[@duytdl]

This is awfully like lucid dreams if you've had them. Very similar restrictions, like you can't go beyond a certain point where it becomes like an invisible border of less details or something obstructing your view

[@badxstudio]

Wait really?? We’ve never had a lucid dream unfortunately. Is that truly how it feels?

[@luiginotcool]

I’ve never had a Lucid dream that had a barrier. Usually I can go wherever I like

[@kevinsimpore2044]

​@@luiginotcool if you used to fly in lucid dream you would have eventually noticed that you can't go beyond a certain altitude.

[@Appleloucious]

One Love! Always forward, never ever backward!! ☀☀☀ 💚💛❤ 🙏🏿🙏🙏🏼

[@badxstudio]

Lets gooooo!!!

[@chrisgutierrez8599]

Great stuff guys always entertaining and learning while watching 🙌

[@badxstudio]

Chris that's the goal ma man

[@chrisgutierrez8599]

Always blessed especially when these videos drop 💯

[@badxstudio]

Good to hear that G

[@Desopolis]

What tools are you using to generate the splats?

[@ORTyOW]

Best 3D/UE edutaiment content on KZbin🔥

[@badxstudio]

Big love to you

[@badxstudio]

Edutaiment love that btw!

[@Sharivari]

That's so cool!

[@badxstudio]

Couldn't agree more!

[@selftransforming5768]

YALL ARLREADY ON A WHOLE NEW ERA!

[@badxstudio]

Glad you think so mate

[@Sharivari]

Just finished the full video. Your video production, knowledge, camera presence and humour will make your channel blow up.

[@badxstudio]

Wowwww thank u so much buddy

[@ORTyOW]

That's crazy, very impressing!

[@dreamzdziner8484]

You guys are excited so much on this concept that its infectious😁 and even I would love doing the exact same stuff like you showed here. What a mind blowing tech 3D Gaussian Splatting is. Will have to upgrade my PC just for playing with this in UE.

[@badxstudio]

Hahahahha It's obvious we are having fun isn't it? I mean it really is exciting being able to create 3D scenes from movies that were shot ages ago. Where does this go next? We've already found videos about 4D Gaussian Splatting which is technically the same tech but in motion so think Gaussians but in video form. Where we are heading is absolutely insane much like video games!

[@dreamzdziner8484]

@@badxstudio Oh man! That would be crazy. Can't still process the excitement after seeing this 3D gaussian splat itself and now you say 4D. Imagine those awesome movie scenes of that golden era re created through our vision. Perfecto!!🤟

[@ferspirit9733]

Man the editing of this video is pure art! And the energy of both of you is contagious, you really got me into unreal 💪🏻

[@user-wc5qq9jh2l]

3D Splatting is unbelievable! The background process is really interesting

[@r.m8146]

Please, do it in VR.

[@badxstudio]

How did you guess the next video title? :D :D :D :D :D

[@sabomarius128]

Yes!

[@JonnyRay82]

This technology is amazing!! But you two playing around in your videos make it so much fun to watch. 😂😀

[@badxstudio]

first of all Jonny it's nice to see you again

[@Kenb3d1]

Brilliant idea.👍

[@badxstudio]

Thanks buddy

[@DaffyDuckTheWizzard]

I think the cool way I would use it would be to build 3d models from miniatures and sculptures for added realism, texture and flavor

[@80sVectorz]

Can't wait for these to be used in "we made ...insert movie title... R-rated" by corridor digital.

[@98f5]

That was 43 years ago you wonderful math geniuses

[@badxstudio]

LOL :D

[@randfur]

In the Trinity shot did you notice the doubling was gone? Also it was squashed, it's like the 3D point cloud stage undid the doubling by squashing them together causing the whole scene to get squashed horizontally.

[@badxstudio]

Yeah we realized that too but in general it made it difficult for the algorithm to arrange the point clouds.

[@OUTTHEMUDVISUAL]

😳 this insane ! I love how you guys break it down with all the information and knowledge ! 10/10 the edits have gotten so much better 🎉❤

[@badxstudio]

Yooo buddy... thank u so much ..glad u like it

[@3Ddex]

I really love the work you guys do. Thank you ❤

[@badxstudio]

Our pleasure!

[@TheTrumanZoo]

imagine filming and recording splat scenes directly.

[@dylanyesenofski5516]

that particular shot in the shining is from the timberline lodge in mount hood Oregon, USA

[@KimAlban]

amazing stuff guys!!

[@badxstudio]

Thanks Kim 🙌

[@TheZackward]

Really enjoying your info and love of the process. Thank you

[@badxstudio]

Thankssss Zackward

[@badxstudio]

Wait till we bring all of this to VR in the next video

[@GranulatedStuff]

holy moly, turn it down a few notches lads !

[@amaddinthenow]

So cool to see brothers working together and having fun together still while adults. Not much of that in the world.

[@andreibezuglov1964]

Well, they are the reason i cant see their videos. Both hairstyles… my eyes cant take it.. and emotions are so fake. Or they use mdma before filming?

[@badxstudio]

Thanks brother!! Well maybe by us making these videos we will inspire more people to have fun as adults 😎😜❤️

[@Anttisinstrumentals]

First thought in my head. I think NEO bullet scene was the second thing I tried after I installed the software. Did not come out so well either. Gaussian Splatting can be used from now on for scenes like this for better effect.

[@Rocksteady72a]

Listening to this on 10% volume and it still feels like I'm going to get a noise complaint

[@badxstudio]

Hahahahahahahah 😜

[@joelface]

Thanks for trying out movies. I was so excited to see how that would turn out. It's a little trickier than I thought it would be, even with some pretty perfect scenes (limited movement within the scene and rotating camera). Video games into VR is a great next idea, since a rotating camera is as simple as moving the right joystick in most cases. I'd also like to see if you can figure out how to edit the capture, delete unnecessary artifacts, etc. Is that possible?

[@badxstudio]

In our UE video, we showed how you can crop these scenes but we still don't have a precision tool for deleting each ellipsoid in UE (devs have already made it for Unit)

[@sissyphussartre2907]

1980 was 43 years ago

[@badxstudio]

oooppsss

[@coisasnatv]

Try to do that with Badge Man photo at grassy knoll.

[@joshfletcher6178]

Imagine if they held off doing The Matrix 4 for just a few more years, they could SHOT for the use of this technology and would have been an incredible revolution from the previous version of bullet time. These tests are amazing but imagine it with a Hollywood budget... someone out there is going to do something insane with it.

[@badxstudio]

HollyWood are u reading this?????? hahaha totally agree

[@DJ-Illuminate]

Imagine taking one of those KZbin videos where someone filmed san fran or LA in 1920 and with the forward motion create a 3D splat or 4D splat. Somehow the forward video would have to be interpreted as side view for at least 180 view.

[@Ardi_0]

This is mindblowing

[@badxstudio]

100% agreed

[@qbert4325]

At this point I am not even surprised to see what it can do

[@badxstudio]

Hahahahahaha I know right!

[@dannyd4339]

"That movie was made in the 80s, that was 33 years ago" what. But then again, you can tell these guys aren't the brightest tool in the shed. They are marveling at something your phone has been able to do for a decade now

[@mystica-subs]

The weirdness you get from trinity in the matrix is framerate/telecine conversion errors. A properly inverse-telecine'd dvd or non-rate-adjusted 23.976 bluray source would be the ideal source for this shot.

[@Edbrad]

Loads of Jerry Bruckheimer films you could have used.

[@jlec9081]

33 years ago? wtf

[@badxstudio]

I knowww, we weren't even born yet

[@ORTyOW]

@@badxstudio we mean it was 43 years ago😅

[@badxstudio]

OH SHIT HAHAHAHHA QUICK MATHS :D :D :D Lol 2+2 IS 4 -1 that's 3 Quick maths @@ORTyOW

[@hanitouati]

2:30 nice math bro

[@badxstudio]

hahhaha ooppss

[@Mowgi]

43 years ago, my friends 😅

[@badxstudio]

Hahahaha fml … we failed maths

[@theodorus321]

this video is a real gem love you guys love from turkey

[@badxstudio]

Ooooof ma man from Turkey! Listen your food is phenomenal!!!! Been there before and love it!

[@theodorus321]

@@badxstudio thx man you are always welcome to my city adana the home of kabab, you guys should make a video about render problems in unreal engine after 5.1.1 movie render queue not working well im handling it somehow but many people cant

[@GamesandBacon]

be interesting to see where this goes. im wondering how long until we see animated gaussian splats and scenes being recorded in 360 from multiple cameras, sync them up and being able to "play" it like a video, or drop it directly into your 3d environment ect.

[@badxstudio]

Definitely going towards that world! We already saw some videos of 4D Gaussian splats which is technically Gaussian Splats in motion

[@DJ-Illuminate]

I actually made a 360 video that could be played and stopped while in VR. I have a KZbin video of it on my KZbin channel somewhere. It is The Cable Center VR Virtual Archive on STEAM. In one part of the building I have a 360 video section. You place the bubble of the video onto your face and you are in the video. The controller changes and allows stop and play. No fast forward though or rewind. That would have been cool. The exhibit was made in 2016 so it is a bit dated.

[@MXK_EDITZ]

Love you guys 💗

[@badxstudio]

You know we love you too right?

[@Uhfgood]

What's kind of cool, is even though AI is used to generate these, you can use some other AI to replace the missing "footage" -- for instance trees and mountain behind. (Sure it wouldn't necessarily be exactly what's actually behind the hotel, but it could generate lost information to give you a reasonable facsimile of what might haave been behind it

[@badxstudio]

We were talking about it after watching Adobe Max, it would be dope to try the video in painting feature. This is definitely something that eventually will be part of the process.

[@ORTyOW]

13:46 it's Trinity calling, because you called her

[@badxstudio]

OH SHIT!!!! WE JUST REALISED THAT hahahahahha imagine if we added that to the script

[@ORTyOW]

@@badxstudio ahahaha

[@MogulSuccess]

Here's my pitch New Industry - Virtual Reality Movies | See "Ready Player One" Allow people to interact with old movies Mode A. Casual observers walking around the scene Mode B. "Karaoke Mode" - Protagonist say lines on cue and be in the scenes on cue - "a la" Just Dance Scoring points Mode C. Party Mode - friends play supporting characters Mode D. Multiverse Mode - GTA worlds of the entire movie available to explore RPG style where user can randomly switch Modes A - C

[@badxstudio]

Yo this is such a cool idea!! Totally can see this happening hahaha so cool

[@MogulSuccess]

@@badxstudioif you have the bandwidth I have some capital resources we could create a prototype and go from there

[@merion297]

Haha, the Batman part suggests that the conversion process should know the camera FOV.

[@juanjosoliz7297]

43 years

[@badxstudio]

ooopppss sorry hahah

[@lukesfilmltd]

looking awesome!

[@badxstudio]

[@DejayClayton]

I wonder how good these results would be if you exactly mirrored the actual video camera path and tracking, and then created 3d models from the point-in-time gaussian splat viewpoints that intersect across all frames.

[@badxstudio]

That is a cool idea in theory! we will have to test that to see if it will actually work tho!

[@DejayClayton]

@@badxstudio keep me updated!

[@jumpieva]

consider this is tech in it's infancy, imagine where we'll be 1 or 5 years from now.

[@MrLyonliang]

amazing ideas!

[@Andimax11]

Commenting to stay on 3D Guassian Splattering algorithm

[@badxstudio]

🤣🤣

[@Because_Reasons]

Very clever idea guys. Love your videos.

[@badxstudio]

Thanks brother 😎🫡

[@felixgeen6543]

video games will be a lot easier since: no lens distortion, no lens blur, no motion blur, no lens aberrations, more FPS... basically just easier to do structure from motion on video game footage in almost every way, except for the HUD which would need to be removed.

[@badxstudio]

We are testing that too! Exactly it’s gonna be much simpler! Thinking about cyberpunk maybe for the test! Any recommendations?

[@lgvivqzt]

Guys, you make tech fun again! I LOVE your videos. I wish you all the best.

[@badxstudio]

Thanks so much!

[@Edbrad]

3:42 now consider the next leap forward where it will intelligently create all the parts we never had a camera record.

[@badxstudio]

Yes exactly!!

[@cg.man_aka_kevin]

Like this is a new technique to make the world stop time! :D

[@BeachcomberFilms]

You guys beat me to it!!! I'm wondering how Gaussian Splatting would go with stereoscopic side by side video at full frame.

[@LutzTeichmann]

looks like a visualization of memorys

[@red4666]

can the gsplats cast and/or receive shadows? like if you put a cube in the middle of the table in that one scene and cast a light, will the table receive a shadow?

[@thevfxmancolorizationvfxex4051]

Would it work for scenes where cameras move across or remain stationary? I'm thinking of using this method to colorize black and white footage, so I'm just wondering

[@badxstudio]

Camera needs to be moving to show clear depth! As of now this Gaussian Splatting tech works on finding points of interest, and if you are not moving your camera the right way you might fail your training!

[@thevfxmancolorizationvfxex4051]

@@badxstudio All I need really is for the scenes to be rendered exactly how they are for what I want to do, as I'd like to edit each object individually

[@badxstudio]

I'm sure it's gonna be doable! You just have to test it out!@@thevfxmancolorizationvfxex4051

[@joelface]

@@thevfxmancolorizationvfxex4051 I think there are other methods you could use for recolorizing old footage. I don't think gsplats will be the right way to do so, especially with stationary camera.. it can't figure out the depth of a scene without multiple viewpoints. It doesn't create any data that isn't already in the video.

[@thronosstudios]

Imagine this thing being able to record every frame properly (instead of a few) and you could literally pause in any of them and look at them from any angle without losing detail. If anything, they should be able to fill in the blanks with AI

[@badxstudio]

Yeah that's the thought! We like to believe that very soon AI will b able to help in generating the missing detail and create a complex scene that looks stunning!

[@HongPong]

you guys crazy for this one. Stanley Kubricks ghost is going to be up and snooping 👻🎥🧿

[@badxstudio]

hahaha hope he is not angry.. we love his movies

[@BrentLeVasseur]

That shot from the Shining is taken at the top of Mt. Hood ski resort in Oregon. There is a glacier there where you can ski even in the summer time. When I skied there, I instantly recognized the lodge as the one used in all the exterior shots in the Shining.

[@badxstudio]

Oh that is so cool to hear!! How was the ski experience there tho?

[@BrentLeVasseur]

@@badxstudioI trained there with the Swedish and Korean national teams during the summer of 1991. Glacier skiing in the summer isn’t the best when compared to normal winter powder skiing, but its good for training on the off season.

[@S41L0R]

I love this.

[@alkeryn1700]

4d gaussian splatting is comming soon you'll be able to do it with moving subject and have moving playback.

[@badxstudio]

yes the github is out.. we are gonna test it soon

[@TheFachen]

this technique would be great for an overhead ring of cameras or partial ring of cameras synchronized similar to the matrix shots - not to take the single output video from per se as you did, but for the filmmakers being able to relight the scene in post

[@hamidvisuals]

Fantastic stuff, I had no clue about this technology until I came across your channel guys. Parcham Balast

[@badxstudio]

Love you dadash!

[@zamadatix]

12:35 isn't "tricks" for motion blur, you've just got a what looks like a telecined copy and your player isn't set to detelecine hence the resulting combing effect being present on only certain frames. If you pull a direct copy from the Blu-Ray (or find a better uploaded copy) it won't have this artifacting.

[@d0rk4ge]

@zamadatix you nailed it. This looks exactly like 3:2 telecine

[@metedev]

Good work!

[@badxstudio]

Thank you! Cheers!

[@MultiMam12345]

No way AI will be able to replace these two hilarious guys in the next 6.......something

[@badxstudio]

Lol 🤣

[@caliberpokwana8872]

Too sophisticated and inspirational at the same time

[@badxstudio]

Thank you brother glad u liked it!

[@shilze1]

I think static and long views of fully CG scenes are probably the best bets of getting anything out of this. Which is at least interesting for the idea that you could then take old somewhat subpar CG shots and recreate the scenes with the same camera movement. At least until it's possible to account for the moving elements completely. Video games are cool and all as a use case but most of them are already 3d and can have their assets ripped, so the only real use case with that is putting funny things inside the scenes.

[@krisserold134]

Y'all are CRAZY and I love it

[@badxstudio]

hahaha thanks buddy

[@larswillsen]

Did he say the Shining helicopter scene was shot 20 years ago? .. Ir's actually almost 50 years ago (approx. 43 - 45) .. :-)

[@cammacpherson.5267]

I love you guys. Talented beyond belief

[@badxstudio]

WE LOVE YOU TOO

[@phill_ai]

OMG this is next level!!! Well done guys!

[@badxstudio]

Thanks Phill

[@midasredblade236]

yeah, it'll be a decade till this is USEFULL

[@lucadorigo1813]

What software are you using for generating/rendering these? Did you write it? Is it on github?

[@339Memes]

This is how to watch a movie in the future, yeah

[@Milaaq302]

Do you have to estimate lens data? That zoom in Batman really screwed it up

[@felixgeen6543]

You guys even explained yourselves why the Batman shot doesnt work: its zooming. changing focal lengths are not supported by COLMAP. You would need to solve the variable focal length in a software like Syntheyes and remove the focal length by scaling the image down inside the frame as the footage zooms in. this would also require you to remove the variable lens distortion present in most zoom lenses. its a complex thing to do.

[@badxstudio]

Yessir!! U got that right 🙏🫡

[@dudelsaladt]

Excellent!

[@badxstudio]

Thank you! Cheers!

[@Pockeywn]

not the unreal default chair lmao

[@badxstudio]

HAHAHAHAHAHAHAH too lazy to drag something else :D :D :D :D

[@deadANTdeadANT]

This is a game-changer 4sure! Lovely experiment guys, keep em comin! (plus what great news to find out that, since i was born in 1983, according to ur math, I've only just turned 30 this year! Thank God, i thought i was 40 years old, **phew** such a relief, i feel 10yrs younger!) 🎉🎂🥳

[@badxstudio]

hahah Thanks a lot buddy.... You are 30 years young my friend. Age is just a number

[@deadANTdeadANT]

@@badxstudio 😊

[@Marco-xz7rf]

Hey can you please try this on apollo 11 or other missions? I tried it with apollo 11, but only with the images. But they aren't overlapping enough :( Is it possible somehow to tell the 3DGS, where each photo was taken and the angle? would that help?