3D Gaussian Splatting in UNREAL ENGINE 5 is INSANE!

3D Gaussian Splatting is one of the MOST PHOTOREALISTIC methods to reconstruct our world in 3D. Now we've done the tests but its no good till we bring them into Unreal Engine 5 to see if we can use it for different purposes. We are doing that using a plugin which is available on Unreal Engine Marketplace. Tested the quality, performance, crop tools and even set up some cameras and even made some tiny cinematic videos.
3D Gaussian Splatting for Real-Time Radiance Field Rendering Paper:
repo-sam.inria.fr/fungraph/3d...
Unreal Engine 5 Gaussian Splatting Plugin Link:
www.unrealengine.com/marketpl...
Shoutout to Maged for the Pyramids Drone footage
Join our discord server:
/ discord
If you wanna see us to do cool things follow us here too:
Instagram: / badxstudio
Twitter: / badxstudio
TikTok: / badxstudio
LinkedIn: / badxstudio
Bad Decisions Podcast 🎙️:
podcasters.spotify.com/pod/sh...
Our personal handles: (if you wanna stalk us)
/ farhad_sh
/ farazshababs
/ farhads__
/ farazshababi
/ farhadshababi
/ farazshababi
#unrealengine5 #3dgaussiansplatting #3dscan #gaussiansplatting #3d #3drender #nerf #unrealengine #blender3d #blender #drone #ai #photogrammetry #vfx #cgi

[@ArtificallyIntelligentAi]

That Unity joke made me fall out of my seat laughing. Super relevant right now.

[@badxstudio]

Hahahahha exactly why that came :D Honestly that was improv not scripted :D

[@mikda360]

..you fell out of your chair laughing? It was THAT funny?

[@ArtificallyIntelligentAi]

@@mikda360 a lil bit =)

[@kaihawaii8924]

I like Unity too, but that joke is badass!!

[@jktech2117]

​@@mikda360bro it is, i laughed so hard exploded into a super nova then turned into a black hole and when it lost its energy over trillions of years the energy released reformed my body and im still laughing. (i used irrlicht engine, btw)

[@chadbonanno5724]

YES! Thank you for going live and explaining some of the questions! you guys ROCK!

[@badxstudio]

No worries brother

[@astr0_th3_man84]

I've been waiting for something like this to happen. What a time to be alive!

[@badxstudio]

It's crazy

[@francesco7800]

ehe, is this a quote from TwoMinutePapers?

[@xXWillyxWonkaXx]

Respect. I think you guys are following into the footstep of the Corridor Crew. I like the flavor that you bring with the video scripts/montage. Subscribed.

[@badxstudio]

Thanks for the sub Tariq

[@joelface]

Looking forward to your next video on G-Splats! So fun to see you experimenting with it.

[@badxstudio]

Thank you Joel! As we speak we have at least 3 more Gsplat videos we are working on so stay tuned

[@OlliHuttunen78]

Cool! Very nice video again! It was interesting to see how Kenjis UE plugin runs in practice. Great job!

[@badxstudio]

Dude thank you

[@r.m8146]

This just keeps getting better and better!!

[@badxstudio]

It's going fasttt

[@tyler.walker]

Really great video, I’m definitely subscribed to continue having fun with you guys! 😂

[@badxstudio]

hahahha man it's gonna be amazing

[@brettcameratraveler]

Very exciting! Have you done any side-by-side tests on if shooting 1080 vs 4K vs 50MP stills vs HRDI vs RAW give and significant quality advantages in the final g-splat?

[@badxstudio]

That's a brilliant video idea we are destined to try! The next few videos are going to be slightly different tho but its coming for sure. Hint for next video: Easiest Method to do 3D Gaussian Splats!

[@Edilyon]

Thx guys ! Looking forward seeing where all of these new techs lead us

[@badxstudio]

You are absolutely welcome and damn are excited about it too!

[@jpmeniconi7818]

You guys are the best, literally! Big hug from Chile!! I will try GSplats this weekend for the terrain of my architecture thesis project, to see if next week I can bring all together to UE5. Hope everything works :)

[@badxstudio]

Dudeeee big hugs to you! oooooh that terrain idea would be so cool! Would love to see how that turns out. Bring it on in our discord

[@danielb.2873]

Exactly what I'm going to use this for. Maybe not in Unreal but web-based

[@JonnyRay82]

Dude!!! This is SO COOL!!!! I can't believe how far back the detail goes

[@badxstudio]

It's mind blowing bro.

[@Mowgi]

I love the enthusiasm of you guys. You keep inspiring me to learn and try new things.

[@badxstudio]

Great to hear that buddy 🙌🙌

[@fnytnqsladcgqlefzcqxlzlcgj9220]

thanks for bringing interest to these new/old techniques!! :)

[@badxstudio]

Glad you like them!

[@bingbang9643]

I wanna use this to keep the same camera angle i intended but stabilize shaky shots. I'm a freak of stabilization and this seems like the ultimate solution. maybe some basic masking the 2 methods might fix the sky? or the colors will differ too much? nice work keep it up.

[@pgjohnson5504]

Yessss.!!!!Awesome!!!Thank you for testing this app. Now I buy Thanks! More Gaussian!!!!

[@badxstudio]

[@nwbrne]

Pretty awesome video, thanks for sharing!

[@badxstudio]

No worries 🖤

[@marcupitzerra6206]

Hello guys, first I want to say amazing work as always! Do you know if there are other methods of cleaning your splat aside from cropping with the plugin? Is it possible to do it with another 3d software like blender?

[@andreyzubkov8435]

Looking forward to making my house and neighborhood a map in a future video game with like a minute of video from my phone. This is insanely cool.

[@lucvandemoortele8544]

Thanks a lot for sharing this ! I was very impatient to find a way to get it in UE (I was searching for it, found your video) I hope we will get it Twinmotion as well in a not too far future... And of course to be able to edit the splats !!! That would be great !

[@badxstudio]

No worries at all! Yesss if we can edit the splats it would be such a game changer!!!!!

[@danielmilyutin9914]

I'm curious if one can do in-game rendering with this. Like do (2 or more)-stereopair for inner rendering -> do GS estimation -> display rendering with GS. Some optimizations come to mind. As I can get GS can allow some extrapolation on multiple frames for static scene when view camera moves. So one can do inner rendering less frequently.

[@Haruchemy]

Amazing content! Keep it up!

[@badxstudio]

Appreciate it!

[@rubberdog]

Totally blown away!!

[@badxstudio]

Right??!?

[@nand3kudasai]

pretty cool vid, thanks. i m new to this channel but it sounds you guys come from film. i'm a gamedev, so i see it from that pov. i find Gaussian Splatting interesting, though i really think mesh reconstruction with nanite could give better results, and speed. specially on the text of the signs (depending on what you want to do). (also reflections, light and collisions). though i'm actually interested to know if actually the performance diff between both. if GS are 100% on niagara it might outperform nanite (given it might be simpler). though on a game, i wouldn't want to use up all my gpu with GS, the game itself would be demanding. It would be nice if you guys could share the gpu consumption of playing in the scene. i wonder if you could do stuff in VR like virtual stuff (like a "walk around somewhere" or "see this stuff"). i think the cool stuff on GS was the part where it captured the "floaters" i think that makes for cool clouds. but i think volumetric clouds are actually better, though maybe less realistic. another point might be how it works with collisions lights and reflections. i think GS visualization might be much less flexible in that regard. i really hope Epic adds GS directly to the engine, it might just be a cool thing to mix with other pieces. Maybe for certain things, like an intro animation (like FFVII e.g.), or mountains on the distance or buildings.

[@Thatindianguy.]

This is getting crazy!!!

[@badxstudio]

Completely out of control haha

[@haikeye1425]

Nice!!!

[@DannyOnateArt]

excellent video! What is the name of the program to preview the frame sequence, when you drag the mouse, move forward or backward in time?

[@badxstudio]

DJV .. you will love it :)

[@darthtechnologies553]

I wonder how well it could render a scene if you got more scans closer to parts of objects and more angles too! This has a lot of potential

[@badxstudio]

Probably more detail then? I mean this is all still too early! But have u seen our newest video? We even turned movies to 3D with this

[@unlockpuertorico]

This is dope! Just learning about this new tech as I would like to find ways to innovate with my drone work in the 3D world 😊. Btw if you want to use some footage from Puerto Rico for these videos I’ll gladly provide one just to see you do some magic with it 🙂 Keep up the good stuff! 🍻

[@badxstudio]

Hey broski! Love it. Sure thing you can always join our discord and share it with us there

[@EazyE206]

Yall innovating at record speeds

[@badxstudio]

Hehehe jus wait till you see the next videos!!

[@runimation]

Hi there. Nice vid but I struggle with a small detail. Maybe you can give me a hint? How do you link the gauss to a cinema camera actor instead of the viewport to render out?

[@LeeBrenton]

So cool guys!

[@badxstudio]

It's so amazinggg

[@Moshugaani]

I think that the issue with the sky generating weird gaussians could be remedied if the software knew what certain parts of the images are supposed be like. For example you could mark parts of the image and tag them as "sky" or "far away background" and it would then know to generate those splats far away instead of how it is now. Someone would just have to code in that functionality first. :D

[@badxstudio]

I hope something like that happens ASAP! NEED IT!!!

[@kgdllc]

This is very nice.

[@joseph.cotter]

So now what we need is for DJI to add a feature to their drones to be able to define a target for calculating a flight path for taking splat maps

[@badxstudio]

hahaha we tried Photogrammetry with our DJI Mini Pro 3 and the trick was to use the target tracker and keep taking photos. Similar thing can be applied here

[@CooperScans]

How are you able to get the GS visible in the CineCameraActor for the rendered cinematics. When I look through my camera the GS is not visible even when the GS is directly in front of the camera?

[@jayAreLive]

in theory you could make a scale model town and use that as your training right?

[@RiverReeves23]

I really want to see more!

[@badxstudio]

2-3 Videos of Gaussians coming your way already this coming week ❤️❤️

[@Mr_Dee]

Supercooool!!! Would it be possible to grab an hdri with rendertargets and then use this hdri to light additional 3D objects in the scene with it?

[@badxstudio]

Hmmm interesting question. Not too sure but would love to try!

[@Krappenschittz]

Thanks for hearing me out!

[@badxstudio]

[@mae2309]

amazing stuff!!

[@badxstudio]

Thank you

[@tombittikoffer412]

For GSs I can open in the python view they don't show up one I import with the plugin and drag into scene with a light. Is there any niagra prerequisites which need installed? It shows when I edit the BP but not in a blank scene, even scaled large and small. Any sanity checks I can perform? Geo shows up fine in the scene, just not these BPs it creates in a blank scene.

[@rajeevnikam6118]

Hello team, just a random question , can we create 3D models using guassian splatting ?

[@Ariesblack90]

Ur tutorials are the best they are so immersive and on point just like your awesome work

[@badxstudio]

HELLLOOO SIR! good to see you here

[@377omkar]

Need some details video how you did it, cabt understand how that splater came or was processed , really confused that how the plugin dose it, dose it take image sequence directly and dose it or we need other software to make the gaussian spaltter ???

[@j.carlosneria179]

i wanna know too, did you find anything about it?

[@GiulianoGolfieri]

Hi guys and thanks for all the great content. I was wondering to use this technique to test infrastructure assessment with drones (imagine capturing a full bridge by flying around and under it). Do you think this could work? And do you think that using a reference object for sizing it would be possible to take precise measures of parts of the structure? Trying to understand if this technology could be useful to replace complex photogrammetry missions to some degree.

[@badxstudio]

Funny enough we were just having a conversation about this between us ... You can bet that eventually these technologies will be used to perform precise measurements but we assume that it is not a 100% there yet when compared to expensive machines and detailed photogrammetry. However since the question was brought up, we are going to attempt to do a measurement test and see how accurate the results are. Stay tuned broski!

[@GiulianoGolfieri]

@@badxstudio great! 😍

[@TheGameChanger-77]

Hey, which software are you guys using for video playback?

[@CannedSoup283]

for certain applications, you could make crazy games like this

[@CyrilPreiss0]

Could you compare the unreal paid plugin and the luma free one ? Is one better than the other one ? Fps ? Yours videos are great btw

[@badxstudio]

Thank you

[@yomamagaming6736]

Wow, just Wow.

[@badxstudio]

Crazy. Right?

[@detcader_]

It seems this technology could also serve as the perfect LOD bias setting, instead of switching out for a lower poly model you can just switch to the Gaussian representation. I imagine it will be very useful for open world games, like assassins creed or cyberpunk 2077. It seems that with the combination of these technologies with things like DLSS and Gaussian rendering we can get games to both look better and run better than native, but easier said than done lol.

[@yvettelin7271]

Hi! Thanks for the sharing you guys are amazing!! but I run into a problem that the Splat Ply I download from Polycam can't load in UE5 with Luma plugin on....the file crash immediately if I drag the file in... anyone knows the solution? Thanks!

[@robmulally]

Nice! Could you use a negative volume or something to crop the unused parts?

[@badxstudio]

Wait didn't we do that in this video? :D I think we did crop it for abit

[@robmulally]

@@badxstudio you cropped it, but I know for a mesh in ue you can crop or do a Boolean, I'm not sure with this point system if it can do same. Like shape a Boolean shape b to leave hole, might be a way to remove the objects in sky.

[@user-hc1vy6wn5j]

I want to know if the size of the trained large-scale scene model is very large? A simple model I trained myself can reach several hundred MB

[@holgi3115]

Here we go)

[@badxstudio]

🙌🙌

[@unparalleledcavern]

Splat only showing in live mode, not in viewport. Is there a fix for this?

[@captainofthewhitetower]

Guys in the drone videos. I wonder if the drone goes 360 how it is going to be ?

[@marlaoutatsgamingcollectibles]

I'm curious, how heavy is a scene like this in terms of memory usage ?

[@ezbench_]

Very intensive on VRAM.

[@marlaoutatsgamingcollectibles]

@@ezbench_ Ah...that's a shame!

[@Atlas_Redux]

Heads up, there is a new plugin out there now that has supports importing edited PLY files, so you don't have to deal with floaters, objects, distortions and areas that you don't want. I can't post link because of KZbin's horrible filtering, but there is literally only two plugins on Marketplace right now, so :P

[@demonhogo]

whats the name

[@Atlas_Redux]

@@demonhogo Just search for them. Again, there are only two 3D gaussian splatting plugins on the store. Also, Super-Splat now can export to native 3DGS PLY, so you don't need compatibility plugins anymore, all plugins will work, including the free ones.

[@arne6617]

hey, i played around with that tool aswell in UE5 but i didnt get to the point to make the different parts of my gaussian splatting scene look seamless. you know what i am talking about? its dependet on where your viewport camera is. it seems as if certain parts of the gaussian splatting scene jump from one resolution to another. were you able to solve this? i looked it up in the documentation but i think i did something wrong. just for now i prefer the "sibr gaussian viewer" from the original github documentation.(just for viewing)

[@badxstudio]

Hey are you talking about a seam (line) in your splat? If thats so check the documentation. You have to reimport it with a new console command to change the import method

[@manishprajapati4582]

It's amazing 🤩, can i use this in unity 🙄 ?

[@RaulFrizado]

I'm having pain because I have to wait for the next video

[@badxstudio]

hahahahah brother it's cominggg very soon

[@Appleloucious]

Dayum you bros have some uncanny valley qualities! : D Only 1m4s in tho really liike it so far! One Love! Always forward, never ever backward!! ☀☀☀ 💚💛❤ 🙏🏿🙏🙏🏼

[@badxstudio]

Cheers hahaha thanks a lot mate

[@Appleloucious]

Thank you for your enthusiastic self and being authentic : D @@badxstudio

[@markinmkn]

Could you provide the link to download the photos to imput, like the piramid ? I love your channel!!

[@badxstudio]

The photos are huge buddy, come on our discord and we figure out a way to give you the .ply file or the footage

[@markinmkn]

ohh thats pretty good!@@badxstudio

[@markinmkn]

I am on discord channel, how can you provide me the ply

[@waitisthatcobalt]

damn, thought you guys were gonna throw a third person character in there real quick over a plane so he can walk, and just make the plane transparent or something. That would've been real simple but cool. either way this is awesome

[@gandepuun]

Absolutely amazing. Link to the Unreal engine plugin please

[@badxstudio]

www.unrealengine.com/marketplace/en-US/product/3d-gaussians-plugin there u go!

[@BenEncounters]

You guys rock!! I don't have a very powerful machine so before when I tried importing NeRFs inside Unreal with another plugin it would crash. Do you think those Gaussian Splatting can load on a less powerful machine? Would you make a test file available to try?

[@badxstudio]

Hey buddd!

[@BenEncounters]

@@badxstudio Super cool!! Gotta give this a shot!!

[@Deuterium0115]

How is performance? Is this something we might see in a game ... or something that I might use to create a VR game that runs on Quest 3? Does this need as much VRAM as GSplats usually need?

[@badxstudio]

Crazy fast and very high FPS. Probably not at this stage but for sure in future yes. It is very fast to view and render. We are going to test it in VR in the coming week.

[@Deuterium0115]

@@badxstudio amazing can't wait to see it!

[@god1994FATHER]

Guys, please, try this in VR! UE+3GS+VR it will be mind-blowing, I sure 🙌🏻

[@badxstudio]

Yeah buddy, that will be the next experiment ...we have them all lined up.. Virtual Production, VR and etc.

[@Dmaradona1997Shady]

Can you do a 360 video inside a city and use this as HDRI/sky dome for a game?

[@GenarAltinisik]

amazing

[@badxstudio]

Thank you! Cheers!

[@377omkar]

What is the software used for gaussian splatter

[@VooDooEf]

very soon we won't be able to tell 3D from real life, wow!

[@badxstudio]

It's getting crazy out here man!

[@shinnokgaming_]

Have to try with 360 cameras to get the whole spots , then u got a really good map

[@arkemal]

Great video thanks! The UE5 plugin costs 99$, has anyone found a cheaper version?

[@badxstudio]

You are welcome brother

[@Juno006]

Ah yes, UEFN my home, can't wait to see what you guys do with this

[@badxstudio]

Dude just a little teaser for ya ... UEFN is on our Roadmap HARD!! Just wait and see over the next few months

[@Juno006]

@@badxstudio sure !!! 😀

[@yuanyuanfeng9118]

It always failed when I train my model, and shows cuda out of memory, anyone can help ? My GPU is NVIDIA RTXA6000 48G, and I input 999 pictures , they are 10G in total.

[@pgjohnson5504]

What about an indoor Nerf vs GSplat. ? I’m going buy this app next Friday and try with the Polycam Splats. And hopefully get around to the regular splat set up. Hopefully these can replace costly reshoot builds where we have to recreate a portion of a Set. And the background is just out of focus anyway Can you test Polycam Splat Vs Original Splats? Awesome channel BTW!

[@badxstudio]

Next video is on polycam splats ;) There is a hint for ya!

[@marinomazor.adventures]

Hei! I have just downloaded Unreal and tryed to connect with Luma but it seems Unreal is 5.3 and plugin is for 5.2? Any help?

[@badxstudio]

Broski this is not luma! This one is different ... its gaussians!

[@Felhek]

Using this technology on security cameras. The videos captured by them. Exploring the environment to have a better look and understanding of the scene

[@TheChrisNong]

Hi. Were these locations shown as niagara particles? Wish you showed how the added lighting would’ve affected these scenes 😢

[@badxstudio]

Yes they convert to niagara particles with this plugin! Don’t worry we are doing a video about it soon!

[@user-ps6mj8pv4h]

We need some Aeria 51 footage to analyse the place better :P

[@SpirusFilms]

can you edit the niagara system?

[@badxstudio]

Yes you can! We actually played around till we enabled shadows on the niagara now! But our experience with niagara is insufficient and therefore we havn't been able to fully utilise it!

[@simonjacksparrow]

Also polycam has now added a feature for this.. but experimental and only 200 photos

[@badxstudio]

Yes, and we are going to test that in the next video

[@hytalegermany1095]

How to create such point clouds?

[@badxstudio]

You can either train your own using your PC or you can now use a website like Polycam that has their own Gaussian splats. .. we have an upcoming video dropping tomorrow night literally about that!

[@Andreas28]

You guys have to try gaussian splatting with a 360 Camera

[@Atharv0812]

Gaussian splatting can be the future of 3d rendering

[@badxstudio]

We are going full speed!!

[@navysealscopypasta9]

Also gaming industry

[@itayhilel2168]

Can you make a video doing this in unity??

[@lobau_]

7:21: that Chrono Trigger "End of Time" vibe

[@badxstudio]

Lololol true!

[@RamadaArtist]

Okay so, hear me out. As an artist, a traditionally trained painter and one who's pretty familiar with 2D digital illustration on a tablet, this has me imagining a new level of digital painting tool. Instead of working on a tablet, you'd wear a VR headset and paint splats directly into three dimensional space with a motion controller. Imagine a 3D rendered scene in Unreal, that's hand painted into space and looks like an impressionist painting. I want that. I want to be able to do that.

[@badxstudio]

Wow that sounds absolutely nuts! That would look soooo coool!

[@lethargicmosquito]

i think you can do that in dreams for ps4/ps5

[@ADUuniverse]

when is the tut coming!

[@badxstudio]

That's part of the plan too! We have a few more Gaussian videos in the pipeline. Next one is gonna be exploring Polycam's Gaussian Splats which doesnt require you to have a GPU with min 24 GB Vram. Another one is bringing these into VR!

[@davekite5690]

'very cool - 'wondering if you had another video, shot from a different angle (eg from the opposite side of a building), would/could it merge the two G-Splats together... what about 3, or 4 vid's or more lol?

[@badxstudio]

yeah that would be awesome to test! Maybe in an upcoming video ;)

[@estrangeiroemtodaparte]

As it stands, while GSplats are visually more appealing than NERFs, the latter appears to have a slight advantage in terms of adaptability for virtual production. Would you agree?

[@badxstudio]

At least it seems like it but there are some limitations! For that very reason we are going to test it next week at a studio here in Dubai on an LED VOLUME to see how it does!

[@estrangeiroemtodaparte]

Awesome, guys. Keep going!! @@badxstudio

[@everythingsfine3064]

Wen export to c4d

[@coreym162]

Just imagine the Gaussian Splats Robots and self-driving cars would produce. They'd be able to "videogame" your life and generate maps for everywhere you go.

[@andrelip1]

Please another video testing in VR

[@enlightendbel]

Imagine Microsoft integrating this into Flight Simulator. It's already scary realistic looking at times. With this, it would become indistinguishable from real life. This as a tool in creating movies, shows and I can even see plenty ways to make this useful in 3D games, if it really takes down the processing power to achieve this level of realism, it's going to be a big thing in all media going forward. One issue I see though, the lighting and shadows from the time of day you record in are baked into it. To make this useful in games, you'll need to figure ways to convert this to a neutral lighting, so you can apply lighting to it.

[@badxstudio]

100% agreed. This is still at a very early stage and we already have Unity and UE Plugin. Probably there will be updates to lighting and shadows in a very near future

[@miranda.cooper]

I tried doing photogrammetry with Google street view domes and it didn't work but I wonder if you could do it with this?

[@badxstudio]

If you tried let us know :D

[@haikeye1425]

Welcome to new UE Plugin: "UEGaussianSplatting: 3D Gaussian Splatting Rendering Feature For UE"

[@badxstudio]

Yo yo are u the dev for the new plugin ?

[@haikeye1425]

Yep, that's me!@@badxstudio

[@badxstudio]

Send us an email or DM on Insta, we love to know how you developed it@@haikeye1425

[@seb3d]

Can you view 3dgs in vr?

[@badxstudio]

You should be able to ... That would be our next experiment :)