Let's Clean the 3D Gaussian Splatting models and remove floating artifacts!

[@BenEncounters]

This is the kind of stuff that would really make sense in VR, it would be so much easier to clean a model this way.

[@stephanelegendre7624]

I had that exact problem and you solved it in a very creative and smart way. You deserves way more subs! Calm relaxing voice + great tips, love it.

[@DejayClayton]

Here's a method for automating Gaussian Splitting cleanup. Generate a 3D grid of evenly-spaced coordinates to be used as vantage inspection points. Visit each inspection point, and from that point, look 360 degrees in each axis, looking outwards. Note which splats are visible from the vantage point, based on how many pixels the splat contributes to any rendered view. After collecting metrics from each inspection point, notice which splats contribute minimally when measured across all inspection points. Then, either remove those splats, or push them backwards away from whichever vantage point saw those splats most visibly, in case those splats contribute to background texture in a way that was just calculated from the wrong depth. Repeat this entire process multiple times until all superfluous splats are processed or removed.

[@AlexTuduran]

Great video Olli, as usual. One mention though: That's not a Unity plugin, it's a Unity project in which the dev made the tooling for creating, editing and viewing the splats. You could say it's a framework component for working with splats. Keep these videos comming, it's great to see progress on the matter.

[@OlliHuttunen78]

That’s a good point.

[@estrangeiroemtodaparte]

I have a feeling I will be visiting this video many times. Thank you, sir!

[@tobiasdierl4557]

Great Video! One sidenote, there is another open source programm called CloudCompare that you can use to trim your gaussian splats, but i have to admit you don´t have a gaussan splat live preview while working in the programm. You see a Pointcloud. One major advantage of the programm is the fact that you can generate a mesh that gets aproximated from the pointcloud. You could use that Mesh as a collision Mesh in a game Engine, etc. And one drawback is the fact that you have to convert your original .py file into a .py file format that is compatible with CloudCompare via the command prompt and when your finished change it back to the original .py file format. (So it can be importet into UE5 for example) Just wanted to point that out. I will try the unity Plugin presented here next because i think it is more user friendly ;) Keep it up!

[@OlliHuttunen78]

Ok. Thanks. I’m collecting all these different programs that can be used around Gaussian Splatting. I’ll check this CloudCompare. Sounds interesting.

[@IXNXMXI]

Thank you for this tutorial, it is really very practical, greetings from Colombia

[@alpaykasal2902]

Great video Olli. Thanks. I did go through this last week (using your c4d living room render actually) and I found that I could never get the walls or cieling (for example) as clean as i want because, once artifacts are removed, then the previously solid/clear surfaces become patchy. Dark camouflage-like pattern throughout. It made me wish for a 'rebuild' tool which closes up some of the damage created by cleaning all the extra floaters. This can be seen in this video at 9:08 . I know you don't have an answer to that :) but it's good to know we're al facing the same issue. Thanks for all your work Olli!

[@indibarsarkar3936]

Great video!! 😃 Maybe try opening the PLY file in a text editor to see how the splats are represented :)

[@andrejdantzler]

Great and clear video. Thank you!!

[@CameronCrichtonVFXArtist]

Unity 2023.1.20f1 does not have the selection tool available.

[@JB-fh1bb]

I’m *shocked* there are no AI tools for this type of work. If we naively assume that walls, floor, and ceiling are flat surfaces at right angles to each other, then we add understanding of doors, picture frames, stairs, etc we could do a two-pass process where structure is first identified then points within them kept as everything in open air is eliminated.

[@Atlas_Redux]

You can remove most floaters automatically now with 3dgsconverter

[@daviddoudouable]

Nice ❤❤❤❤ It would have been fantastic to see at the end a side by side comparison of the quality of the walls after and before cleaning the gs 😊 the blurs are removed but it should affect too the quality of the global rendering, isn't it? ❤

[@Olstman]

Great! Thank you for tutorial! Are there any filtering methods? Obviously, each splat has X and Y coordinate, perhaps there is a filter that finds the most stretched on one axis or two axes splats and selects them. As a rule, they are the ones that clog the whole view. As a Guru in Gaussian Splatting topic, perhaps you have heard sometigng of such tools?

[@hgatmit]

great tutorial. I haven't tried yet, but it appears that in v0.8.0 you can subtract from selection by pressing CTRL key

[@OlliHuttunen78]

Yes. Developer fixed it pretty soon after this video. Which is great. I recommend also test the Super splat service in web. It is very versatile tool for cleaning Gaussiann models.

[@hgatmit]

@OlliHuttunen78 I'v been trying the supersplat after your good review of top editing tools, but as I recall supersplat don't save the spherical harmonics back in the ply/splat. I also noticed a lower fidelity in the models that were previously saved after a "clean-up", so this is why I'm trying the Unity plugin..

[@OlliHuttunen78]

Well. Now the Spherical Harmonic data issue is fixed on super splat. In Unity I opened and used the project file which comes with the code. In there the selection tool button is visible.

[@bryanalban7868]

i cant see the Scene GaussiansSplatts after the Export. I dont know what is the misstake =(

[@RiccaDiego]

Mazing tutorial! But when I try to import the gaussian splat on the tools menu, there is an error that I can't find anywhere on internet: "output folder must be within project, was "C:/User...". Don't matter wat I do, this error keeps comming. I'm using the 2022.3.21f1 version of Unity.

[@arjuneswarrajendran]

Can you share the scanning workflow for the apartment? how did you mange to scan the entire aprtment including roof with minimal artifact, and also about the pattern of your orbit to capture. Thank you

[@bharath2788]

Hi Olli thanks for the video! i'm using 2022.3.7. I can't see the selection tool. Is there something I'm missing?

[@mrlopre]

thank you

[@RobertWildling]

Wonderful! I am really happy to have found your channel! - Question: Isn' there also an Unreal editing tool? I think it is not free, though...

[@OlliHuttunen78]

In fact, Luma AI has just released a version update of their plugin for Unreal. it's free and very fast to render Gaussian splat models.

[@RobertWildling]

@@OlliHuttunen78 But is it possible to EDIT gaussian splatting with that plugin? Would love to see you do a video on that 🙂

[@OlliHuttunen78]

It seems that it has only the cutouts tools and you cannot delete individual splats like in this Unity tool. Perhaps I will make a video about it.

[@Atlas_Redux]

@@OlliHuttunen78 You can't edit in Unreal Engine or Unity itself, but you can use Super-Splat which with SH (spherical harmonic data) enabled output to standardized 3DGS PLY which can be used in UE and Unity with all plugins. Or you can use CloudCompare with 3dgsconverter. You can also try using 3dgsconverter on your incompatible PLYs, as despite it is supposed to be made only to work with CloudCompare and back again, it worked with Super-Splat before it had its own compatibility.

[@jag24x]

This is brilliant! Thanks for sharing! Can you reload this back into sibr viewer? Also have you done any rendering in Unity with a camera path?

[@OlliHuttunen78]

Yes it can be reloaded into Sibr. Just check my previous video about that. There is a one config file trick that need to be done before sibr viewer will open the ply file. I haven’t done any video rendering in Unity but I’m sure it can be done.

[@CyrilPreiss0]

Very interesting ! any way to upload the modified ply to Luma ai ?

[@modme1market]

Hi, thank you so much for your tutorial. I'm getting an error when importing the PLY from LUMA that says: PLY vertex size mismatch, expected 248 but the file has 27. Thi sis happening with all of them. Not sure what to do from here.

[@modme1market]

All fixed now. luma changed the layout

[@alvydasjokubauskas2587]

Me too still getting same errors, maybe I should reupload the file once again?

[@PeeP_Gainz_DEV]

what about flexicubes from Nvidia?

[@aaronjaggars321]

Is there an unreal engine 5 version

[@OlliHuttunen78]

Yes. There is at least 4 different Gaussian Splatting plugins for Unreal 5, but none of them has this neat selection tool to clean artifacts. But most of them has cut out boxes. Check my other videos where I go through them. For example this one: kzbin.info/www/bejne/b6G6fKGvrJxngrssi=sW7o7NF_2BAQo3fD

[@randfur]

Very sad that the interfaces are so unsuitable for common uses.

[@chiliu9967]

yeah I found it really look good in Unity but can't be exported as obj or add interaction to it like normal objects in unity.. can it?

[@greenstonegecko]

this is the worst that this software will ever be. It will only get better from here