Understanding 4D Gaussian Splatting: A Practical Overview

[@midzasaur]

been following you for a while now and have been waiting for something specifically about this. THANK YOU OLLI! cant wait for more

[@leono83]

Amazing work! You are definitely sailing into the future.

[@MrCranebay]

Thank you! You are doing amazing work with these and all the tutorials and tips are so to the point!

[@DavidHeath-u8m]

Ollie, great job staying ahead of the curve! Very good video, exciting future for 3DGS and 4DGS!

[@buroachenbach703]

As always excellent,, video. The camera array tool is a great service to the community, and the explanations in the videos are always very clear and helpful. Can’t wait to test this workflow.

[@mitroer]

your videos are as always good storytelling about a technical subject. Dynamic splats are interesting but for real word capture it is a challenge. I wonder which camera has the best “just enough” performance and the lowest price point - so we can buy 20 of them. Someone needs to take up this niche market and also implement some reliable synchronization of capture.

[@IVANLLUNELL]

Spectacular video. Thank you very much for the effort in explaining how the 4dgs works. I am experimenting with Trellis, a Gaussian Splat generator through images, post-shot, and then viewing it in MetaQuest via Gracia VR. I am eager to try it.

[@reageleo744]

You're a legend Olli 👏

[@foxy2348]

Amazing content

[@hugocalder7982]

you are the goat

[@Hebrideanphotography]

Fantastic. 👏👏👏

[@rafalj83]

@OlliHuttunen78 Interesting fact is that you can export camera colmap data and point cloud only for one frame and just exchange images while training in postshot.

[@OlliHuttunen78]

Really? Interesting! How do you do that? How you change the images during the training?

[@rafalj83]

@@OlliHuttunen78 No, images need to be exchanged prior to importing them into PostShot. For example, you can export images for frame 1, along with the PLY point cloud and COLMAP camera poses. Then, you can copy images for, say, frame 10 into the images folder. If they are named in the same manner and imported into PostShot, you should be able to train successfully on frame 10, even if your PLY point cloud does not match. I'm not sure if this has any practical use, and I'm surprised it actually works.

[@ClemKlootShoops]

Love your videos. Legend

[@oke_doke2894]

I thought for sure that Gracia was going to get a mention here. It's rough but it looks like they're on the way to the sort of compression you mentioned.

[@OlliHuttunen78]

Yes! Gracia is interesting VR solution.

[@r.m8146]

Next level

[@UnrealPixelation22222]

Digital Gold 💙💯🙌

[@waynegrabowski3180]

Awesome, thank you. Now how can I get enough cameras to do this with real video....

[@omankz]

thank you!

[@urkoVR]

Thank you very much for sharing your extensive knowledge. My question is, what is the point of using synthetic animation with Gaussian splatting when traditional poligonal animation is already visible with any game engine like Unreal Engine or Unity?

[@OlliHuttunen78]

Yes. I’ve been thinking about that point also. Currently there are not much use to generate dynamic Gaussian Splatting for games since polygon solutions can do it so much better and lighter at the moment. I can only see that these gaussians could come usable on simulations like smoke or explotions or something that is very heavy to render with traditional methods.

[@IVANLLUNELL]

Spectacular video. Thank you very much for the effort in explaining how the 4dgs works. I am experimenting with Trellis, a Gaussian Splat generator through images, post-shot, and then viewing it in MetaQuest via Gracia VR. I am eager to try it.

[@urkoVR]

@@OlliHuttunen78 thank you Olli!

[@reza2kn]

very cool! but how do the 18 generated PLY files of animated Batman compare in size / memory usage to the 22 PLY files of the plane that were 300MB each? 👀

[@OlliHuttunen78]

True! That's a good question. I forgot to provide a file size comparison. Since the Batman run loop was created using the ADC profile, the PLY files are comparatively much lighter. approx. 7Mb per PLY file. If you want to download the animation sequence from this sample package, you can get it here: nappiboxi.net/downloads/BatmanRunloop_4DGS.zip

[@reza2kn]

@@OlliHuttunen78 Wow! awesome thanks!

[@DavePorkins]

Is there also a way to automate the training process of regular Splats? I have a bunch of Scans that piled up and I'd love to just let PS do it's thing while I'm at work. Could you point me to some info on how to use the command line or to scripts I could use? :)

[@OlliHuttunen78]

Sure the batch Trainer which I made works also for regular images. It just takes longer time to train them because postshot needs to make a camera tracking phase to them. Just put your images to numbered folders and the batch trainer will automate this task. But be aware also that error issue which I mentioned in video. Postshot is not that stable yet. It can crash during the training. So you at least need to check out the process on time to time.

[@DavePorkins]

@OlliHuttunen78 I see, thank you for your reply :) This works well if you already have the image sets and only want a .ply file in the end. I was more interested in using video as an input and I would like to get a .psht file (with "Store Training Context" enabled). Is this possible with a script?

[@OlliHuttunen78]

Yes. This batch trainer of mine is designed to export PLY sequeces. But you can save Postshot own native format .PSHT trough the command line also. Just check out the dokumentation in Postshot page here: www.jawset.com/docs/d/Postshot+User+Guide/Command-line+Interface

[@codyosullivan6830]

Hi Olli - I'm wondering if it would be possible to use the output data of the training from each frame as the starting point for the training of the subsequent frame, considering that the points in the cloud are persistent? That way every frame after the first would require fewer training iterations.

[@OlliHuttunen78]

I understand the idea and such a technique would certainly be exactly what the 4DGS compression method is intended to be. In this sequence technique, however, all the Gaussians are new and the model is completely retrained in each frame. And that is currently the only way to produce these PLY sequences for regular Postshot users at least. This thing will certainly develop and change soon and we will get a similar process as in video compression and only moving Gaussians are changed. Those that are in place will stay in place. Thus, the size of the Dynamic Gaussian Splatting file will certainly decrease as soon as such a format is developed.

[@maratkhafizov07]

Hi! I bought the camera array tool version, but it doesn't have animation data. is it not available for purchase yet?

[@OlliHuttunen78]

Like said in video it is a upcoming feature. The update where this 4DGS feature will be is coming soon. If you already bought this add-on you will get the update for free when it is ready.

[@maratkhafizov07]

@@OlliHuttunen78 thank you

[@胡炳扬]

cool

[@xpez9694]

NEXT LEVEL...but alson next level hassle! LOL

[@MyKiruha]

Лайк неглядя

[@MyKiruha]

Лайк неглядя