RealityCapture to Gaussian Splatting using PostShot

[@PatrickHogenboom]

Massive thanks for sharing! I haven't kept up with gaussian splatting and this brings me up to speed and gets me fired up at the same time 🙏

[@lilianplume2809]

thank you for your tutorial I have finally understood why i couldn't do it

[@fuffilwl]

Thank you, this greatly improved our results and made the process faster!

[@DownhomeTraces]

At 7:01 you say "If you want to use the mesh from photogrammetry but the visuals of gaussian splatting, importing them and having them be aligned at the right scale is trivial because they're based on the same alignment data, so that to me is a massive benefit." Sorry if this is a dumb question, but are you just talking about how RC is useful for alignment, or can you actually combine the mesh from RC and the splat from Postshot in some other way? Is there a way to combine them beyond alignment data?

[@AzadBalabanian]

certainly. You can combine them in engine and create a seemly blend of the two. See: kzbin.info/www/bejne/Y5SkfWywhqyFjKc

[@obseraft]

Thanks Azad! You are always ahead in digital twins!

[@vrguytokyo]

Never heard of this app. Miss seeing your posts on twitter (since I quit it a while ago) but will certainly check out your blog more often!

[@DurgaPrasad-ks9em]

How did you shoot and import it to RC please let me know 🙏.

[@OlliHuttunen78]

This is really good stuff. I've been trying to get a decent point cloud built in Reality capture but it always creates so many different components. Do the components have to be merged together before the PLY file is exported from Reality Capture or how is it done? I tried to look at the settings in your video and it looked like you had many separate components too.

[@AzadBalabanian]

Correct, everything needs to be merged into a single component for things to be aligned properly. In terms of your work (combining multiple cameras or views from a 360 camera), there are some tricks to making sure everything aligns like, making sure all cameras "see" the same feature at the start/end of a loop. You could even print out AprilTags that will ensure RC will recognize them as shared features across the cameras: wizardofaz.medium.com/how-to-use-my-3d-printable-apriltag-scale-markers-for-realitycapture-6e3244da050e I have a few more tricks listed out here: wizardofaz.medium.com/reality-capture-alignment-tips-fixes-d49371ee6643

[@olgaforce6678]

Excellent! I was stuck on exactly what/how to export for import into Postshot.

[@AetherScout]

Helpful??? - I's fkn game changer for me man. you have solved a MASSIVE issue I had. Thanks, Top man!!!

[@gussanchez-perez5477]

Pretty cool workflow. Thanks for sharing.

[@gelisob]

thanks for the video, got my little scene to unreal too after this workflow. Any tips on model or settings for export/import, to have scene appear normal in VR? I imported to ue5 vr template but in vr-review, the splatting model seems to be non-stereo, something is off.. feels bad when looking with both eyes.

[@vrbased]

are you using the Apple Vision pro in this? the headshot is sick brother

[@AzadBalabanian]

Indeed!

[@Alec.Jerome]

when you say its trivial to have the same coordinate data for photogrammetry and g splats what do you mean? do you have the option to export either one once you import all your clips?

[@RiniKunduInteriors]

You mention that you need to export registration and point cloud files to the same folder as your images. I usually have my drone images in a separate folder to handheld images following a component workflow in RC and then merge components. Is there a way to make this workable with your process? Thanks

[@AzadBalabanian]

Currently, I don’t believe so (and it’s an annoyance for me as well for org purposes)

[@PrinceBrosnan]

I would like to do PostShot to RealityCpature with my original video RC failed and built a broken model PS did great and I have a very good quality gaussian. ply I would like to convert it into a mesh inside the RC Is there a way to do it? Thank you for your videos

[@AlexRanedo]

Hery, cheers for the tutorial, it seems a great workflow. However I am trying to import a RC export but with a lot of images, 2350 including the .csv and .ply. But the moment I hit import it freezes and crashes. My specs: I9 121900K/RTX 40490/128GB DDR5 Ram. I also tried just running the GS render using the import of the images but after 24 hours and it at 65% in step 3/4, I shut it down since that is simply to slow. Is there perhaps a limit on images used? Also I'm not sure if this would affect it but not all images in the folder (for the Postshot import), were registered in RC. I imagine that's what the registration file is for though possibly. If you can help it'd be greatly appreciated ;)

[@AlexRanedo]

I was going to delete this but in case someone encounters the same issue. I just now dragged the contents of the folder (images and .csv and .ply), instead of dragging in the folder itself and it seems to have worked. 2 minutes in at 5%...

[@arifyakupogullari691]

Hey man, is it possible to use this without PostShot. I am trying to use the imagine alignment of RealityCapture and make it work in Gaussian Splatting in the same time.

[@realitycapturehk-danny915]

Thanks for your excellent tutorial! Will you share this dataset sample in youtube for downloading?

[@jakemartinez8580]

Thank you for the scripts , i have managed to make a python Gui that make it easy to point to the directory and it makes the scripts and has a run button. The only issue i am having is I don't know what the command to toggle in postshot the store training context so training can be resumed if needed. Can share python script if you want. Thanks again for any assistance

[@srdrazic]

Failed to import files: PlyReader: unexpected table size? what is this mean?

[@Grigoriy360]

Great tutorial, like always! Did not used PostShot a couple weeks, I see they added new option " Splat MCMC" - does it work better compare to just "Splat" ? Thank you

[@AzadBalabanian]

I believe that’s now the default splat model. There’s another model that focuses on density. Haven’t tried it much though.

[@Grigoriy360]

@@AzadBalabanian Thank you

[@henry-louisguillaume532]

Really cool tutorial and thks for the script, but I tried with nef files (Nikon Raw) and postshot don't recognize the format (need to try with jpg)

[@jorcher]

thanks, awesome tutorial!

[@fis8640]

Thanks, this is great! Love your avatar :D

[@AetherScout]

I don't suppose you have a tut on how you use control points to create real world relative accuracy?

[@AzadBalabanian]

Here ya go ;) wizardofaz.medium.com/setting-1-1-scale-in-realitycapture-aadeaafc3737

[@xorgeai]

Thank you, excellent content!

[@markinmkn]

Why will you go on RC first if you can train the photos directly on PostShot? Can you help me understand, on RC you will need to create the model, texture, unwrap, and export, it's a large workflow just to get a 3DGS.

[@AzadBalabanian]

No, you don't need to create the model, unwrap, texture, etc. You just need to align and export. The reason you'd do that is because RC's alignment is much faster than PostShot's COLMAP. Watch the intro again.

[@markinmkn]

@@AzadBalabanian Thank you for your help, did you already try that with laser scan data inside RC?

[@AzadBalabanian]

@@markinmkn I dont understand. Yes, you can process LiDAR data in RC. No, I don't think they'll work in PostShot for gaussian splatting

[@markinmkn]

@@AzadBalabanian I tried your process on Postshot just with Alignment and the photos, after a while on Postshot I received the message "Reach the maximum 32k' but the Gaussian result is ugly. I think the process isn't finish. I have a 4090 don't think the error is about Vram

[@AzadBalabanian]

@@markinmkn I imagine it's due to the accuracy of the alignment isn't very high? Try using the default "compute from images" setting in PostShot instead of importing RC alignment

[@大支爺]

It won't work while the target is in actions from a single camera.

[@张超-l2e]

按流程做了一遍，到postshot提示，plyreader：unexpected table size

[@ChrizLizt]

Veery cool, thank you

[@ChrizLizt]

Yes please, I subscribed and am super keen to see more videos. Your shortcuts are lightning as well.

[@hexagonai212]

tq bro share

also RC alignment success is like 10x lower than Agisoft

[@AzadBalabanian]

Depends on the data. However, I agree that Metashape can have a more robust alignment than RC's sometimes. There is also a PostShot workflow for Metashape as well. But RC is my preferred software and is now free to use

[@jinggao-c9s]

I have 100% CPU usage and 0% GPU usage in RealityCapture1.4 alignment, here's why.