Photogrammetry Mesh vs. Gaussian Splatting point cloud

Comparison of the same data set processed with photogrammetry and Gaussian splatting.
Data: 110 images, 42mpx
Photogrammetry: 6million polygons, 10 4096 textures
Gaussian Splatting: 30k iterations, 172k points, 7 million splats

[@realthing2158]

Wow those gaussian splatting results are amazing and your recording quality is very smooth. I like the background audio ambiance as well.

[@DJ-Illuminate]

Maybe make a point splat mesh like this: Direct Mapping: Each splat directly corresponds to a point (or set of points) in the mesh. As the splat undergoes any transformation, the mesh point follows suit. If the splat moves to the left, the point moves to the left. If the splat grows in size, perhaps the point-mesh around that splat could expand or become denser. Fluidity in Motion: Imagine the splats as buoys floating on water. As they move, the net (or mesh) attached to them adjusts and moves with them. This creates a fluid, dynamic motion in the mesh that's directly influenced by the movement and behavior of the splats. Temporal Dynamics: If the splat data is changing over time (like in a sequence or animation), the point-mesh would continuously adjust and evolve frame by frame, creating a moving, dynamic representation. Overlapping and Merging: If two splats come close or overlap, the corresponding points in the mesh might also come closer, merge, or influence each other in some manner, depending on the design and algorithm in place. Interactivity: This approach also opens doors for interactivity. A user (like a designer or animator) could directly manipulate a splat and see real-time adjustments in the corresponding mesh, providing a tactile and intuitive way to shape and design 3D structures or animations. In essence, by directly linking the behavior of the splats to the motion and structure of a point-mesh, you can create a dynamic, responsive, and potentially more intuitive system for mesh generation and manipulation. It's a concept that harmoniously blends data representation with visual aesthetics and user interaction.

[@afleast]

The main issue here for me is the 3D model itself. Currently, there is no way (although I believe it will be resolved) to give gaussians appropriate colliders, shading, and responsiveness to lighting.

[@smashing_times]

hey! thank u for video. can u tell please wich soft did u use for splatting?

[@MatthewBrennan]

repo-sam.inria.fr/fungraph/3d-gaussian-splatting/

[@haikeye1425]

Great!

[@tazztone]

would there be a way to clean up those floating splats sometimes blocking the view ?

[@MatthewBrennan]

Yes - the current version of the unity project by Aras-p has the ability to use volumes to "hide"/cut the floating splats

[@dimaananyev5336]

Hello, colleague! I'm quite impressed with the results you're achieving, and I subscribed to your channel some time ago, which I'm very pleased about. In several of your videos, you mentioned that you have a large database on classical photogrammetry, and now you plan to use the captured materials for visualizations in Gaussian Splatting technology. You're a pro, and I really appreciate what you're doing. I find your comparison videos fascinating and very informative. Thank you so much for your hard work! If you find the opportunity to reply, I would be very grateful. You mentioned that you use Agisoft Metashape in your work and transfer camera location information from there to a file that can be further used by Nerf for data processing. Could you explain how to transfer camera position data from Agisoft Metashape to a service file? Currently, developers are using a very slow and inefficient tool, Colmap. Yes, it's free, but it seems to have its limitations. Half of my reasonably good photogrammetry sets are not understandable to Colmap; it loses camera positions, making it impossible to work with them further.

[@MatthewBrennan]

You can export the camera positions from Agisoft as an XML file. Then I’ve been using NeRF studio for nerfs, which can interpret the Agisoft file. For Gaussian splatting there is a script I’ve been using that will export Agisoft in COLMAP format: github.com/agisoft-llc/metashape-scripts/blob/master/src/export_for_gaussian_splatting.py

[@dimaananyev5336]

@@MatthewBrennan Thank you very much for your answer, I will try the path you suggested!

[@trollenz]

​@@MatthewBrennanthanks for that one 👌🏻

[@lunabeige]

can you convert gaussian to a 3d model / mesh?

[@MatthewBrennan]

In theory yes, because the Gaussian "splat" file is a PLY point cloud with splat information. However, you would lose that color/viewing angle information when you meshed it, and likely be stuck with vertex coloring, or have to create a texture from the imagery.

[@dibbidydoo4318]

@@MatthewBrennancan't you combine the color information from gaussian splatting with the mesh information created from a separate algorithm?

[@MatthewBrennan]

@dibbidydoo4318 yes - there's something called "SuGaR" (github.com/Anttwo/SuGaR ) that aligns the gaussians to a mesh, I haven't tried it yet though.

[@dibbidydoo4318]

@@MatthewBrennan thanks!