3D Gaussian Splatting + Insta360 RS 1" - Impasse Saint-Eustache, Paris

gr...
linktr.ee/grad...

[@ゾカリクゾ]

Wow, this could be huge for gaming or interactive experiences in general. Lots of artistic and overall potential.

[@danilluzin]

If the world has to be static and lighting cannot change i can't see how this is useful for an interactive experience

[@mgntstr]

yeah taka a million high def photos of an alley to get janky simulacra

[@wladefant]

@@danilluzin under interactive I mean for example museums where you can walk around in your pace and look at objects, which is much more interactive than a 360 camera

[@brazni]

It is more on the order of 10's of pictures @@mgntstr

[@khanch.6807]

​@@danilluzin for generating world map and spaces. Everything dynamics can be generated separately.

[@pixelreconstruct]

This is ridiculously good.

[@RikMaxSpeed]

The last 5 seconds where it breaks down into 3D gaussian are fascinating. Can you create new special effects with this?

[@simontist]

Truman Show vibes

[@Jowanoofy_ZO]

Check the complex game

[@whatsappvideos9665]

Mesmerising

[@Powertampa]

Reminds me of a movie where they said they used all the cameras from a cctv setup across the entire city to look into the past when in reality it was an actual time machine of sorts. Can't remember the name. Though if you imagine using all the cameras in London to record and build up not just stills, but full timelines of images this way you could experience the city as if you were walking through it in real time. Which has zero practical application

[@alkaponas]

It resonates with some spiritual experiences :)

[@juizel8008]

drone + Insta360 + 3D Gaussian Splatting = real VR world

[@liuliu-ec5gf]

你好，可以和你交流一下吗

[@gaussiansplatsss]

What gaussian splatting technique is used here?

[@MuteObserver]

Stunning

[@chinaLH007]

What I want to know is whether your 360 - degree images are obtained by frame extraction from the original dual - lens 190 - degree fisheye cameras, or by splitting and extracting 6 frames at random 120 - degree intervals from a 360 - degree view. As far as I know, original dual - fisheye cameras can be calibrated. If 6 frames are split from a 360 - degree view, there will be camera distortion that cannot be eliminated, so the quality won't be very good. Could you tell me whether you use independent fisheye cameras for frame extraction?

[@woodenmonkeyfox]

At the end it's like walking away from a dream where the invented part is blurrier than the memory part

[@_boux]

Google maps is gonna be wild pretty soon

[@melol69]

no doubt that google will use their supercomputers to make 3d gaussian splatting google street view

[@TheBoeboe]

@@melol69 we would still need commputers to be able to run it, and immm also guessing this takes up a lot of space

[@Setupthemabomb]

Give or take 10 years, when ordinary computers could move data few terabyte per seconds we'll get nice game made using this method

[@PySnek]

​​@@Setupthemabombthe 5090 will be able to push > 1 TB/s, some OC versions of 4090s already reach 1000 GB/s of bandwith. It's complete nuts

[@relaxvideo3d]

possible to watch this in VR? And look around? How?

[@dialectricStudios]

Been wanting to try out this Gaussian splatting thing, seems like the real deal for real time + high quality. How are you liking it?

[@theonlyartist1992]

google maps gonna be lit

[@gaussiansplatsss]

Is it captured video or photos and if photos, how many 360 images

[@ahmadhasan3258]

bro the reflections!!!!!

[@EmixMotoTech]

Hi, Beautiful video!! I try to use insta360 for Luma Ai, with Insta360Studio for reframing, what is the LENS ANGLE setting for best results with nerf3D? wide, ultra wide, linear, flat or others? Thank You

[@george6ix9ine20]

can I order a video from you? I need a video shot from the center of Paris on a 360 camera in the morning (before people arrive) how many euros could this cost? haha, sorry for my impudence, but I'm serious

[@ian_watts]

Did you record a video then extract the frames or take a bunch of photos?

[@stickmanunivers5si9d]

excuse me，how there is not a hole ，how did you capture the entire scene

[@martinhotinchan]

How would you turn this into a typical real case use? Like run this as a game on your phone? I would imagine using browser based “game” or maybe an app?

[@askeladden450]

Can this handle dynamic/animated objects at the moment? Also, can it be used to create fictional worlds?

[@Moshugaani]

This rendering technique is just taking its first baby steps. At the moment it seems like people are mainly using it to render environmental scans that are obtained with photos or video. But I suppose there's no reason why a handcrafted high fidely 3D environment couldn't be converted into data that can then be renderes with gaussian splatting. The issue with it for interactive applications is the static nature of it. I haven't seen anyone use dynamic lighting or reflections with this tech. These photo scanned environments have "baked" lighting that just renders everything seen in the photos, and reflective surfaces are rendered as just space like looking through a window. And I've seen no one do moving parts in the environment. At the moment I supposed that any interactive application would need to mix different techniques. Reflective surfaces would need to be done with common methods and moving objects would need to be made from polygons. But all of these issues will probably be solved relatively quickly!

[@rafox19]

How computational expensive was this? Is a 3080TI good enough to calculate something similar in a reasonable amount of time?

[@historyphotogrammetry2218]

While much less expansive and striking than the remarkable demonstrations here, I found it was reasonable on an 11700 (non-k) and 3060 12GB (from less than an hour to maybe six hours, depending on settings and total number of images), e.g. with some Sony Xperia 1 (II) imagery, kzbin.info/www/bejne/j3nYc5yIba6bfZI

[@jimj2683]

Why doesn't Google Street View do this? It is so cool

[@theneonbop]

it runs very slowly, requires orders of magnitude more images than they already have, and doesn't really work on a large scale.

[@baglayan]

They will, in a few years.

[@fqdn]

It's not as advanced as the demo, but Apple's "Look Around" functionality has smooth transitions, much nicer than Street View. I read somewhere that their cameras are equipped with LIDAR, so they have depth info to work with, Google also has some depth info I think, but it's not as advanced.

[@jimj2683]

@@theneonbop Check out their latest paper on SMERF. It is supposed to be much more efficient.

[@talkingape86]

I am trying to recreate the space with the same method and equipment(insta rs one), but my result is fuzzy and messy compared to yours. Do you have any tips or pointers to reduce the noise? I would appreciate ANYTHING. Thanks in advance. 😢

[@captteemo9133]

Maybe try denoise with Neat Video before

[@360_SA]

great work is this done in windows or unix and can you help the step to make

[@TheKronos1011]

Did you need special pre-processing for the images? I did the tutorial linked on the gaussian-splatting github yesterday and it said the images need to be converted to pinhole style cameras. is this done automatically by the convert script?

[@gwentarinokripperinolkjdsf683]

If I had to guess it just means the images shouldn't have too much lens distortion

[@prizmbreaker]

The basically is basically gonna be real dude

[@pierrelemoine6766]

Let's make IA merge this technique with all the data possible and find all the disappeared

[@matlarasta]

That's crazy, do you guys think it's possible to put it in a game like UE5 ?

[@loonywalker6729]

yes it is possible. but you need data first.

[@somethingtojenga]

I mean, this is nowhere NEAR real-time rendering... to say that a process of scanning replaces tracing light paths when you can see that the shadows are baked INTO the surfaces, is ridiculous. Do this at night while ambulances are racing down the street and a fire is burning in a trash can near a wall drenched in rain lol it's not even capable of ANY of that

[@Kjhd9987hy]

very scary world in 5 years.

[@DJVON]

we are in a simulation... one day we will find the singularity... then...