NVIDIA Research showcases QUEEN - An AI Model for Free Viewpoint Streaming

[@henriksundt7148]

To add some context: These videos are shot with multiple cameras and combined into a "3D" video-stream, which in turn can be rendered from an arbitrary viewpoint.

[@4.0.4]

That is a lot less magical and a bit more practical of an explanation, thanks.

[@divyarajsinhrana6397]

localization might be used to streamline these applications.

[@haseebasif100]

I feel like more explanation is needed for how it works.

[@pandoraeeris7860]

What exactly is this?

[@user-yh8ce7tg8l]

SLR has entered the chat...

[@sekinVR]

Wow 👏👏👏👏👏

[@sansmojo]

YASS QUEEN

[@alexbasic9776]

dammn Nvidia, congrats guys

[@ElectronaMusic]

good

[@SoulAir]

so its making stuff up? doesnt count as seeing the other angles IMO

[@MFSCraft]

But you are able to see the video at different vantage point, at the expense of seeing false/generated scenes in places where there is no available information. The goal of the tech is to take one or more 2D videos at slightly different angles and produce 3D videos and real-time playback at 350 fps. Useful for VR/AR.

[@AalapShah12297]

To be honest it is not very clear what they are doing. Based on the blog it seems like this is using a multi-camera set up. But AI-reconstruction is still needed for interpolation at angles between the cameras. And what they are saying is not that they achieved this reconstruction (it was already acheived) but that they reduced the bandwidth required for these kinds of livestreams.

[@4.0.4]

I'm sorry, who is the target audience of this video? It's clearly not developers. Probably not the layman audience either.

[@StiekemeHenk]

Researchers.

[@edward_fox]

Braindance is coming

[@brandoncramer8400]

Is this capable with just one camera lens?

[@notthatrkelly]

It is not. It's like NERF or gaussian splatting for dynamic 4D capture.