Point Cloud Visualization + VFX (Unity)

OAK For Unity
🎆This week time for second big shout out to Keijiro Takahashi (Unity Japan)
✔ Few weeks ago I showed initial work on point cloud visualization inside OAK For Unity (OAK Unity plugin)
🔥 Now he released PoC working with OAK and let us to include all this work inside OAK For Unity 😊
👉So working on basic streams with support for point cloud and VFX
📢 Unity developers/artists wanted to develop visual arts demos? Please follow/star OAK For Unity repo and join beta testing 🔥github.com/lux...
Keijiro repo: github.com/kei...

[@stenfeio]

Would be great to get a tutorial on setting something like this up.

[@Luxonis]

Please follow/star the repository: github.com/luxonis/depthai-unity will push soon demo scenes for basic streams and point cloud vfx

[@amartious2123]

0:42 the man moves his arm and it looks like the arm's shadow is cast on his chest. Is it really a shadow or just points that disappear? If it's really a shadow, does it mean your point cloud is affected by lighting like a classic mesh object would? Or is it something else ?

[@tiagobrizolara]

It's a shadow, but in the context of the video capture. It's not a virtual shadow cast by a virtual light. This is because the 3d points are calculated from the estimation of the distances of the points in the image to the sensor (I see that they are using their own DepthAI platform to estimate depth directly from video. But it can be done with dedicated hardware like a Kinect, a Lidar, etc). So, we have what we call a depth video, in which the values of the pixels represent a distance (depth). Then we convert this pixels to 3d points in space departing from their x,y in the image and their distance value. So, if the arm stands in between the chest and the sensor, the depth captured will be the closest one, and so we'll have a shadow of the arm projected in the chest - no 3d points in the chest can be calculated at this very moment. If you are not familiar, searching for kinect depth map and kinect point cloud may clarify🖖