Virtual Windows in Passthrough Mode | Selective Passthrough Shader & Stencil Shader

Hi XR Developers! In this video we will learn how you can easily create these amazing virtual windows inside your room, like we have seen from many viral videos lately, like Ocean Rift! If you want to take this experience one step further and smash realistic wholes into your walls and ceiling, check out my exclusive tutorial on Patreon, on how to create your own “First Encounters” Meta Quest experience!
💥 [EXCLUSIVE] Mesh Destruction System - Recreating 'First Encounters' by Meta!: / exclusive-mesh-99965227
Resources:
🔗 ShaderLab command Stencil: docs.unity3d.com/Manual/SL-St...
🔗 Explore your own aquarium in 360 for VR!: • 【360°VR】Explore your o...
🔗 ShaderLab Culling & Depth Testing: docs.unity3d.com/2019.1/Docum...
🔗 Selective Passthrough Sample: developer.oculus.com/document...
🔗 Vintage Ship Portholes Type A: sketchfab.com/3d-models/vinta...
Other Videos:
📹 Mixed Reality Utility Kit: Build spatially-aware apps with Meta XR SDK: • Mixed Reality Utility ...
📹 Meta’s Building Blocks | Develop Mixed Reality Apps lightning fast: • Meta’s Building Blocks...
📹 How to use Lights & Shadows in Passthrough Mode: • How to use Lights & Sh...
📹 Get Started with Meta Quest Development in Unity: • Get Started with Meta ...
📹 Meta Quest Scene API: • Meta Quest Scene API |...
📹 Meta XR Simulator: • Meta XR Simulator | Sy...
Join the Discord!
👾 / discord
Support the Channel:
🫂 Patreon: / blackwhalestudio
🐦 Twitter: / xrdevrob
🤝 Join this channel to get access to perks:
/ @blackwhalestudio
Need professional XR & Multiplayer development for your project?
👨‍💻 Get in touch with us: bento.me/blackwhale
Chapters:
0:00 Intro
0:48 Project Setup
1:26 Create Video Skybox
2:30 Scene Setup
3:11 Passthrough Shader
3:54 Stencil Shader
6:10 Setup Render Object in URP Render Settings
7:00 Script to Apply Layers to Walls
8:16 Create Window Prefab
9:06 Script to position Windows
11:04 Outro

[@leynenslucker2991]

Damn, you can't stop giving! Keep going!!

[@blackwhalestudio]

Thanks! I'll keep trying my best! What would you like to see next?

[@NathanPuozzo]

You deserve way more views ! You rock !

[@blackwhalestudio]

Thank you so much! Honestly, it's not about the views, I'm quite happy already. It's just great to see reactions of people when they save time or manage to do something thanks to my videos! :)

[@JasonSGameDev]

Excellent Tutorial. I've been looking forward to more information on this. With some experiments on top, I learnt if you want a raycast to shoot through the wall and hit objects behind you will have to make a collider for the window object, the one that lets you see through the wall, big enough that it is thcker then the wall, and when the ray hits it has to create a new raycast to continue in the previous cast's trajectory (I used a ~Wall layermask, to avoid the ray from picking up a hit from the wall layer, on the new "continuation ray"). My only issue is with now using Canvas layers, and other alpha effects, all seem to be rendered behind my walls until the walls are blasted away and all there is, is the 3d world behind. My guess it has to do with sorting layers? ** update to that question. I figured out that if you replace the materials for your UI elements to be Opaque materials, then you will not have that issue. Transparent and Alpha textures are drawn from back to front. So, for all your effects and UI elements that are happening inside your living space, you might want to use Opaque materials so that they are drawn in front.

[@blackwhalestudio]

Thanks so much for all this info, im sure it will help a bunch of people!

[@rogerpingleton9199]

First of all THANK YOU! This is so cool! Second. I am having an issue that is preventing me from implementing this. When I get to the 3:42 point where I should be able to see just the passthrough to the walls, I see passthrough in my left eye and the skybox in my right eye while using the link to test. Additionally I get the "...it's required to do a runtime permission request" issue, and when I hit fix, it just comes back, and the OVR manager never asks permission. Any ideas. Again thank you for all your videos. They are so amazing and so helpful!

[@wallawAllwlla]

I bumped into the same issues, but I just kept following up the tutorial, and the final outcome works as normal (build to the device, unplug the quest link, and run on the device alone)

[@blackwhalestudio]

Yes unfortunately it is normal to see two different things on two eyes. I am not entirely sure what this is caused by. Is there still an issue you are facing?

[@grimsbaldruk4014]

Hello ! Currently facing this problem in Editor and can't seems to make it work on standalone headset (Passthrough not showing at all). Did you encountered the same issue, if so did you manage to fix it ?

[@anmaavr8315]

Great tutorial! Can’t wait to try it! I have a question. Do you know if it’s possible to have multiplayer option to the passthrough mode? Can I have it other people join (remotely not in local) when I am in the MR environment?

[@blackwhalestudio]

Technically yes, the passthrough shouldn't have any impact on your multiplayer features!

[@LucasHorgan-go5ln]

Great Tutorial man, very straight forward and easy to understand ! Keep up the good content ! I have a question is this functionality currently only doable using the OVR toolkit ? Or is there a way to do it with OpenXR aswell ? The MRUK has to be used in connection with the OVR set up I guess ?

[@blackwhalestudio]

Technically you can just use the same shaders with OpenXR. But yes, to use MRUK you have to use OVR!

[@LucasHorgan-go5ln]

I ended up taking a different approach BUT your tutorial is the only one I saw on youtube which mentioned checking for the renderer in ProjectSettings ->Quality, I was stuck for days with it not working on the Quest but now it does... So thank you very much this really is a helpful tutorial !@@blackwhalestudio

[@mrbanana101]

Amazing tutorial, thanks so much! I was able to successfully get it up and running. Though, do you know if there's any way to enable rendering of transparent objects with alpha (materials that aren't completely opaque) in passthrough? Another commentor below said that they had to just not use alpha materials, but for the game I'm developing, the particular art style takes a bit of a hit if I can't render alpha materials in passthrough mode. Thanks!

[@blackwhalestudio]

Can you try setting the render queue to 2999?

[@mrbanana101]

​@@blackwhalestudio thanks for the quick reply! Almost there - I set the vr passthrough material's render queue to 2999, and alpha works now in passthrough, thanks so much! But unfortunately, the stencil shader no longer seems to take effect, so it's not cutting out to display the VR world behind it. I have the stencil material at render queue of 3000, so I attempted setting it higher without any luck, I tried adjusting the shader and material settings for a bit with no success yet

[@kylehentschel4858]

great tutorial. this method has a lot of flexibility. But I am stuck, the default UI material is rendered underneath the passthrough material. any tips to fix this?

[@blackwhalestudio]

Have you tried adjusting the render queue?

[@Filmblanc]

How can I get particles to render in my actual space and not as a part of the "outside" skybox rendered world?

[@blackwhalestudio]

MRUK allows you to spawn things within your actual space, check the MRUK API

[@user-id5rx4xu8m]

Thanks for this, is there a version that doesn't use URP? What would I need to change to make it work in core?

[@blackwhalestudio]

URP is required unfortunately

[@user-id5rx4xu8m]

Ok I achieved it in Core but it took a lot of experimentation :)

[@blackwhalestudio]

@@user-id5rx4xu8m Great! How did you do it?

[@user-id5rx4xu8m]

I'll post to your Discord channel when I have the time.

[@yoyopei3793]

@@user-id5rx4xu8m Look forward! I use built-in too

[@mdelight31]

Very interesting tutorial! Thanks heaps! My only problem is that the raycast hits much closer from the user than the actual room wall thus placing the window offseted from the wall instead of flush to it... Any idea ?

[@blackwhalestudio]

In the editor or on your device?

[@mdelight31]

⁠thanks for the quick reply! In the device, and also realized it was not a problem with the raycast but more that my room was all offseted and rotated weirdly…