Setting Up Meta XR Passthrough For AR in Unreal Engine

[@esphyxia]

THIS IS WHAT IVE BEEN LOOKING FOR. THANK YOU!

[@GDXR]

I have a 13 part tutorial series on my patreon walking you through almost the whole proccess for setting up a mixed reality game.

[@thiagogold6896]

IF YOUR VR PREVIEW STOPS WORKING ONCE YOU ENABLE METAXR. He forgot to mention this in the tutorial but you need to disable the openXR plugin if you want the metaXR to work.

[@somecheesychips0529]

Thank you so much! I was beginning to think that I was the only person in the world with this problem

[@BoopDood]

Great video, big pain that Meta broke OpenXR compatibility till UE 5.4 releases

[@jyashi1]

I still cant see the passthrough. On VR template i can only see a black screen and floor after taking off walls Also does anyone know why after enabling Meta XR plugin the VR preview stops going in VR mode when using from Oculus Link desktop mode?

[@RonnyBTokin]

project settings > meta xr > launch meta XR project setup tool then clear up the errors

[@TheTgolProductions]

He forgot to mention this in the tutorial but its because you need to disable the openxr plugin to use the metaxr plugin. I had the same issue

[@cenavirtual]

@@TheTgolProductions but when disable open XR the unreal project not open more. What i can do ?

[@TheTgolProductions]

@@cenavirtual i didn't run into that issue, I would try making another project from scratch and trying again

[@Khudiakov]

I would only add that you need to disable OpenXR when MetaXR is on. That was very surprising to find it out

[@calvinwayne3017]

i only got it to work by disabling OpenXR plugins (dismiss the error missing after restart dont enable) if that helps anyone, but idk what that broke disabling it, before everything was black

[@alexjamesbilodeau]

This is still a requirement. Bumping.

[@yunaseo7805]

thank you!!

[@Khudiakov]

Thank you so much! Great content. Helped me a lot!

[@V4C38]

Thanks for this explanation - got it to work! :)

[@maximkrioukov4597]

For MAC users. Meta XR ist not working with UE5.4 yet. Use the integrated VR Add-on instead. Otherwise your app would crash immediately after starting on oculus quest.

[@GDXR]

Hey, unfortunatly Meta is not supported on a MAC. It requires windows to show up.

[@saemranian]

Thank you so much! It was so useful.

[@ozymandias3735]

It works perfectly in UE 5.3, but when I try it in 5.4 it doesn't work. You can only see the external camera, any solution?

[@mtc]

followed everything, but doesn't seem to work on 5.2 :(

[@DanSmith-cz4oy]

Thanks for the tutorial mate! Do we want this running along side OpenXR plugins? Or should we turn those off to avoid conflicts?

[@libortelupil7647]

Hi, I have a little problem with that - the passthrough is visible only for like 1 second, then it gets black and stays black. Any idea why? I can still see 3D objects that are in the scene, only the passthrough background is black. The trick with putting the headset on after the window appears does not help. Thanks.

[@GDXR]

In Oculus Desktop, go to Beta and enable Public Test Channel. it might restart, wait a moment and it should say there is an update avalible. try that and then restart unreal. Also check that you XRPassthrough component is a child of the camera although it's not full required. Right now it seams like Oculus Desktop and Unreal Engine dont like each other.

[@ChrixB]

Thanks for showing that config, I would love to see how you build assets that inbteract with the surounding, any cues on that or documenations to go throught about it ?

[@deckchair76]

Great tutorial, thanks for sharing. One question, regarding passthrough. I can start the VR preview and see passthrough, but all I see is passthrough! I was expecting to see the objects that I can see on screen, is this correct? I can see when I move my head and look at the VR preview that I am looking around in the environment so it is working to some degree.

[@deckchair76]

Further to this I found a post on atmeta that pointed me to In Project Settings > Rendering > Postprocessing change Enable Alpha Channel support to "Allow Through Tonemapper." Whilst this didn't work initially, when I turned off the Quest 2/Quest Pro settings in Project Settings > MetaXR then it seemed to work! A slight aside, it's frustrating that in the SteamVR itself, you cannot record video and even a photo snapshot doesn't capture the working passthrough!

[@qingyiwei1753]

@@deckchair76 Passthrough is all i see ,too. Turned off the Quest 2/Quest Pro settings in Project Settings > MetaXR seemed to work?

[@CharlesNadenLamb]

Hi , I have been trying to set this up following the steps but when I run the vr preview there is crazy lag with the passthrough and often just crashes. PC specs are ryzen97950x RTX 6000 and 128GB Ram so im sure it cant be that. Any help would be appreciated as Im just not sure whats causing it.

[@GDXR]

Is there any chance you have steam vr running and meta link. also do you ahve them set to openXR (Also passthrough) should only be used for mobile aps and tested on an APK to get past the streaming to the pc issue the camera feed has.

[@yimingli9419]

Hi, I tried to package it to Android and let it run on Quest 3, but the sky is still black. My question is - does it work in a standalone headset?

[@GDXR]

it does work in a standalone headset. It's how ive been running/testing mine.

[@captainspudd]

@yimingli9419 Did you manage to make it work in standalone? Mine works in PC preview, but weirdly I still get black sky in standalone.

[@DanielSmith-rt1wz]

Hey Mate, im having an odd issue where the passsthough works, the on screen viewport shows everything working as expected but in the headset everything is static. Did you experience anything like that? Its like the headset captures the first frame of the scene and maps it to the headset, but its completely static.

[@XVel]

for some reason my vr preview launcher in unreal does not send to my quest 3? i can package and launch it though.

[@Aero3D]

Disable OpenXR plugin but leave MetaXR plugin on

[@generico]

thx for the tutorial, i got a really slow performance on the Passthrough. This is normal or it's something that I'm doing wrong

[@GDXR]

passthorugh is a heavy proccess as your device is rendering the passthrough layer on the screen at runtime and compositing it to the View port. That said it should still run pretty fast but it also depends on the amount of content and how it's coded.

[@ThatTechGuy123]

the video is unlisted but the playlist is on your channel

[@GDXR]

ahh good to know, thank you.

[@TIHiHubMixedRealityLab]

can you please tell is it possible to have dynamic shadows in vr using unreal engine because we can do it by unity

[@GDXR]

Check this out it might help. forums.unrealengine.com/t/tip-how-to-enable-dynamic-shadows-correct-reflection-maps-on-mobile/30528/3

[@kokashking]

Thx for the Video. Do u know if anybody tries to create an XR PC Streaming App, by utilizing some video stream codec with alpha channel?

[@Fabio-g1g]

Do you think its possible to use Passthrough on the Quest 3 for PCVR smoothly? Because when I use it in the editor or even in a build for windows its just a laggy mess with my quest 3. I also waited a bit but it just doesnt seem to get better. As long as your in a standalone VR apk build its fine but not in exe for some reason. Maybe its because its trying to restream the passthrough to pc?

[@GDXR]

No, the passthrough on pc is really broken right now and we are hoping for it to be fixed with 5.4 and the next MetaXR Plugin update. But mixed reality isn't supported on PC anyway and is only available with an oculus developer account. Any and all AR apps built right now should be standalone. As it's the only way to access scene data positions.

[@GDXR]

I was wrong when recording this, You DO NOT need Oculus Source For scene/Room setup on the Meta Quest 3 In UE5.3 just the Meta XR Plugin needs to be installed.

[@atreyu-9132]

the same @@5万回視聴回数

[@qingyiwei1753]

In vr preview , i only could see the passthrough,no any mesh actor

[@ibrahemabdelaty-r6v]

i made aall things put it giving me the desktop view not vr

[@ibrahemabdelaty-r6v]

any help pls

[@5万回視聴回数]

Like many of my siblings, the VR preview is pitch black and you can only see the outline of the actor. By the way, where do you store the unzipped Meta XR Simulator files? Could this be related to this problem?

[@GDXR]

simulator is unlikely to be part of the problem, make sure you have OpenXr plugins disabled and then try again. Also make sure the xr passthrough component is a child of the camera.

[@5万回視聴回数]

@@GDXR It has been successfully improved! thank you.

[@Casual_Capybara]

Really looking forward to the next tutorials on this! Quick question, at 4:08 when you're adding the Oculus Passthrough Layer, at the top you can see Oculus XR Room Manager Layout and Oculus XR Scene Global Mesh components. I have been trying to get these two to work, but I can't figure it out.. there's literally no documentation/tutorials online for these two components and I imagine it's what I'm suppose to be using to try to make a roomscale MR/VR game. Any insights on how to get these components to work on Quest 3? Keep up the good work, you're a legend for the UE VR/MR community!

[@PAVANSAIEEDARADA]

we have to disable the OpenXR plugins to work with it

[@張柏皓]

With the "MetaXR" enabled, and done all steps in the video, I encountered 2 situations. 1. Disable "OpenXR": there is nothing especially the passthrough showing up. 2. Enable "OpenXR" & Set the "XR API" to the not recommended one: passthrough doesn't show up, but actors do. Help... how can I see both of em... btw my version is 5.3.1.

[@nm_675]

How do you deal with distortion, etc?

[@GDXR]

You dont. We cant modify the camera out put at all. So to put it simply. We just have to deal with it.

[@KRX01]

in Vulkan mobile preview my pass tru is inverted. All the 3d objets are passing trough, and the other space is dark. Any tips?

[@stanislavgulamov5796]

Please explaine for a lame man. why when I hit button VR preview, my carracter just laing on the floor and nothing happen inside vr headset. I mean there is just a meta home page. 🤤

[@V0L4T1L3]

I followed the tutorial exactly but I still see black (no passthrough), any advice? Quest 3, 4090, UE5.3.1 Tried disabling OpenXR Double Checked settings Copied project to another machine... No dice

[@GDXR]

In Oculus Desktop, go to Beta and enable Public Test Channel. it might restart, wait a moment and it should say there is an update avalible. try that and then restart unreal. Also check that you XRPassthrough component is a child of the camera although it's not full required.

[@JunwithoutE]

Thanks for the easy-to-follow tutorial..! I have a few questions. 1. When I press Play in VR, passthrough works, but it laggs like crazy. Is it normal? 2. When I package it into APK file, and install on Quest3, passthrough does not work.. Any idea why this is happening?

[@GDXR]

I think its normal when using Play in editor if you wait a couple seconds the link kicks in and everything becomes smoother. I think it has something to do with passing the feee through oculus desktop to unreal. I typicall just build the app as an APK and test it out there. Make sure to accept any permissions which apear and possibly check your headsets settings to enable passthrough mode. I think Q3 does it automatically but Q2 may not. im not sure. If these dont help, drop by the discord and we can see what we can sort out.

[@JunwithoutE]

@@GDXR Thanks for the reply!

[@SystemUnderSiege]

Did you ever sort number 2? I can see passthrough when doing VR Preview through PC but when I run the app on my quest after installing it, passthrough doesn't work

[@JunwithoutE]

@@SystemUnderSiege I think I had to turn off other vr plugin or something. Also make sure you have the passthought support on the meta xr plugin and stuff.

[@SystemUnderSiege]

@@JunwithoutE Hmm.. I Definitely have pass through enabled, it works on PC, it's set right.. I don't see any other VR plugins to disable. :/

[@diegohamuq]

Great video... a friend of mine followed it and make it work with the QuestPro but only disabling OpenXR (he says that it works perfectly).. IN my case with a Quest3 even after doing the Public Text channel update, when I launch the project, and put the headset on, I don´t get any singal to the headset, I see the Oculus VR home screen, it´s like if the Unreal project is not running. If I change in the MetaXR API to the not recommended option, it works, but there is no passtrough. Then I did as my friend and disabled the OpenXR plugins and after launching the app and putting my hmd on, I see ONLY pure very lagged passthrough and in the unreal preview windows in my monitor I can see that it shows the 3D scene without passthrough... hehehehe very weird !!..

[@diegohamuq]

I reinstalled Unreal Engine 5.3.2 and at last it worked... but the video is very laggy ( I imagine because of the back and forth between the HMD the PC and back to the HMD)

[@moltycat7475]

Same(

[@diegohamuq]

@@moltycat7475 Now it works better, with cable, check that is good and a fast cable too.

[@Cristian-twd]

It doesn't work for 5.2 right?

[@GDXR]

Should do. It's the same process.