Virtual Production in Unreal with Composure Green Screen, Garbage Matte, and Vive Trackers

[@NC17z]

Thank You Jake for taking the time to share this with us. I really enjoyed setting up my camera using the green screen. I really learned a lot along the way. I look forward to any additional video tutorials you provide regarding Unreal. Thanks again!

[@donhsi]

simply the best all -in-one UE tutorial with everything I need: vive, tracking, virtual camera, green screen, matt... hands down! thank you , thank you , thank you! subscribed!

[@lewkmiller]

I am trying to do this exact thing. This is by FAR the best explained process! Thank you.

[@JakeGWater]

Excellent! I have updated a lot of this in more recent videos too.

[@Myrmillion]

Incredible tutorials you have, and I don't mean it lightly. I've watched countless others where they basically did the exact same kind of example as yours, but you still managed to teach me a lot of useful tricks, tips and basic understanding of it all. Thank you !!

[@JakeGWater]

Thanks! My approach has been "show everything, people can skip forward if they want".

[@skipatrol43]

YES! I've been working through major issues with my vive and this solved all my issues, Thank You!

[@VanDerKeelenBenny]

Only place markers outside the image to be recorded and a film with a larger image that you can resize later so that the markers cannot be seen in the video image, so you don't have to remove them later!

[@MeinVideoStudio]

thank you for all your progression

[@DRDINC]

This is exactly what I've been looking for. Thank you for helping me connect the dots between Composure and the Vive Tracker.

[@ninomancuso7021]

yeah i agree, I was trying to use some of the LL blueprint tutorial setups and was crashing UE

[@НикитаСергеев-н2ы]

Great work. you explain the material clearly and easily. thank for it again

[@riverknoll1471]

Great tutorial Jake! Thank you. Did you ever fix the jitters?

[@JakeGWater]

I did! I'm still using the Vive trackers too. I'm not 100% sure which things made it work but I do have a list and if I can find some time I'll do a followup video.

[@DougHogan]

Did you figure out what actually fixed the jitters? I’m struggling with this right now. Genlock maybe?

[@OmkarJagdaley]

Hey, it worked like magic! Thanks a lot mate, keep posting! Hope we get to collaborate soon!

[@AdamSmith-pn5hk]

Great info once again! Just wanted to ask if you ever made the follow up on taking the camera green screen footage with the rendered unreal footage and matched in post? I’ve been looking for a tutorial on the best way to do this within After Effects for color grading,etc. cheers

[@crummudge3123]

Dude , DO NOT sell yourself short, you are doing WAY better at this than I am, you are at least tracking your background simultaneously with the head, genlock may be your issue at this point, but this is great, keep it up!

[@JakeGWater]

That’s so kind! I’ve actually gotten some good results lately. See here twitter.com/JakeGWater/status/1426963970028556289?s=20

[@NewWayIndia]

It's a wonderful tutorial from start to finish other than jittery tracking. May I ask one question where I got stuck for few days before finding this tutorial regarding using simple HDMI Capture Card USB 3.0. Did you connect your BMPCC 4K HDMI to HDMI Capture Card USB 3.0 directly (OR) used anything in-between (like any converters). I tried a month back but it didn't work and I left it there only. After seeing your tutorial, I got energy to try once again with HDMI Capture Card USB 3.0 to start learning the workflow. Please reply to this question as I got stuck for many days. Thanks for wonderful tutorials.

[@JakeGWater]

I forget what I used here, but I have a Decklink and will never look back, it works so much better than the USB captures. It also supports timecode and custom time stepping in sync with the incoming video.

[@NewWayIndia]

@@JakeGWater Thanks Jake for your reply. I saw your tutorial with DeckLink and HDMI to SDI converter. I'm planning to buy DeckLink & Converter in a month or two. Meanwhile wants to try with Capture Card USB 3.0. Thanks again.

[@JakeGWater]

@@NewWayIndia good luck, if you have any luck I'm collecting info for vpifg.com about what works, so feel free to let me know how the setup goes.

[@tyldarprod1399]

Love your demonstration. Have you thought about using an external keyer like the Datavideo 4K Real-Time Chroma Keyer or the Blackmagic Ultimatte system?

[@JakeGWater]

I'm not doing live content, so I plan to re-key in post-production using Davinci Resolve or Nuke.

[@oruam474]

Let me know it please, if you should get rid of the jittering, when using a HTC tracker

[@MrGnagnagnou]

Did you ever figure out why Vive introduces such shakes given that the sensor is smooth and on a slider here ? I was minutes away from buying into Vive and I intend to use it on sliders and jibs for slow movements almost exclusively !

[@mayapower9899]

Thanks for sharing ..did u mange to fix sliding using Htc Vive tracker ..(without mars system) .. do u recommend Htc system or (zed-2i-intelrealsense)

[@MNadeem-vs7df]

Hello, Thank you for your complete tutorial, I want to try a small virtual production from my small room with some initial equipment like Green Screen, DSLR, HTC Vive tracker 3, with Unreal Engine. I want to know, virtual production can work without Base Station if need can work previous model Base Station (lowest price available) with HTC Vive tracker 3 and also can work 1(one) base station? I want to start setup after your guide and your tutorial, please let me know. Thank you.

[@JakeGWater]

I have heard someone say they used their iPhone instead of the HTC Vive. However, I think its' hard to do without a Decklink card.

[@PaRloFilm]

Hi Jake, I have been trying to connect an HTC Vive Tracker to SteamVR for a couple of days but without success. I wanted to ask you something that is the doubt that has perplexed me for 48 hours. Is it possible to connect and use only one Vive Tracker? That is, can it be the only Vive device that is connected to the PC on which Unreal runs? Aside from yours, in all the tutorials I've seen on YT, vloggers had more than one Vive device connected to SteamVR, I wouldn't want it to be such a thing not to have my Vive Tracker connected to SteamVR. Thanks in advance.

[@JakeGWater]

Sorry I didn't get back to you. Did you have any luck? Still stuck? I'm looking at doing some proper motion tracking tutorials next.

[@matthewaina4234]

I was able to get everything running in Unreal Engine, the tracking via LiveLink xr, keying, timecode, and genlock, except that I can't get my actor to align with the virtual environment, I mean to make my live footage align with the 3d world. Is there something I'm not doing right? Could you please help me out? Thank you.

[@JakeGWater]

Sorry for the delay. Were you able to figure this out?

[@riddlezhou9411]

Really a perfect demo! But I would like to know whether there are any ways to play the composure while in play mode. Thank you very much!

[@hytalegermany1095]

Do you know if there is a way to get the live camera information into unreal with being able to live colorgrade and chroma key before it reaches unreal (because the chromakeying tools of ue are kinda bad)?

[@nguyenmarces9032]

i got a problem that its kinda glichting on preview , anyway to fix it

[@sinanarts]

Dude thank you for this.

[@LiveUPMedia]

how do you render this composure in seqencer

[@hethfilms]

Hi Jake, very interesting stuff you are showcasing on your channel, gladly subscribed a while ago. I'm thinking about investing in the vive system but I am still not really 100% clear about what the bare minimum is I need. Would it be sufficient to get 3 of the old base stations 1.0 (for better accuracy) and one tracker to get a good camera live track going in unreal? Or do I need anything else to make this setup usable in unreal? Normally you see in the instruction video that they use the vive controller and measure the room etc. Is this required to make the vive setup work? I strictly want to use this for virtual production in unreal, so I don't want to spend anything more for a full virtual reality system if I don't have to. If this works fine, I would consider buying more trackers to also get full body motion capture going using that system later down the road. I would appreciate your insight on this, thank you! :)

[@JakeGWater]

Hey, awesome. I use a Vive 1.0, which has a maximum of 2 base stations. I am not sure if you can setup Vive without the headset, but if you can then all you need is 1 base station and 1 tracker to get started. Beware that many Vive seems to have a small jitter. I'm not sure if it's fixable, or if the only solution is something else, but I'm still using the Vive for now. I might jump to 2.0 base stations, which allow 4 stations and you can use a physical wire to sync them (hopefully improving accuracy).

[@hethfilms]

@@JakeGWater Thanks for your answer, I was a bit confused as I read about one base station max. for the old version, but I realized you edited your post already to 2. That was something I was still trying to figure out, as HTC states only more than two base stations with the new 2.0 version. Which also means higher price per station and per tracker as the old trackers apparantly don't work with the new 2.0 stations. To my understanding there seems to be a workaround to get steamVR working without the headset by tinkering with a some settings files. Not sure how reliable this is... Right now I'm using my iPhone 11 and the unreal remote 2 app via live link to track my camera. It does work but it´s really unreliable and often just freezes up, the GUI is very small and often does not react correctly and if you accidentally press too high at the top of the screen you open the notifications of the phone which immediately freezes the app connection to unreal. I was hoping for something more reliable with the vive but it seems to me this is still very much experimental even though you see a lot of virtual productions using a vive system. But the cost of entry is rather off putting especially if I don't need the headset or the controllers for my projects. Anyways, thank you for your input, I'm looking forward to more content from you on this interesting topic. :)

[@supacinematics5582]

Excellent video!!

[@apencilneckdesigns]

Thanks so much for this.

[@krishnaavrilcg1136]

can you please mention the list of hardware required for the camera tracking? can Vive tracker and base station is good enough for it?

[@dthang3008]

Awesome, thank you tutorial

[@nod.p]

Thank you! This is practical.

[@mxaxolotl]

u can try aximmetry

[@ananya.s]

Can you please explain why are you suggesting Aximmetry here?

[@MrYoungAndrew]

Yes very helpful!! Thank you!!!

[@dadiyao827]

Thanks for your video,it's very useful.And have you fix the jitter?I watch your newest video and do the genlock but also have jitter.My camera was 30fps and i set unreal engine timestep 30fps,but if i did this the whole viewport will jitters,just like when you play a video game but the game only 30fps.So i don't use timestep or fixframe to sync the engine frame and camera frame,when i don't sync my unreal engine is very smooth and i have 120fps in engine but the real video is very jitter.Do you know how can i fix this.thanks very much

[@JakeGWater]

Is the motion tracking jittering, or is Unreal having trouble rendering?

[@dadiyao827]

@@JakeGWater When i don't fix the engine frame i have 120fps，i move the tracker and everything is good and smooth except the real video is jitter

[@JakeGWater]

@@dadiyao827 That totally sucks!! I am not sure what the problem is. I am doing some work with motion tracking again, so hopefully I can share something useful soon.

[@dadiyao827]

@@JakeGWater Thanks very much

[@ibrahimmomoh]

Hi what is your pc setup?

[@JakeGWater]

AMD 32-core 3970x with a RTX 3090. I call it the monster.

[@ibrahimmomoh]

@@JakeGWater true very true

[@thezombiefriedchickenshow2253]

As much of a pain as the Vive is you should be able to get much more fluid tracking with it; something is off (something is always off tho huh lol) - the Vive is aweful compared to systems like Antilatency tho because of the massive amount of time having to dial it in

[@JakeGWater]

Agreed! Something was (and still is) definitely off. I also only have the Vive 1 base stations, and they're not synced via cable. It's a barebones setup.

[@thezombiefriedchickenshow2253]

@@JakeGWater I would think that having a second base station might resolve but I would never go VIve for camera tracking; way too much work and buggy to get it set up - I did invest into antilatency, but I also have been using an Unreal Remote 2/ipad mini (latest) and 4.26.2 and not only is it really smooth for tracking but all the options for lens settings are controlled by the ipad ; its kinda hard not to just go that route

[@JakeGWater]

@@thezombiefriedchickenshow2253 that's really helpful to know! I will have to give it another try. I recall trying the iPhone before and the tracking wasn't good.

[@thezombiefriedchickenshow2253]

@@JakeGWater supposedly iphone/ipad tracking was choppy with the first version of unreal remote but much better w unreal remote 2

[@dy6079]

You need the new vive Mars.