VR Mocap for Unreal Engine - Character Movement Blueprint

[@onethingeverything]

super cool!

[@dpredie]

This is great! thanks for sharing

[@shihhu_phileVods]

I have a vrm character with a skeleton, how can I make it load successfully?

[@Megasteakman]

I don't believe it would work. You would have to make a a new listing in VRMocapActorRigProfile_Enum, rework one of the other control rigs and animation bps for that skeleton and, make sure the the MocapActor Component stores the important meshes and the Player Pawn casts to those meshes and their animation bps. I have a mode in the works that should use any skeleton, but it hard crashes at the control rig phase.

[@shihhu_phileVods]

@@Megasteakman Thank you, I have been testing for a week and also found this kind of problem

[@Megasteakman]

@@shihhu_phileVods I'm cleaning up the new locomotion options now, but I might take another look at the 'unknown' skeleton type that just gets the closest bone to each tracker. I feel like it's somewhat dependent on bug fixes in control rig though. Hopefully by next weekend I can either make a VRM preset or write documentation on how to add a new skeleton. Most of the important info is made in the MocapActor Component, and can be traced to the player pawn from there.

[@shihhu_phileVods]

@@Megasteakman Thank you very much, I really think this program will be very helpful for some 3D VTUBers

[@XothikaOfficial]

Any chance at all you can do an update for Metahuman set up. I seem to be having a hard time.

[@Megasteakman]

Here's a more recent video with the Metahuman set up, but I think I sent you another video on discord? Both processes are the same: kzbin.info/www/bejne/maPZZ3R_l8aXidE

[@nayslayer1563]

Looks awesome! Can this be used with webcam mocap (live link) or do you need trackers? I currently capture landmarks with Wongfei's Ue4 mediapipe plugin on GH. Thx

[@Megasteakman]

It's only designed to work with vive trackers, but if you could re-route the relative goal transforms from a webcam mocap to the trackers in the player pawn then it could work. I'd be curious to see how that works.

[@nayslayer1563]

@@Megasteakman thanks! Will give it a go. Mediapipe opencv has a lot of noise jitter so perhaps that could work well together. Will dl your plug-in thank you.

[@Megasteakman]

@@nayslayer1563 In the VRMocap_PlayerPawn_BP You will find a macro called UpdateTrackersFromLiveLink. I think it you rewire that to get the transforms from the webcam mocap plugin it should work fine, but you will probably need to find a way to trigger the CalibrateVRTrackers event. Near the top of the Player Pawn Event Graph there is a 'IF YOUR CONTROLLER INPUT IS BUSTED' node that you can use to calibrate the trackers after a set amount of time.

[@nayslayer1563]

@@Megasteakman very inform. Will try. You are hard working man. Thanks

[@sociamadness233]

Hi, there is a new vr called slime vr. Does it work with this method?

[@Megasteakman]

I would say try it out!

[@imaginationrabbit]

Great work- this is so cool- making me want to buy the suit/setup to try it.

[@Megasteakman]

Hey thanks! Yeah, the cost on PCVR is pretty expensive, and Valve might be working on a new headset so it might not be a good time to buy, but considering you get a mocap volume for the price it's really great. I think with a valve index/lighthouse/controller kit and 5 vive trackers would be less than 2k, which is pretty insane for the versatility of the kit.

[@WorkbrandAssets]

This is amazing! HMD less FBIK ✨

[@romesvonwolf]

Hm, can't get the trackers to show in the play window, got the trackers showing, linked up, etc.... but when i play, just nothing seems to show for the trackers. i can spin the model with the joystick, that's it.

[@Megasteakman]

Is this steam vr plugin installed? The github has a wiki, but you can also ask in our discord. It's a little hard for me to tell what is going on without seeing more info. Thanks!

[@romesvonwolf]

@@Megasteakman no! i was looking at that plugin though... and wondering if i should try it - it said depreciated in 5.1.1 and to use another one instead, (i forget it's name) but i'll try that! thanks!!

[@Megasteakman]

​@@romesvonwolf I am looking into moving the system to open xr, but so far in my experience it hasn't been ready unfortunately.

[@romesvonwolf]

@@Megasteakman the steam VR made it all register. :)

[@WorkbrandAssets]

Lol @4:41 basically my reaction anytime I have to smooth move around in VR 🤢

[@Megasteakman]

Oh 100%, hahah. I have to do it for future generations though.

[@hellohogo]

excellent!

[@Megasteakman]

Oh thanks! Yeah, I'm finding it really fun! My mind is already working on some ideas on how to make VR work. If I can figure that out, you could basically make a game with this bp (well, you'd have to change up livelink and the way I determine the actor forward, but it's possible!)