Unity XR Hands Custom Gestures Tools Are Here!

[@dilmerv]

📌 Below are some good resources I used when learning about XR Hands Custom Gestures: Building Blocks: docs.unity3d.com/Packages/com.unity.xr.hands@1.4/manual/gestures/building-blocks.html Finger Shapes: docs.unity3d.com/Packages/com.unity.xr.hands@1.4/manual/gestures/finger-shapes.html

[@cirtoyt2403]

This new tool is of great impact for me! Thanks a lot for a clear breakdown of how their systems work. It's interesting to see identical parallels between the options the pose/shape classes use to build the gestures and the gesture system I have been working on over the last 1.5 years in my free time! Seems we've come to similar solutions including offering tolerance controls, pose relative options, and component-based built-up poses/shapes. The tool I'm working on specifically utilises gestures from various input sources (OpenXR controllers, Hands tracking, etc) for building Magic systems, so I'll have to add another input kit to support the XRHands Gestures within my system! That way, users will be able to use the Unity XRHands Gesture system and build Magic system on top of them utilising my tool still :D Planning on starting promo vids and such in a couple month's time if you're interested!

[@dilmerv]

This sounds amazing, and I like the saying that great brains think alike 😉 which is more likely why you arrived to similar solutions. Let me know when you have a version available, I would be more than happy to test it out and provide feedback. Thanks for your comment and for watching 🙏

[@cirtoyt2403]

Haha thanks! I gladly will when the time comes! 😁

[@dilmerv]

@@cirtoyt2403 awesome 👏

[@jackwang3561]

This tutorial helps me getting my own XR custom gestures to work! Absolutely fantastic! Dilmer how do you detect a gesture that requires both hands together (like a heart shape)?

[@dilmerv]

That’s a great question, I would say you could detect each independently and then through code reference both and check when both are detected.

[@namename-o7k]

Thank you! It was very helpful for my studying. Is there any way to use XR-Hands and gaze tracking simultaneously in Quest Pro, by any chance? When I tried, gaze and XR hands worked on different plugins.

[@dilmerv]

If I remember correctly, Gaze / Eye Tracking features with Quest Pro requires a Meta adapter which has a dependency on Oculus Integration / New Meta XR packages, therefore it is not compatible with OpenXR. Meta has its own OpenXR implementation which makes it incompatible. Also, thanks for your feedback and I am glad this video helped you!

[@DavidZobristGames]

Hey Dilmer. The unity xr tookit by default lets you grab the interactables via pinch could you make via of how to make them grabables like a human intuitivly would grab an object? Its wield to pinch a sword or gun etc instead of grabbing. I guess it must be doen throught the custom hand gesture and than be mapped so it triggers the xr grab. But in detail i dont find any videos abou it. So its not abou the handpose when its grabbed but about the pose detection to trigger a grab. And pinch is just not how all my test player want to grab objects. I guess i can do this based on this video but I think a lot of people will wonder about this specific case.

[@kjspaces]

Thanks for sharing this awesome tool! Can it save or export any data/config files of the hand gestures?

[@dilmerv]

Yes and great question. Each hand gesture is configured by using a hand shape scriptable (which is a file you can export anywhere) you could also have poses by using a hand pose scriptable, also a file.

[@sigifredovalenciaensinales7080]

Hello Dilmer, thank you very much for this very useful video, I appreciate your dedication. Just one question. Have you added 3D hand models or something similar? When I test it in Unity, the panels indicate that they are detecting the gestures, but I don't see my hands. Regards and thanks in advance.

[@dilmerv]

Have you tried the HandVisualizer demo scene? I would suggest running that first before doing anything with custom gestures, that way you can verify that hand mesh is working. Great questions and thanks for watching!

[@sawsanalraisi972]

Hi Dilmer, do you know how I can create a steering wheel for VR hands (Meta) to drive a car?

[@dilmerv]

Hello there, I would suggest looking into Interaction SDK essentials which has a variety of examples that you can tweak to accomplish a steering wheel mechanic: assetstore.unity.com/packages/tools/integration/meta-xr-interaction-sdk-essentials-264559 thanks for your comment.

[@flexcg675]

can we use it for vision pro? or can i move my camera view with hand gestures?

[@AndersSandell-q1o]

Hi Dilmer, great video. I've been working my through your Vision Pro Videos. Can you use this package together with Polyspatial to create custom gestures for the Vision Pro?

[@dilmerv]

I am not 100% sure to be honest, but I will be testing that very soon. Great question!

[@byulight]

Thank you for this tutorial! I'm still finding it hard to make custom gestures - I'm not sure if I'm understanding the debugger wrong but the values I use don't seem to work. Also, how does the tolerance range on the debugger work? The thumb shows two blue lines different to the other fingers.

[@asimdeyaf]

Does Unity XR hands still work for Apple Vision Pro or does Apple require you to use their own SDK? I can’t imagine anyone using Apple’s SDK if Unity XR hands works just as well, unless you were exclusively developing for the APV.

[@dilmerv]

That would be amazing if this could work with Apple Vision Pro, but I don’t believe Apple supports OpenXR which is required by XR Hands. I wouldn’t be surprised we see changes but as of now it is not supported. Great question!

[@blackwhalestudio]

Unity XR hands will inded work for Apple Vision Pro through PolySpatial! Besides, Apple doesn't offer us their own SDK for Unity.

[@dilmerv]

@@blackwhalestudiothat’s correct, XR Hands exposes native visionOS hand tracking features through PolySpatial, thanks for the correction. I think my 3 weeks away from XR due to 😷 made me forget a few things.

[@blackwhalestudio]

@@dilmervNo worries, glad I could help! Get well soon, we missed you!🥽

[@dilmerv]

@@blackwhalestudio thank you 🙏

[@dre_purba]

Hello, can this hand tracking implemented in full body avatar system ?...

[@dilmerv]

This hand tracking can be implemented with any system, you would be to implement the providers based on the XR Hands implementation. Here is more info about it docs.unity3d.com/Packages/com.unity.xr.hands@1.1/manual/implement-a-provider.html

[@gagagu01]

Thx for this tutorial. I am looking for the Info how to use a special gesture to grab something. The gesture detection is working but I could not find out how to grab with it. Is there any Info available for this? Thx forward

[@dilmerv]

Look into Unity XR Toolkit grab interactions and you will find a lot of info about it.

[@De-Panther]

Cool! Works on WebXR Export as well

[@dilmerv]

Awesome and thanks for your feedback. I haven’t tested it with WebXR, did you test it and is it available for me to check it out?

[@De-Panther]

@@dilmerv yes, I have a video on my channel, how to set up WebXR with XR Hands, but without the details you have in your video on how to create a gesture

[@rolex-vj]

While using XR interactions toolkit, Im getting duplication of my XR hands. 4 hands projecting in my scene. I checked everywhere .. Can any one tell whats the issue here?

[@dilmerv]

Are you using a brand new scene or an example scene? Maybe creating a scene from scratch could help?

[@rolex-vj]

@@dilmerv🙌!! Previously i dragged rig the example scene.. Now i dragged prefab to scene from scratch. it worked!!! 🎉Thanks Dilmerv

[@dilmerv]

@@rolex-vjawesome man ! I am glad you figured it out and thanks for sharing what you did! Have an amazing day!

[@celaldogangunes]

Does it work with MRTK?

[@dilmerv]

I believe it should since they are both running on OpenXR if you use MRTK3.

[@celaldogangunes]

@@dilmerv what about mrtk2?

[@dilmerv]

@@celaldogangunes it won’t work for sure with MRTK2.

[@zzw6299]

我不知道如何在 hololens 上运行它，尽管它可以从支持的 SDK 运行。但是我无法配置它，我希望在这方面有教学。谢谢

[@dilmerv]

Thanks for your feedback, I haven't tested this with HoloLens2 but yes you could look through my early MRTK tutorials where I go over hands.