3d Hand Tracking in Virtual Environment | Computer Vision

[@fttx6673]

One of the best projects, and we can also facilitate the work so that we can associate a 3D character with the engine. This is a wonderful thing

[@LittleFifth]

Very excited about this video, I was looking forward to 3D stuff since a 2D pose estimation video you did- thank you and I'll be sure to try this

[@FernandoMoreiraR]

This video won you a subscriber!

[@bobjefferson2829]

E là lớp phó gương mẫu, e chúc thầy và cô Thuỷ mạnh khoẻ, hạnh phúc và có nhà mới thật đẹp ạ

[@angelojojo3338]

Hey man, great video. But use this line will yield a better control float x = float.Parse(points[i*3])/80-8; float z = float.Parse(points[i*3 +1])/80-5; float y = float.Parse(points[i*3 +2])/80; You need to transform the coordinate from OpenCV to Unity. You use: float x = 5 - float.Parse(points[i*3])/100 in your video However, this will reverse the coordinates of your hand. To transform the coordinate, you will need to turn the middle point in OpenCV from (0.5x, 0.5y) to (0,0) In this case, you will need to subtract the x value by 80, and then minus 8. and y value by 80 and then minus 5

[@rafaroeder]

Hey ! thanks for this. Did you figure out how to make te hand go back and forth? The "point0" does not move on the z axis. tanks!

[@oezbro]

You're a real life saver. Thanks a lot! Best tutorial for handtracking I've seen on KZbin. Keep up the good work!

[@zimpel35]

Super Video, super project. As always from you! one of the best channels on yt.

[@12qwe-di3tf]

and super socket 15:43

[@abhilashh3904]

best tutorial i have seen related to computer vision and unity. thanks a lot. please do more videos

[@VINESHRAJPUT]

Everything is perfect . One question Can I grab the cube also?

[@055pq2]

You can, but its extremely diffcult!

[@BurhanAhmad-s4e]

@@055pq2 did you figure out how to?

[@marshallmann7620]

You're awesome, Murtaza.

[@giancarlopappalardo]

you have really outdone yourself this time

[@pramobunnythebun]

Wow amazing... I'm watching your all videos. Really appriciate your knowledge. Please give us more tutorial like this. Can u give more tutorials for starting how to code augmented reality? It will be a great help. God bless you

[@surajpoudel755]

BROTHER, YOU ARE THE BEST!!! You oooh really helped me!! THANK YOU VERY MUCH!

[@calvinthapro616]

Hey, good video. Could you post the code on how to move the hand on the z axis? That would be very helpful.

[@f1l1p10]

u can use code from this video kzbin.info/www/bejne/oKTSlmSDZ6adfM0 but using point 0 and 17 from your hand

[@MLOps_]

@@f1l1p10 thanks

[@tokiotv2843]

Anyone has it?

[@호모사케]

Hello! Can you give me a solution to build project as a one exe file?

[@SassyPantsy]

Incredible tutorial. Very informative without being redundant. This is what the internet was for!!! Thank you so much for this knowledge. Subscribed!

[@fiogar7006]

How do I get the virtual hand in unity not to scale when I move my real hand forward and backwards to the camera

[@alexchan252]

a very nice tutorial

[@pickapack4704]

Nice project sir 👍

[@rhezafatria]

Many thanksss!!!!!🤩

[@loveronthesunid]

keep up the good work

[@1979zeesh]

awesome project bro....amazing. one question: are u using the usual unity collidor components to detect collision or what? that part is missing in your video. how u detected collision?

[@lucker7579]

ArgumentOutOfRangeException: Index and count must refer to a location within the string. Parameter name: count

[@Hekaro]

the possibilities are endless. will certainly play around with this

[@cybersecurite_CPT]

wher can I download UDP receive Please??????????

[@edbic4223]

This is awsome! I want to try this on my own and develop it! Do you think, if I make the unity 3d enviorment complex, is the detection or all the other process gonna get laggy??

[@worldtimefactechzwtf4857]

Great stuff thanks!

[@aniketkhadse2002]

Wonderful.... Can we do this with full body?

[@MrBrains_]

07:18 min AttributeError: partially initialized module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline' (most likely due to a circular import)

[@anantchandak9574]

he is going to make revolution in computer vision just tell me can't we cover our points like it we have red point and green lines as attachment, so we can increase the thickness and change the color in unity. so by this can't we create a real looking human hand

[@JavieraPaulinaArgandonaOlivare]

great video:) is it possible to show both hands at the same time?

[@dxgarmy1373]

pls tell me can I use this to grab a object how im trying its not working pls pls tell

[@notnow3631]

thank u so much !!

[@Soccertrash17]

Has anyone a solution for the z coordinate estimation?

[@asurstark]

May I know sir, how you have integrated the camera?

[@dxgarmy1373]

all done points working but lines not , error is coming the variable origin of Line code has not assigned in the inspector but I had linked all the line and so many errors for many lines

[@flioink]

There's some kind of error in the "extend" part of the code, I keep getting: IndexError: list index out of range Edit: OK, I narrowed it down - the lmList list only gets 2 dimensions(x and y) and not the z. Why would that be?

[@allamerican9414]

Hey, I am having the same issue, did you ever end up figuring out why lmlist is refusing to give z values?

[@yade1234]

@@allamerican9414 Upgrade your cvzone package using pip

[@nv7287]

Can this work on mobile? Amazing video tutorial!!

[@FruitBK]

Thank you~~!!!!!

[@deltakingoffisho3410]

great, thanks you

[@trash2treasure786]

Does it work when published to android?

[@rohitchabukswar7266]

Can we make the hand look like actual hand?

[@NotnotADHD]

I have a issue. I dont know what the app that you used to code python in is called or where to download it. please help

[@josippardon8933]

Great tutorial. Acctually, there is automatic way to put points in origin and destination variables of line objects. Have points named "Point (...,...)" and lines "Line ...-....". Then write this code: public class LineCode : MonoBehaviour { LineRenderer lineRenderer; public Transform origin; public Transform destination; // Start is called before the first frame update void Start() { lineRenderer = GetComponent(); lineRenderer.startWidth = 0.1f; lineRenderer.endWidth = 0.1f; //THE CODE: string[] pointIndexes = gameObject.name.Split(' ')[1].Split('-'); origin = GameObject.Find($"Point ({pointIndexes[0]})").transform; destination = GameObject.Find($"Point ({pointIndexes[1]})").transform; } // Update is called once per frame void Update() { lineRenderer.SetPosition(0, origin.position); // 0 => starting point lineRenderer.SetPosition(1, destination.position); // 1 => ending point } }

[@aisteelmemesforaliving]

this is the way.

[@coderserabutan]

really cool, but this project need python server to run the hand tracking. so, it cannot separate with the server when the unity build into mobile platform, I mean it cannot run only using unity. thats sad.

[@Lightyboii]

Theres a system to use python in unity

[@coderserabutan]

@@Lightyboii really? run python server? how?

[@Lightyboii]

@@coderserabutan theres a Python addon in unity. And AFAIK it does support modules. Not sure about Android though. Havent used unity in a while.

[@adityasprojects7126]

Can you make a video where you do evrything in Unity. There is Opencv module in UNity also right. This will be easy for deployment and more advance project. I couldnt find any proper video explaing it.

[@holoments]

There is an opencv package at asset store what you can play with

[@nemeziz_prime]

Hello Sir, I had a question. Can we deploy DL models inside a Desktop App for Windows/Linux? I searched for solutions but couldn't come up with any reasonable ones

[@felixtechverse]

Where can I find the udp script? I didn't find it on the website.

[@YigalBZ]

Thanks for the nice and interesting project. How will it work on a SBC like Jetson Nano? What I don't understand is how this software will be able to utilize the GPU?

[@dinukachathurangashow8571]

this will only work in x and y axis right ? is there any other way to get depth information for z axis ? so we can use it in 3d space.

[@aakash723]

is 2gb ram of respberry pi 4 is sufficient for making opencv Projects.. like find contours

[@flickermarc6308]

Does anyone know a way to implement two hands in Unity I already made it so it can track two in OpenCV But I'm having trouble implementing it in unity

[@adhviklalthoppe2214]

Hey, great video. I was wondering how you could implement the python so that it opens up whenever the unity project opens up?

[@bceg3961]

I need it!

[@dante_zu]

Hi! You can Import "os" library to python and use "os.startfile(path to your .exe file)" in order to execute the compiled Unity Project.

[@bceg3961]

How to create a build of this application? Is it necessary to run the code in python and in c#? I don't understand.

[@luigiesposito2481]

hello Friend would you help me do it for online game capture? people use CV to detect enemies and attacks on aim does this do the same?

[@umam.ytchannel9464]

now i am in a good mood

[@modelad3d]

What logic can I use so that when separating or moving the hand away the same thing happens in the 3D environment instead of the hand being scaled? I've tried everything and I can't find a solution.

[@林裕锋]

Hello sir, I want to ask why the effect line I made is separated from the sphere

[@deejaysushant]

What the performance in using in a mobile?

[@mrdotss4967]

Thank you so much, but I have a question? Will it work on mobile android/ios?

[@justfeta8409]

amazing....

[@lakpatamang2866]

can we build this in mobile?

[@iqrobotics8960]

Hi bro ap Indian sy ha ya Pakistan sy

[@abdelhaksaouli8802]

what is the z coordinate in hand track output ?

[@RoboTron_Channel]

Ты очень крутой, спасибо!

[@yuchen3587]

Can this algorithm run on mobile devices like iOS or Android?

[@howtogamedev_]

how can i make this work if I wanted to make a android build from unity and use the phone camera to do the hand tracking

[@ShivGp-tr3vk]

Hai...thankyou so much for this video, this was so helpfull for me to make a project with this method. its work for me but can u give me another video about how the hands can grab the object like do some 360 move for the object, hope u saw this message, thankyou

[@zoterodisuke4398]

Is it possible to do this but for fullbody?

[@rajputnehu05.]

Work bot, thanks broo

[@BestMakerHackerCreater]

how we can use phone camera ??

[@adhamymagdy8211]

what is the algorithm used in this project

[@menaeem]

Can we build apk out of it???

[@yoshirovilchez3392]

Hi! I'm new ! How can I start in this field of Computer Vision? My dream is work in this kind of projects :)

[@LeDiop]

Did someone find how to move on the z axis ? I'm trying it so hard ! because the z is based on the wrist point so it stays at 0 ...

[@calvinthapro616]

Hey, did you find a solution?

[@itsjustepic5029]

@@calvinthapro616 i didnt find one can you sent me the code my english and python knowledge is noct so good

[@yuchundehanguo5555]

Orang Vrindavan pada pinter pinter yah

[@balajinaidu9482]

hey sir, You done a great job but can you guide or make a tutorial for the hand tracking android application using mediapipe and unity

[@RC8Retro619]

Does anybody know how I can use the hand tracking as input so instead of colliding with the cube at the end, moving my hand left or right will move the cube left or right. Im trying to make a game with openCV using Unity for a school project and I am a bit stuck!

[@alidkjfk]

for z , in lmList dictionary you have a center position yo can use math.hypot to calculation 0 position with center position and with numpy.interp you can make a relation with this space and plus it to z positions , my language is not English and i can explain this good but I hope you understand how is it work!

[@mariusarnbjerg8864]

Can you elaborate on this? What would the code look like?

[@ry2453]

hello sir, can you explain about udp receive scripts? thanks

[@AlexSilva-cn1gp]

Hello, nice work! I wanted to know if there was a way to create a system for grabbing objects that are in unity, could anyone tell me?

[@dilaymenemencioglu1865]

How can i get APK for this project?

[@samma-pixelstudio]

Only if you integrate the hand detector in the unity project ung the opencv plugin for unity

[@ajayysingh4976]

Can u plzz say wt is the objective and scope of this project

[@kristijan1991ful]

Is hard implement to recognize more hands. Like both hands, or 4 hands. For example 2 players use both hands to play game.

[@salloookhan1783]

Can we bring this hand tracking on tft display or other display so we are able to use it without computer like tracking face tracking hands tracking whatever but with out computer pc how can we do this

[@rafaroeder]

amazing tutorial! .. Ive been trying for the last 2 hours to make the hand move on z axis, but the point 0 wont go back and forth.... any hints?? thank you

[@RuiMeloViPeR5000]

Excelent

[@nickdev]

can tihs work on android too?

[@Ivystra]

When linking the lines, I create the LineCode as a new component for all the lines, but I can't change them individually. Changing the origin and destination points for one changes the origin and destination points for ALL THE LINES. HELP

[@spaz3834]

can you use it in vr ?

[@samma-pixelstudio]

Yes

[@yumyum_99]

Hello sir, i cant figure how do you add the z position of the hand tracking, i have tried using triangle similarity algorithm method to detect distance using single webcam but i get nowhere close to the solution. Since u mention the answer for moving hand backward and forward is easy can u explain on that?

[@vadox-6807]

how i can save the python script in unity ?

[@Cola_andy]

How can i make it two hands instead of one

[@itsjustepic5029]

how do i add 2 hands to unity

[@Jeetu0702]

is there any way to execute file without running python file?

[@toxicvaloplayers3210]

sir collision in unity is not so perfect in this way the hand many times enters the box bcz of hard code points it get from python

[@pranshugraak3106]

Good Afternoon Sir. Sir i am using a depth camera that is intel realsense d435i instead of a normal webcam for this hand tracking but this error ie (index and count must refer to a location within the script ) is showing to me when i write the code ie: data = data.Remove(0, 1); data = data.Remove(data.Length-1, 1). This error is not shown in the case of normal webcam but in the case of intel realsense depth camera this error is shown. Hoping for a fast reply need urgently.

[@hamzathassan7080]

Use this `data = data.Trim(new char[]{'[',']'});`

[@ThatDodu]

i am gonna guess that means the hand script is not getting any data, try putting all the code in an if statement. First set the variable "string data = udpReceive.data;" and then add this if statement, "if ( data.Length != 0 ) {Put_All_Code_Here}"

[@farisdirar2976]

thank brooooooo !!!!!!!!!!

[@judetenga2148]

Hello there seems to be an error with KeyError: 'lmlist' do you know what the reason of how can resolve that?

[@SuperEvilTeam46]

lmList is keyvalue. Propabably what is wrong in code is that u are typing second L in 'lmlist' in small case

[@flioink]

The "z" component is missing, not sure why.

[@judetenga2148]

@@flioink ok but what if this issue pops up as well as an error. lmList = hands[0]['lmlist'] KeyError: 'lmlist' [ WARN:1@24.978] global D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (539) `anonymous-namespace'::SourceReaderCB::~SourceReaderCB terminating async callback

[@judetenga2148]

@@SuperEvilTeam46 ok but what if this issue pops up as well as an error. lmList = hands[0]['lmlist'] KeyError: 'lmlist' [ WARN:1@24.978] global D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (539) `anonymous-namespace'::SourceReaderCB::~SourceReaderCB terminating async callback

[@flioink]

@@judetenga2148 This is a bug with the windows camera driver. I got the same thing occasionally - unplug & plug the usb connector and try again.

[@harshahir5901]

make video on 3d face recognition android app