Custom Hand Gesture Recognition with Hand Landmarks Using Google’s Mediapipe + OpenCV in Python

[@cesarams0292]

Hey! Just wanted to say thank yoy, because of your video we are making a project of universal hand gesture recognition applied in cobots, it´s really exciting and you really helped us out, greetings from Mexico! I hope you upload more videos soon

[@StefaanHimpe]

interesting tutorial... and with classification on the point history one should also be able to recognize movements like "swipe left", "swipe right", "clockwise turning" etc which makes the system even more useful

[@CodeBrewCS]

One of the best and more enganging videos Ive watched on this topic ! Good stuff

[@lolopupka]

Hi Ivan I was researching to make the same thing as your student! I want to make a sign language learning app as well, for fun. If your student completed the project, could you share a link? I would love to see what a finished project could look like.

[@jamesgalima1463]

I've developed image processing apps because of you, thank you for these tutorials

[@IvanGoncharovAI]

That's SO cool to hear!! Awesome job man!

[@ramanasai7025]

Kudos to you for explaining very clearly . I mean it , and kudos to the github repo owner as well. 👏👏👏

[@itsmemattagain9841]

This video helped me greatly with a game I'm doing as i was planning to implement hand recognition to play it

[@IvanGoncharovAI]

Awesome to hear that buddy! Really glad you liked it and it helped :)

[@itsmemattagain9841]

@@IvanGoncharovAI I mean, I thought i'd have to implement this all by myself. It already does all I wanted and let you train new hand gestures in such a easy way... I wish there was a way to train both hands at the same time tho, like touching hands and such

[@IvanGoncharovAI]

@@itsmemattagain9841 I can see how that could be implemented though it'd be a rather serious rewrite and would take some solid familiarity with what's going on the code. Probably would need to encode hand gestures into a vector from both hands and then modify the neural to be trainable on more keypoints - that's just a really broad view of what could be done

[@luisaurso]

Thanks for sharing this with us Ivan, very well explained and actually it will help me on a real use-case project I am working. Thanks a lot ! very well done.

[@IvanGoncharovAI]

Thanks bud!

[@ilonanietosvaara7090]

Hi Ivan! Such a great tutorial, I love that you really get into the details of the code and explain what is going on concretely. Perhaps a stupid question but is it possible to do transfer learning with the Mediapipe framework (for example by training it with images of hands with more keypoints added for more accuracy)?

[@zombiesoul9740]

thank you for your great video ... I have questions how I can use my existing dataset Image for labelling and training ?

[@ZINK863]

Hi Ivan! This was a really helpful and amazing tutorial! The only question I have is how to change the model so that it can take more than 10 gestures. What/where/how should I change the code? Can you explain?

[@germanaspinelli8525]

In logging_csv function you have to change writer.writerow([number , *point_history_list]) to writer.writerow([number + increment, *point_history_list]) where increment = 10 allow you to save symbols from 10 to 19 (button 0 = 10, button 1=11, button 2 = 12 and so on), increment = 20 allow you to save symbols from 20 to 29 and so on. But you will press button from 0 to 9. To see that you are pressing and saving the correct button, in draw_info function you have to add increment to number value. I don't know if i am clear

[@user-ng9nn3kk6i]

SOME IMPORTANT KEYPOINTS: - Custom hand gesture recognition using Google's Mediapipe and OpenCV in Python is explored in-depth. - Two primary approaches, pixels-first and landmarks-first, are discussed for hand gesture recognition. - The necessity of machine learning or deep learning for recognizing complex hand gestures is emphasized. - Mediapipe simplifies hand detection and landmark extraction through its pre-trained models. - Steps like pre-processing hand landmarks, converting to relative coordinates, and training models are detailed. - Training data stored in dot cc files, normalization of values, and Jupyter notebooks for classification are highlighted. - The video serves as a comprehensive guide for developing a reliable hand gesture recognition system.

[@shireesh2209]

so..... we are using numpad (0-9) to save each different gesture right. can we only save 10 hand gestures????? how to save more than 10 gestures???

[@user-kc7gm9ty5o]

FOR: Training Mediapipe models using custom datasets.

[@animehd2628]

Hi Ivan! Masterpiece video u have made here! Loved every part of it! Keep it going! Would love to see how you can build onto this and make it even better for real world application

[@IvanGoncharovAI]

Thank you! I've answered that several times in the comments, could you look through them please? If you'll have more questions, feel free to ask :)

[@animehd2628]

@@IvanGoncharovAI What about a sign gesture that requires motion or 2 hands. For instance, Sorry, Where, Name and etc

[@animehd2628]

Is there a way to train the model to identify between the left and right. For instance, left and right hand is doing a total different sign but to form 1 word, like Help

[@IvanGoncharovAI]

@@animehd2628 With two hands you'd need to rewrite some of the code and the neural net to capture input from two hands (and normalize it in a similar way too), and I think it's doable with some technical expertise on the subject. With motion, things are quite a bit more complex because you'd likely want an encoding algorithm that gives inputs to a neural net to work across several frames and potentially a very different neural net architecture with it.

[@IvanGoncharovAI]

@@animehd2628 I think you could, like, two gestures for example "piece sign" and "rock sign" and then, say, if left hand is "piece sign" and right is "rock sign" you trigger some response that says that it's the "help" sign or something like that

[@alankuo2727]

20:00 i tried running app.py in vscode yet it didn't work, giving me this error: ValueError: Could not open 'model/keypoint_classifier/keypoint_classifier.tflite'. any idea why? Thank you so much!

[@gouthamsuresh5672]

Hi, I'm also facing the same issue while executing app.py . where u able to find any solution for fixing this?

[@alankuo2727]

@@gouthamsuresh5672 i havent :(

[@gouthamsuresh5672]

@@alankuo2727 I found the issue. It's because of library versions. If u install these versions specifically , u will get it working: OpenCV - 4.5 MediaPipe - 0.8.5 Tensorflow - 2.5.0

[@alankuo2727]

​@@gouthamsuresh5672 I downgraded opencv to a older version and I've got it working!

[@shwetabhagat8920]

​@@gouthamsuresh5672 hey! converter = tf.lite.TFLiteConverter.from_keras_model(model) converter.optimizations = [tf.lite.Optimize.DEFAULT] tflite_quantized_model = converter.convert() on executing the above the kernel is crashing each time Any help would be appreciated!

[@inson_piano]

Hi Ivan! This is a fantastic tutorial! Really helpful for starting to learn gesture recognition. I heard the palm detection method and Mediapipe for the very first time thanks to your video. Just want to know how to add new gestures to the current model that allows the machine to recognize them?

[@IvanGoncharovAI]

I think I explain that in the video. Like, the first training that I do is exactly adding new classes to the existing model. And the second training was training a completely new model from scratch.

[@IvanGoncharovAI]

Hey folks! This video was filmed weeks before the recent events in the region that I'm from, which almost gives a double-meaning to all the "peace sign" references in the video... I mean, I just really hope that things can normalize, and that if you or your loved ones are affected by it that you can stay safe and make it through. Lmk if I can help in any way, too.

[@hudahamed7693]

Please can you help me to solve error install pycocotools Note: I implemented all your instructions with an algorithm yolov5 video

[@IvanGoncharovAI]

@@hudahamed7693 Maybe this'll help github.com/cocodataset/cocoapi/issues/393

[@febiefebriansyah]

@@IvanGoncharovAI Can we read #multipleHand (3, 4, ...), sir?

[@sudhanshudambare7162]

Which screen recorder do you use please help me ..

[@kiranbabu4414]

Can you please help me with the project regarding stone , paper , scissors where input is given using hand gestures

[@ilhanylmazkaradeniz8875]

Hi, this is a really cool tutorial for sure. But what if I have to use both hands to perform a hand gesture such as heart sign?

[@user-xm9fd9yi1n]

This video was really helpful! Thanks for the tutorial :)

[@FaisalKhan-gx5eg]

20:01 after running my camera window is not open in this code ..... i tried it with basic code it is working there ... plz help

[@readingnlearning176]

I want to make project when a person writes K in air it should be converted into text K , how can do this

[@creased528]

Hey Ivan! Great stuff here! One question from me is how do I make the Hand Gesture Recognition model more robust and accurate, as compared to the provided simple sequential model. Cheers!

[@IvanGoncharovAI]

Thanks! So, the thing here is that I don’t think that a larger model would be necessarily better because the input size is quite small (40 x and y points) and you could risk overfitting the model. You could experiment though with adding more neurons to the sequential model in the case that you train it on a lot of gestures or you aren’t satisfied with the model performance now. I’d say the safest bet to is always to focus on more quality data in general

[@astra6329]

Traceback (most recent call last): File "C:\Users\AA\Desktop\hand-gesture-recognition-mediapipe-main\app.py", line 545, in main() File "C:\Users\AA\Desktop\hand-gesture-recognition-mediapipe-main\app.py", line 170, in main keypoint_classifier_labels[hand_sign_id], ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ IndexError: list index out of range Process finished with exit code 1 . How can i fix this error? Pleaes help.

[@Danny_davito_mk2]

I have the same problem 😂😭

[@Danny_davito_mk2]

I found something! Try making a new symbol and avoid using the old one, then train it again and it might work

[@astra6329]

@@Danny_davito_mk2 ty bruv

[@Danny_davito_mk2]

Oh nvm you need to save all files before running the code 🤣

[@logesh6904]

Hey! I tried this project and it works. Thanks. I tried to train using both hands but it only saves one hand gesture. But the the gesture is combination of 2 hands. Can you me out?

[@SophiaGalelli]

HI, it is not clear to me what types of neural networks are used in this project, because you mentioned CNN and feed forward, but in the project there is something about LSTM neural net. Thanks in advance

[@sillius]

Great video, although I am having trouble when running app.py, it seems as if the tflite inepreter has trouble opening the tf.lite file in the folder. Here's the error: File "d:\Project\lib\Lib\site-packages\tensorflow\lite\python\interpreter.py", line 464, in __init__ self._interpreter = _interpreter_wrapper.CreateWrapperFromFile( ValueError: Could not open 'model/keypoint_classifier/keypoint_classifier.tflite'. Any help would be appreciated.

[@sillius]

Well, I was able to solve this issue (maybe not properly), by replacing "model_path" in interpreter.py, with the actual path to keypoint_classifier.tflite, but now, while the video shows up, it crashes when it "sees" a hand, and gives this error: Traceback (most recent call last): File "d:\\hand-gesture-recognition-mediapipe-main\app.py", line 543, in main() File "d:\\hand-gesture-recognition-mediapipe-main\app.py", line 153, in main finger_gesture_id = point_history_classifier( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "d:\\hand-gesture-recognition-mediapipe-main\model\point_history_classifier\point_history_classifier.py", line 30, in __call__ self.interpreter.set_tensor( File "d:\\lib\Lib\site-packages\tensorflow\lite\python\interpreter.py", line 720, in set_tensor self._interpreter.SetTensor(tensor_index, value) ValueError: Cannot set tensor: Dimension mismatch. Got 32 but expected 42 for dimension 1 of input 0.

[@aaquibtayyabi]

were you able to resolve this ?@@sillius

[@user-jh5os5mn2t]

Can it detect sign language?

[@shwetabhagat8920]

KERNEL IS CRASHING Initially I was getting the error that the model name must be in ".keras" format later I commented the part which was throwing the error and then I executed the cells of the keypoint_classifier.ipynb in the vs code in the kernel(that is the virtual environment that I created). But now the kernel is crashing and if I run the cells on the jupyter notebook then it is not identifying the modules,i.e. giving errors like tensorflow not found. Do you have any idea how to fix any one of the issues, i.e. either the crashing of the kernel or the issue of jupyter notebook?

[@aishezsingh7004]

hey did you found a fix for that keras error ?

[@user-kc7gm9ty5o]

Yep. Facing the same issue

[@abuzerkomurcu9997]

Friends, did you find the solution?

[@user-kc7gm9ty5o]

@@abuzerkomurcu9997 nah man

[@EVANSSAMUELBIJU]

@@user-kc7gm9ty5o apparently.hpf5 is legacy and instead change it to .keras

[@bceg3961]

Thanks, I needed exactly that for my project, I'm working on a Libras translator, which is the Brazilian sign language.

[@IvanGoncharovAI]

Glad to hear it! Good luck with your project!

[@alexandragoncharova9997]

❤

[@francomedin]

Hi Ivan ,Thanks for the video!

[@zaidtariq4]

Hey Ivan, Awesome video. Can you please tell me if you came across a way to fine tune the mediapipe hand landmark detection using ones own dataset. Some how I cannot find the model subgraph at their documentation or github. The application that I am working on has the hand close to the screen which I sometimes results in inaccuracies.

[@IvanGoncharovAI]

Hey! Interesting question that I haven't really explored yet. My guess would be do kind of get this thing to work locally and then look through mediapipe files until you find those tensorflow lite neural nets, and then get them to work in Keras/TF or something like that and try fine-tuning? Maybe I am overthinking it but not sure how else to approach this if - as you're saying - they're not explaining that in the docs

[@-LTUIiiin]

Great vid. I would like a pointer one from you too!

[@shawnloo8577]

Hi Ivan! I wanna ask why do I get an UnicodeDecodeError when I try to run the app.py? The error states that 'charmap' codec can't decode byte 0x81 in position 15718: character maps to

[@freenomon2466]

hey thanks for sharing. : ) Is there a way to get the wrist rotation data from this? I'm trying to retarget the finger motions (which I got it working somewhat), but I can't find a way to get rotation data of the hand itself.. (ex: hand facing palm up or down). any tips? thanks!

[@IvanGoncharovAI]

I am not sure if such data is present (but can be wrong as I haven't thought about it in some time). I think if you treat hand rotation data and train it as gestures, you can get something reasonable in this way.

[@rryann088]

Hi Ivan! This is a fantastic tutorial! ♥Really helpful for starting to learn gesture recognition as you have clearly explained everything!! :) I heard the palm detection method and Mediapipe for the very first time... thanks to your video. Could you tell what can I do for recognizing a motion based gesture rather than static gestures? I am trying to do it as a part of a research and really want to solve this issue. It would be very very helpful if you can give your valuable guidance. Also is it possible to connect with you in discord or any other platform? Thank you very much!! 😊

[@IvanGoncharovAI]

Hey Raiyan! Thanks! Motion gesture recognition would be quite a bit more of a complex task. You'd need to classify for each frame the static gesture and then combine that with the data about how the hand is moving. I honestly haven't dove into the topic that deeply myself but it would be quite a bit different from the approach in the video. Maybe you'd even need some sort of 3d hand landmarks estimation thingy.

[@rryann088]

@@IvanGoncharovAI Thank you so much for your input Ivan 😊😊 I am actually struggling to figure this out. Please let me know if you come up with something that can solve the issue or if you have plans to make a video! It will be highly helpful :) Love your videos anyways! 😃😃

[@lailagaber5434]

@@rryann088 Hey Raiyan! Did you find a motion based gesture recognition tutorial?

[@rryann088]

@@lailagaber5434 sorry but no, I havent. Are you working on it?

[@StefaanHimpe]

@@rryann088 i think you can use a classifier on the point history to do that (actually the repo seems to contain code for this already)

[@mukilankrishnakumar5591]

Appreciate your work bruv

[@IvanGoncharovAI]

Thank you Mukilan!!

[@Nicholas_Carey]

Hello Ivan, is there a way to add audio to the gestures? Like can we make it so when we sign "hello" can the AI say "hello?

[@yoggjoshi07]

That could actually be a really good addition might try doing it with sapi5 and os modules

[@gouthamsuresh5672]

Hi Ivan, the tutorial was amazing and worked perfectly. I have a small doubt, i wanted to add the accuracy of detected class on screen along with class name. But I wasn't able to figure out how to get the accuracy value from the model. Could you please help?

[@IvanGoncharovAI]

Hey ! responded to your other comment. P.S. it's really easy to do and you may also look through other examples of how it's done with Keras nets to apply it here easier.

[@gouthamsuresh5672]

@@IvanGoncharovAI Thank you!! , will try it

[@aliahbarus489]

thank you sir for the video very helpful, may I ask something sir? there is a file keypoint_classification and point classification history, I see both of them use different models, namely NN and LSTM. does it compare the two algorithms? please help explain

[@IvanGoncharovAI]

Hey! I am not covering point classification in the video but it makes sense that it uses something different for making predictions as the task itself is different

[@quanganhnguyen3012]

Sir! Thank you for a very awesome video. But when I did like you, I have trouble in Jupyter notebook. I have an error in "model.fit( X_train, y_train, epochs=1000, batch_size=128, validation_data=(X_test, y_test), callbacks=[cp_callback, es_callback] )" It said "InvalidArgumentError Traceback (most recent call last) Cell In[32], line 1" I don't understand why and how to fix this problem, can you help me with this error? Thank you very much!

[@swamynathan2772]

I too got same problem InvalidArgumentError Traceback (most recent call last) Cell In [11], line 1 ----> 1 model.fit( 2 X_train, 3 y_train, 4 epochs=1000, 5 batch_size=128, 6 validation_data=(X_test, y_test), 7 callbacks=[cp_callback, es_callback] 8 ) can clarify this???

[@NIHAL_THAKUR]

hey could you pls tell me how do i integrate a volume reduction function when the clock-wise movement of the hand is detected , thanks

[@fuadwidiatmoko6803]

Hi Ivan! This is a fantastic tutorial! what if I need more than 10 classes? is there any method to do it? thanks

[@IvanGoncharovAI]

Yes! thanks! I've answered that several times in the comments already, if you don't mind looking through them

[@shreyaprakasan3891]

Hey! Very informative video!! Had a query - is it possible to use Mediapipe for detecting Dynamic Gestures as well ??

[@vailankasimrandias4158]

Hey are you working on dynamic gestures

[@user-oo1le7yx4x]

what about when you want to train a gesture that incorporates two hands?

[@jyotiprakash5137]

there is a certain early stopping while training the data is there anything i can do to solve it

[@joart333]

Finally Ivan is back😎😎😎

[@IvanGoncharovAI]

I am! Thanks! :)

[@rohaneesenanayake145]

Great explanation! I want to remove the thumb from the Hand landmark tracking Mediapipe model to only track my 4 fingers without the thumb. How can I do that? So far I was able to remove the unwanted CONNECTIONS by defining a custom connections list. But still landmark dots/points visible. How can I remove them?. Thank You.

[@IvanGoncharovAI]

So, in the video where I show on the mediapipe website there are indices of which points correspond to which numbers, I think that you could do something around that since the thumb would always have the context index at which its points are located in the Python list we get with those points

[@aureate_ir]

Thanks, it was really useful.

[@user-ni4fk6uk2y]

Is there any method to get voice output along with this detected text,ie how to convert the text displayed on bounding box converted to speech....

[@abdullateefhussain6521]

Hi Ivan, great video although i have a question about which algorithm did you use for prediction, if anyone knows please help

[@IvanGoncharovAI]

I think it's just a simple MLP architecture of a neural net implemented in Keras

[@noah4769]

Hey Ivan, super helpful video. I got stuck on one part for a project I'm doing. How would you write an if statement for no hand being detected. I am trying to reset a counter when there is no hand on screen, but cant seem to figure it out so right now it just resets when it sees a different gesture.

[@IvanGoncharovAI]

I think mediapipe somewhere in the code must return somewhat of a list containing landmarks for different hands, so you can see if the length of that list is 0,1 or 2. For no hands, one hand and two hands. Or something like that is one way to do it.

[@rohanp7177]

Thanks, man No errors ✌

[@melvinshibu3755]

I am unable to train the model by pressing the key 'k' I am not sure what is the issue can someone help me

[@IvanGoncharovAI]

Sometimes if your language on the keyboard is set wrong or if you haven't clicked on the OpenCV window this can happen

[@germanaspinelli8525]

Hi what type of neural network have been used?

[@nagatouzumaki8750]

Such a helpful Gesture Recog video! And thanks for the TimeStamps!

[@IvanGoncharovAI]

Glad you appreciate it man!

[@rashadal-hamhami8064]

Hey Ivan. Any recommendations on how to achieve this project in a web app ?

[@IvanGoncharovAI]

Well, it's a two-part project. First you could see if Mediapipe can be used within JavaScript, and then you could see if you could get that small Keras neural net to work there as well. The latter sounds like the most likely one to and the more, I think, may-or-may-not-work one is the Mediapipe part. After that, probably, you could try to pull other algorithm pieces that tie the both parts. Alternatively, I think, that if you develop, like, a website with Python (maybe, like, Jango or Flutter or something) you could have a much easier time probably integrating this project in it as it's itself in Python

[@lintingwei]

This is awesome! Thank you!

[@IvanGoncharovAI]

Glad you like it!

[@jeffreyhao1343]

Great, Ivan. But I have a question: the default hand skeleton of mediapipe looks weird. If I want to replace it with a real virtual hand model, what should I do? Thanks

[@IvanGoncharovAI]

Thanks! So, there are a bunch of simple OpenCV drawing functions under the hood (drawing lines and circles one by one). You may wanna look into somehow building up on top of those drawing functions but in a different way.

[@davimoreira170]

nice video ivan! thanks a lot!

[@IvanGoncharovAI]

Glad it helped man!

[@srijitpal5126]

Hi Ivan ! Can i use this concept as my college project? Can you suggest me something to do more interesting my college project with this project?

[@karthikarumugam121]

I have to feed an input video other than a selfie cam from my laptop, but when i do so it recognizes left hand as right and right hand as left, i guess it has something to do with mirroring, how can i change the code so that it works on video which are not from a selfie point of view and recognize right hand as right and vice verca?

[@noah4769]

I believe he mentioned that somewhere in the video

[@filipesantos7086]

Ivan, thank you for the video! Great job! Is it possible to dockerize this script to run on a rasp with a usb webcam? If so, how would you do it?

[@IvanGoncharovAI]

Thank you! I haven't had experience with docerizing things, so can't answer that. But it's basically TensorFlow and OpenCV running in the background, so if it's possible to docerize, then maybe

[@noah4769]

You should make a video on Yolov3 Ivan, this was incredibly helpful!

[@jorgesanchez4831]

Great video!!

[@ahemeida3712]

Hey! I'm running issues with the model and I was wondering how to reach you for some advice or if anyone else is willing to help. I need to get this working in the next 2 days.

[@otkrivlenie]

How to find the direction the hand moves? I have signs where it needs to go to the right to be one label, and still to ramain as other label. Is there a way to do that in this program?

[@saivikramangk8720]

Did you find how to do?

[@TechWithShinOfficial]

can you tell me how to improve the distance for detection? when i moving far away the camera, above 2 to 3 meters, the hand disapeared.

[@IvanGoncharovAI]

do some sort of crop with the area of interest with your hands are and pass that through mediapipe and the rest of the pipeline in general as an option

[@cookiemonster6963]

Hi Ivan! Is it possible to calculate the accuracy of this model?

[@rentaroiino1789]

How would I be able to get a confidence percentage for each recognized hand gesture?

[@rentaroiino1789]

or possibly thresholding data?

[@Argusom]

@@rentaroiino1789 Heeyy hie did u get the codee?? or else atleast the datasets running for it?? If so could u please help mee i need it for my project plss

[@rentaroiino1789]

@@Argusom im sorry for the late reply but i did

[@user-tw3rn7bv7b]

Can I use this one to control a DIY music media player?How can I sent the recoginiton result to the player ?☺

[@ashif_107]

cant we add 11 classifier label .. if yes how so .. plz explain

[@bixpii]

Hi. Can i use rtsp link from ip camera to recognition?

[@arpy8]

anyone else getting this error: TypeError: 'PointHistoryClassifier' object is not callable

[@zephy01]

which interpreter should i use for this project

[@bibekshah1757]

hello ivan , I want to integrate this Custom Hand Gesture Recognition into a video calling system , how do you think I should start it and can you give me some ideas to what exactly I must do ?

[@syeddaniyalbokhari502]

What are the algorithms you have used here?

[@devadevu6353]

the part where jupyter notebook is opened from folder using cmd, im geting error in it .. its showing 'jupyter' is not recognized as an internal or external command, operable program or batch file. what to do ?

[@IvanGoncharovAI]

Maybe you need to sort out jupyter module installation in Python, should be easy enough, look it up

[@devadevu6353]

​@@IvanGoncharovAI thank u i sorted it out.

[@joart333]

Great Tutorial 👌👌🙏🙏

[@IvanGoncharovAI]

Thank you!!

[@sameershahbaz8821]

thats very very helpfull sir, sir i am trying to create a sentence on the screen while gesturing, is that possible if it is so can you tell me how to do that. ill be very very gratefull

[@IvanGoncharovAI]

For sure! You can put text on screen with cv2.PutText (you can look up the exact command) and before that you need some sort of algorithm written in python where, say, you press some keyboard key (OpenCV can help with that too - meaning getting input from the keys pressed when you click on the window that shows up) and that records the key, adds that to some variable in Python and that that variable gets displayed with cv2.PutText on screen somewhere. That's kinda how I would do it probably.

[@sameershahbaz8821]

@@IvanGoncharovAI Thank you very much sir, but last thing i wanna ask i know u r soo busy but ill appreciate if u help me with that, sir the thing that i wanna ask that how would i increase number of classes and which line of code in app.py i would have to change inorder to increase the classes.

[@IvanGoncharovAI]

@@sameershahbaz8821 Sure! You can just the variable in which the input is sent from cv2.waitKey and then add, say, +10 to it. Thus, whenever you press 0 it'll be 10, press 1, it'll 11 and so on

[@sameershahbaz8821]

Um u mean to say I add key = key + 10 after key = cv.waitkey(10)?

[@IvanGoncharovAI]

@@sameershahbaz8821 Yeah, something like that. The result should be that they're written to the csv file like that

[@hudaasaad7862]

Hi ivan, that helps alot! I'd like to ask if I can get the full code because I'm trying to build something similar on my robotic arm project.

[@IvanGoncharovAI]

What code parts are you missing in particular?

[@adagakiaki9642]

Thank Ivan, that helps alot. I have a question that I want to put the accuracy rate (detection_confidence ) on the review time screen. How can I do that

[@IvanGoncharovAI]

Glad to hear it. You can use cv2.putText to print things on the opencv window. You can find how to use it with a quick google search

[@gouthamsuresh5672]

Hi, i was looking for a way to do the same. To print accuracy along with the predicted class name. Do you know how to get the accuracy value ?

[@IvanGoncharovAI]

@@gouthamsuresh5672 Well, I guess you're referring to the activation value of a particular neuron in the output class aka the confidence score. (Idk, I typically think of accuracy as being a training/testing evaluation metric rather). So, what you do is you look through the code and there should be the output of the Keras neural net. Now what it's likely doing is using argmax to get the index of the neuron with the highest activation. What you can is also get the value of that neuron and there's your confidence score.

[@sreerag.a8859]

How can i add more hand guesters???

[@bikramghosh3167]

Hi Ivan! I just wanted to know suppose i have that android solution API for mobile devices running fine will i be able make changes like u did in the video thanks mate for the content :)

[@IvanGoncharovAI]

Hey! Do you mean whether you'll be able to run the same stuff on Android? I think it's possible as TensorFlow Lite stuff can run on Android but you'd be probably making many changes on the OpenCV part

[@ahmedabdallatif7464]

Hello Ivan! Great video keep going . I just have a question, what if i want to add a voice function to make the model speaking. I did wrote the function but i don't know where to put the function? cuz when i put it anywhere the model work but it's being so slow and lagging, plz help.

[@IvanGoncharovAI]

You would need to put that function in between frames changing probably and, ideally, work out a way for it to be voiced in a different thread so that the model doesn't have to wait with computing stuff until it's finishing voicing

[@NAVINOISYBOY]

Здравствуйте, Иван. У вас в начале показывается видео (00:10), где программа при показывании жестов выводит русские буквы. Подскажите пожалуйста как это можно сделать?

[@IvanGoncharovAI]

Привет, только что на англ ответил твой коммент

[@user-uv9zi5cq9d]

Hi Ivan, thank you for an amazing video! Could you tell me please if there is a way to train the system to recognize not only static gesture as you show in tutorial but like a motion? I mean motion by hand. For example, like an entire hand making circle, or hand moving on screen on the left or on the right. If yes, what is a possible way to do it? Заранее спасибо

[@IvanGoncharovAI]

Hi! Interesting question. So, in the repo that I talk about there's also point history recognition option where if it detects you doing a "point finger" gesture you can draw something on screen and the model will recognize it, so you could potentially hack around with it and have it detect point history of any hand gesture moving in some way. Also, if the task is really simple you could possibly program something yourself (if we're talking, like, "detect if the gesture upon appearing has moved right in the first 10 seconds" or something like that). Пожалуйста)

[@user-uv9zi5cq9d]

@@IvanGoncharovAI thank you very much, Иван, I'll try! 🤩

[@alexandragoncharova9997]

❤

[@a_09_shreyabhalgat25]

hey @@user-uv9zi5cq9d ...did you find anything for motion detetction?

[@Cordic45]

hello, thanks your intro. What is the purpose of training if we aleard can use landmark to "recognize" the Gesture ?

[@IvanGoncharovAI]

We use training to add new custom gestrues

[@Cordic45]

@@IvanGoncharovAIyes I know. what is the advantage of traing the AI model instead of useing landmark to do it. For example, if the 8th point is lower than the 6th point, it means that the forefinger is not lifted.

[@inessebti1239]

Hi, amazing video ! I was wondering how do you access to raw data of each "sessions" and if you can transform them to just have a csv file with the classifications e.g: Ok sign, Open sign, instead of coordinates

[@IvanGoncharovAI]

Thanks! I don't think the raw data of each sessions is saved automatically, what is saved is when you press a key and log it to a .csv file - aka your training data. You can probably work out a way to save all of that stuff in the code with a bit of trial and error though, but I don't think it's present there right now.

[@inessebti1239]

@@IvanGoncharovAI Cooll thansk ! do you know which key to press to log it to a .csv file ?

[@IvanGoncharovAI]

@@inessebti1239 I think I mention it in the video. "k" if I am not mistaken; it's defined in the code.

[@IvanGoncharovAI]

@@inessebti1239 It says it in the repo docs

[@inessebti1239]

@@IvanGoncharovAI Cool thanks! Other question, do you know how mediapipe gives timestamps, I understand that it is a real time calculator but do we know whether It gives a row every 10ms or 20ms or is it by frames ?

[@VandanaRD]

Hi Ivan, thanks for tutorial, how to run mediapipe in windows with gpu

[@ECEAtulDhiman]

i want to add more than 10 gestures,how can i add them when we can usly press 0-9

[@germanaspinelli8525]

In logging_csv function you have to change writer.writerow([number , *point_history_list]) to writer.writerow([number + increment, *point_history_list]) where increment = 10 allow you to save symbols from 10 to 19 (button 0 = 10, button 1=11, button 2 = 12 and so on), increment = 20 allow you to save symbols from 20 to 29 and so on. But you will press button from 0 to 9. To see that you are pressing and saving the correct button, in draw_info function you have to add increment to number value. I don't know if i am clear

[@hrushyang4441]

Hey hi, great tutorial. Really helped get through my major project. Can you please share the github repo of your student's project which included recognizing the russian sign language. It be really helpful. Thank you in advance!

[@IvanGoncharovAI]

I'll see if that's possible, great suggestion

[@_SyekaBagusP]

@@IvanGoncharovAI Hi Ivan Can you please share the github repo of your student's project which included recognizing the russian sign language. It be really helpful. Thank you

[@sriram-kj2gj]

Sir, Can you make a video using any ML algorithms for hand gesture recognition?

[@shireesh2209]

Hey bro . What type of neural network did you use in this project?????

[@IvanGoncharovAI]

Hey, it's a Keras neural net + Google's Mediapipe (which is several neural nets working together)

[@nileshchandrawanshi6376]

How to resize the camera

[@alankuo2727]

Fantastic video. Is it possible for the code to read a video instead of use webcam? I changed line 56 to cv.VideoCapture('\video address') yet it doesn't seem to work

[@IvanGoncharovAI]

Yeah, it's definitely possible and how I would do it is close to how you're describing it. I'm not sure what's the problem in your case specifically, but, yeah, I would pass a video path and I just suggest that you do more debugging because it can be, like, a 100 little silly error thingies that are wrong. Maybe filepath is wrong, maybe you forgot something additional. But yeah, it should be possible.

[@alankuo2727]

@@IvanGoncharovAI i fixed it!! i used OBS's virtual webcam and put my video through that and it worked

[@burakkorkmazcan7658]

@@alankuo2727 Hi Alan, I have the same issue, can you explain me in detail, how you did it with a video? I would really appreciated!

[@riyadahiya1737]

Hi Ivan! How can we save more than 10 gestures?

[@a_09_shreyabhalgat25]

how to generate point.csv?

[@NAVINOISYBOY]

Hello Ivan. You have a video at the beginning (00:10), where the program, when showing gestures, displays Russian letters. Please tell me how can this be done? And one more question. How can I add more than 10 gestures?

[@IvanGoncharovAI]

Hey man, the "more than 10 gestures" Q was asked many times in the comments, just look for it. In terms of the letters, I think OpenCV supports Cyrillic alphabet, so it's just the matter of naming your gestures as Russian letters in this way.

[@NAVINOISYBOY]

@@IvanGoncharovAI Я уже пробовал назвать жесты русскими буквами, но это не работает и я не знаю как это можно исправить.

[@IvanGoncharovAI]

@@NAVINOISYBOY а в чём проблема?

[@NAVINOISYBOY]

@@IvanGoncharovAI Проблема в том, что, когда я меняю название жеста с английского на русский, то просто выводит знаки вопроса над жестом.

[@IvanGoncharovAI]

@@NAVINOISYBOY погугли это связано с тем как OpenCV в твоём случае почему-то воспринимает кириллицу.

[@abigailarun1449]

The video is a useful one but what shud i do if i wanna add a 10th gesture...can u please help me out!!

[@IvanGoncharovAI]

Hey, thanks! I've answered that already in some of the comments, could you look through them?