Real Time Sign Language Detection with Tensorflow Object Detection and Python | Deep Learning SSD

Language barriers are very much still a real thing.
We can take baby steps to help close that.
Speech to text and translators have made it a heap easier.
But what about for those that maybe don't speak or can't hear?
What about them?
Well...you can begin to use Tensorflow Object Detection and Python to help close that gap. And in this video, you'll learn how to take the first steps to doing just that! In this video, you'll learn how to build an end-to-end custom object detection model that allows you to translate sign language in real time.
In this video you’ll learn how to:
1. Collect images for deep learning using your webcam and OpenCV
2. Label images for sign language detection using LabelImg
3. Setup Tensorflow Object Detection pipeline configuration
4. Use transfer learning to train a deep learning model
5. Detect sign language in real time using OpenCV
Get the training template here: github.com/nicknochnack/RealT...
Other Links Mentioned in the Video
Face Mask Detection Video: • Real Time Face Mask De...
LabelImg: github.com/tzutalin/labelImg
Installing the Tensorflow Object Detection API: tensorflow-object-detection-a...
Oh, and don't forget to connect with me!
LinkedIn: / nicholasrenotte
Facebook: / nickrenotte
GitHub: github.com/nicknochnack
Happy coding!
Nick
P.s. Let me know how you go and drop a comment if you need a hand!

[@oraredd3042]

For those who came here after seeing the Indian girl's post, yeah Priyanjali Gupta plagiarised this repo 😂

[@ritwikagrawal1951]

😂😂 exactly here for that

[@user-sw2nn7fw6o]

Exactly, am just arriving from threads

[@sharjeeljavaid8151]

can i get a link please

[@aneenaandrews8005]

But she has mentioned that she used his tutorial for making this model. I think she was just trying it out.

[@alanmcoll101]

The most logical and understandable guide on the subject I have seen. Great job.

[@NicholasRenotte]

Thanks so much for your kind words @Alan 🙏!

[@samrogers806]

dude. you. are. sick. thank you man. keep pushing. you could really become something big. super high quality content. cheers

[@NicholasRenotte]

Thanks so much @Sam 🙏!

[@shankhabhattacharya8921]

Awesome tutorial. I am really glad that I found you!

[@dedisupardi2815]

You are my Prof in python. Thank you so much for all, the codes you share, the video and the tutorial you gave to us. 👍

[@irugal8964]

Dear Nich Sir, I was a software engineer student, yesterday my results has come out and I got a first-class honor for my BSc degree, For the final project, I got 80 marks. My final project was Sign language detection in my language (Sinhala language). That project was kinda Advanced, but your sign language detection videos series inspired me to make a better project for my finals. So Thank you soo much and wanna say this, again and again, YOU ARE THE BEST AI TEACHER IN KZbin, Keep making these kinda videos and make others' life brighter like this. I think you should pin this comment. ⭐⭐⭐

[@NicholasRenotte]

This is amazing news!! Congratulations!!!

[@irugal8964]

@@NicholasRenotte Thank you Sir

[@sukanyadessai6238]

@@irugal8964 hey how did you do it? I'm not able to use tensorflow. I'm error version error. What should I do? Please please help me🥺

[@irugal8964]

@@sukanyadessai6238 kzbin.info/www/bejne/movLkKOpbcx-ndU this is the video i followed and setup all correctly. Also for the data training i hv used colab.

[@prathi.editss]

hey i'm working on same project and got few errors can u help to solve it

[@johndoe4073]

This is really impressive! Can't wait to try it out.

[@davidskopalik9073]

I was looking for helpful vids on my project and this guy already did my project. Nice

[@NicholasRenotte]

Ayyyeee, glad you found it @David!

[@adamputra8288]

How do i contact you?

[@aankitaaa]

hey @davidskopalik9073 did you do this project i am facing some issue could you help me out i'll be grateful if you could

[@davidskopalik9073]

@@aankitaaa i mean it has been years not even sure if i have the files but i can try what's your issue

[@AamirMaqsood-sn7gq]

Can you please help me in executing the same project?

[@rewriteending]

this guy is awesome, hope to work with you one day Nicholas

[@NicholasRenotte]

Oh thanks so much man!! ✌🏽

[@NicholasRenotte]

Hey y'all! Wrote up a couple of steps to setup your environment. I'm planning on doing a video on this but until then, here's a crash course post: 1. First up install Python 3.7.4. Download and install the package for your OS that has the words 2019.07 in it from here repo.anaconda.com/archive/. This should give you 3.7.4 to work with. 2. Then install Visual Studio C++ 2015 from here: go.microsoft.com/fwlink/?LinkId=691126. Tensorflow needs this in order to compile 3. OPTIONAL IF YOU HAVE A GPU - Install Cuda and Cudnn. Install Cuda first, then install Cudnn. - Cuda: 10.1 - developer.nvidia.com/cuda-10.1-download-archive-base - Cudnn: 7.6.5 - developer.nvidia.com/rdp/cudnn-download Once Cudnn is installed you need to copy the Cudnn files into their respective folders inside the Cuda directory. I used this as a guide: towardsdatascience.com/installing-tensorflow-with-cuda-cudnn-and-gpu-support-on-windows-10-60693e46e781 4. Then install Protoc 3.13 from here: github.com/protocolbuffers/protobuf/releases. For windows, download the repository and then add it to your PATH file. 5. Then install python packages using the pip command: Tensorflow: 2.3.1 - pip install tensorflow==2.3.1 OpenCV: 4.4.0 - pip install tensorflow==4.4.0 6. Install the object detection API. To do this, run these commands from a command prompt or terminal: git clone github.com/tensorflow/models cd tensorflow/models/research protoc object_detection/protos/*.proto --python_out=. cp object_detection/packages/tf2/setup.py . python -m pip install .

[@rashmigaikwad1116]

I have followed all these steps ..but still i am getting error as object _detection module not found... What should i do.. Please help me out here!!

[@enamulkarimadib]

@@rashmigaikwad1116 Same here. Did you manage to solve it?

[@kaansevinc6182]

OpenCV: 4.4.0 - pip install tensorflow==4.4.0 (is this a Typo do you mean pip install opencv-python )? at step 5!

[@kenneths9839]

will this work with later versions of python?

[@zaheersuhabuth2677]

I'm loving the contents of this channel!

[@NicholasRenotte]

Thank you so much @Zaheer!

[@datapro007]

Nick - Love your channel - Thanks for this video. I was still struggling after the "face mask" video. This one clarified a lot of the important points. It gave me a much better understanding of what is going on than just watching the first object detection video twice... ok ten times ;-) . Speed of training - I have an AMD RX 580 video card instead of NVidia so I'm stuck with training via my AMD 3700X CPU (or am I?). No hindrance to the learning experience of course, but if I had only known last year when I built up my systems... sigh. Thanks again, you are truly a terrific teacher!

[@NicholasRenotte]

Ohhhhhh I know the pain of getting the GPU to work. Yep, unfortunately at the moment only NVIDIA cards seem to dominate the deep learning space. I think there's work being done on the AMD cards but it's still a while away. SOOOOO pumped you got it all working though!!

[@datapro007]

@@NicholasRenotte Your videos are among the scarce few worth watching, so yeah!

[@nyharenafitahiantsoaraheri3557]

Wouaouhhhhh... Man, your amazing :) ... This is what I was searching for after some months of Machine Learing study

[@NicholasRenotte]

🙏🙏🙏 thanks so much, glad you enjoyed it!

[@nyharenafitahiantsoaraheri3557]

​@@NicholasRenotte No, thanks to you with your awesome works, awesome technics, ...But could you make a video on how to save this model ? ( Save the mobel in tflite format, tf.js format for example?)

[@NicholasRenotte]

@@nyharenafitahiantsoaraheri3557 definitely, it's coming in the next couple of videos!!

[@nyharenafitahiantsoaraheri3557]

@@NicholasRenotte Ouuuupiiiii ,,, can't wait to those videos :)

[@nyharenafitahiantsoaraheri3557]

@@NicholasRenotte I have already test an object Detection project in flutter application that uses the SSD mobilenet and YOLO and if I am not wrong I think by changing just the .tflite model and the label.txt by this one, I think it will work because I think your model extends to SSD mobilenet

[@ishmeetahuja9343]

I finally completed this project Thanks Nick for your wonderful explanation and super simple guide lines 👍 Amazing video

[@NicholasRenotte]

YESSSS! Amazing work @Ishmeet!

[@shubhamnagalwade4642]

@IshmeetAhuja Can your help for this project. I got error during running the tutorial code.

[@ishmeetahuja9343]

@@shubhamnagalwade4642 can you tell exactly which step

[@shubhamnagalwade4642]

@@ishmeetahuja9343 I got error during config file. When I run code config_util.get_configs_from_pipeline_file(CONFIG_PATH) Error is:- Message type "object_detectio.protos.TrainConfig"

[@NicholasRenotte]

@@shubhamnagalwade4642 hmm, that's the full error? Sounds like the protoc step might not ahve been run during installation. Can you try running through this: kzbin.info/www/bejne/movLkKOpbcx-ndU

[@alifia276]

Excellent explanation... Thanks for sharing!

[@andrescubides3387]

Just in time, I'm developing a project for my master's degree about this topic, thanks for the content

[@aankitaaa]

hey did you do this ?? i am facing some issuues could ou help me out?

[@josephattia6040]

How do you not have more views? This is amazing content!

[@NicholasRenotte]

We're getting there @Joseph! Either way, a tonnnn more videos coming! I've got 285 tutorials left on my list to do (and it keeps growing each week) 🤣!

[@ypm7369]

Finally a video that makes sense !!!

[@NicholasRenotte]

🤣 Oh I know where you're coming from, it was a nightmare coming up with the walkthrough for this. The content and documentation is haphazard as hell. I just need to make an installation video now!

[@basics7930]

@@NicholasRenotte True

[@ypm7369]

Haha I bet it was laborious, by far thought this the best video in KZbin that implements some sort of tracking hand gesture that makes sense all the way through lol

[@NicholasRenotte]

@@ypm7369 thanks so much 🙏 , it honestly means a lot you saying that!! I'm working on how to export the models and work with them in Tensorflow.JS apps right now...there's also sooo little documentation for how to do it properly.

[@NicholasRenotte]

@@basics7930 better documentation anyone 😅?! #amirite?

[@travelfreak1246]

Really good explanation.. and real time experience is awesome.. thank a lot.. 👏

[@sunkekishorekumar5900]

is it working model ?

[@yonipaul6077]

Nick, you are amazing! All thanks to you for making a well organised video with soo much clarity 💥💥 .... I did my first deep learning project based on your teaching and safe to say that it was a HIT among my professors.... I look forward to make more of such projects... Thank you so much Prof. Nick .... You are the BEST⭐⭐⭐⭐⭐

[@rajasrithiruvarasan3164]

@yonipaul6077 how did you train the model with tensorflow 2x??

[@rohilkulshreshtha2875]

This is amazing Great content....Really helped me with one of the projects I was working on Keep making videos like these

[@NicholasRenotte]

Plenty more coming @Rohil, sooo glad you enjoyed it!

[@sumit2670]

@@NicholasRenotte please help me

[@DhanunjayaSrisailamTTT]

This tutorial is great. Love the explanation

[@NicholasRenotte]

So glad you enjoyed it @Dan Jay!

[@_buffer]

Hi! Is it possible for me to generate a *.pb file so that I can use it the way I'm used to? By specifying a config path and settings it to the *.pbtxt and the weightPath which would be *.pb?

[@NicholasRenotte]

Heya @BinaryBunny, should be possible, I actually walk through an example of saving a frozen graph and coverting to TFJS here: kzbin.info/www/bejne/kIW2g42qZZl9obs I believe there are some other converters available as part of the ODAPI in the models folder!

[@nicolaicaruana9950]

Dear Nicholas, I would first like to thank you for showing us these amazing tutorials, they are nothing short of educational and fun. I came here asking for help though, if it is ok with you. I am developing a similar project to yours where the Object detector would figure out which technique in Judo it is observing. I am trying to run the project and I followed the tutorial to a T. I am able to train the model, but the loss value always starts at NaN (technically I am not able to train the model, but step 6 is within reach, because I can execute the command). I have gone through the comments and have done all the suggestions, which include: 1. Double checking the labels 2. Decrease the learning rate (although that is only viable if the loss value does not start at NaN) 3. Used 2 different datasets (1st is the Judo dataset, the other is the image dataset that was created, the same as yours). 4. Followed the installation to everything on your previous videos (Tensorflow installation + CUDA/CUDNN and Face mask detection tutorial) It would be greatly appreciated if you do end up helping me. But if not, this is still a very interesting video.

[@gauravsontakke5891]

same issue bro if get the solution please tell me

[@tibtrader]

This is lit 🔥 - keep it coming!

[@NicholasRenotte]

Oh you know it! A tonnn more stuff coming in this space @Tenzin!

[@Luca-yy4zh]

Hi Nicholas. Thank you. Do you also have the final model that you trained with your images? I'd like to convert it to tflite and try it.

[@NicholasRenotte]

Heya @Luca, I don't have it saved anymore unfortunately. I've got this one available but it only detects hello (not trained for that long though): github.com/nicknochnack/TFODRPi/blob/main/detect.tflite

[@azgiraffe]

Thank you so much for this wonderful tutorial

[@SriHari-xu6zz]

Can you help in solving the error no module named object_detection in creating tf record

[@danieldominguez8135]

Thanks for the tutorials. Really helps and they are amazing! Wanted to ask you what when you want to run it without having to use the commands at jupyter notebook. Like, just a normal app or computer program? Can you help me with that step?

[@vedanggoswami2540]

Thanks for this amazing tutorial Nick. You explained everything so nicely. A quick question, if I trained this model previously for 20 pictures of each class, and now I wish to increase the training dataset to around 50 pics. Will I have to retrain the model from scratch? (Cause on CPU, it took me around 5 hours to train for 20k steps!)

[@NicholasRenotte]

You don't have to but you need to tweak your code if you want to pick up from where you left off. If so, you need to freeze your model, export it, then update your pipeline.config file to pick up training from that checkpoint (fine_tune_checkpoint parameter in the pipeline.config file!)

[@matannagar]

@@NicholasRenotte Hi! Is there a way where I can see an example of this? We are trying to train an Hebrew Sign Language model thanks to your video :)

[@akshikamde6764]

Hi I am also training the model on cpu since I don't have GPU. And 2500 steps were done in 2 hours! So should I continue or stop because my pc has become super slow...will it be ok to leave the pc overnight?

[@Zellpampe]

Great video. Appreciate your effort a lot! Two questions: If you would perform two gestures using both hands at the same time, would the model have detected both gestures at the same time? Is it possible to train for one class only? Best regards

[@NicholasRenotte]

Hi Raphael, in this case it would as I labelled images for each class using both hands. If I only used one hand per class e.g. right for hello, left for no, right for yes and trained for a while. Then there's an increased likelihood that it would only detect the class that matches the hand.

[@rockerhlm]

@@apnadekhtu How are you goin to train the model? For this the best is to use any dns to your routher like Open Dns Family Shield. Sorry for my poor english

[@jees__antony]

Your videos are so encouraging... 🙌🙌🙌

[@user-tr4bd6zu1o]

thx u so much for taking this tutorial video. I just done my program from packages installation to real-time detection. but the I think i need to change another cnn/rcnn model and train it again. because my accuracy is wired lol.

[@NicholasRenotte]

Anytime!! How long did you train for? It really varies, one of the other peeps on the channel was saying it took them 2000 steps but it might take longer.

[@farzanevafaii5820]

thank you for this great video .I just have one question. this code just recognize every sign that we give to it. if we want to have a code with all sign to recognize every sentences and words , what should we do?

[@NicholasRenotte]

You have to train your model to detect the signs that you want. So, you could add in all of the additional words you'd like to have recognized as part of your image collection and labelling @samaneh.

[@sarbajeetpaul7116]

Hello Nicholas, I know you made this project long ago, but still I would really be glad if you let know what are the algorithms used here. Can we use other algorithms to this for its better accuracy and results like KNN, logistic regression or decision tree, if yes, please guide how. And last, how to generate the classification report here. Please help as I`m really new even to Python language as I have been seasoned with JAVA by far.

[@madhavsharma288]

Hi Nicholas, I had a question, how do you convert our trained model to tensorflow lite to be used on microcontroller.

[@wassimchakroun4170]

Hi Nicho! Great tuto as usual. I wonder if it's possible to use videos instead of images for training because I noticed that many signs requires a sort of movement? Thanks in advance.

[@user-tf7yj7dj2d]

The video will be divided into many photos for training based on the frame rate, they are the same

[@raghavchawla5632]

Thanks for this amazing tutorial. I used it to train a model exactly as you did. Do you know what the best way to load the model I have trained into the C++ API of tensorflow? Thank you again!

[@NicholasRenotte]

Could try loading the checkpoints that were saved to C++. In theory it should work, although haven't try it out yet man.

[@SriHari-xu6zz]

Can you help me in solving the error no module named object_detection in crete tf

[@neelcrews]

Hey one girl has copied your code 😂 she is getting more fame than this video literally she is 😂😂😂😂😂😂😂😂😂😂😂

[@pandyasohamhareshbhaib20ee90]

Chutt bhai

[@user-pb9qw4tq3s]

had a great learning though i got a lots of errors in the way but that make me go deep into subject and taking this from scratch is awesome and again expectinga a lot like these from your side nick

[@rajasrithiruvarasan3164]

How did you train the model with tensorflow 2x?

[@silentcoder5134]

Awesome tutorial!

[@juliam.lampreajulian.lampr5318]

Are you awesome. Greetings from Colombia.

[@NicholasRenotte]

Thanks so much @Juliam, what's happening from Sydney!

[@sualaksharma3297]

Thank you so much for this tutorial Nicholas ! One more thing I wanted to ask that is it possible to read out the phrases that gets recognized by the system? If yes, please do guide !

[@NicholasRenotte]

Planned!

[@JamilKhan-hk1wl]

you can use a tts library for python

[@HK-jw2et]

@@JamilKhan-hk1wl Hey. I'm halfway through the project. Can you pls tell me how many nodes are being used in this Project? And about activation function, if possible. I would really appreciate your reply.

[@ananyadixit09]

could you include the audio output in your project? if yes then can you tell me how you did it?

[@thejuanpa88]

Amazing work Nicholas

[@NicholasRenotte]

Thanks so much @JohnDoe!

[@choijongho2]

Nicholas! I have solved all errors. Although it was not easy, all codes worked well . Thank you so much for upload this tutorial!

[@NicholasRenotte]

THIS IS AMAZINGGG to hear! Thanks so much for powering through! Deployment to RPi next?

[@simonconneely1615]

Im having an error with cv2.imwrite. any tips? thank you

[@NicholasRenotte]

@@simonconneely1615 got some errors for me?

[@meeraaltamimi4628]

can you help me please

[@abaan.a_]

Please, solve me, SyntaxError: invalid syntax how to ?

[@mvaabox7092]

Excuse me, I want to ask. At 11:01 minutes, in the program to take pictures automatically, my program has no errors at all, looks good. The camera is detected successfully, the frame appears, and the frame can display the image from the camera. After the program finishes (camera off) a new directory appears, but none of the images are saved. What happened? May I know the explanation? Thank you before!

[@____________________________.x]

That was really well explained, thank you. What I'm confused about is how do you make a Pass/Fail object detection system? I can see how you'd spot say a normal widget on a conveyor belt, but how would you detect a broken widget, or a misplaced cat on the conveyor belt?

[@PragatiChauhanMAI]

Very useful video. Thankyou so much .

[@Jandoesrun]

Hi I'm glad I found your video! First of all, You are awesome man! Keep it up! I'm planning to do a similar project as a research proposal for a graduate school application and I was wondering how can I train my model if the sign language uses hand movements rather than still poses.

[@NicholasRenotte]

Heya Jan, you can use a slightly different architecture to do it! I've been researching it lately and it looks like there's already a model out there that handles it, check out Tiny Video Net: tfhub.dev/google/tiny_video_net/mobile_1/1

[@g.satwikasattu233]

Hi iam also searching sign language research topic urs dun research topic

[@galaxxyz777]

Hi Nicholas. Thank you for this amazing video! How long did it take you to train this model? I'm going to train a model for detection about 10-15 gestures, but I also want to make train dataset for each gesture slightly bigger. If you remember the approximate amount of time I think I can roughly estimate how long the training session will take for me.

[@tejeswanisenapathi2708]

I am doing the same thing, can you help me with this? please. That'd be very nice if you offer to help me.

[@uniquescience7047]

@@tejeswanisenapathi2708 hello, did you do it?

[@Vicky-zl6wd]

Amazing Nicholas, love all your videos. Had similar project in Matlab, but loved your python-tensorflow version the best...Amazing keep up the amazing video and content work!!!!

[@SriHari-xu6zz]

Can you help in solving the error no module named object_detection in creating tf record

[@syphin2286]

Hello Nick, I had a question regarding the last step where you ran it and it opened the camera and showed the detections. Is there a way to record which object is being detected and then do something if a specific object is detected (eg. print hello to the console if it detects hello, or print yes to the console if it detects yes)?

[@NicholasRenotte]

Definitely, I actually show how to do something similar in the Automatic Number Plate Detection tutorial. It shows how to add a logic gate as part of the detections!

[@ypm7369]

Hi Nicolas, finally I have been able to run everything. Although my recognition is not that accurate . I guess it would be nice to automate the part of selecting the gesture in every picture 🤣

[@NicholasRenotte]

Oh, that's part of the fun!! Here's some stuff you can do to boost your accuracy: 1. Train for longer 2. Add more images (possibly some with different backgrounds and clothes) 3. Add images with varying poses and levels of blur or distortion!

[@ellysavatia]

For some reason, my code isnt running. Can you share the complete code I have a look

[@akshatachoukimath3534]

Well installing the object detection api messed up. Can you please help me

[@ypm7369]

What version of python, tensorflow and opencv are you using?

[@NicholasRenotte]

Heya, just updated the pinned post, all the install steps are there! These are the versions and source locations: Python: 3.7.4 - Windows: repo.anaconda.com/archive/Anaconda3-2019.07-Windows-x86_64.exe Mac: repo.anaconda.com/archive/Anaconda3-2019.07-MacOSX-x86_64.sh Linux: repo.anaconda.com/archive/Anaconda3-2019.07-Linux-x86_64.sh Tensorflow: 2.3.1 - pip install tensorflow==2.3.1 OpenCV: 4.4.0 - pip install tensorflow==4.4.0 Protoc 3.13 - github.com/protocolbuffers/protobuf/releases. If using GPU - Cuda: 10.1 - developer.nvidia.com/cuda-10.1-download-archive-base - Cudnn: 7.6.5 - developer.nvidia.com/rdp/cudnn-download

[@diegofernandomorillovelepu2829]

in step 5 someone gave this error? 96 callable (pipeline_config_path) 97 with tf.io.gfile.GFile (pipeline_config_path, "r") as f: 98 proto_str = f.read () TypeError: 'module' object is not callable

[@riya2603]

Have you completed and executed this code? Because there are certain errors that I am getting and I need some help

[@rimichowdhury8035]

awesome!!!!! too happy to learn

[@InkDoMink]

Hey, do you think its possible for the raspberry pi 4 to deploy the model? Meaning I give it the pre trained model from my pc and using the Pi Camera it can detect gestures or is it too computational for the Pi Hardware?

[@nigaralishzade3206]

Thank you so much, sir! I wonder what if I train about 100 different signs and gestures? Should I alter the layers in that config file? How to deal with overfitting and underfitting problem?

[@NicholasRenotte]

Heya @Nigar, all you need to do is ensure you have examples of all 100 signs and gestures as part of your images, update your labelmap so you have them listed and change your number of classes to 100 in the config.

[@SamirAliyev771]

Salam Nigar. I am from Azerbaijan. Are you also doing the project for Azerbaijani sign language? If so, could you please contact me for cooperation in some points? I am doing the project.

[@boozebag2000]

Hey Nicholas, in my first run I trained my model to 12000 steps. There is still a relatively high loss (0.25). How can I alter the training command to retrain starting from my latest checkpoint?

[@luvkaranwal7511]

Same here

[@NicholasRenotte]

Heya, try changing your checkpoint in your config, point the line below to the folder where your new checkpoints are located: pipeline_config.train_config.fine_tune_checkpoint = PRETRAINED_MODEL_PATH+'/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8/checkpoint/ckpt-0' Should be something like, change it to the appropriate checkpoint number based on what you've got. Then rerun the same training command. pipeline_config.train_config.fine_tune_checkpoint = CHECKPOINT_PATH +'ckpt-6'

[@simonconneely1615]

Absolute YES!

[@anjalik3826]

Hello @NicholasRenotte Sir, Thanks for the amazing tutorial! I am glad that I found this savior tutorial last minute....Thankyou so much :)

[@mickjagger4548]

Thank you so much for this content, you saved me with my senior project- which is ASL alphabet recognition (turns out MATLAB is terrible for someone new to this field)... however, how can we implement this style detector for letters like "j" and "z" which require movement? I have seen many papers about combining lstm's with cnn's but it seems like everyone is keeping their cards close to their chest and their are no actual good tutorials on this matter... what are the chances of you doing one of these within the next month or so lol?

[@NicholasRenotte]

MICK JAGGER! Yah, definitely it's on the cards. I don't know if it'll be out within the next month but I've already got action detection down in the next set of tutorials!

[@geethamadhuri1985]

Who are here after that LinkedIn post ? 😂

[@SatyamMiri]

mee :D

[@radhaa9048]

me too...lol

[@BackpackerSiddhant]

haha 🤣

[@geekXgamer]

@@MohanSinng vit full form?

[@sumanth2669]

😂😂

[@davidfuquen6660]

What a great video!

[@frodobaggins2852]

Hi can we use the same tutorial if i already have images taken instead of taking them using opencv?

[@tasneembanu9882]

Thank You so much for explaining the project code by code. I am facing a small issue. My TF records are not getting created. Do you have any idea why? I've gone through many videos of yours to understand where am I going wrong.

[@prathi.editss]

hey have u got solution for this?i'm getting same issue

[@chitranshsrivastava5880]

​@@prathi.editss I am stuck on the same issue.Do you know its solution now?

[@nishiparekh1921]

hey, were you able to solve it? I'm stuck on the same issue

[@amruthashetty3546]

Hey there! I have issue with this "2. Create TF records" step. I am trying to run this and it is processing from a very long time without any output. Is there something I need to do ?

[@NicholasRenotte]

Heya @Amrutha, are there any errors? How many images are you processing?

[@amruthashetty3546]

@@NicholasRenotte Heyy. Thanks for replying. I am processing around 1000 images . The error is stated as follows "AttributeError: module 'tensorflow' has no attribute 'gfile'". (version of tensorflow is 2.3.1)

[@NicholasRenotte]

@@amruthashetty3546 if you run pip list is tensorflow AND tensorflow-gpu 2.3.1?

[@amruthashetty3546]

@@NicholasRenotte Hii . I resolved the earlier issue . Now there's a new error "Message type "object_detection.protos.TrainConfig" has no field named "fine_tune_checkpoint_version"." and yeah if I print config, there is no such field.

[@lorenagonzales8032]

@@NicholasRenotte I also have this issue. pip list shows I'm running version 2.4.1

[@betterbrained]

Nicholas, thank you, that's a 100% subscribe! May i ask you a little question, please? I know that optimal number of num_steps parameter depends on the task and on the model. Is there a way to implement early stopping in TFOD to get rid of this hyperparameter? Thank you so much, mate! Hello from Italy!

[@sid4752]

[Errno 2] No such file or directory: 'Tensorflow/workspace/annotations\\label_map.pbtxt' I'm getting this error, can someone help me? (doing this on Collab, MacOS 10.15.3)

[@NicholasRenotte]

Heya @Siddaharth, try removing one of the slashes after annotations and flipping the other so you have: Tensorflow/workspace/annotations/label_map.pbtxt'

[@sid4752]

@@NicholasRenotte Would it matter if I did this in Collab instead of regular Jupyter notebook? I'm getting another error when doing it in Collab.. (I've already installed object-detection through pip) Traceback (most recent call last): File "/content/generate_tfrecord.py", line 29, in from object_detection.utils import dataset_util, label_map_util ModuleNotFoundError: No module named 'object_detection' I'm really sorry for bothering you with this, I just am really thankful for your videos and time!

[@NicholasRenotte]

I'm always here to help, no stress@@sid4752. It should be relatively similar, that error is because the object detection API hasn't been installed, try running through the install video: kzbin.info/www/bejne/movLkKOpbcx-ndU

[@sunidhichaudhary2455]

Hey! Thank you for this amazing tutorial, I'm trying to create a video conferencing project and want to include this as a feature, can you please guide me on how I can do that?

[@NicholasRenotte]

Haven't dug into integration with VC tools that much unfortunately @Sunidhi.

[@dhivyarooparavi1108]

hey Sunidhi, I am also working on the same project. Could we discuss?

[@sunidhichaudhary2455]

@@dhivyarooparavi1108 Hey Yuktaa! Yes sure, pls let me know how I can get in touch with you!

[@michhedeberg]

Good stuff dude. I'm and embedded software guy trying to learn some of this stuff to control my TV with sign language. Haha

[@NicholasRenotte]

Hahahah, that's awesome, shoot me a vid once you get it working! That'd be sickkk!

[@haifaalromaih1370]

Hello Nicholas, I hope you’re doing well❤️ I would like to thank you for your video that helped us alot💘. Now I’m actually in the training step, I tried to train the model in my laptop “which doesn’t have a GPU “ so it took 10 hours to train 20 photos. Is there a way or a device that I can buy that will make the training process faster in my laptop?

[@anaashaikh7342]

Heyyyy, can you please help me with my project. I'm stuck at this place. If you could help me please.

[@NicholasRenotte]

Heya @Haifa, you can try using an external GPU (eGPU) but these tend to be a little more expensive then just getting a basic gaming computer with a half decent GPU.

[@NicholasRenotte]

Definitely, what's up @Anaa?

[@anaashaikh7342]

@@NicholasRenotte heyy, thankyou so much for the reply, here is my error Traceback (most recent call last): File "Tensorflow/scripts/generate_tfrecord.py", line 29, in from object_detection.utils import dataset_util, label_map_util ModuleNotFoundError: No module named 'object_detection' Traceback (most recent call last): File "Tensorflow/scripts/generate_tfrecord.py", line 29, in from object_detection.utils import dataset_util, label_map_util ModuleNotFoundError: No module named 'object_detection'

[@NicholasRenotte]

@@anaashaikh7342 this looks like the OD API hasn't been fully installed. Try running through this first: kzbin.info/www/bejne/movLkKOpbcx-ndU

[@DhruvPatel-mg6ou]

Hey!! Mr, can you make video on how to Deploy object detection model online, plz😇.. And love your content as always🤗🤗

[@NicholasRenotte]

Here you go kzbin.info/www/bejne/aJXahIuOgqubhrs 🙂

[@DhruvPatel-mg6ou]

@@NicholasRenotte Is there name length limit for object in detection model? like in label [ ] ,'I Love You'.. if not then we can have labels like 'Hello How are you there.'

[@NicholasRenotte]

@@DhruvPatel-mg6ou I don't believe so, I think you should be fine to include longer labels!

[@DhruvPatel-mg6ou]

@@NicholasRenotte thank you for the reply..

[@vanamuthuk1706]

Frame (Not responding) ???

[@hunar.abubakir]

Hello Nicholas. I have all the steps in your other video and trained the model. Our images have taken and labeled then moved to train and test folders inside the images folder. The model can recognize some photos but when use the webcam the model unable to detect and recognize any of the signs. Could you please gives some recommendations? We have 500 images for each of our 7 signs but we use other sign language not yours.

[@pavanichadalavada1598]

I was looking for helpful vids on my project ,I would like to know how this project is different from other existing ones and how we can improve when compared to the other projects. Thanks

[@zarakjahan8430]

Sir, when i capture the images they are not showing in the collectedimage folder. Please help

[@sreelakshmideshpande6553]

Same problem!

[@NicholasRenotte]

Normally this is because the file path isn't fully defined. Do you have a folder called collectedimages (I think that's what I called it in the vid).

[@priyampratapsingh7072]

@@NicholasRenotte i did everything same, all label folders are created inside collectedimages but there's no image in there. Please help sir!

[@nigaralishzade3206]

Hi Nicholas! Thank you so much! I got this, please help: error: OpenCV(4.4.0) /tmp/pip-req-build-dglzv4yn/opencv/modules/imgcodecs/src/loadsave.cpp:738: error: (-215:Assertion failed) !_img.empty() in function 'imwrite'

[@NicholasRenotte]

Heya @Nigar, at what part of the code is this occuring? :)

[@tumtum6910]

@@NicholasRenotte I got the same error for the line cv2.imwrite(imgname,frame) (code shown in 8:58 of the video)

[@tumtum6910]

How do we resolve this error?

[@NicholasRenotte]

@@tumtum6910 heya, let's check a few things 😁. 1. What do you get if you print frame before the cv2.imwrite line 2. Is your webcam activating?

[@chazhiyong4402]

maybe could try to check for the "cap.release", it needs to be at outside the for loop

[@Sim-rm2dm]

Hi - it sounds really exciting, I wondering if the model needs to be retrained to detect movement with a new person. I missed what type of information the model is using to make the detection so I am not so sure how much generalizable it would to different people with different hand/upper limb size, skin color or objects wore on the hand (ring, wristband, watch...).

[@KratiChoudharymusic]

Hi Nicholas! Love your videos. I had a quick question, how do I tweak this to create a model that can recognize from a preloaded dataset? Should I just add the dataset in the images folder? And can I use video dataset for the same.

[@NicholasRenotte]

Yup, just throw the images into the folder. For video, gotta split it out into frames and label them!

[@SriHari-xu6zz]

Can you help in solving the error no module named object_detection in creating tf record

[@shaimaahossam1306]

Hi nicholas. Thank you for the amazing tutorial!! Im facing a tiny problem and I need help. When I run the cell 'config = config_util.get_configs_from_pipeline_file(CONFIG_PATH)' I get the following error: 'module tensorflow has no attribute gfile' I tried replacing every tf.gfile in the config_util with tf.io.gfile and it still doesn't work. How can I fix this please?

[@NicholasRenotte]

Heya @Shaimaa, this looks like a tensorflow install version issue. I'm releasing a setup video from scratch tomorrow which should solve it all!

[@shaimaahossam1306]

@@NicholasRenotte Thank you so much! I'll be waiting for it

[@NicholasRenotte]

Its out got for it :) - kzbin.info/www/bejne/movLkKOpbcx-ndU

[@shaimaahossam1306]

@@NicholasRenotte Much appreciated, keep up the awesome work!

[@NicholasRenotte]

@@shaimaahossam1306 anytime, will do!!!

[@BackpackerSiddhant]

VIT wale full credit lekr marketing kr rhe. koi to roko unhe 😂

[@rohitshailesh3449]

i was wondering why that girl is getting so famous 😂

[@amessit10]

exactlyyyy

[@belle4486]

Hello! I would like to ask does this code work for recognizing moving gestures as well?

[@DrShaikAhmad]

Hi Nick great fan of your work. Can you do a video on temporal action localisation on an untrimmed videos. I saw a lot of interest in that topic in the industry. But there are a very few repos and blogs about that. The repos that are out there doesn't cover many things in detail. This will help a lot of people and get good attention.

[@gurpreetdeveloping]

Bro some girl has copied ur code and she is getting more fame than u.. I also came on ur video after seeing her video..

[@meryy.3779]

ValueError : 'images' must have either 3 or 4 dimensions Why :/

[@meryy.3779]

In the comment below, you wrote that rerun cap.release (). But I tried it and it didn't work. I use colab :/

[@meryy.3779]

Pls help🤦🏻‍♀️

[@NicholasRenotte]

Ah, you can't capture the images using colab. It's not able to access your webcam through the notebook. Try capturing the images locally and then training on colab if you need to!

[@meryy.3779]

@@NicholasRenotte i already have a test train. Education etc. everything ok. I came to the end. The last code have such an error. Detection-opencv part. If I do it locally I have to go back to the beginning : /

[@NicholasRenotte]

@@meryy.3779 awesome, ah you don't, you just need to bring your checkpoints over to your pc and run it from there. There is a way to access your webcam from colab it just takes a little more work.

[@yaredabraha6134]

Hey NIcholas, Can I jump to this tutorial after doing the installation part from the playlist? Would it work, cuz its saying no module found cv2 when import. Please tell me easily

[@user-vw3et6ks9e]

Hello, I am a student studying with your video in Korea. The tensorflow version is not correct, so the error keeps coming out. I want to use gfile in tensorflow2, which code should I do??

[@hamzaazert6326]

great JOB man, I'd like to thank you it's very nice video

[@revathipolampalli6689]

Nick sir...this session was really helpful.. keep on rocking with coding🎉

[@rajasrithiruvarasan3164]

How did you train your model with tensorflow 2x?

[@iklilzafirah5641]

hi sir, may i know what deep learning model did you use for this sign language detection? Cnn or something else?

[@zombiesoul9740]

thank you for this great video, I have questions how I can use my existing dataset?

[@manjunath7472]

Super useful!

[@vrcanimatorking716]

i really apreciate your help with dowloanding this software

[@jonatapaulino]

Hello Nicolas. I'm a fan of your channel, and I wanted to congratulate you for the video. Could you adapt this code to recognize water drops or do you think that another video of yours would be better adaptable? Thanks.

[@vishwavadalia707]

Thank you so much for this amazing video. I have one question. Can we add voice 'Hello' along with text 'Hello' in real-time detection? If yes please guide me. Thank you again.

[@nurulaishah9695]

Hi, may I know what version of Tensorflow did you use in this video? Best regards

[@rachelyongyj]

Hi Nicholas, I ran into some issues as my folders ended up having the name : tensorflowworkspaceimagescollectedimageslabel, the photos are also not appearing in the folders. I am using a macbook pro on i9, any idea what could have gone wrong? Thanks so much!!

[@viewview6687]

How did you manage your time, both for you full-time tech job ( that might be overtime ), youtube project, and your life activities. Love your channel ❤

[@D_z859]

Hi Nick, what Python version did you use in the video?

[@vijayreddy193]

Hi Nicholas How can we print out the prediction class for a particular frame? Any idea like in which variable its stored.

[@NicholasRenotte]

Definitely @Vijay, check this out, it shows you how to do it: kzbin.info/www/bejne/mXncnoCqZriEpJo

[@reyyna9426]

Thank u so much for the video! I have an error when I’m trying to collect the images. It says “The syntax of the command is incorrect”, when my code is exactly the same as yours. Could you help me with this? What could be the problem?