How to Build an Object Tracker Using YOLOv3, Deep SORT and TensorFlow

[@TheAIGuy]

Let me know what cool classes you are creating an object tracker for here in the comments! Also if you have any issues feel free to comment them and I will try to help solve them! Thanks.

[@mahfoozalam6977]

Hi sir very very thankful to you , sir please request to make a video of object count.

[@snehitvaddi]

Hey, try something like tracking number of cars crossed a line (or) number of people entered into specific area!!

[@TheAIGuy]

@@snehitvaddi This sounds like a great idea, thanks!

[@mahfoozalam6977]

@@TheAIGuy Hii Sir, I wish your next video will come to the object counting, so thank you

[@mahfoozalam6977]

@@snehitvaddi thank you but no idea please if you have able to do then upload video I wish

[@vigneshjothilingam7624]

I wonder why there are only a few subscribers to this amazing AI man.. He deserved to have a million subscribers.

[@Meego_007]

Finally someone told the real fps ... Finally. Ty

[@teetanrobotics5363]

One of the best tutorials i have watched by far

[@ThiruSankaravelu]

Thats a awesome work man . Thanks a looooooooooooooottttttttt . Thanks a billion times even more . Yours is the only tutorial on internet to make custom object tracking in real time for multiple objects

[@randiko]

best channel to learn YOLOv3

[@TheAIGuy]

Appreciate it! Thanks

[@mangaenfrancais934]

I have learned a lot from you, kepp doing videos

[@tytofficial4180]

you are explaining very cool sir,

[@al-farabinagashbayev5403]

Keep Doing, AI Guy! you have great content where everybody can learn something new!

[@sysadmin9396]

Also! Haha. I’ve learned TONS from you man. You make things easier and more organized. Thanks a ton bro.

[@hansgabrieldaduya2565]

I hope you get more views and subscribers!!! I'm really learning a lot even if I'm a noob in deep learning.

[@sohailawan77]

awesome content waiting for your vehicles counting tutorial.

[@mattiastancred4391]

Am a total beginner and can say that your videos are very informative and helpful. A way to summarize results in a good way is my next goal to achieve - so if you could make a video where, for example, the number of passing cars / pedestrians can be calculated would have been absolutely perfect. Also, if there is a way to calculate [mean] speed of a moving object between two points and get this in a similar summary, this would have been very helpful. Thanks again from Sweden - keep up the good work.

[@aswathganesh2113]

eye opener for me in field of AI....thanks

[@y031962]

thank you so much for posting this

[@ImranAli-zo8bi]

Hi. I have a model trained on Faster R-CNN. Can I use the deep SORT tracker with it? Thank you

[@sysadmin9396]

Hey ! I’m currently following your videos to help me with building a python 24 game solver using a webcam. So far I’ve gotten the coco dataset working following your yolo and tensorflow video. I am now working on creating my own data set following your lblImg video and almost have that all ready. Now if I’m understanding this correctly? I need to feed those images to darknet ? I’m having some trouble installing darknet on my windows pc. Is there another way I can feed my own data set using your yolov3 and tensorflow conda setup?

[@TheAIGuy]

Use my "YOLOv3 in the Cloud" video to train your own custom model. It will eliminate the need for you to install darknet on your local pc!

[@sysadmin9396]

The AI Guy YOU ARE LEGIT THE BEST

[@Sebasuraa]

Thanks! I have a question: What if I wanna track just people and ignore all the other classes?

[@TheAIGuy]

You can easily add an if statement to the object_tracker.py file. Just wrap the tracker part in an "if class_name == Person"

[@mohammadmoaddi2268]

@@TheAIGuy Or he could use weights that only detect people? is that right?

[@אריאלקנטורוביץ]

Hi thank you for great tutorial. I have a question this repo can run on nvdia jetson xavier?

[@abdshomad]

Love the content and explanation. Please consider to convert the model from TensorFlow to TensorRT for FASTER inference. Strongly recommended.

[@TheAIGuy]

Thanks! I will look into this.

[@alankarshukla4385]

This is good man.

[@TheAIGuy]

Alankar Shukla thanks man! Check out my most recent video to see yolov4 with deepsort!

[@alankarshukla4385]

Sure i will do.

[@Davedav84]

Good tutorial, it's might work using jet4.6.1 with CSI camera?

[@tawfeqkamel9989]

how can i get the speed and the direction of the objects that tracked

[@김재현-m6z9y]

Thanks this helped me for my project!! but I have a question for you. There was a problem running the custom weights file... I have put the python load_weights.py --weights ./weights/yolo-obj_45000.weights --output ./weights/yolov3-custom45000.tf --num_classes 3 into running, Traceback (most recent call last): File "load_weights.py", line 34, in app.run(main) File "C:\Users\Kim\.conda\envs\tracker-gpu\lib\site-packages\absl\app.py", line 300, in run _run_main(main, args) File "C:\Users\Kim\.conda\envs\tracker-gpu\lib\site-packages\absl\app.py", line 251, in _run_main sys.exit(main(argv)) File "load_weights.py", line 21, in main load_darknet_weights(yolo, FLAGS.weights, FLAGS.tiny) File "C:\Users\Kim epos\yolov3_deepsort\yolov3_tf2\utils.py", line 76, in load_darknet_weights assert len(wf.read()) == 0, 'failed to read all data' AssertionError: failed to read all data This error has occurred. Could you tell me in detail how to run a custom weights file??

[@svendenexter79]

Nice video, all worked out for me! Could you also make a video where you explain how to make an app on iOS with your custom object detector?

[@TheAIGuy]

sounds like a great idea! will add to my list.

[@sysadmin9396]

Also, what is the difference of using this vs the tensorflow yolov3 method?

[@TheAIGuy]

This video is for object tracking while the previous video was purely object detections. Hope that helps.

[@sysadmin9396]

The AI Guy thanks!

[@lowqchannel]

btw, how do you save the video results to mp4 format with the code

[@hemantpema]

@The AI Guy, Loving your videos, nice and simple to follow. Will try to go through the others as well. What I would love to see if you can is getting your apps running on a Jetson Nano with Jetpack 4.4. I will try to get somewhere on my own, but I know with your help it would be faster - sorry a noob when it comes to the AI stuff, but learning a lot. Please keep these videos coming, love them - im working on a security system for home so would love to see more around that. This tracking video is great and would love to see it track people from one camera stream to another. Also using Blueiris so picking the streams from there and working on the models would be great.

[@adityach454]

Did you get it running it on the Jetson Nano? Even I'm trying about to run it on the jetson nano but stuck with deepSORT itself. The current version aren't supporting the code in the repo

[@donhsi]

Would be great if you can do a tutorial using EdgeTpu object detection in place of YOLO3 with Deep Sort!

[@QihanYang-b9o]

Hi, I am just starting with yolo and may I ask is there any way to display the number of object per frame? like Car: 10 Person: 15 Bikes: 10 (It would be nice if it also works for the custom detector) Thanks!

[@palanikumar9352]

yes, its possible

[@vivekpandian08]

How to calculate the speed of the object that we are tracking? TIA

[@tawfeqkamel9989]

i got a question , how can i get the values of boundray box for the object that i detect

[@TheAIGuy]

Inside the code for object_tracker.py you can print or save the variable boxes or converted_boxes depending on which format you want. These variables are on line 96 i believe.

[@chinuzoka6087]

Is it possible to save the result as a .txt file, with object names and IDs?

[@leonallen4688]

我正要用你这个项目，谢谢了

[@nivedithakatta9382]

Does the object tracking work for non stationary camera as well

[@snehitvaddi]

Facing error like--> KeyError: "The name 'net/images:0' refers to a Tensor which does not exist. The operation, 'net/images', does not exist in the graph." I am running yolov3-tiny version in Google Colab

[@seldom89]

Me too.. Have you found a solution yet bro?

[@seldom89]

Found solution.. Target tensorflow 2.1.0.. Colab latest now is rc 2.2

[@TheAIGuy]

@@seldom89 Yup making sure proper tensorflow can help fix this!

[@snehitvaddi]

@@seldom89 @The AI Guy Thank you bro, everything is working fine

[@snehitvaddi]

@@seldom89 Bro, are you able to play .avi file. Because I tried to downloading .avi file and converted to .mp4 using online tools and it is prompting, the file is corrupted.

[@ahmadullahkhan2578]

Dear sir What is the latest research problems in object tracking? Please help me.

[@maloukemallouke9735]

Thank you, do you try Digits recognition No handwriting one electronic or street numbers .... , combine with Yolo detection?

[@lucacirillo3830]

Hi! Thanks for the video, I would have a couple of questions though. I have been training my own yolov3 model and I am having pretty good results when it comes to detection, however I am facing some issues with tracking. My problem is mostly due to the fact thar my objects "transforms" into different classes as the video plays, which the detection algoritm clearly does not like. Do you think I can just extract the xy coordinates from the yolo prediction and then run a simple tracking of them, like a good ol' neirest neighbour? What would you recommend to do it?

[@lucacirillo3830]

Hi, great video, thanks, any luck extracting the data from the prediction? In particular I would be interested in the coordinates of the bounding boxes. Cheers LC

[@bakamiku9428]

Thank you for your time and effort to make this, it's really appreciated. One question I've been wondering about is how do I make a separate counter for each object? And print on the detector demo increasing total of each one?

[@TheAIGuy]

Will be having a video on this within the next week! Stay tuned!

[@inesines1119]

thank you so much for this tutorial can i open this code and run it with jupyter please

[@aaravpatel4478]

How can I pause training, change the batch size, and resume training using the same loss curve?

[@ibrahimaa9]

Can you show us how you will the last video project About yolov3 and webcam

[@NobertoWellington]

Is there an easy way to convert this code for yolov4 tracking?

[@TheAIGuy]

They are not quite the same for yolov4 as the conversion is slightly different. I will have a video out soon to go over how to convert yolov4!

[@jokosusilo303]

Hi. Thanks for the very inspiring video. Can we also get crop images from each class object detected inside of the bounding box?

[@TheAIGuy]

Working on this for my next video!

[@exoticme4760]

@@TheAIGuy where is this video? thanku

[@linkduda]

Is it possible to make a light version to run on a raspberry PI and track a mall door and the flow of people?

[@rog0079]

hey, im getting this error : File /yolov3_tf2/utils.py", line 68, in load_darknet_weights conv_shape).transpose([2, 3, 1, 0]) ValueError: cannot reshape array of size 209204 into shape (512,256,3,3) can u help me with it. I dont know whats wrong, my custom yolov3tiny works good but this happens when im trying to convert with the help of load_weights.py Thank you!

[@akashsahu410]

You have the problem in .weights file

[@nicoleiva5498]

I have a question related to performance. I have a GTX 980 and I can process most of the videos beetween 28 and 30 fps, so great! I've been developing a project to track soccer ball on free kicks and to meaure the shot speed. I know that YOLO recocnize really well the "sports balls" but: wouldn't be better to train a custom object detector that only detects soccer balls? Or is just better to stick to YOLO's accuracy? What I mean is: If I train a custom model like you did for just 1 object, the detection would be faster than the conventional YOLO? Thanks!!

[@DaveRepairs]

hi, just doing research right now. I got a project that is a sport that requires ball tracking and possibly players position, i presume i need fast camera shutter and possibly 2 or 3 cameras that need processing. It be interesting to see what the position data that comes out .. so yes I started researching if this is indeed possible

[@davidh6915]

Is it possible to track an object/person with an assigned ID and then go in front of the webcam again and be identified with the same ID?

[@TheAIGuy]

Yes that is part of the deepsort algorithm, you can play around with the code in deepsort folder, mostly track.py and tracker.py. You can adjust the age etc, and it will allow for ID's to be remembered longer etc before they are forgotten.

[@davidh6915]

@@TheAIGuy Thank you!!

[@exoticme4760]

noob question can deepsort & yolo be used for offline videos?

[@007jsteeve1]

sir why does the output vedio automaticaaly shuts down and the code stops running after 2-3 min ,is it supose to work like this??? pls help me out of this problem

[@TheAIGuy]

This might be due to the way the code decides if the video is done playing. I have it set to close when it has seen three empty frames. Maybe when you run the video for a long time it randomly sees empty frames sometimes. Try changing line 86 of object_tracker.py by increasing the count > 3 to a higher number.

[@satyam171]

Will this tutorial work if I have a Yolov4 model trained for custom objects?

[@peerfunk]

This is the first of about 20 tutorials where I actually get something done well done, only thing it could be better if you would give us a dev env in docker...

[@adilrana969]

Plz guide me to make object detection alarm on custom trained weights THANK YOU

[@TheAIGuy]

Will make a video on this soon!

[@abhinavreddyk4954]

Hey! Can you also show specific person re-identification in single camera and also multiple cameras

[@dandanny7402]

Thanks for another great video! I actually followed your previous video to make my security cam run realtime object detection through rtsp stream. One thing I found out is that using the h264 codec, the stream is about 10 secs delayed, but when I switched to MJPEG compression, it was realtime. Anyway, I have a question. Is there a way to use Google Coral to accelerate YOLOv3 detection?

[@spfcchoir]

Can you share some pointers on how to take rtsp stream please?

[@dandanny7402]

​@@spfcchoir It's quite easy. I watched this video for rtsp in python: kzbin.info/www/bejne/mX_Hp2WJp5abhqc Then in AI Guy's detect_video.py, find 'cv2.VideoCapture' and replace the mp4 files listed with your rtsp url.

[@dandanny7402]

I mean replace 'FLAGS.video' not mp4 files.

[@TheAIGuy]

@@dandanny7402 Wow this sounds really cool. I'll definitely give this a try!

[@TheAIGuy]

You might be able to accelerate YOLOv3 on Google Coral, you would probably have to use a yolov3-tiny model and tensorflow slim. Check out Google Coral's pre-compiled MobileNet SSD model, its already prebuilt to easily download onto Coral with all COCO classes.

[@emonemo2704]

Hi how do i extract each of the detected ID? Cant seem to figure out

[@hiteshsingh3439]

i am not getting bounding boxes on test image

[@ahmedzaki2668]

Hello, thanks for this great video. do you think it's possible to use these data with a third party system? like node-red? thanks.

[@khurramshafiq416]

@The AI Guy , How can we print prediction on screen and play with the value? and which variable is storing the prediction value on object_tracker.py file??, sorry i am pretty new with all AI, any help will be appreciated. Thanks.

[@TheAIGuy]

If you want to print values to the console window you can easily do this by adding a print line in object_tracker.py. There are several values that are stored relative to each object detected. Look at lines 120-130 for the variables for bounding box, class name and tracker ID number. Hope this helps!

[@khurramshafiq416]

@@TheAIGuy,Thank you so much for the quick response,I really love all your videos, I just checked the lines 120-130, they don't have the detection scores with them, I was looking to print scores with those class names as like in object detection with yolov3(in your other videos), also it would be helpful if you can tell me which print line will be able to print scores with tracking as well as detection in object_tracker.py file. Thanks.

[@khurramshafiq416]

@The AI Guy?

[@lilianleonidas9565]

Hi members, what if i need to use yolov3 spp instead of the original yolo?

[@jellabin]

Hi, It worked perfectly for the example video (downloading tiny weights), but when I use a custom model... it runs the video, but makes no detections at all... python object_tracker.py --video ./data/video/DJI_0008.MOV --output ./data/video/results.avi --weights ./weights/yolov3-tiny_704.tf --tiny -num_classes 3 .data/labels/cafe.names Am I missing something?

[@awthura9599]

Can I use Yolov4 pretrained weights instead of v3?

[@swethamanonmayee9079]

Hello Sir , I am thankful to you for the explanation as well as the repository . Can you please help if we can plot precision vs recall graph for the tracking obtained sir ,

[@sangvongoc9035]

It’s really impressive, can you please show me instructions to custom deepsort with other detection model, :33

[@stevencarrell83]

Training a model to detect if a driver is using a phone or wearing a seat belt. Out of curiosity, what GPU are you using? I've ordered a 2080ti, so hoping for a good frame rate.

[@stevencarrell83]

Also... thanks for the great videos

[@feyzadogan2388]

Hi, Thank you for your great tutarial. I tried this code on my custom dataset however objects' id does not follow each other sequentially. I saw same thing in your github repo at real time detection. What is the reason behind that?

[@ruksharalam173]

Great video!! I learned a lot. I wanted to ask how to output the class name along with the coordinates in the detections.txt because detections.txt is only showing coordinates.

[@manuklinkz2691]

Can we do this if we have an Radeon graphics card (RX 570) instead of Nvidia ?

[@TheAIGuy]

Cuda is for Nvidia graphics cards only. You can run this tutorial with CPU only as well.

[@manuklinkz2691]

@@TheAIGuy ah ok, i though that maybe with ROCm and Ubuntu that might work. My mistake thanks for clarifying. Great work by the way.

[@tamzidalam8766]

Can you please tell me from which portion of the code I can get the total number objects detected?

[@houston3414]

can we do this on google colab?

[@snehitvaddi]

Hey @The AI Guy, is everything fine?? Why are not uploading any videos??

[@shinmccold]

Can you include a video on TFlite version, how to convert the YOLO model to tflite

[@jajagege8267]

Thank you!! I have a question why we use app.run? Because i want to call main method in an other script but impossible if you have a solution.

[@BetaSh4dow]

how i close de windows when im using webcam to tracking?

[@TheAIGuy]

Bet4Sh4dow just hit the “Q” key on your keyboard to quit and stop it!

[@BetaSh4dow]

@@TheAIGuy you can explain me how to only detect determinate objects?

[@FirstNameLastName-fv4eu]

Brother Thanks, Can you help on the detection.txt. How can I add time and object name into it ?

[@manuklinkz2691]

Hi Ai Guy, Is it possible to use A.I. to generate music ? Like if I make it listen to many musical style, could it generate a new song ? If so could you show how to do it ? It think it would be something like deep dream but for music. But it would much more easy if you could show us. Thanks I love you channel.

[@TheAIGuy]

this is definitely possible. I know AI has been able to synthetically create music. I will look into making a video on this!

[@manuklinkz2691]

@@TheAIGuy Hi Ai Guy, I did some research on net, have a look at musenet . But it not clear how it work or even how to use it. I like your approach way more better and you explain this way more clearly. Love your channel. Thank you for teaching us.

[@ishamandrekar5041]

Nicely explained. Can you walk us through your object_tracer.py file code?

[@roguejrn]

Hey, amazing video man! I need to know how I can include direction sensitiveness like including whether a person is entering or exiting a particular area using the approach as a baseline?

[@feyzadogan2388]

Hi, thamk you for great tutorial i have an error bu t i cannot figure it out File "/home/afeyzadogan/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_utils.py", line 990, in check_steps_argument input_type=input_type_str, steps_name=steps_name)) ValueError: When using data tensors as input to a model, you should specify the `steps` argument. What should i do? Thank you...

[@espeneilertsen9274]

I am having the same issue, let me know if you solve it and I will do the same :-)

[@sunnydragon5379]

Gotta a problem when saved model, it says ValueError: Duplicate node name in graph: 'packed'. How can I handle it??

[@akbaraliotakhanov1221]

Thanks for the tutorial. I have a problem with saving output video from the webcam (--source 0 ). It's detecting well, but it's not saving output video from the webcam. python object_tracker.py --video 0 --output .data/video/first.avi How can fix this issue?

[@sebastianortmann4728]

my costum model results in: OSError: Unable to open file (file signature not found), What am I doing wrong?

[@tfkcheetay4965]

I want to track only one object in a frame using YOLOv3 and I am also using pan-tilt tracker to follow the object. Let me know about the changes in code.

[@minhdinh3814]

How to detect and track only one object person.please help me

[@hebanajem7185]

Hello how are you ? Thank you for this great channel and your excellent efforts. I benefited from the videos a lot. If you don't mind, I want to help you. I am working on a graduation project, which is to help the blind by using the object detection with the vocal feedback . I mean, the text of the detected class (object) using the Yulu algorithm is converted into sound. Please help me because I did not find any sources and I do not know how this can be implemented. Thank you very much in advance

[@TheAIGuy]

Send me an email, you can find it on my About page!

[@hebanajem7185]

The AI Guy i did please check ur e-mail

[@malsafhi3201]

I learned a lot from your videos, keep working on it, it's amazing. Is there a way to use object tracking on the computer screen record? Real-time not on video?! Thank you

[@TheAIGuy]

Like to run the tracker on your computer screen? I haven't really tried this but I can look into it.

[@malsafhi3201]

@@TheAIGuy I found small path for that 🙄 , by using Multiple ScreenShots "mss"

[@TheAIGuy]

@@malsafhi3201 thanks a lot! Really appreciate it and the support.

[@tomeastem6926]

great example, thanks! Could you make a tutorial on coding tracker counter of objects crossing a line? Something like number of objects entering area and number of objects leaving the area?

[@davidmoodyonline]

I'm getting an error: "The name 'net/images:0' refers to a Tensor which does not exist. The operation, 'net/images', does not exist in the graph." Do you know what might be causing it? edit: Never mind, I fixed it by making sure all the modules were the right version

[@exoticme4760]

how

[@shaishavgupta2124]

how to count objects need that !!!!!! can you please provide a detailed overview of the code it would be really helpful. thanks you doing great job.

[@ismailelhaoumi2673]

hey bro , first of all i wanna really thank you for ur amazing tutorials caz really i learned a lot from u ... i have a request to u , could u make for us a tutorial about how to link our object detection model to an android application or to raspberry pi for exaùple ?! i would really gratful to u if u made it

[@TheAIGuy]

Yes I think a lot of people would appreciate this. Will add it to my list of videos to make!

[@RizwanAli-jy9ub]

Nice tutorial, Can you please help me in accessing the webcam in Google Colab.

[@deepaksahoo9336]

is it possible to do in google colab?

[@TheAIGuy]

I'm looking into this at the moment. You can't do it right now because Colab is hard to run video in.

[@deepaksahoo9336]

@@TheAIGuy what about Google cloud platform?

[@TheAIGuy]

@@deepaksahoo9336 I believe it is possible on GCP, youd have to look up other tutorials for that.

[@deepaksahoo9336]

@@TheAIGuy please make one video on tutorials on Tensorflow in GCP

[@fahadalharbi945]

hey man, thanks for a great video ! i was wondering if its possible to count the total objects in each frame and the count of each class. and displaying it !? Thank you.

[@NobertoWellington]

How to train our on feature extractor .pb file?

[@matkoschlebikom3630]

hello your videos are amazing, very helpful for my project and i need print coordinates x , y from webcam detect object can you hepl me?

[@mahfoozalam6977]

sir how to count Object using yolov3

[@brofessorsbooks3352]

Hey there, im running two 1080ti on ubuntu and getting this error File "/home/paul/anaconda3/envs/tracker-gpu/lib/python3.7/site-packages/tensorflow_core/python/eager/context.py", line 945, in _compute_gpu_options raise ValueError("Memory growth cannot differ between GPU devices") ValueError: Memory growth cannot differ between GPU devices Exception ignored in: Traceback (most recent call last): File "/home/paul/anaconda3/envs/tracker-gpu/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 765, in __del__ AttributeError: 'Session' object has no attribute '_session' Any clues? Thanks!

[@TheAIGuy]

I think you need to choose one GPU for the program to run with and not try to use both.

[@brofessorsbooks3352]

@@TheAIGuy Thanks ! The solution to make it only run 1 gpu is CUDA_VISIBLE_DEVICES=0 python object_tracker.py --video ./data/video/test.mp4 --output ./data/video/results.avi