YOLOv4 in the CLOUD: Build Object Tracking Using DeepSORT in Google Colab (FREE GPU)

[@TheAIGuy]

Hope you enjoy! Let me know if you like the Google Colab videos or if you would like to see more of them? :)

[@oscarfrausto4502]

Very cool, could you show how to use multi/cameras ? I want to track persons with differents cameras

[@sriadityab4794]

I enjoy this colab videos. I am interested in custom object re-identification integrated with YOLO. Can you make a video on that?

[@yuchenzhao2663]

thank you so much! realy enjoy your videos! instance segmentation from annotating data to training custom model maybe a possible one?

[@slann2k]

Colab tutorials are really useful to quickly play around with the code quickly. What would also be useful would be more documentation explaining what are some of the thresholds/metrics/flags we can tune in Tracker.py/object_tracker.py according to our needs. For example, I am trying to modify this code to count the number of people who have walked past a certain checkpoint within the video, so I don't necessarily need the tracking boxes to be always present (I guess some false negatives are ok) but I really don't want ID switching which would cause double counting. Would be helpful to know which are the thresholds/metrics I should be tuning.

[@utkarshay]

pls make video on crowd analytics, counting people in a crowd and uploading results on cloud...

[@tibigumi]

Great video and tutorial! Just an FYI: I noticed today that my Google Colab had tensorflow version 2.4.0 with which the converted weights file was smaller (9 MB instead of 11MB) and the object detection + tracking were not working i.e. not found any persons, cars, etc. Then I reinstalled tensorflow version 2.3.0-rc and with that it was working properly. So if anyone were to run into the same problem check your tensorflow version in Colab.

[@ritikvaidande4683]

thanks that worked! keep contributing :)

[@ThePlymouth37]

Great tip, this fixed it for me as well, Thank you!

[@ThePlymouth37]

Here is the code that worked for me to backdate tensorflow: !pip uninstall tensorflow -y !pip install tensorflow==2.3.0 import tensorflow tensorflow.__version__

[@karthickraja1026]

@@ThePlymouth37 Thankyou, now its working

[@automationcontrols9141]

@@ThePlymouth37 Thanks for the tips. Its working now.

[@jeffdryden7069]

Just wanted to comment in case anyone else has this issue, while attempting to use my own custom weights DeepSort would not detect anything. I had to uncomment the dependency line, restart the kernel, re-convert the weights again and then it worked! Also wanted to say that I love this channel, thanks for everything!

[@rajkumarayyalsamy1971]

i have the same issue, could you tell where to uncomment to depedency line? and how to re-convert the weights?

[@abdirahmaanyuusuf7687]

@@rajkumarayyalsamy1971 uncomment the "!pip install requiremts " line, and once you restart the session you should execute the save_model.py again to convert yolov4 darknet wights into tensorflow model.

[@aliighph]

thank you so much

[@fauzimakarim3818]

Great video and very good channel, I'm a final semester student working on a project using YOLO and this channel is very helpful.

[@houston3414]

finally, the video I was waiting for, thank you for understanding me, going to follow carefully step by step, thank you again.

[@TheAIGuy]

Hope you enjoy! Thanks for the support.

[@KS-gy7hn]

Extremely interesting videos, would love for you to tackle multi-label image classification using google colab next. Thankyou for these tutorials.

[@rick28971]

Thank you very much!!! Is it possible to add custom classes to this object tracker just like you did with the object detector???

[@shubhamshah6645]

Make a Discord server, so everyone can connect with you and share some naive or good doubts which can be solved with collective intelligence. Thanks for putting so much efforts in making great content as always.

[@TheAIGuy]

I will definitely look into setting this up. Thanks for the great suggestion.

[@shubhamshah6645]

@@TheAIGuy that would be awesome.👍

[@jshr3]

Hey TheAIGuy, you're awesome!!! You're saving in my job so hard haha. Thank you so much

[@thanosbountliakis9487]

Thanks a lot! you have an amazing channel. Keep on the good work!

[@pradyumnjain1578]

Hey can you make a video on configuring deepsort to train it on a custom data set ?. Love your content .

[@adjejeremiealagbe4037]

Hi AI Guy! Good job!!! One question: Where are saved the output BBox coordinates, or how to extract them and write to excel sheet? If anyone can help please do me that favor. Thanks!

[@peterr2237]

Thanks for sharing this wonderful video. I was wondering what is the resolution of the videos that were in your experiment, and how you computed the fps as both the yolo detector and DeepSort tracker also need to take time to unpacked the bounding boxes objects particularly when the numbers are relatively high.

[@nilreml1]

Thanks @The AI Guy for your nice work on yolo and deep sort! I really enjoyed the videos you did on the subjects. Best youtube content on the topic! Quick question: I can see that if I try to limit FPS in my yolo+deepsort pipeline, I limit the capacity of deep sort to guess similarities between bounding boxes in order to group them under the same id. Do you have an idea in order to improve the quality of this part of the pipeline? I tried to improve the FPS gradually to guess kind of the limit for now. Is there a temporal parameter in order for deep sort to perform better with longer wholes between frames, or is it just a question of training?

[@DiogoSanti]

Cool stuff, just one intriguing thing is that, if deep-sort is trained in MARS with only humans, how the model worked really well with cars??

[@HuHu3169-s7o]

Thanks for the tutorial! It was quite helpful. But could you make a video on how to build Object Tracking using DeepSORT? I mean I have followed your video "YOLOv4 in the CLOUD: Build and Train Custom Object Detector (FREE GPU)" and "How to Build a Custom YOLOv4 Object Detector using TensorFlow (License Plate Detector)". They work just fine. But when I try to deploy that self-trainied yolov4-obj_last.weights with DeepSORT, it just didn't work. Have no idea what's going wrong about it. Thanks anyway!

[@jesskelly9935]

Thank you so much for your video! So easy to follow!

[@shubhamvashisth9518]

@TheAIGuy Thanks for the video, how can we do standard detection on colab using Yolo for Live video

[@dipteshchakraborty9163]

if you are using this repo, you just need to change your input video path as "0"

[@boypotus]

I have a collab request for soccer shot detection on custom datasets, been having trouble -- any way to contact you ?

[@marthagichuki4204]

Great work @TheAIGuy, please someone help, My tracker does not show label names, neither Ids, What i happening? Do I need to bring the detector model into the tracking model?

[@marthagichuki4204]

Awesome video. I had to uninstall Tensorflow version 2..0 and installed version 2.3.0 after suffering for a week

[@diyamourya1743]

Great video with nice explanation Question: How to apply YOLO in AirSim for object Detection and Tracking

[@twoscholars]

Great video!!!!!!!!!!! Great work!!!!!!!!!!!. Just a quick question if anyone is also working on it. How do I get the coordinates of the bounding box getting tracked for the last car video he just ran.

[@carloselilopeztellez7802]

Thanks man! Excellent video, keep it up

[@nicholaslim9708]

Hi! Great Video! Can i ask if it is able to use the weights/model from custom detect (colab) and use the weight for deepsort (colab)? If so how can i do so.

[@marthagichuki4204]

Hi Lim, did you get a solution to this problem?

[@talhayaqoob889]

amazing video! please make a video on segmentation like segnet and mask RCNN

[@TheAIGuy]

Great suggestion! Definitely will be covering this in the future so stay tuned.

[@mrconnardtroispointzero7176]

What about doing this in real time with IP webcams ?

[@neelusingh2467]

great explanation with a good understanding thanks!!!

[@amitdivekar9562]

Great Work..... One Query.... How to integrate Faster RCNN object detection with deep SORT

[@mwaikul]

My video has no trackers. My log is not showing any trackers either Frame #: 1 FPS: 0.03 Frame #: 2 FPS: 24.59 Frame #: 3 FPS: 27.17 Frame #: 4 FPS: 27.05 Frame #: 5 FPS: 27.36 Frame #: 6 FPS: 27.42 Frame #: 7 Don't see any errors. Can you please help!

[@siwarriahi7441]

same problem 😪😪

[@mario977777]

Para crear la red en base a tu propio dataset: Descarga tus 'pesos' obtenidos en el entrenamiento de YOLOv4 en este nuevo entorno, pon este archivo en la carpeta /data, después deberás crear en esa misma carpeta otro archivo '.names', en mi caso "obj.names", con los nombres de las clases, justo como los pusiste al entrenar YOLOv4. Por último deberás ir a la carpeta /core al archivo 'config.py' y deberás cambiar la linea "__C.YOLO.CLASSES" para que ahora apunte al archivo con los nombres de tu dataset: = "./data/obj.names" en mi caso.

[@biahoi87]

Great job, I am thinking about the motorbike thief detection project. Would you please advise me between 2 options: first one is use posenet and yolov4 then base on the wrists position to detect the malicious chance (this part is coding condition). Second option uses RNN to detect the series of action through time and predict the thief (this one is 100% AI but data is my concern).

[@meliha9250]

Thank you. My question is we need to train yolov4 and deepsort separately, right (if we want to fine tune both of them for our application)?

[@shuto9946]

this better than my computer class which costs $500

[@bohanho5173]

Why did I follow every step of your video, and the final output video didn't produce a correctly tracked bounding box like you did?

[@야니스-l6b]

what a wonderful tutorial..!! i really appreciate it, Thank you...!! may i ask one thing? i wonder that RealTime Tracking Possible with DeepSort? i tried to set to 0 for webcam but not working in colab.

[@akhilareddy4259]

Thanks for the awesome tutorial Can we add custom ROI lines for counting the number of cars for upstream and downstream traffic

[@nina_________________]

Thank you so much for the video and code! Unfortunatelly i've had some problem with the requirements that stopped the making of the video: the tensorflow-gpu requirements and pip v19.0 are in conflict with other versions or settings in my colab. How can i solve this?

[@TheMetajose]

Hey there! were you able to solve this issue, I'm having the same problem :(

[@VTwin4Christ]

Can calab listen for UDP video stream and return JSON objects for each frame being processed??? On a callback URL?? UDP to my local machine - forward UDP video traffic to calab -calab makes url callback with JSON object of what was detected.

[@sid1r]

Great video! Thanks for sharing it! It is very clear and precise. I wanted to ask how do you extract statistics about the object being tracked such as distance the object has covered, velocity. Is this part of deepSORT result that can be specified in the return call from the function? Also how does one add custom classes and bounding box for a new custom class?

[@hansalas]

For those having issues, pls try the below ! pip uninstall tensorflow ! pip install tensorflow==2.3.0

[@AshishMishra-wb3ki]

Can you help out, how to estimate the speed of the cars which was in Video.

[@athirapmohan6496]

How can I train with my dataset (tracking fish)

[@TheAIGuy]

You will need to train your own custom Yolov4 model using your dataset. The tracker can then be configured to run with your custom model.

[@athirapmohan6496]

@@TheAIGuy it will be great if you make a video about tracking using custom dataset

[@TheAIGuy]

@@athirapmohan6496 it is very easy to do. You can find the detailed explanation in the code’s Readme.

[@giorgossartzetakis8771]

Hey did you figure this out? Is it necessary to train deep sort for the custom dataset as well. I track falling rocks..

[@athirapmohan6496]

@@giorgossartzetakis8771 no. Just need the weight file of yolo object detection trained with our custom dataset

[@erikektrada5158]

hai thanks to tutrial yolov4 i have i question can this running in webcam video in realtime?

[@vitalyzagrebelnyy5903]

Hi . Appreciate your videos. Well done. Could you answer one thing ? in one of the videos you have shown the use of OIDv4_ToolKit to download labeled pictures . However I didn't get how to download all labels for one picture. For example I'm doing detector of cars and people, and i want both labels if they exist on one image, not separately labeled two categories of images . Is there the way to download them?

[@amiy24]

Hi Ai guy..Thanks for making such a mind-blowing video..I was just curious if we can tracks bubbles captured using high speed camera. Is there any class available for bubble detection?

[@lilianleonidas9565]

Am failing to execute same code on colab, it requires to change the video format. After running the code it does not detect

[@SadatSafawi]

in 100th iteration of object detection is gives error and don't save the weights in Drive

[@siddhikhanvilkar9890]

Hello Sir, I have learnt use of YOLO by watching your videos. Thank you for providing such videos. Sir can you please provide us video on how to distinguish between static and moving object using YOLO in google colab?

[@mattgillespie2743]

Hi I'm running this implementation on Google Colab and getting only 0.5fps processing speed. It would seem the GPU is not being used even though it's available. Tensorflow 2.3.0cr0 has been discontinued but it's still listed in your requirements.txt. Is there an update to get this working? Thanks

[@shahrinnakkhatra2857]

Which one is better for real time detection? Using YOLO per frame or using tracking algorithm like deepSORT? I got 1 FPS while rendering the output in my local machine with DeepSORT (GT 755M 2GB GDDR5 GPU)....But my end application will be on webcam video and on my local machine it gave me like 1-2 FPS on webcam lol...Hope it will be better using colab

[@mahmoudbenltaifa3896]

I watched your video about how to build and train our custom object detector and it was very helpful but i have a question I used google to train the model then i downloaded the weights and used opencv function dnn_detectionmodel it was good at detection but it's slow and i tried the same thing using frozen inference graph .pb(pre-trained) as weights path and it was fast and i tried to have .pb file from my yolo weights (using save_model.py)then i tried doing the same (using cv2.dnn_detectionmodel ) but it doesn't work So the question is can we extract frozen inference graph.pb from our yolo weights and use them with opencv to have best results Thanks in advance

[@denysmarques7144]

Hello, how can i save the output ids and positions in a JSON file?

[@holidayjo2706]

What is the benefit if I buy a GPU? Looks like I can code on Colab.

[@briangrey4849]

I got error 'Video has ended or failed, try a different video format!' please help!

[@Vissou]

I tried the same techniques on google collab to make it work (cloning, pip install with your requirements file, placing the weight file in the correct folder ect...) before you posted the video, but it wasn't working when i was trying to save the model (an operation failed because of a problem of dimensions). Now it works perfectly but I am wondering why it couldn't work before, do you know what had to be done ?

[@TheAIGuy]

Earth Shaker no i didnt change any of the commands. That is strange!

[@michaelmutekeri8770]

I am fan...can you elaborate on the join.. what comes with each package.. if there is a schedule so we can use it to its fullest if it fix my schedule... we would love to support this channel

[@TheAIGuy]

Overall, the join is just a way to show your support of the channel. The basic tier is just people who want to support the channel and encourage me to make content. The 'Big Fan' tier is for those who get a lot out of the channel. I will be posting photos of projects as well as project updates for that tier.

[@michaelmutekeri8770]

@@TheAIGuy Will do that

[@TheAIGuy]

@@michaelmutekeri8770 Much appreciated! Means a lot, thanks :)

[@drydenstewartenterprises]

I love your videos, they are really easy to follow along with. I'm kinda teaching myself about robotics and computers, and I still have a lot to learn. I'm trying to build a home assistant robot and I have a couple microcontrollers (esp32 cam, and an Arduino Uno) I was hoping there was a way that I could stream video to google colab and have it do the processing and then send the results back to the (robot) to execute a command... Is it possible to do this some how? or is the robot going to need its own GPU for this??

[@nirmalhasantha986]

My GPU Memory doesn’t seems to go high and stuck with around 5 to 7 FPS. I ran real time gpu memory tracking code beside and it shows only 6GB of gpu memory is used by this 12GB GPU. Any advices?

[@orimoladeeyimofe3940]

Hi please if i want to run this on a different video how will I go about it. I tried inputting the video but it didn't work

[@umar_muhammad_yaree]

How can I display the total number of people on the frame?

[@eliastemponi6139]

Thank you for sharing this high level content videos. On Step5 output i get this WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. `model.compile_metrics` will be empty until you train or evaluate the model..... And on STEP6 i didn´t get any detection. Any one with same problem ?

[@lalaaduc7240]

I followed the instructions but yolo is not working. can you check it again?? Thanks.

[@diwakarraju2976]

hi... Is it possible to combine two custom dataset

[@JackMagicOfficial]

Hi there, first of all thank you for this tutorial. Is it possible to change the code so I can select ROI on the first frame of the video manually and then let the tracker do its job? Or what if I only need to track few people in the video and not the rest of them?

[@akshayjaryal8930]

Please make a video on semantic segmentation or on Detectron 2.

[@jazzinho_futebol]

Anyone experiencing an issue with th object tracker not working. It worked before but recently the object tracker doesnt want to run

[@ucanhvu274]

Please make a video how to add new classes in this yolov4 DeepSORT model

[@jamesee3929]

Hi TheAIGuy, I enjoyed your tutorials very much. Would appreciate it if you could do a tutorial on converting darknet yolov4 to tensorRT and run the obj detection with deepsort on jetson nano/xavier nx. Many thanks in advance

[@youtubeuser9372]

Great video Please do a video on object detection using Densenet

[@dipteshchakraborty9163]

Hi, First of all you provide a very good demonstration how it works and all the codes. Currently I am using your colab notebook and all the videos of it, here the detection is not working on the first step. Nothing is detected. I haven't done any modification in the notebook Please check for the same.

[@marthagichuki4204]

Hi Diptesh, did you get a solution to this problem? I need the same assistance

[@ashutoshshukla4680]

Great video I want to learn more about yolo can yu tell me some resource

[@TheAIGuy]

Ashutosh Shukla a great first resource is the official YOLOv4 research paper! You can find a link to it in the description of this video. It covers a lot of cool information.

[@dancronin403]

Thanks for the video! I got this working but it's using CPU and only getting 2 FPS 😳 I have a GPU but don't know how to activate it. Can you help? Thanks!!

[@ngochuynh3229]

Hello sir, When I follow your code in the colab, the model can't predict. That means the output video looks like an input video without a bounding box with a label. Thank you for your reply.

[@bohanho5173]

me too ，did you solve the problem?

[@8eck]

colab is not working for me, it is just making video copy, without any bounding boxes around objects.

[@AlejandroDiaz-kc5lq]

Hey is it possible to run this with tensorRT in Jetson Nano?

[@shriyapotdukhe33vc56]

can you pls explain how cloud is used in this video ?

[@TheAIGuy]

Google Colab notebook’s use a free Cloud VM with a GPU. So when you run the code it is running on one of Google’s machines and not your local one. Hope this helps!

[@mertkiremitci7463]

Hi, I really enjoy with your works. Thanks for these works. I have stucked with some multiple object tracking problems. I don't know how to prepare a train and test datasets for evaluation of multiple object tracking process (for example, I am using deepsort). Do you know how can I prepare a test set in order to evaluate yolov4+deepsort?

[@mertkiremitci7463]

@The AI Guy hi :)

[@aqsag5997]

Hello Thank you for these amazing videos. Could you please make a video about communication between 2 agents, detecting objects?

[@utkarshay]

pls make video on crowd analytics, counting people in a crowd and uploading results on cloud...

[@TheAIGuy]

Sounds like a great idea! Thanks for the suggestion!

[@shriyapotdukhe33vc56]

this will be amazing

[@mangning1107]

another fabulous viedo from AI guy btw, could you do some video about Mask RCNN, it should be fun and helpful for many people

[@jeipastrana7998]

Hi. hi can I implement this but with social distance tracking and facemask detection? I hope someone can help me out.

[@maritaeques]

Thanks for the detailed video! However, for some reason no classes are detected on step 6. I can see the frame and fps which means that the video is indeed processed, but at the end it says 'Video has ended or failed, try a different video format!' and nothing has been detected. Could you please help? Thanks in advance!

[@eliastemponi6139]

Hi Marita. I got the same problem. Did you manage to solve?

[@maritaeques]

@@eliastemponi6139Hi Elias! I actually didn’t, but I used his other video on yolov4 in the cloud (YOLOv4 in the CLOUD: Install and Run Object Detector) and it worked perfectly! It does not use DeepSORT or anything like that, but it works for me. Also in the next video of the one I mentioned, he describes in detail how to train on a custom dataset. Good luck on your project!

[@briangrey4849]

I too have same problem, did u get solution up to yet?

[@tungaoduy5596]

Amazingggggg, u are my idol

[@AndreF0H]

Hello, I have been executing the cells without making any modifications but it does not detect anything at all, nor does it show any error and the same happens to me in the yolo v4 crop video

[@lilianleonidas9565]

did you get solution?I got similar problem

[@AndreF0H]

@@lilianleonidas9565 my problem was that my custom weights were poorly trained, I tried a test with the originals and it worked and after training my own weights properly it also worked.

[@lilianleonidas9565]

@@AndreF0H am still failing to execute same code, as it tells me i should change the video format. can you help how you ran the code?

[@lilianleonidas9565]

@@AndreF0H Also for custom dataset which place needs changes apart from weights folder?

[@55_it_kuldeepkumarsaini26]

how tu use in yolov5 ?

[@siddhi1359]

Pls make a video on Crowd counting... 🙏

[@HaoyangPei]

cool! but could we run object tracking on the webcam live on colab using yolov4?

[@HaoyangPei]

when I build my own deepsort tracker on colab. it always says"UnboundLocalError: local variable 'ANCHORS' referenced before assignment"

[@jokosusilo303]

Coooollllllllll.

[@gemymarry7738]

Can anyone help me with how to add up and down function to this.

[@divyamsharma6647]

its giving an opencv error when i do it on my own videos !

[@muhammedatmaca4303]

how it works with webcam, does not work with webcam

[@miftahq1]

What gpu?

[@gallegostankiefer6641]

Is it possible to do object tracking live? If so, can you teach us?

[@aiosv4168]

use the streams from rtsp cameras

[@nfaazmi3508]

hye the AI Guy, can you help me to solve the code count person only?

[@AbhishekBagul]

update the gpu-requirements.txt tofollowing: tensorflow-gpu==2.3.0rc0 opencv-python==4.1.1.26 lxml tqdm absl-py matplotlib easydict pillow numpy==1.19.2 tensorboard==2.4 folium==0.2.1 imgaug==0.2.7

[@abdirahmaanyuusuf7687]

bro could you explain why we need to do that?

[@nfaazmi3508]

can do in image

[@xueyang112]

Hello, first thank you so much for this great work. I got a problem when I ran the code as followings. Could you please help me? ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==2.3.0rc0 (from versions: 1.13.1, 1.13.2, 1.14.0, 1.15.0, 1.15.2, 1.15.3, 1.15.4, 1.15.5, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.7.0rc0, 2.7.0rc1, 2.7.0) ERROR: No matching distribution found for tensorflow-gpu==2.3.0rc0

[@ahmadhassan8560]

try using 2.3.1

[@xueyang112]

@@ahmadhassan8560 Thank you so much. It worked.

[@adeeshbhargava8136]

Currently the code creates a video that doesnt track any object !