How DeepSORT Works?

So in this video, Im going to give to you a clear and simple explanation on how Deep SORT works and why its so amazing compared to other models in this computer vision lecture. We implement this in OpenCV in the following course:
⭐FREE DeepSORT+ YOLOR Course - augmentedstartups.info/YOLOR-...
⭐YOLOv8 - www.augmentedstartups.com/yol...
But to understand how DeepSORT works, we first have to go back, waaay back and understand the fundamentals of object tracking and the key innovations that had to happen along the way, for DeepSORT to emerge.
Now tracking assumes that we have an already detected an object of interest. For detection as you may already know is done with YOLOv4. Once we have detected the object, it is assigned an id and is tracked using Deep SORT. We use an example with Elon Musk and SpaceX
So in this application, we have applied deep sort for tracking of vehicles on a highway for traffic surveillance applications.
------------------------------------------------------------
Learn Advanced Tutorials
►Augmentedstartups.info/Teachable-AI-Bootcamp
Support us on Patreon
►AugmentedStartups.info/Patreon
Chat to us on Discord
►AugmentedStartups.info/discord
Interact with us on Facebook
►AugmentedStartups.info/Facebook
Check my latest work on Instagram
►AugmentedStartups.info/instagram
------------------------------------------------------------
0:00 - Introduction
0:42 - Object Tracking
2:39 - Optical Flow and Mean Shift
3:01 - Mean Shift
3:58 - Optical Flow
5:10 - Kalman Filter
7:00 - Simple Online Realtime Tracking (SORT)
7:55 - Detection
8:21 - Estimation
9:00 - Target Association
9:31 - Track Identities life Cycle
10:30 - DeepSORT
11:27 - Deep Learning
12:13 - The Appearance feature Vector
13:03 - Climax of the Story

[@Augmented_AI]

To Learn more on YOLOv4, then you can either watch more FREE YOLOv4 videos here►bit.ly/YOLOv4Playlist or you can get support by enrolling in the comprehensive YOLOv4 course on Augmented Startups Here ►augmentedstartups.info/yolov4release

[@Skhillz_FN]

So I wanna make a macro that uses image based tracking of what's going on in "program" Different colors indicate different outputs Glad to see I've found the right place The only thing that seems to change and Possibly cause some difficulty is the camera essentially becomes a moving object But basically is it possible to create altered outputs or a series of executions using the appearence discriptor? After all the camera movements are an output

[@brianyoon815]

Phenomenal explanation that helped me understanding MOT in a single video after pondering for a month

[@AbdulAhad-eg3se]

This is the most intuitive video on the topic ! Loved the teaching style

[@agrimarora111]

Best video on object tracking !! 👍🏻❤️👍🏻

[@aliabdari5640]

This video is really beneficial. Thanks a million!

[@haniyounis720]

Simply, I love you 🤟 Big thanks for a charming tutorial like this give you a thumbs up 👍 🆙 ⬆️

[@priyankasaini665]

This is the best learning video I have seen on youtube. What an impressive way to explain DeepSORT, It cleared my doubts regarding Kalman filter. The musk story you make holds us till the last second of video. you are funny and smart! Thanks for this video :)

[@Augmented_AI]

Hey Priyanka, I'm really grateful for comments like these. It really makes my day 😁 I'm currently experimenting with visual storytelling to teach complex topics, so if you have a topic that I should cover next, Please let me know? Thanks again.

[@itaynave5770]

Amazing video, thanks for the knowledge.

[@mrOmar432]

What a great effort, many thanks for being a wonderful human being!

[@Augmented_AI]

Thank you Omar. I really appreciate the comment 😁

[@meomeo4852]

Nice video. What a cool way to learn something complex like Computer Science! Thank you

[@Augmented_AI]

Thank you Meo Meo :). I am really glad you enjoyed the video. If you have anything that you want me to make a video about, just use the hashtag #augmentedstartups with your comment and I'll see what I can do. If you haven't already, check out my channel page to see all of the topics I've covered so far www.augmentedstartups.info/KZbin or check out our courses here www.augmentedstartups.com/store or you can consider becoming a member of Augmented Startups and get access to Advanced Tutorials :) bit.ly/Join_AugmentedStartups I look forward to seeing you around! 👊

[@michaelschulte2399]

Okay, so in this video I did a project by direct order from Elon Musk, there were rockets launched, Kanye West memes and an exceptionally great storyline accompanied by beautifully detailed animations. The technical content was brilliantly conveyed, I was exceptionally entertained and took away more about the topic than after two weeks of literature research. You, sir, have single-handedly beaten the entire educational system! I love your content from the great humor - over the timestamps in the description - up to the immense quality of the videos! Much love from Germany!

[@Augmented_AI]

Hey Michael it's really great to get comments like these. 😁 It really makes my day and I really appreciate it and I'm glad that I can help clarify this topic for you!

[@abdktefane858]

I have been working on research in my university about tracking people for five months and I have watched many videos on the topic, but this video summarized everything wonderfully. Thank you from the bottom of my heart. Can I get the images, video slides, or the name of the application that the graphics are working on in order to present the project at the university?

[@Augmented_AI]

Hi Abd, Im really glad that you enjoyed this video and it was summarized well. Lets discuss over facebook messenger. Please add me augmentedstartups.info/FBMessenger and we can discuss.

[@08ae6013]

OMG...there is so much to digest from this video... I need to watch this video at least 10 times to grab all...

[@Augmented_AI]

Enjoy :) it's a complex topic

[@allboutapex7742]

amazing content, thank you!

[@patryklaskowski1085]

Outstanding! Clearly explained

[@Augmented_AI]

I'm really glad you enjoyed it . Please consider sharing the video if you will find it helpful to others

[@anushabajracharya9135]

Very helpful content👍

[@nurali2525]

greatest! keep it the same way! never seen more interesting explanation than this! liked and subscribed after this video))

[@Augmented_AI]

Welcome aboard! Thank you for your support and Im really glad you enjoyed the video :). Let me know what you would like to see next?

[@rajivdesai5709]

Very informative video bro thanks

[@raminghorbani2073]

Thanks...It give me a good picture of deep sort

[@ilia9872]

Amazing video!!! Thanks a lot!!!

[@KevinKuei]

Thanks for the GREAT video! I have one question. How does DeepSORT associate tracking objects and detected objects? We have both appearance distance and IOU distance. Does it combine both distance metrics?

[@katerynachumachenko9483]

Thanks for the video. I am curious for the sources for the fps of Tracktor++, JDE, and DeepSORT. E.g., how is tracktor++ 3 fps or how is JDE 12 pfs and low res (even on their official github they report 22.2 fps at 1088x608)? And when you say that Deep SORT is the fastest, do you consider the additional detection step?

[@papuce2]

Very good and funny overview, thanks :-)

[@soumyadbanik]

To get the Kalman gain, we need measured error. Right? what will be the measured error here ?

[@ChetanAnnam]

It's so nice that you've read the SORT paper aloud 😰😰

[@mithunchandra8221]

Nice Explanation,Sir.Needs More

[@Augmented_AI]

Hi Mithun more in terms of?

[@cbrao5967]

Amazing content !!!!!

[@Augmented_AI]

Thanks bro 😁

[@hirenvyas1234]

Using deepsort can we find velocity of object?

[@fjturner123]

Really informative and funny video. Thanks!

[@Augmented_AI]

Thank you :) glad you enjoyed it

[@BSelm05]

outstanding! keep it up!

[@Augmented_AI]

Thank you so much :)

[@praveen9083]

Outstanding!! :)

[@Augmented_AI]

Thank you Praveen 😁

[@toannn99]

Interesting video !! Very helpful and funny :D :D !!

[@Augmented_AI]

Thank you Như Toàn Nguyễn :). I am really glad you enjoyed the video. If you have anything that you want me to make a video about, just use the hashtag #augmentedstartups with your comment and I'll see what I can do. If you haven't already, check out my channel page to see all of the topics I've covered so far www.augmentedstartups.info/KZbin or check out our courses here www.augmentedstartups.com/store or you can consider becoming a member of Augmented Startups :) kzbin.info/door/FJPdVHPZOYhSyxmX_C_Pewjoin I look forward to seeing you around! 👊

[@jackki1008]

I am glad it has a happy ending! I thought about the end when the model did not work, and Elon tided the guy into to the next rocket and send him to Mars

[@mostinho7]

@3:50 optical flow

[@robins341]

Boet, this fantastic! However, what I did not get was whether DeepSORT does the detection or not. You seem to imply that you need to use e.g. YOLO to detect (i.e. detect + classify the Falcon 9) an object and then DeepSORT handels the tracking. Is that correct? If so, what if we do not know what object might appear in the frame? What if we wanted to track the rocket, without knowing its a rocket, as well as the pigeon or any aircraft that might appear in the video feed? Can DeepSORT do this, or we will need models to detect the bird, an aircraft, and the rocket?

[@Augmented_AI]

Thanks bro 😁. So deepsort can work off the bat without yolo. But it works better with yolov4 or the better yoloR. So that it uses the classes that you trained with yolo. You do get class agnostic object detection models but those need to be tested. The best is to train your own custom yolo model and deepsort will work automatically

[@magiusmajere2602]

we need help... am using yolov and deepsort with avermedia PW513 wide angle webcam, the pc just kept rebooting. Any insight on this?

[@Augmented_AI]

Which hardware are you using?

[@termoulabderaouf4118]

can you do norfair tracker please

[@suyashchougule5358]

6:30 when you said we can put weight on sensor data and motion estimation. But while tracking an object with by means of only computer vision (i.e. without using sensor data) how does kalman filter works there ? cause when we use deep sort there is no sensor , just the visual information. It will be helpful to understand if you could comment on this.

[@muhammadnaufil5237]

same question. Do you have the answer now?

[@roboticsforunitedindia9421]

Please make video on 3d object detection in yolor

[@addieira03]

One question, how's the memory usage of DEEP-SORT? Is it suitable for edge devices or not?

[@Augmented_AI]

It's quite high and not recommended for edge devices at the moment. I would rather do this on a cloud or server platform

[@addieira03]

@@Augmented_AI Thanks for the quick response! Do you happen to know any papers that mentions that?

[@morne1317]

joh my bra soo funy dankie man you made it so interesting, well done

[@Augmented_AI]

Lekker bra! ;) Glad you enjoyed it!

[@MrMatijadam]

Hi, Liked your video a lot and have been using DeepSort also in my research. Unfortunately it doesn't handle sudden path change as I hope it would. Do you think, or you may have tried already, those other trackers you mentioned (Traktor++, Track RCNN, JDE) would handle object erratic movement better that DeepSort? Keep the good work

[@Augmented_AI]

Thanks for sharing! I will add this to my blog and lectures on deepsort. :D

[@ravikumarthakur3038]

What are you trying to track? If you are able to detect the objects moving suddenly, then you may track it. The original paper puts a lot of emphasis on feature descriptor. Training the siamese model (github.com/abhyantrika/nanonets_object_tracking/) might help

[@Ip_man22]

@@Augmented_AI First of all thank you so much for the video, you explained it really well @matija Almost all tracking methods are not 100% accurate, I've used DeepSort for one of my project and it has major ID switching problem but the same is with other methods as well, so you have to think and decide which works best for you, for some projects a simple centroid based tracking works too!

[@cbrao5967]

JDE is the best, faster fps rate , better MOTA score ,handles occlusions and ID switching very well than deepsort that I implemented. Repo for JDE: github.com/Zhongdao/Towards-Realtime-MOT Paper: arxiv.org/pdf/1909.12605.pdf

[@tungtt4915]

haha this is the best video, love it

[@tkts8782]

God I love this channel

[@Augmented_AI]

☺️😊 I really appreciate the comment 🤟😁 thank you.

[@sunilscorpy4594]

God I am so confused! Lol 😂

[@tkts8782]

@@Augmented_AI NP! Have you thought about placing objects in ARCore using GPS (as in the player spawns items and those items are all hooked to a GPS coordinate)

[@tkts8782]

@@sunilscorpy4594 XD

[@TheBigLou13]

14:28 - then the shockwave from the rocket-launch hits the camera and the signal gets corrupted and we get a freeze frame for a few seconds. The tracker corrects its estimated course - velocity approaches 0 since the sensory data clearly shows a rocket, floating in still-stand. When the image of the camera updates the rocket is already far out of the tracking frame and you thought "I should have implemented a reasonability check for the velocity (shouldn't be zero after launch) - or add an estimation for changes in velocity, with abrupt changes being treated as visual occlusion - or a feature to zoom out after a timeout of not seeing the rocket for longer - or include more sensors...". Elon looks at the blue sky video feed and asks: "Where's my rocket?"

[@bharath5673__]

💘

[@bachhuynh1562]

Deepsort quite heavy one

[@user-rt6bl3tj4f]

这个马斯克的故事好有趣

[@Augmented_AI]

Thank you so much 😀

[@Augmented_AI]

⭐ Also if you enjoy my work, Id really appreciate a Coffee😎☕ - augmentedstartups.info/BuyMeCoffee

[@jh6643]

Just watched the first 3 minutes. Dude, you're funny. :D

[@Augmented_AI]

Thank you Jibanul Haque :). I am really glad you enjoyed the video. If you have anything that you want me to make a video about, just use the hashtag #augmentedstartups with your comment and I'll see what I can do. If you haven't already, check out my channel page to see all of the topics I've covered so far www.augmentedstartups.info/KZbin or check out our courses here www.augmentedstartups.com/store I look forward to seeing you around! 👊

[@theboomguy1426]

🎉

[@ThePaypay88]

Any deepsort code to check?

[@Augmented_AI]

You will find that in my course :)

[@P4el1co]

Cool story but I don't think you know the theory behind the framework.

[@Augmented_AI]

Thanks. Yeah there's a lot of depths to cover in just one video. But this video helps to give an overview of the technology behind deep sort. You may want to research each individual topic to gather a better understanding of it

[@yufeixia3073]

haha, you are so funny

[@amritanjanthakur6510]

1 like for using elon musk .hehehe

[@MrDeyzel]

Great video and explanation, but the Musk workship is so cringe.

[@Augmented_AI]

Thanks for the feedback

[@bolzanoitaly8360]

just due to one word almighty Kalman, I disliked this video. use simple and true words, why you use wrong words, your video will go lose each and everything, so be careful in future.

[@bobbyintheshop]

good videos but imo your attempts to be funny and cute are too much and detract from trying to learn the topic. just one single opinion though

[@Augmented_AI]

Thanks for the feedback

[@RightlyFree]

Ganja 😂