Deep SORT | Object Tracking | Perception for Self Driving Cars

[@JordanMasters-s4w]

amazing video! Finally, was able to understand MOT concept. thank you!

[@bakshivishal895]

This channel is so underrated. The explanation is soo good, soon you will be reaching 100k as the evolution of Artificial Intelligence is improving in a rapid pace. ❣

[@8eck]

Thank you for the explanation. You helped me to understand some of the parts of this complex algorithm.

[@RoboticswithSakshay]

Glad to help!

[@peterr2237]

I like your explanation about the DeepSort which combines a series of techniques that were hard to understand. As you said that the Kalman filter did not perform well in the DeepSort, I was wondering how the DeepSort consolidated the score of cosine similarity of those embeddings and the socres of the covariance distribution based on the Mahalanobis distance.

[@AnmolKumar-so8lh]

One of the best 👍🏿

[@8eck]

So the deep appearance descriptor is helping to extract features (embedding) from a detected bounding box so that they could be used for KNN-like algorithm, to find matching bounding boxes, to tell us who is the same person on the image compared to the previous image(s)?

[@RoboticswithSakshay]

Yes, correct.

[@thefirewolf7478]

Really good explaination.

[@user-hi2hb2ny2p]

Good explanation, thanks, upvoted

[@8eck]

16:25 yes, i was trying to understand why is this happening as well.

[@RoboticswithSakshay]

Yes, this is strange. Did you make any progress?

[@8eck]

@@RoboticswithSakshay kind of, but still researching, will reply later

[@8eck]

@@RoboticswithSakshay here is my ticket in github github.com/mikel-brostrom/Yolov5_DeepSort_Pytorch/issues/371

[@8eck]

@@RoboticswithSakshay I have posted my thoughts about it. Still investigating deeper.

[@none-hr6zh]

cosine similarity is taken between detected object passing through ccn that feature vector with previous frames detected??

[@8eck]

So we are using kalman filter to know where to look for the next bounding box of the tracked object?

[@RoboticswithSakshay]

In theory yes. But, the researchers found that even without using Kalman Filter, and only using the Deep Appearance Descriptor, they were getting good results. So, although the explanation includes Kalman Filter, during run time it's weight is set to be zero.

[@8eck]

Mahalanobis distance is used to compare two different Kalman Filter results from different tracked objects?

[@RoboticswithSakshay]

No, the distance between the actual position (single point) and the predicted position (Kalman Filter result). In summary, Mahalanobis Distance is used to compare the Distance of a single point with a distribution of points. Kalman Filter outputs a probability distribution of points where the object would be next. Therefore, Mahalanobis Distance is used to compare the actual future position and the possible future positions.

[@aasheesh6001]

Thanks bro for this.

[@doodle_video_scribes1226]

Hi, Thanks for the very informative video. I am looking for a tracking solution for a parking slot. I have a surround view from 4 cameras around the car and I am using Yolo to detect the parking slot. Then during the parking maneuver, I need to track the parking slot in each frame. So the parking slot is detected and is a static object while the car is moving backwards now try to park in the slot. Can this be done with SORT algorithm? Or can you point me in some direction that may work for my problem?

[@jaisuthan1]

Good explanation and very informative

[@RoboticswithSakshay]

Thank You! 😊 Happy to know you liked it.

[@Laddu225]

Hello. Thank you very much for your video. Can we estimate the lateral and longitudinal position of the cars using Deepsort?

[@aryanrao948]

great buddy

[@8eck]

Couldn't understand Cascade Matching and Mahalanobis distance.

[@RoboticswithSakshay]

Let's say we have 2 cars, and we have predicted their future positions using the Kalman Filter. Now from the camera, we observe what is their actual position in the future step. We don't know which actual position corresponds to which of the future position. Here, we have what's called an Assignment Problem. Based on the Mahalanobis distance, we need to assign the predicted state to the actual state. Simply done using the Hungarian Algorithm. Now, consider time as a parameter as well. There are objects that are more frequently seen compared to others. We can use this information to better solve the Assignment Problem. Cascade Matching accounts for this. For more details, why we actually use the Cascade Matching, refer to the paper: arxiv.org/pdf/1703.07402.pdf, section 2.3 Matching Cascade.

[@ChetanAnnam]

great video, thanks a lot

[@RoboticswithSakshay]

Thank you Pavan! Glad you liked it!

[@thefirewolf7478]

Hi I had a question. What if the car goes out of frame(you overtake the car) and later the same car comes back in frame(the car overtakes you). Will you be able to still track the car with the same ID number?

[@manojsamal7780]

yes some id's are stored in memory for few seconds and if cars comes in frame it will detect

[@moodiali7324]

did you make any progress yet on the IT assigment problem? the algorithm is not only missing the count when an oclusion is happening but also when new objects appear very similar to previous objects

[@RoboticswithSakshay]

Hello! No, I have not made any such progress on this. Beck, who commented on this video was working on it. The algorithm we discussed in the video, was more for demonstration purpose. For it to be useful in real world scenarios, there may be a lot of modifications required.

[@alykhaled6678]

It's really a good video explaining deep sort algorithm , but I can't take my eyes of your tom & jerry shirt 😂😂

[@8eck]

Have you managed to solve that occlusion problem?

[@RoboticswithSakshay]

No, I haven't really returned back to this project. I think by doing a little parameter tuning, this would work. It may be possible that, for this specific problem, we would need to increase the weight of the Kalman Filter part.

[@8eck]

​@@RoboticswithSakshay In short, problem is where new tracks are created, when missed/unmatched detections appear. Then newly created tracks overtake previous track for some reason and previous track is not overtaking new track anymore for some reason. I suspect that ReID features saved in the first track are diluted/polluted while objects overlapping is happening.

[@RoboticswithSakshay]

@@8eck Can be. How did you discover that this would be the problem?

[@theboss73104]

Bruh how to track a recognised person? With his name

[@bahaeddinehemmemCHAREN]

Khouya l hendi lizoum