Football Players Tracking | YOLOv5 + ByteTRACK | Google Colab | step-by-step Tutorial

In this object tracking step-by-step tutorial, we show you how to combine power of YOLOv5 and ByteTRACK to track players on football field.
Chapters:
0:00 Introduction
0:36 Object Detection, Tracking and much more
2:15 Download the dataset from Kaggle
4:11 YOLOv5 pre-trained COCO model inference
5:41 Train YOLOv5 model on custom dataset
6:24 YOLOv5 custom model inference
7:48 Installing ByteTRACK
8:47 Custom annotators
10:41 Outro
Resources:
🌏 Roboflow: roboflow.com
🌌 Roboflow Universe: universe.roboflow.com/
📝 "Track Football Players with Computer Vision" blogpost: blog.roboflow.com/track-footb...
📓 "Track Football Players with Computer Vision" notebook: github.com/roboflow-ai/notebo...
🧑‍💻 YOLOv5 repository: github.com/ultralytics/yolov5
🚀 Find football-players-detection and other grate datasets on Roboflow Universe: universe.roboflow.com/roboflo...
📓 Learn more about YOLOv5 and other Computer Vision models with Roboflow Notebooks: github.com/roboflow-ai/notebooks
Stay up to date with the projects I'm working on at github.com/roboflow-ai and github.com/SkalskiP! ⭐

[@jaredpereira8487]

Hello! I saw your content on data is beautiful via Reddit. Thank you so much for all your hard work. You’ve inspired me to TRY and learn some of these tools! Wish me luck 🍀

[@Roboflow]

This is awesome complement. Thanks for sharing that. 🙏🏻 You on the other hand inspired me to push forward and publish more.

[@zackshainsky2637]

This is awesome!

[@muza886]

really impressive work , PETER Sir

[@SkalskiP]

Thank you very much! I needed a bit o motivation this morning :)

[@cecijos]

Great, thanks for sharing

[@hedibendaoud4831]

Great job!

[@jongwukson273]

great video. thanks so much

[@JonasZaoui]

Wonderful works thanks ! It's possible to improve the tracking of the players ? Id is switching when player disappear and then appear on the frame, i have the same with all tracking methods

[@liellplane145]

So cool and bless you for sharing

[@roboSkalskiP]

Oh! It's Peter here :) Thank you for such kind words!

[@pareshkamble7888]

Nice work.

[@mehdijamey]

This is a great video and tutorial. Can you upload your custom model's weights for pytorch 2? current weights are not compatible.

[@jonimatix]

Can you please show how you custom annotated the data set and trained the model ?

[@aFuzyyy]

Thank you so much, but the weights provided are not working fine, is there something wrong from my side or the weights are not compatible so i should train the model ?

[@ANSIPHD]

Can we count the passes and change theplayer into 2D and also export the events into csv ?

[@Raulvic]

Thank you for sharing!

[@Roboflow]

Thanks for watching :)

[@rafael.gildin]

Great video. Did you test the same process in yolo v8 object tracking? If so what was the output?

[@jasonjasonkim]

Hi! I'm having trouble connecting to the GPU as a Mac user (specifically nvidia-tensorrt. Are you aware of methods to get around this problem?

[@eranfeit]

Very good !!!

[@LookItSnows]

This is awesome! I'm trying to do something very similar with hockey games. In the end you mentioned that you managed to automatically classify the players by team by looking at their jersey colors. Can you upload or have you already uploaded a notebook somewhere showing how this is done? Thank you!

[@Roboflow]

Hi 👋! Yes, I shared the link to the Kaggle notebook as well as result examples here: github.com/roboflow/notebooks/discussions/113

[@linxuer]

There is a data set in your kaggle that is already private, can you give me the data set? I can't run all your code on kaggle

[@adityasprojects7126]

Can we build someting with kalman filters, eg: where the player is taking the ball, where he would shoot it, how players would move with respect to ball's position etc. This would be a great analytics to any team to get good insights. It could also summarize player's general behaviour in many situations

[@roboSkalskiP]

Hi, it's Peter from video. Does it need to be with kalman filters? I personally feel that conversion to bird-eye-view (BEV) is the key to all those advanced analytics.

[@adityasprojects7126]

@@roboSkalskiP Ofcourse not. And to be honest I don't even know if we can use KF here. I had done couple of projects on my own wherein i track and predict ball's position in next few frames using KF. I thought we could use it here as well. And yea, we should be able to get some analytics from BEV. I'm just curious if such videos are even available. If yes, can actually do some like i stated earlier? That would really be awesome!!!

[@roboSkalskiP]

@@adityasprojects7126 I like challenges ;) but you are right it is super hard! for now I just have some ideas nothing more.

[@74Gee]

@@roboSkalskiP How about if the people were not footballers in a uniform but people in a crowd wearing random clothes. How should I go about re-detecting the same person based on their clothing attributes and vecctor, maybe some sort of color map with a decay since last seen would do it?

[@shreynalode60]

I have a doubt about how to calculate mota and motp of ByteTrack used

[@dhamaragudelo2684]

Hello, thanks for the video. I got the final video but I would like to know if I can access the x, y positions of each player. To the data what is used to generate the video. As it appears in the model that is in roboflow

[@Roboflow]

If you mean x, y coordinates on frame, than absolutely! You can access that data.

[@nhuquynhnguyen7632]

Thanks for your sharing ! I want to ask about the type of file that we use to store the annotations. Which one should I save ? (txt, coco json...)

[@Roboflow]

Hi 👋! I'm confused. Please help me to understand, what do you want to save? Tracking results? If so then I'd suggest CSV.

[@adamabubaker4784]

facinating

[@Roboflow]

We are supper happy you like it :)

[@YongTang-rh4si]

This is awesome! I want to know whether having a tutorial about multi-cameras MOT?

[@Roboflow]

This is actually a great idea! Let me add that to my TODO list.

[@redabenallal4695]

@@Roboflow i am really interested too , it would be awesome

[@anadianBaconator]

@@Roboflow I would be super interested too!

[@9876farhan]

wah great explanation. but one question, can this bytetrack use for a hand movement recognition like handsign?

[@Roboflow]

ByteTRACK is a tracker it can track everything as long as you have a detector capable of detecting it. We have plenty of datasets that you could use to build such detectors at Roboflow Universe: universe.roboflow.com/search?q=handsign

[@ranamohtady2658]

How to get the api of this github code

[@omarfarooqahmad]

Hey Love your videos if i wanted to do jersey detection inside the players what approach would you recommend

[@Roboflow]

But only detection? Or you want to cluster those players somehow?

[@vc1351]

Hello! Your video truly is awesome! Do you know some courses (online or not) in order to get into the field of Image detection/Object Tracking ? Thanks

[@Roboflow]

We have this curated playlist: kzbin.info/aero/PLZCA39VpuaZajiCtgDDwU8ghchtqx347R Hope it helps!

[@ammararazzaq132]

is it possible to build a goal keeper detector as well? will an object detector be able to differentiate goal keeper from rest of the players?

[@SkalskiP]

I'm sure it is! It is only question of providing anough examples. In my dataset there were some images with golies but not enough to do it reliably. But given the fact that referees are usually correctly detected, I'm sure we could do the same for golies.

[@trappedinme1883]

Hello, I am completely new to computer vision have been spending the last week having a look, I am trying to get my head around the process, I modified this to use YoloV8, had some slight issues at start due to using a Macbook with an M1 Pro chip and the GPU not being accessible but sorted now, I am running the first stage to use the pre-trained COCO model, in the video you cut, I am about 6 hours in, was just wondering how long this should take roughly? Trying to create something that can be used by grassroots teams to record games as the current offering from Veo and Pixellot have gone way beyond the price of most clubs and keeps rising (nearly doubled in price within a year), think it would be really useful to help reduce violence towards ref whilst also hopefully catching and create memories of moments in matches, in an affordable way.

[@sleepyshoes]

I'm attempting to do this same idea. Did you manage to get this working? Thank youn

[@ANSIPHD]

Can you share your chanel for twich

[@user-uq1pc8lw8q]

Hello, thanks for your video. I am just entering the world of tracking and your video has helped me a lot. But I have a question, in the chapter YOLOv5 pre-trained COCO model inference to view the video you mention at minute 4:40 you mention that thanks to cinema you can view the results immediately. What is cinema? because I ran the lines of code and it prints this but I can't display the video. someone please help me

[@Roboflow]

Oh I just downloaded the video from Colab.

[@user-uq1pc8lw8q]

@@Roboflow Can you explain to me how you download the video from Colab? please

[@silakanveli]

Great work! How fast is the inference fps for whole loop?

[@Roboflow]

It is around 3 fps. But we use 2 heavy models on quite weak GPU. It is quite possible to bring it up to 15-20 fps :)

[@henrydonahue9153]

awesome!! the line where you download the clips file is not working for me.

[@Roboflow]

Do you mean download from Kaggle? What is the error you see?

[@gbo10001]

is there some support team that i can consult with for solving my use case?

[@Roboflow]

I wouldn't call that support team. But you can post your question here: github.com/roboflow/notebooks/discussions/categories/q-a and we will try to help you :)

[@paulpolizzi3421]

Amazing, can this track touches, possession percentage?

[@SkalskiP]

Yup it is possible. If you want to be super precise then it is still long way to go. If you just want to have a estimation it is pretty much ready.

[@khadijakhaldi6468]

can we compute the speed of the players based on the video ?

[@axaxnaji5368]

I think it's doable. However pitch size vary...

[@avamaeva7999]

Can you amend the code to work out how much each player has moved? eg Player X moved 3 kilometre, Player Y moved 4 kilometre..etc

[@Roboflow]

Hi. This is something I want to do, but to be honest it is a lot of work - probably a whole separate video. So it won’t happen to soon.

[@automatedvideoingassistant6192]

@@Roboflow would you do this as a blog post if you do it? If so we will look out for it as this would be very interesting and exciting.

[@5-minutepython]

Wow, what a wonderful project. But I think the ball tracking won't works well here. Because the balls are placed outside the field as well. Am i correct.

[@Roboflow]

Ball tracking is indeed really complicated. And there are multiple reasons that contribute to it: - Extra balls outside football field - that can lead to multi ball detection. To prevent any accidental "ball possession, we discard all ball detections on frames where there are more than one. - When ball is being kicked by the goalkeeper it flights really high, and the flight path leads across the stands. Model don't know how to distinguish ball from all those heads. - There are occlusions, where player accidentally hide ball from camera. - Ball is relatively small, so IoU based trackers may miss it when it travels fast.

[@5-minutepython]

@@Roboflow yeah, it’s complicated. But i solved these issues after a long research🤗

[@Roboflow]

@@5-minutepython congrats! This topic is super interesting and inspiring!

[@mahmoudfarid1288]

Can that be done with yolov7?

[@Roboflow]

Hi! Nope, with YOLOv5

[@akashrakshit7842]

Not being able to download the kaggle dataset at the moment! Will that be only possible after the competition results?

[@roboSkalskiP]

Hi! It is Peter from the video. Few other people in comments have the same problem. I could access data because I was participating in the competition. I'm talking to people inside Roboflow, we will try to host part of that dataset for you to use.

[@roboSkalskiP]

Hi! You should find zip with 200 clips here: drive.google.com/file/d/1khi9E-uC6nfmOn-Vqhh1SYiTsynK7lUy/view?usp=sharing

[@akashrakshit7842]

@@roboSkalskiP thanks a ton Piotr. ❤ You are doing an amazing work.

[@roboSkalskiP]

@@akashrakshit7842 thank you very much :) I'm really happy you like it!

[@NEZERSEY]

how can yolov5 inference be remodified

[@SkalskiP]

Hi it is Peter from the video. Could you be a bit more precise? I'm afraid I don't understand.

[@Premiercoaching]

This is brilliant. I've been looking for something like this. Can I track players from my own full game footage?

[@Roboflow]

I think if camera angle will be similar it should work. If you want to confirm, pick one frame from your video, and drop it here: universe.roboflow.com/roboflow-jvuqo/football-players-detection-3zvbc/model/2 It's slightly less powerful version of my model.

[@Premiercoaching]

@@Roboflow Thanks for the reply. I dropped an image in. What am I looking for?

[@Premiercoaching]

I also uploaded the video. The image/footage is from a drone so it is a similar camera angle.

[@Roboflow]

​@@Premiercoaching It should basically be a detection test. Notice if model: - detected ball - detected all football players - distinguished between players and referees - not detected any additional people that shouldn't be detected If answers for all of those questions are positive than you should be able to successfully use code in Google Colab to track players on your own football game footage.

[@Premiercoaching]

@@Roboflow ah, OK. It has not detected all players, the ball and referee. It has detected most players and some people at the side of the pitch.

[@temitopeibrahimamosa2885]

The kaggle dataset is not accessible... because no new entry is allow to the competition! please did anyone as the data and can share

[@Roboflow]

Oh. Thanks for pointing that out. That whole competition looks like it is frozen. It should be ended like two months ago, but it looks like it is still on. Do you need the whole dataset or just some example videos?

[@temitopeibrahimamosa2885]

@@Roboflow Thank you for your response.. some video is okay

[@temitopeibrahimamosa2885]

I also want to ask, apart from the video did you inference the model with other video

[@akashrakshit7842]

@@Roboflow Yeah a few sample videos will be ok for now. Would be highly thankful if you can provide that.

[@roboSkalskiP]

@@akashrakshit7842 Hi! It is Peter from video. Let me just ask around in @Roboflow, we will try to help you ;) stay tuned

[@MassimoArteconi]

If we change the videa with a different one what are the point of attentions?

[@Roboflow]

- First of all, make sure that the videos that you plan to use are without any cut scenes. We trained our model on this dataset: app.roboflow.com/roboflow-jvuqo/football-players-detection-3zvbc/4 so the camera perspective should be similar. - We can detect referees and goalkeepers so make sure you test and validate if the model can detect those. Depending on the league model may have problems If, for example, refs are wearing a different color of shirts. - Make sure to run detection with as high inference resolution as possible.

[@Roboflow]

If you'd like to discuss more, feel free to start a thread here: github.com/roboflow/notebooks/discussions/categories/q-a

[@MassimoArteconi]

@@Roboflow done :) lot of questions come to my mind

[@Roboflow]

@@MassimoArteconi great! Let's have a chat there :)

[@mohamadghaithalzin9318]

Is it possible to use the model on an Android app using Java or Kotlin?

[@SkalskiP]

Short answare is yes, but it would be really complicated. How about running that on backend and streaming result to app?

[@mohamadghaithalzin9318]

@@SkalskiP Well, Do you mean sth like triggering a Lambda function of Python code on AWS that receives image data and returns the result of the model?

[@SkalskiP]

@@mohamadghaithalzin9318 for example, if you don't need real time processing

[@mohamadghaithalzin9318]

@@SkalskiP Indeed I do need real-time processing!

[@SkalskiP]

@@mohamadghaithalzin9318 hm... I'm not sure about the performance, but you can definitely convert detector from pytorch to tensorflow light and run it on android device. Tracking can be a problem. I'm not really familiar with any implementation in Java or Kotlin :/

[@awezsheikh8962]

Cant load kagle dataset any another way

[@Roboflow]

What’s the problem with Kaggle?

[@awezsheikh8962]

​@@Roboflow it's showing out of date can't use the loading dataset command. Which is !kaggle competition files -c dfl-bundeslinga-data-shootout | grep clips | head-10 I checked in kaggle it showing no dataset available for dfl- Bundesliga

[@smartfatorynumberone2760]

The important thing is the detector

[@Roboflow]

Exactly! Without training the model on custom dataset the result wouldn't be as good.

[@combustion2479]

I have no idea why my questions have been deleted.

[@Roboflow]

Did KZbin deleted your comment? We haven't done that.

[@combustion2479]

@@Roboflow I don't know. Btw, if it is public, I would like to ask about some details of the custom model training settings. Many thanks!

[@Roboflow]

@@combustion2479 Sure thing! If possible create issue or discussion topic on github.com/roboflow-ai/notebooks It is much easier to have this type of conversation there.

[@jacobjonm0511]

These tutorials are only good for making videos on youtube using and simulated data. Once you deploy them in real world football then you see they are hopeless.

[@Roboflow]

Hi Jacob 👋🏻 This data is not simulated. It is coming straight from Bundesliga games.

[@ifyugwumba8120]

Thank you very much for this beautiful content I got hold of your content from LinkedIn.I would like to ask if someone wants to be like you what path would one take I guess you are an ML engineer 😊 Thank you

[@Roboflow]

Oh! Thanks a lot 🙏🏻 I aus great to start weekend reading something like this. It is quite long story. But on Monday I’m point to be guest of a podcast and we will talk among other things the path I took to be here. If you follow me on LI you will definitely see me posting about that next week.

[@ifyugwumba8120]

@Roboflow Alright thank you so much for your response and time looking forward to your podcast am all down to listen

[@Roboflow]

@@ifyugwumba8120 not mine :) I’ll just be guest ;)

[@alexandergrifith7786]

This is awesome!