Football AI Tutorial: From Basics to Advanced Stats with Python

[@giancarlocestarolli6049]

That’s an amazing video! Few people in the world provide such great content for free. 1 hour of pure learning.

[@Roboflow]

I’m soooo glad people can see the effort I put into this video.

[@schneidershades]

Thank you so much. I got into Object detection and python with your guidance and help in just 2 months to solve problems in the football infrastructure. Thank you

[@Roboflow]

so awesome to hear that those tutorials really help people!

[@VideoScopeAlpha]

@Roboflow They certainly do, and I've been working on something during weekends for the last few weeks. I'm hoping to finish it soon and will share the results here.

[@aishidiot]

This is absolutely GOLD ! Haven't watched it but it's on my list. Have been following your updates on TW !

[@Roboflow]

haha if you read my X posts you probably already know everything haha

[@mind6861]

Very true 🤣 Nothing is free except your tutorials Thank you for your efforts 🌹🌹

[@Roboflow]

I see you decided not to skip model training section ;)

[@prometheus6001]

Tell them ohhhh!!! It's called Football!!! FUUTBALL... The sports where you use your foot to play the ball. Not your hands.

[@Roboflow]

EXACTLY!

[@kof3317]

thanks a lot. I am currently applying this to field hockey. I have completed player object detection and am working on pitch keypoint detection.

[@Roboflow]

wooooow! I’d love to see some results!

[@kof3317]

@@Roboflow haha, I'm currently focused on building up the dataset, and once I finish with the keypoint detection, I'll share the notebook. The biggest issue with field hockey is the rolling substitutions. As mentioned in the Q&A video, when players disappear from the screen, they are sometimes recognized as different players. When a substitution happens, I would have to manually switch the player, but I still don't know exactly how to handle that.

[@vagdrak6575]

the moment he said it is called football --> insta sub

[@Roboflow]

That was a risky move. I bet it can work both ways haha

[@calebcauthon1117]

your initial tutorial was my intro to computer vision and helped me get pretty far. this one has lots of unblockers. I was doing homography before, but I didnt know about keypoint detection. another unblocker is the specifics of how you did classes for training. should help me a lot!

[@Roboflow]

Awesome to hear that! You came back after almost 2 years for part 2.

[@FranciscoBerkemeier]

You could probably now train player movement to deduce a realistic model for video games! Probably moving away from Python I guess, but that would be extraordinary, with tweakable speed and intelligence, even if resolution is relatively low. Suggestion is better than perfection. Wonderful stuff!

[@Ghost21PG]

This will be one of the best projects I could've ever wished for❤️ Thanks a lot Sir

[@Roboflow]

I love sports + computer vision combo!

[@skipius]

Thank you so much for this video. Im currently working on getting it to work for my volleyball matches.

[@Roboflow]

Really? I’d love to see your results!

[@RyanGooch-t5n]

Amazing video, thanks for sharing! I did my postdoc research in computer vision, using similar techniques, including even the perspective transforms, to ultimately automate error calculations for solar mirrors. I feel your pain on the keypoint labeling, that takes forever :'). Even though that was only 4 years ago, it's amazing how much progress has been made and how much easier it is to manage these models, datasets, and transformations. My postdoc work would have been so much easier in 2024 than it was in 2020, thanks to these developments and tutorials like yours 😁

[@Roboflow]

Thanks a lot Ryan! 4 years in AI space feels like forever. I can’t even imagine what will happen over the next 4 years. It’s also super validating to hear you used similar strategies, and super interesting you’ve done this in completely different field. We are organizing community session next week. Is there any chance I could show some visualizations from your work on solar panels?

[@RyanGooch-t5n]

@@Roboflow Sure thing! Please send me an email and I can share images and more info

[@khanhtuongtranlam9141]

Thank you. I am researching for my final project at my university. My topic is the same one you made. I hope we can discuss it more. Thank you so much, bro.

[@Roboflow]

you can always find me on LinkedIn and X

[@alvarofelipe1273]

Amazing video, thanks bro. You are my hero in CV

[@Roboflow]

thanks a lot!

[@Deutschzeit]

Thank you for encouraging me to pursue a career in this field with your videos :)

[@Roboflow]

It is really cool field to have career in!

[@RicardoBennesby]

I could't wait to watch this video! Thank you for sharing!

[@Roboflow]

Let me know if you like it and if you have some questions or cool ideas

[@pratyushkumarkarna9666]

wow. Even though I have not that much knowledge of ML, the video was great

[@Roboflow]

I was trying really hard to make it easy to follow even to people without ML background.

[@ericjones7300]

Outstanding video. Oh....and Go Barcelona!!! :-)

[@Roboflow]

thank you very very much!

[@maveriktn]

Thanks Peter this video is very useful!

[@Roboflow]

Great to hear that!

[@proximus323]

Piłka nożna 😊

[@Roboflow]

nie da się ukryć haha

[@flasssss]

this is hilarious huge project !

[@Roboflow]

thanks a lot! so cool to see people are still excited about Football AI even several weeks after release

[@tylerweirtube]

Amazing, thank you for recording this.

[@Roboflow]

my pleasure!

[@LouisDuran]

Thank you for this tutorial. I plan to run through it soon!

[@Roboflow]

Let me know how you liked it

[@alvarozambrana9661]

Amazing!!

[@danielisflying]

You are blessed! Thank you for this great effort.

[@SBARTSTV]

Great!

[@GisleHoelAarhus]

Thanks alot!! Greatly appreciate it

[@Roboflow]

my pleasure!

[@brmgha]

thank you from Egypt

[@sinavahid5194]

youre a life saver

[@yunqiguo9100]

Very nice tutorial! Hope to see the parts with Player ReID!

[@Roboflow]

Judging by the frequency of my work on football AI it will come in 2 years. Haha I hope not. ReID is really something I want to do at some point.

[@AbhishekAgrawal-dv1id]

Good you cleared it out, never understood "soccer"! Lol.

[@Roboflow]

Haha! I decided that important things need to be explained right at the beginning.

[@salahidin]

It would be great to have this in real-time

[@Roboflow]

There are 100% easy optimizations that can bring us closer to… 15 FPS. Going faster than this can be challenging :)

[@antonzizic3483]

thank you very much for sharing this knowledge !

[@Roboflow]

My pleasure!

[@ingush1989]

God bless you brother

[@Roboflow]

Thank you!

[@psalav]

I would really like if you can do something similar but with tennis!!

[@Roboflow]

Tennis is 100% on my TODO list along with basketball. I just need to work on other projects in the meantime:/

[@user-gg9ou2iz1j]

I dont understand where did you get the footage of the game, because normally a television stream would not show the action in such a wide angle and we wouldn't be normally able to see all the players at once

[@Roboflow]

I actually tell exactly where I took it from in the video ;)

[@VideoScopeAlpha]

When training the initial model for player detection, how does the model distinguish between a player and a referee? - If the model's distinction is based on the color of clothing, will it still work if the clothing colors are changed? - For example, if referees typically wear yellow shirts, what happens if players in another match wear a similar color? - In some training annotations, I noticed that a person (possibly a coach) standing outside the field wearing black clothing was not labeled as a player and, therefore, is not detected (as expected). I'm curious how the model understands these differences.

[@Roboflow]

VERY GOOD QUESTION! In general hard to tell how exactly does it work, but during data collection I tried really hard to include refs in different outfits - red, black and yellow. I think the model just learn that refs have outfits visually different than most of people on the field. As for refs on the sidelines, they have flags and I think this is how model knows, people one the sidelines without flags are not refs.

[@PetreDiaconescu]

If tracking is lost in a frame for a player, then the next track_id that’s assigned to the same player is different, which makes it impossible to track the same player for the whole game. Is it possible in any way to fix this? Or even assign the jersey number as an alternative id? Great video btw

[@Roboflow]

Jersey number assignment is a lot more complex task than it seems, so I would treat it as something extra and not a potential solution. At the end of the video I shortly talk about next steps in tracking - using ReID models or more advanced trackers like MASA. In short we need trackers that take into consideration object location but also its appearance.

[@fernandodutra3788]

Really great content, congrats! May I ask why did you choose to detect the key points every frame instead of using optical flow, knowing there are many features in the video (incl the pitch) which are static? Additionally, how would you approach tracking of the ball in 3D if you wanted to have high accuracy for the ball position?

[@Roboflow]

Cool questions. Correct me if I’m wrong but optical flow won’t take me all the way. It only shows me that „something moved there” but does not tell me what it is. So if I want to know where are my reference points it won’t be able to do it. Because it does not know what points are interesting to me. Second of all pitch is not really static. You see pitch all the time but only some part of it. And you need a solution that will tell you what part of pitch is it. I didn’t really thought deeply about what model would I use to support 3D ball detection. But I recently was playing with 3D keypoint detection models that gave me 3D coordinates of human pose. I’d start there. But I’m open to other ideas.

[@fernandodutra3788]

Thanks for the swift answer. What I thought with optical flow is that given an initial state (1st frame) we can use anchor points known to be static in the world in order to estimate how the camera is moving. With that you can calculate how points in one frame relate to the next. But you are right, this approach is very fragile because it relies on having a set of anchor points which are always visible in the footage. For the 3D ball trajectory, because we know the physics of the ball movement, I’d probably start with trying a Kalman filter. One model for when the ball is in the air and another one for when on the ground. Use ball size in pixels as an estimate of how far from the camera it is. This is evidently very noisy but hopefully the dynamics of the kalman filter would smooth the trajectory

[@zakazaka]

00:00:49 - damn straight!

[@SkalskiP]

population of subscribers from Europe is increasing haha

[@JasonLiu-w3v]

@@SkalskiP You also got subscribers from China haha. In China, We call this sport 足球 which literally means football. 足 shares the same meaning with foot and 球 means ball.

[@bahwateengezeur7516]

Amazing video! I was wondering how much features you could add ≠ how much more data can be extracted. Lets say you have competitions where besides goals / assists nothing is tracked, but I do have access to full matches. Would it be possible to for each player individual extract data using this particular software, shots on target, passes completed, interceptions etc?

[@Roboflow]

Short answer is yes, but that project would need a lot more work to get there. The most problematic part is actually player tracking and avoiding tracker ID swaps. That can happen especially if you want to process whole video. Do you want to run it real time or process games offline?

[@bahwateengezeur7516]

@@Roboflow I believe offline would work best since it isn't always easy to gain access to full live matches of these types of competitions. To run it all automaticly would be a lot more work as well. If you have any ideas on how to realize this project I'd love to discuss it with you!

[@ErcoleBellucci]

Crazy tutorial, what are the specs of your pc? better local or cloud setup?

[@FelipeArrudaMoura]

Thanks for the video. I am checking that, for some videos and some players, the IDs are changing between the teams, even very different teams' colors. Any tips to increase clustering performance?

[@modanmohanmanna1693]

ohh that's starting make me subscribe u 😂

[@Roboflow]

Gooood decision ;p

[@RanaUmar-m4c]

Quick Question ! I'm working on a system. Some person on Reddit provide me with your football video that will help me build my project that is to count how many customers come into a retail store and also track employee activity, especially their working hours. One problem I’m facing is that the system assigns a new ID whenever it loses sight of a person, which throws off the customer count and causes issues with tracking employees too. Also, since the store uses multiple stationary cameras (unlike moving cameras covering whole feild), it gets tricky to keep track of the same person across all the cameras. How can I solve this issue, especially syncing the cameras to avoid assigning new IDs?

[@p.logesharavind3528]

You have created one custom model, with this model, will we able to use for other match videos ? for tracking

[@Roboflow]

Model is open so you can use it. As for „will it work” I discuss it towards the end of the video.

[@Marc-up3gg]

This wouldnt work very well without a high overhead viewpoint right? The visual noise of crowd and sideline players would botch the CV from working properly I assume?

[@Roboflow]

Yup! We cover all important considerations at the end of the video. If you want to use ground-level cameras this problem becomes borderline unsolvable.

[@QuarkLearns-in7uc]

I'm reviewing the scripts, and I've noticed that some of your Collab codes don't appear in main.py. Am I making things too complex? I'm little bit confused bc of that

[@Roboflow]

The code in script might be vastly different than the code in colab. I recommend Colab as the reference.

[@sachinperera4230]

curious to know how the models would perform when the teams wear kits with similar colours?

[@Roboflow]

This rarely happen as there are rules to prevent that. But we can test if you have any video we could use.

[@mushafsibtain]

Thats an amazing video!! Is it possible that i can discuss you about my project personally and you can guide me how to do that thing? I would be so grateful..

[@Roboflow]

Unfortunately not… I used to do it but now I get too many requests like this. It would be a full time job :/ sorry

[@TheBradleydwyer]

Nice video. Really enjoyed it. I don’t quite understand why we use SigLIP for the team ID vs using an additional class in the object detection model (“player team 1” vs “player team 2”). Is there some complicating factor that makes that not work well?

[@Roboflow]

This is something that I clearly did not explained because you are not the first one to ask this question. So the problem is that every time different teams play. In one game red play against blue team. In different yellow against white team. There is no way you could annotate data in a way general enough you could apply it to any game in the future. Solution that I presented is general. And does not require you annotate all data with more info.

[@TheBradleydwyer]

Ah that makes sense. Is soccer similar to other sports where the “away” team usually wears white/light jerseys and home wears dark/color jerseys? Wonder if home/away classes would be enough to generalize on

[@SababAdad]

Hi. Can you please tell me in which folder you're saving the output result video that you've shown in the tutorial?

[@Roboflow]

If you run it in Colab, I save in `/content` which is the default output directory

[@SababAdad]

@@Roboflow So, all the output videos that you've shown in the tutorial will be saved in '/content' folder, right?

[@Rock_Film]

Thanks Bro❤

[@rossanovinicius7373]

Basically I should create a new project for every team or new video I would like to analyze?

[@Roboflow]

Hi! No. Everything can sit in a single project. Why do you think you need to split data?

[@DavidSantos-ms2bz]

Thank you so much for stating the sport's correct name hahaha

[@Roboflow]

Haha oh that was the easiest part of making this tutorial

[@AnandaMuhuri]

Visca Barca ✊

[@ALOKSHARMAMD]

oh wow, what a nice find. can you tell me what techniques is required to make this real-time assuming a 30fps input and same output? do i need to implement this in deepstream?

[@Roboflow]

Stream processing is not the reason here. You’d need to train smaller architectures not YOLOv8x but YOLOv8s or m. But most importantly make embeddings calculation faster.

[@ALOKSHARMAMD]

@@Roboflow thank you, i will try that my goal is to do live video feed processing. the concept is same where i need to capture keypoints and map it to a 2d map & mark trajectory but for a different project.

[@knobico1337]

Great video!! I'm trying to put this project together based on your video, but I'm stuck because I haven't figured out exactly what TeamClassifier's predict() method does. I searched in the Colab code but was unsuccessful. Could you describe the exact code for that method?

[@Roboflow]

All the code for TeamClassifier is here: github.com/roboflow/sports

[@knobico1337]

@@Roboflow Thank you very much!!!!

[@Roboflow]

@@knobico1337 pleasure!

[@froukehermens2176]

Instead of the embeddings I have used image classification for a different but related task.

[@Roboflow]

Let me explain why I went with embeddings and not classification: problem with football is that every game is different. In one game you have white vs red. In other you have blue vs yellow. So I’d need to have separate classes for all of them. Embeddings is general solution. It doesn’t matter who plays. And it does not require annotation.

[@ניסאביטן]

Hi man, great tutorial! I would like to ask you a few questions. How can I contact you? Tnx!

[@Roboflow]

Leave your questions in the comments. I answer most of them.

[@ניסאביטן]

@@Roboflow

[@ניסאביטן]

@@Roboflow

[@felipemoura7461]

Thanks for the video. Fantastic. Is it possible to run in our own GPU, in WIN10 system?

[@Roboflow]

It is possible to run it on your own GPU (100% if you have Linux). I’m not sure about the windows part. I did not installed anything on windows in 15 years. :/

[@raphaild279]

Can you tell me how the yolov8 architecture works? I would like to ask how to train the model to recognize circles and output, instead of the width and height of the bounding quadrangle, the x/y location and radius.

[@Roboflow]

we have dedicated YOLOv8 tutorials on this channel; did you have a chance to watch it?

[@raphaild279]

@@Roboflow Yes, and what about the second question. who can help?

[@TalebTuradi]

Great job.I have implemented the code how can I save the video to view the shot from the match

[@Roboflow]

We explain saving and downloading video from Google Colab in the video.

[@pysoft]

🔥🔥

[@user-wx6hw8hm6q]

Hi quick question, so I'm trying to use the best.pt model from your yolov5 + bytracking video and it's not really tracking my own dataset, does the best.pt model you made only work on the dataset from kaggle?

[@petarpetkovic5275]

Hi, I will try to answer on this question, best.pt is yolo v8 model fine-tuned on specific kaggle dataset as he mentioned, so the best.pt model is output from training showed in 10:00. If you want to use the same video, this model should work fine, but if you want specific football game, you should fine-tune model on that game. You can label data on roboflow with label assist tool and then fine-tune model on your dataset. There is plenty tutorials about that on youtube, check it out!

[@Roboflow]

hi all :) I cover this topic towards the end of the video. The distribution of train dataset and data you run the model on should be similar. So if I use TV footage and you use for example video made with phone standing on the sideline, the model would need to be fine-tuned.

[@jeremykim1236]

incredible video! I copied your notebook and added a way to correct the ball tracking when the ball is in the air. Can I connect with you to discuss on Linkedin or twitter?

[@Roboflow]

big if true! absolutely! what's your X handle?

[@jeremykim1236]

it’s jeremy9k27 - I also replied to one of your threads on x with a screenshot of my results 👍

[@ParkiOfficial]

is it possible to make a pass counter for both teams ?

[@SkalskiP]

yup! that would be a natural extension of that project, we know the accurate ball path so it should not be overly complicated

[@MuhammadMominRauf]

Recently i started a similar project but my main concern is real-time analysis is there any way i can connect with you either email or linkedin

[@Roboflow]

Just leave comment under my LinkedIn post and I’ll invite you

[@estadisticamatematica8737]

"What real application is it used for? I mean, what are the benefits of this? Thank you for the video!"

[@Roboflow]

Real football and basketball teams use data like this to for examples trade players looking for this that suit their play-style the best. Another usecase could be data overlay during game broadcasts or automated referees.

[@thewatcher4940]

After training when I run: %cd {HOME} Image(filename=f'{HOME}/runs/detect/train/confusion_matrix.png', width=600) I got error FileNotFoundError: [Errno 2] No such file or directory: '/content/runs/detect/train/confusion_matrix.png' what is wrong ?

[@Roboflow]

Is it possible that you interrupted your training somehow and restarted it once again?

[@thewatcher4940]

​@@Roboflow Yes you had right. This is result of my lack of knowledge of Roboflow environment, but I still learn.😅 Thank you for clue

[@Roboflow]

@@thewatcher4940 uuuuf! I was scared for a second haha

[@neerajr1582]

I am currently working on a project to identify players using their jersey numbers. I trained a YOLO model to detect players, another YOLO model to detect the jersey region, and a third model to predict the jersey number. After making predictions, I swap the track ID with the detected jersey number. However, the issue I'm facing is that the track ID keeps changing throughout the video. How can I maintain the detected jersey number consistently throughout the video?

[@Roboflow]

Awesome project! Would love to take a look at some visualizations from project like this! I’m currently working on tracks stitching. That would allow you to maintain same tracker ID.

[@neerajr1582]

@@Roboflow Waiting for a good outcome

[@rithikkumar7683]

using this as base knowledge in which fields we can apply this for a day2 day life prob?

[@SkalskiP]

the most obvious choice is traffic analysis in large-area stores; people want to know how customers move, where do they stop

[@rithikkumar7683]

@@SkalskiP thank you sir

[@DiffQ_Bro]

Can you detect someone taking a shot?

[@ernestaddo9578]

Quick question. Assuming i want to learn how to do this but have no idea about coding, can i still do it? Cos i want to do it

[@Roboflow]

I think so! You can certainly try it and see how it goes for you! Google Colab does not require any installation.

[@EgoIrreverensTV]

It wonder if it’s possible to do it realtime

[@Roboflow]

I cover this topic at the end of the video. I think it is possible, but would require a lot of optimization.

[@DomiSiki]

Can you calculate distance covered for each player?

[@Roboflow]

Yup. At this point you have everything you need to calculate distance

[@DomiSiki]

@@Roboflow perfect. Would this work for futsal? This could be great app. Once finished, what you need to do, upload the video from the game for example? Would that be enough?

[@livetag-q2t]

i receive this error when i run custom training : [Errno 2] No such file or directory: '$HOME}' /content /bin/bash: line 1: yolo: command not found

[@Roboflow]

Hi 👋🏻 looks to me like you didn’t executed all cells in order specifically missing one at the top that creates HOME variable.

[@livetag-q2t]

@@Roboflow can i contact you via email please ?

[@suhailk3958]

Can u explain what are the data it will track/store. i mean the practical application of this project. Anybody Please take a moment to explain that you are explaining to an 10 year old.

[@Kokouvi_el_mejor]

❤

[@Roboflow]

first comment in first minute 💜

[@QuarkLearns-in7uc]

when i try to do this locally it occurs an error can you help me please?

[@QuarkLearns-in7uc]

im trying to implement keypoint detection etc to my project

[@Roboflow]

@@QuarkLearns-in7uc does the error accure when you run our code or your code? :)

[@deepanshumishra7518]

what the technogolies used in this project? plz reply

[@Roboflow]

You ask about specific Python libraries?

[@deepanshumishra7518]

@@Roboflow yes. I'm new in it. So, i am taking to overview of the project. That's why ask for the technology used

[@mck5311]

So basically u ask us to train it and use roboflow to update our results , hard pass

[@Roboflow]

not sure what you mean, as I said multiple times in the video, already trained models are available for free, so you can skip the training part and go straight to inference

[@abshirahmd]

Correction: Soccer AI Tutorial: From Basics to Advanced Stats with Python 😊

[@Roboflow]

Hahah ;) I don’t think so

[@1b0o0]

Correction, you are uncultured. 😊

[@prefixerable]

Please never come back

[@zssgabe]

ur gay

[@gustavojuantorena]

Great!