How to Scrape SofaScore for Football Data

[@Goonerilla]

Awesome video. Is there a possibility for a part 2 where you show how to recreate the shot map in Python?

[@mobhamjee786]

Yea would like to see that

[@santiagonieto5602]

Hello McKay Johns, great content. I have a question, is it possible to scrape the Match Momentum of the match as well? Thank you very much!

[@RelaxingNature6]

I am very happy Johns about your vedeos you are doing for us , to learn more about how yo scrape Football Data from websites , but my question is , how we can organize the Data as Table not like that ??

[@FiorenzoMaccarone]

Hi, is it possible to extract all the Serie A players and the statistics for each player? Are there too many requests and does it block? In the video I see that you take the URL from Fetch but can it be done automatically to take the XG of each player for example? Thank you.

[@mikearchman]

Hello,, this was pretty helpful. I've been trying to get some data from SS using selenium, but every now and then something changes on the pages,, have to keep changing paths and selectors,, just pure pain,,, this is much pretty neat and easy to maintain,, thanks

[@bernardooliveira7184]

Hi! Great video! However, I have a question: wouldn't it be easier to use the SofaScore API?

[@dmousouroulis]

Thank you McKay, that was quick and easy to follow! Appreciate the great content

[@McKayJohns]

Glad it was helpful!

[@RUOHONGMA-m9p]

Thanks, it's definitely an impressive application on football data scrape. By the way, can we get data from the Wyscout database using the similar method? Many 😊 thinks

[@sailithish4579]

Hey bro I have an issue when I’m trying to scrape data for statistics it’s getting restricted to one league can you help with me how I can tackle this issue and make it so I can scrape statistics of players from all leagues

[@joeosborne980]

New backdrop is clean

[@McKayJohns]

The pikachu lights up

[@constructionisdangerous]

Look man is it possible to figure something out where I can find out stats for team in the last 5.games I can't find websites that do this

[@nadirstivan6088]

I want to ask about the previous program that I explained more than two years ago, which I did, I think between Fulham against Liverpool, how can it help me get good expectations? Is this possible?

[@o.gustavoneves]

Hello, my friend. I Nedd a Help With the shotmap endpoint, for each request I make, the api returns different data to me even if the game has already finished. Has this ever happened?

[@vaggkkk5483]

I do all the steps but get 403 response. Has sofascore closed access or i am doing something wrong

[@McKayJohns]

Hm the only people I know that have gotten a 403 have been using google collab is that your case?

[@danads4988]

​@@McKayJohns Same problem and I use a jupyter notebook. Please can you give an up dated tutorial on how to bypass the 403 on sofascore? Thanks in advance.

[@uttaranbera8573]

Hi, Just wondering, which do you feel is the best website to scrape defensive actions?

[@McKayJohns]

probably look at sofascore to see if it has anything or maybe fbref. If not try whoscored

[@CoachJuan]

Hey is it possible to scrape the webdata to google sheets? I want to keep track of the player stats thats provided by SofaScore

[@McKayJohns]

Yeah you can either save them manually, or if you want to play around with googles sheets api then you could insert it directly

[@RubenMartins-x4c]

this only work for live events? the external api requests

[@OkenTheReal]

Hi, thank you for your video, I recently saw that sofascore integrated "honeytoken" mechanism into their apis, which means that they randomly send you fake data instead of blocking you. Do you know how to solve this?

[@McKayJohns]

Yeah currently working on a updated video. Probably out sometime next week

[@ryanfeasey3725]

Anyway to turn the shot locations to a Data Frame?

[@McKayJohns]

you should be able to do pd.DataFrame(shots) and it will load

[@ryanfeasey3725]

It works but it is still in a weird format any idea on how to get proper coloumns and rows.@@McKayJohns

[@McKayJohns]

@@ryanfeasey3725 you might have to unpack each individual row. I would look into maybe something like this: stackoverflow.com/questions/61847791/how-to-explode-nested-json-in-pandas-as-rows or try and ask chat gpt haha

[@ryanfeasey3725]

@@McKayJohnsthanks

[@Doloufoks]

Hi !! Thank you for the video! I've been using this for a while now, and I really find it excellent. Since yesterday, I've encountered an issue: an HTTP 503 Service Unavailable error appears. Have you experienced the same issue? Do you know if this is temporary or if they have implemented a block on the API calls? Thank you in advance for your help.

[@McKayJohns]

Yes I noticed it today. I need to remake this video with the new method I founs

[@Doloufoks]

@@McKayJohns That would be great! Thank you in advance.

[@pankajjadhav3969]

Hey mckay, thanks for doing these tutorials. Just one question : how do i export it to a CSV?

[@McKayJohns]

if you load it into a pandas dataframe, and your dataframe variable is called df you can do df.to_csv('location you want to export.csv')

[@pankajjadhav3969]

@@McKayJohns Thanks man

[@vartanpashayan297]

Do you have to write the headers if you get a 200 code on the response the first time you try it

[@McKayJohns]

Technically no, headers just help for authenticity of the request

[@davidrohan6098]

Would it be possible to scrape a player's shotmap for an entire season

[@McKayJohns]

Yes totally possible

[@agdaltarek]

i'm finding a hard time plotting the shots on target locations, i guess it's the 'draw' column that gives the coordinates but don't know what goal dimension they have set. do you have any ideas ?

[@macaveliypm]

Hello bro, can you scrape data between date range? like from 2023-02-01 to 2022-08-01

[@macaveliypm]

Hello bro

[@joaopompeo]

Hey Mckay, is there anyway I can put on together all the data from where a player shot and his mean throughout an entire season? Was thinking as well if there's any way for me to enter in contact with you to see if you can do one for me and how much you would charge for you to do so, thanks a lot in advance!

[@sigfigronath]

hey Mckay, can we code using visual studio as well and not stick to jupiter

[@McKayJohns]

Ya I prefer Jupyter for tutorials just cause it’s easy to use but you can use whatever

[@nadirstivan6088]

Just by chance, I saw you in your videos explaining a program between two teams, Fulham against Liverpool, for more than two years... and today, you also explain another program. Can you explain more... Can it help me to have good expectations?I play satellites.... especially football

[@user-nl4gl7gf2h]

Hey my friend, can you create a football predictions script for me?

[@Madib21]

Oh god where on earth did you get all this from..amazing work. Have one question. Do you work as an analyst anywhere in the industry?

[@McKayJohns]

Hahah I work as a data engineer for a startup so no I don’t work in sports just have been doing it as a hobby and to learn new skills for a while now

[@Monsalvo888]

Really useful video. thanks for sharing!

[@Thibsalm]

Great video ! Did you know how to scrape MLB results with all innings scores from Sofascore ?

[@McKayJohns]

You should be able to. I've never done it but process should be roughly the same

[@Thibsalm]

I tried to do it without succeded... could you do a quick video please ?

[@myanmarfootballanalysis]

This really helps me.

[@LanusStats]

Amazing as usual!

[@marcelotrinkard2255]

After yesterday’s 503 errors, the API queries are working again today. I’ve been making some attempts to consume various APIs as you explain in the video, and I’ve noticed that the returned data is inconsistent, even without changing anything in the headers or the query URL. The data changes, and logically, these are queries for data that isn’t changing. For example, when I check the league statistics, one query returns 567 accumulated goals, and the next one returns 1059 accumulated goals. It’s as if it’s providing fake information. Is this a defense mechanism of Sofascore? Have you had the chance to verify the reliability of the returned data?

[@McKayJohns]

Yeah I noticed this and am remaking the video to show how to get around it

[@vascosilva9582]

I'm having the same issue, using this to get info on a team's next match - for each request (using the API), the parsed data shows a completely different timestamp pertaining to the start time, consistenly inaccurate

[@adityakhare8942]

Thank you so much for this great video!! Would be Very Helpful❤🙏

[@atlasgunther8947]

Can you do a detailed video to scrape sofascore's historical score data, i currently have to scroll manually to scrape it. Thank you.

[@vinothnk5591]

thank you very much for this video!

[@TLFootball123]

Good tutorial as always, any chance you can do one on Zones of control? Thanks

[@McKayJohns]

I can take a look 👍

[@TLFootball123]

@@McKayJohns Thanks!

[@TLFootball123]

If you see this, is it possible to convert the JSON into CSV and if so how do you do it@@McKayJohns

[@McKayJohns]

@@TLFootball123 Ya you can put the shots in a pandas dataframe... pd.DataFrame(shots) like that then export it as a csv if you wish

[@Pavle_Savkovic]

Great video thanks a lot!!!

[@victorpaulinodesousa8400]

is there a option for R?

[@McKayJohns]

I don't have the code for R, but the concept is the same

[@scoreadvise_official]

Very nice!

[@maniac_q]

Make a tutorial on how to get trough a chat ban on sofascore please

[@timokorpershoek1893]

chat ban? what do you mean?

[@maniac_q]

@@timokorpershoek1893 my messages don't show up in the chat

[@Muiloko2000]

You're the man

[@everflores9484]

I have the intuition that if i wanted to scrape every match of the PL I'd have to create a list of the 380 matches, but I'm sure there's an easier way. Same with say, ManUtd. Sofascore doesn't make it very easy to scale I feel

[@McKayJohns]

Yeah it’s tough but you should be able to programmatically make a list of match urls

[@bakhodiryakubov3981]

@McKayJohns, do you use playerCoordinates or draw coordinates for the shotmap? I followed your tutorial for shotmaps using data scraped by following this video but it's not aligned, goalscorer is somewhere in the middle of the pitch :)

[@akawonde4044]

I have this thing

[@nadirstivan6088]

Hello, I would like to ask, is it possible, is it possible, is it possible, for example Hello, I would like to ask, can such a program help me, such as the way you choose the results of football matches, especially since I play or any program you talked about I play in the field of sports, note, frankly, can you help me what you play in it and how it is

[@Eizengoldt]

What’s wrong with you?

[@SeifAjax04]

response.status_code is 403 While I am using the latest version of Google Chrome and i followed the instructions exactly :(

[@McKayJohns]

Hm what does your code look like

[@SeifAjax04]

@@McKayJohns hi bro First, I want to thank you for your great content,, I was using DataLore, but when I used jupyter It worked well, but I haven't finished the video yet, and I still lived your exciting experience, But I want to send you an email to inquire about something. Do you have time for that?

[@poeticscales]

Hi Mckay, first of all I've really learnt a lot from your videos thanks for all of your informative videos. I do post Football Data Analytical & tactical videos on my channel really would like to have a chat with you so that I can tell you more about my content, if possible I'm from India so if anyone is planning to visit my channel there will be language barrier 😅