Working with APIs in Python [For Your Data Science Project]

[@jaggyjut]

After a lot of search, I finally found this tutorial which shows how to get data from an API and then convert the json into a dataframe. This is pure gold. Thank you

[@stratascratch]

You wouldn't have thought it would take so long to find a video that does? =) glad you found this channel.

[@jaggyjut]

@@stratascratch there are tutorial on how to use API and then different tutorials on converting json to data frame. So far haven’t found a tutorial showing everything end to end.😀

[@purvilimbachiya1501]

what is the next step after importing a database??

[@vijaykumarcn8425]

How about applying read_json method ?

[@Thuvu5]

Hi Nate, thank you for this great video! Just a quick tip, if any wants to grab the video data for multiple channel IDs at once you can directly use this method (from KZbin API doc): request = youtube.channels().list( part='snippet,contentDetails,statistics', id=','.join(channel_ids)) response = request.execute() where channel_ids is a list of channel ids (e.g ['id_1', 'id_2', 'id_3''] and youtube = build('youtube', 'v3', developerKey=api_key)

[@purvilimbachiya1501]

what is the next process after importing database ??

[@farhansarguroh8680]

Thanks a ton

[@johnwig285]

OMG you're here anyway thank youuu thu vu 🥺

[@joaofernandes6349]

The best part is how you show in a easy way how to do the whole process and then you clean up the code according to good practices. This was really good, thank you!

[@prateek2159]

Hey Nate, your videos are just too good. I love how your channel is so dedicated towards real word data science. By the way I noticed that you started a video series, "For your Data Science Project" and I really want you to continue making videos for this particular series because there's literally no one on KZbin with such guidance on DS projects and I have been looking for one since a very long time because I have my placements just after 12 months and I really want to make a full stack data science project. Thank you.

[@adityaahire2743]

I cannot comprehend right words to express how great this video is and all the content that you create. For all those who dream, plan or wish to make great career in data science you are simply an angel. The most important part which I want to request is a complete playlist of all the components that you mentioned in the : "The one and only one Data Science Project you need". This video covers the data collection part through API...and if other components are covered and all together that playlist would become the gold mine for aspirants. I hope you get time from your busy schedule to consider my humble request... Thanks a lot for all your videos which keep enlightening the Data Science aspirants.🙏

[@stratascratch]

Definitely will create this playlist or at least write an article that will provide you with resources for each part of this project journey. Thanks for the suggest. It's a great idea.

[@alfatmiuzma]

Next highly expected topic can be .. setting up cloud to store the data. We can have a playlist of videos on topics Nate mentioned in the end to end data science project 😊😊

[@CodyJelinek]

This is brilliant. I took way too much time searching and googling and watching hours long videos to find pretty much everything I need in a clear crisp straight forward 30min video. Minus a tiny bit of API beginner information and knowing the basics of Python, this was perfect. Well done. I imagine the next video will be everything I need and nothing that I don’t to take the next step of processing/analyzing the data. I am hitting play on that……. Now!

[@Networkprofessor]

I thought my video speed was on max, but it wasn't; there is just a lot of information being presented. Very comprehensive. Thank you.

[@stratascratch]

Thank you. We love to share what we know. Glad we have helped you.😀

[@tuhindas6745]

I have been struggling to learn data science from multiple sources and there are hell lot of content.. After scavenging through the dirt pile, garbage of useless information. I can finally say that I have found the most valuable resource available on the internet ! Thank you Nate ! I can finally see light at the end of the tunnel. Love from Delhi, India

[@stratascratch]

Thank you for your kind words! More to come!

[@Delchursing]

Made more api progress today than with days of Internet and ai. Thank you. I love how pragmatic this video is.

[@MrHyde_evol]

Thank you so much for the detailed breakdown and clear explanations of your code logic! As someone new to Python, I understood each concept but was struggling to piece it all together. This video was incredibly helpful, and you've earned a like and a new subscriber. I'd really appreciate it if you could cover the different types of API authorizations in more detail. A video explaining the requests module, especially how to work with GET, POST, PUT, etc., and their parameters would be great. Also, some guidance on using virtual environments would be very helpful!

[@rohitprajapati2303]

Amazing video ,finally get the understanding how to extract data from the API ,Thanks man

[@stratascratch]

I appreciate your feedback! Thank you too.

[@azamatbagatov4933]

This channel deserves million subscribers and views. Keep it up!

[@soumodeepsen3448]

This is probably the best video i watched so far on use of api in data science cause everywhere else the videos are so long and i could not understand a lot of those tutorials. Thank you so much for this video content.

[@badsanta01]

Thanks for the video Nate. Update : One might get an error in the dislike count part. "The statistics.dislikeCount property was made private as of December 13, 2021. This means that the property is included in an API response only if the API request was authenticated by the video owner. See the revision history for more information."

[@Davidkiania]

What I love most about your content is how you simplify the building process and then refactoring …. I am hooked.

[@menchu56]

As a beginner, this really helps me understand how to work with JSON files and API calls. Thanks for the great content!

[@sitanshumishra6468]

Thanks Nate, this is the next level of simplifying a so called complex issue related to an API

[@czr372]

I spend couple hours figuring out the url's and the api key, but this is a great help, Thanx a lot!

[@grimoireen]

Wow, you're really give a huge knowledge for data science projects for free, big thanks man

[@ivlivs.c3666]

Now this is real-world, practical knowledge. After watching your video on the one and only data science project you'll ever need, I knew I had found something amazing. After watching this one, I was sold. Just subscribed. Keep these gems coming!

[@ujjwaltarway1438]

One of the best videos regarding API call. Subscribed.😇😊

[@naanaaabdelmalek]

I really wanna thank you so much cause those are the real skills that we need as entry-level data scientist, i'm an artificial intelligence student and everything they are teaching us is a pure academic which will not cover the practical side.

[@clockfixer5049]

Incredible how concisely this whole topic can be presented. Happy to very soon become a StrataScratch subscriber.

[@1cutecouple]

THIS IS ABSOLUTE BLISS TO WATCH!!! THANKS FOR THE AWESOME CONTENT

[@stratascratch]

Glad you enjoy it!

[@darkflamedoctor9023]

@@stratascratch now that we have collected the data how are we going to apply ml to this specific task(yt api) My Guess is we need little more info to predict something with good accuracy these are the things I thought for this data: I think we can cluster the video with less views or likes or something like that, one more thing I can think of is predicting views or likes with the help of regression models I cant think of anything else can u please tell me something else

[@danielmoss7133]

Best video I have seen on APIs. Thank you so much.

[@stratascratch]

Thank you! If there are other projects you're interested in, let me know!

[@avanishpank7572]

Ever since I started my data science learning journey,this might be the most valuable video I have come across,thanks a lot mate,this is extremely helpful.

[@stratascratch]

Thanks!

[@cristiansoto7581]

More videos about this, please. Great information. I loved it!

[@javierjdaza]

simple, short, and efective. Keep going bro, amazing content

[@stratascratch]

Thanks so much for watching. Glad people like these python videos!

[@luzestrada9536]

You are the best! Thank you so much for sharing this valuable information

[@stratascratch]

Thank you. We are happy that we were able to help.

[@BasicoftheBasilisk]

Really appreciate the tutorial, tremendously helpful getting my own api project started.

[@chiomaeze8245]

wow, this is amazing! Thank you so much. It was concise and robust. Learnt a lot from it

[@stratascratch]

You are welcome. Happy that you found our video helpful.

[@inigocuervo]

Hey there! Just a quick tip for anyone trying out the code: make sure you're using a version of pandas earlier than 2.0. Otherwise, you might need to use a different method instead of .append, or consider adding the data to a Python list first and then converting it to a pandas dataframe. Hope this helps! 😊

[@lme918]

Even if you will use append method, it's strongly adviced do not use append as in his video as it's awful for performance. Don't do it!

[@jiangchengli2191]

good work man, best tutorial of collecting data from api !!!!

[@stratascratch]

Glad you think so!

[@stainxyz3373]

thanks. this thing took me an entire day to finish.

[@stratascratch]

I'm glad you did the project! It's a good one day project

[@PATRICKCHUAD]

Thank Nate for this very clear video on API. It is a great help for me as I'm studying python API , flask etc. right now.

[@mazhar0001]

Covered all the basic and important points is a well structured manner to make it simple enough to follow and understand. You have great teaching skills! Thank you for spreading your knowledge.

[@JMinster]

Thank you so much for your videos, I just recently discovered this channel and it is amazing!

[@hafsawajid2793]

You are doing an amazing job. It is what's required to understand the programming side.

[@codehorror6743]

the amount of knowledge in this one video!!!!!!!!!! God bless you

[@vinutnasai584]

Very good and easily understandable educational video! Thanks Nate, for making the tech stuff interesting as well !

[@bactran7799]

amazing tutorial. Thank you for your well organized video, your voice and everything you talk about are absolutely clear and help me easily to follow. Please keep your great works

[@wwmmff]

Huge thanks! I really like your video. Systematic and clear in your explanation. hope your channel grows! subscribed!

[@stratascratch]

Thanks for watching! And glad you found it educational!

[@mariumbegum7325]

Fantastic tutorial, easy to follow and presented in a way that is easy to follow

[@sitiismi7231]

this video-storyboard on point. very clear, explain the scope and summarize. thank you!

[@joelngige5776]

Super informative video on working with API's

[@rajgopalmanoharan]

This is an amazing session, thank you

[@sajjad5345]

I love this playlist and look forward to seeing more videos in it.👍

[@stratascratch]

Awesome! Thank you!

[@guoyitang4001]

love your video! Clear explanation and straight to the point

[@procorepierre3124]

Thanks a million Nate for such a great content. You are an inspiration

[@rolandr97]

I forgot to thank you in April: awesome, thanks!

[@Paper-Z]

You just gained a subscriber Bro 😃 Nice content

[@jackpeters1598]

Brilliant! Great explanation of the process.

[@69nukeee]

AMAZING tutorial, thanks for sharing!

[@Mars.2024]

This video : highly recommended 🍁

[@rj_nelson_97]

This is great, thanks for providing. Sometimes, the apikey is in the header, so it has to be passed that way. I used this to pull the data in from the API I'm presently working with (ATTOM): url = 'the get url' headers = { 'Accept':'application/json', 'apikey':'your api key', 'accept':'application/json' } response = requests.get(url, headers=headers) response.json()

[@stratascratch]

Very cool! Thanks for sharing.

[@jeh5842]

Thanks Nate for making this easy!!

[@elhipster1]

This is definitely the best youtube API video I've been able to find. Thank you so much. I actually went through two other videos before this and this one was the clearest that actually moved into a full pandas dataframe. Thanks!

[@stratascratch]

Hey thanks so much for the kind words! I tried my best to clearly explain all the steps and theory. I hope to create more in the future.

[@elhipster1]

I really appreciate how you created the 'for' loops, THEN went back and restructured the code to be more 'fundamentally' sound. bravo!

[@stratascratch]

@@elhipster1 Thanks for noticing how I do it. I always work in this way where I will write code to make sure the function is there. Once the entire script is finished and it's able to function, I'll go back and refactor everything and try to automate things to make my life easier the 2nd time around. It's a good practice for your self. And if you work on a team, you can even share the code with your team so they can use it for their purposes.

[@ellenycnyc]

Wow this helped so much! Super straightforward thanks!

[@supriya1316]

Thanks for your video . You make the things easy . Thanks a ton :)

[@victorloureiro_bodyboard]

6:11 that's basically why data engineers are needed

[@alirezakaramali1844]

It was very clear and useful. Thank you so much.

[@eulerthegreatestofall147]

Great video!!!! very easy to follow through!

[@poo872]

Perfect Explanations. Very useful. Thank you so much.

[@stratascratch]

Thank you for watching!

[@djjiang3718]

Good stuff and Thank you Nate for sharing!

[@stratascratch]

Thanks for watching!

[@chiuvinson5035]

great video! thank you so much! my results are showing only 50 rows of the df but the channel has over 200 videos. how can I get all of the others past the first page? i feel like it has to do with the pageToken maybe?

[@zahedinima732]

FANTASTIC tutorial!! What can possibly top this?

[@omginever]

this is pure gold!

[@mercantilism954]

Oh my god. Your video is a treasure

[@sidechain0]

Great tutorial, thanks!

[@samchimaobi3398]

Thanks for this video. was really very helpful.

[@RememberQuranAlways]

Just Awesome

[@stratascratch]

Wonderful. Hope you find the video helpful.

[@awesomeGuss]

THANK YOUUU man...woow, love this

[@soujanyamukkala1052]

Hello Nate, Nice Video, love it! I have a question here, instead of making a second API call for the stats, can we use fields parameter in the URL of the first API call to get the stats. Pardon me if I am not understanding the fundamentals properly. Thanks

[@stefano_er]

Thanks! Really appreciated it

[@senyotsedze3388]

Interesting video. I was able to create my own api key from KZbin. But since I have no posted videos, it is difficult to conceptualize your teaching. I wish we could get a generic api key to see data populated in the dataframe.

[@jcokonkwo]

Thanks again Nate, I appreciate the helpful content on understanding/navigating the API through Python. I know you didn't have time in the video to elaborate, but how can we continue looping through pages via 'pageToken' to extract all of data than just the first page?

[@stratascratch]

Great question. I actually have the code in the github repo. There's a link in the description. I use the 'pageToken' to do this as you mention. Check out the implementation!

[@jcokonkwo]

@@stratascratch Thanks Nate! Appreciate the help. Currently in interviews and I've gotten far with your content!

[@ЕленаЯкунова-б7с]

@@stratascratchWhen I add the "pageToken" some conflist with response['item'] arises. What could be the reason? ----> 4 df = get_videos(df) in get_videos(df) 6 response = requests.get(url).json() 7 time.sleep(1) ----> 8 for video in response['items']: 9 if video['id']['kind'] == "youtube#video": 10 video_id = video['id']['videoId']

[@michelchaghoury870]

i really like your content a lottt can you make more SQL and Databases vides (More Advanced) please and keep going

[@its_me7363]

Hi Nate, Thanks for making this video and I really enjoyed and learnt from this video. I have a question, do all big/medium companies/startup make their data public via API?

[@stratascratch]

Hey! Thanks for watching. The next video is coming out in 1-2 weeks to close out the series. Sometimes companies make their data public but they usually won't make internal data like user engagement or financial stats public. But they usually do make whatever data you own or whatever is deemed public, accessible to all. Like Twitter has an API you can use to collect tweets because all the tweets are publicly accessible. Hope that helps.

[@saaqibmhd]

Love this tutorial!!

[@ChrisMaringka]

Hello Nate, thank you for sharing this tutorial I appreciate it! I just had one quick question about the second API calling: when I tried to call the statistics, every now and then one of the stats (viewCount, likeCount, dislikeCount, or commentCount) might be missing and I could not assign them into a variable. Do you by any chance know the work around for it? Thanks!

[@sarwatzabeen8523]

This is very helpful! Just a quick question, since the second set of variables were defined for the first video only with index 0 and no for loop, I am getting a dataframe with only one row. Am I missing something?

[@chiomaeze8245]

cause you used 0 index so you will get only one row. Your loop should be able iterate through the each index of the Json["item"] object to grab the features you need.

[@myselfandpesit]

Thanks for the video. Nevertheless, some python libraries such as Tensorflow talk about APIs when they are actually saved already on our hard drive. I want to ask you how does, a web application such as tensorflow API function here ?

[@stratascratch]

Hi. If I'm understanding correctly, these APIs that are local are basically just ways of transmitting your data locally. You're just using an API locally to build your web app. You can move the API to another server or to the cloud and your web app would still work.

[@kashifrana6798]

Amazing content !

[@digigoliath]

Awesome!!! TQVM!!

[@pauljordan6346]

Hi Nate, I love the video, super clear and explains a lot. I am struggling with pagetoken, you said it would be explained later, is there another video I need to watch to get a handle on this? Thanks

[@konstantinpluzhnikov4862]

Good old days, you can see dislike_count for any video. Great tutorial, many thanks!

[@manarma7536]

what if i don' t have a videos in my channel, how can i work with channel ID ?

[@karanbhosale5390]

Can we do same for the Spotify ? Basically i wanted to see all songs heared in previous 24Hrs

[@stratascratch]

Yea most definitely. Just use the same principles as outlined in the video. Looks like Spotify has 2 python API wrappers (developer.spotify.com/documentation/web-api/libraries/)

[@jzh4950]

Very helpful! thank you

[@rishabkhuba2663]

How do I follow along this tutorial, if I don't have a youtube channel?

[@stratascratch]

You will have to sign up for YT to be able to follow us. You may also sign up for our newsletter to get updates on our blogs ,www.stratascratch.com/

[@rishabkhuba2663]

@@stratascratch I meant that, I don't Have KZbin videos to get video likes, comments etc to follow along with your tutorial. I think I will try to follow along by trying to access API's of other apps Thanks for the reply though ☺️

[@sprinter5901]

@@rishabkhuba2663 You can use others channel ID. KZbin how to get channel ID.

[@shuang7877]

That diploma looks very familiar - hello Big Red!

[@stratascratch]

Hello Big Red indeed!

[@chronomorrow1511]

Could you have made the dataframe appending more efficient with a loop and index [0]? Instead of writing out all 7 key value pairs.

[@kelechiasianya4496]

Your tutorials are cool

[@DylanOkyere]

You never linked to the api key document

[@alexwoodard2759]

good stuff, sir!

[@maurocruz1824]

What are the differences between API for collecting data and doing webscrapping?

[@stratascratch]

An API is specifically created to retrieve data. Companies create APIs to do this task. Webscrapping just pulls from elements on the webpage and often isn't legal to do. I've never really needed to webscrape anything working for a company. I think it's much more valuable to learn how to work with APIs.

[@faustopf-.]

Thank you so much for this amazing video! I got just one question: Could you list all the most used APIs by you in your daily basis routine as a DS/DA?

[@stratascratch]

I think most DSers use internal APIs if their companies build their pipelines as a microservice concept. Otherwise, you'll use DB queries to pull your data from dbs.

[@faustopf-.]

@@stratascratch Gotcha, thanks again! 💪🏻

[@wombatwombaffle]

Good work!!!

[@learnwithme3596]

Can you please upload an end to end data science project which includes ml model working on streaming real time data .... Thankyou

[@EzinneOgwo-Ude]

Please I am building a recommendation system for diet and exercise. Where can I get my data set or API. A link will be appreciated.