How to Scrape YouTube Comments with Python 2024

[@analyticswithadam]

Updated Video with Replies kzbin.info/www/bejne/d2KsenyVebaIaqssi=_8-d4OpoIHpr2jJm

[@Fallkhar]

Thank you, man, you really made it work for me. My initial approach was HTML parsing with beautiful soup which proved to be very difficult because not all comments are loaded and I think google limits accessing that way as well.

[@analyticswithadam]

Glad it helped!

[@Fallkhar]

@@analyticswithadam Yeah, it really helped me get into the KZbin API. I've now expanded the code to get all the comments from an entire KZbin channel by using the uploads playlist.

[@al-naqeep]

Can I use Python library youtube comment download

[@Fallkhar]

@@al-naqeep You mean like the code I wrote?

[@L_Only_Cans]

Great video! Do you know how much of your quota this used up or roughly how many comments you can extract per day with the current call limit quota? Thanks again!

[@analyticswithadam]

Each call is 1 . Quota is 10k per day. A million comments perhaps but haven’t tried to max it out.

[@L_Only_Cans]

@@analyticswithadam excellent thanks

[@JamesDeanDesigns]

This really helped me out. Thank you

[@analyticswithadam]

Awesome 🙌

[@oitik]

Great video. I have a question i am doing a research where i need to scrape comments from a live stream. How can i do that can you help? because your code cannot scrape the live comments.

[@yuxrazafar8532]

I want to extract the comments from a playlist. do i have to gather all the IDs manually or I can automate it through some API?

[@analyticswithadam]

I have a video on the way to do just that.. Here is the code in advance, pulls the replys as well. github.com/analyticswithadam/Python/blob/main/Pull_all_Comments_and_Replies_for_KZbin_Playlists.ipynb

[@yuxrazafar8532]

Oh Thank you so much!@@analyticswithadam

[@SabineWildemann-l9q]

Hi Adam, Thanks for this tutorial series, very helpful! If i don't have the IDs of the videos but a phrase I'd like to analyse (like "Die letzte Generation"), how can i proceed based on your code? Thanks for your time!

[@analyticswithadam]

Would this phrase be in the video title?

[@SabineWildemann-l9q]

@@analyticswithadamyes

[@mohamedmowena353]

Can u make a video in how to scrape the captions/subtitle. Thank u very much.

[@omerfaruksezeerr]

hello, I just want to pull the comments on a yotube channel, how can I do this, do I need to do a separate process for each video or is this possible only for a yotube account please help me thank you very much

[@analyticswithadam]

Hi Omer, What you need to do is collect the video ids and them put them in the for loop of the code discussed at the end of the video. Two ways to do this. 1. Manually open the video and pull the codes. For example the url of this video is kzbin.info/www/bejne/ZnfXlHugfpqfo9k&lc=UgwJNvziZkqzm3m__J94AaABAg the code is after the 'v='. 2. Use the youtube API to pull all the codes. . Instructions in the url below. developers.google.com/youtube/v3/docs/videos/list

[@sapphireluv2705]

thanks for the tutorial !!! btw is there any daily limit for using the API ? and also how much does the API cost ? I'm planning to build a model for sentiment analysis based from yt comments of a movie trailer. I want to use Oppenheimer trailer as the data source. The trailer has approximately 32 thousand comments.

[@analyticswithadam]

The limit is 10,000 a day per request, I think this is one request per 100 comments so you should be good but let me know if you get them all. The API is free, I show how to get setup in the comment in the description.

[@sapphireluv2705]

@@analyticswithadam thanks for the reply. Okay so if the limit is 10.000 per day, how can i scrapped the rest of it in the next day so that the script doesn't scrapped the same 10.000 comments again? How can i modify the script for that purpose?

[@analyticswithadam]

@@sapphireluv2705 1 credit should pull 100 comments so I think you will be fine, if it doesn't. The pageToken=nextPageToken should store the last page token used so you can start with that Token next time

[@sapphireluv2705]

@@analyticswithadam okay. I will try it. Thank you

[@sapphireluv2705]

@@analyticswithadam hello. I got some news. So i've tried to do some scraping but i only got 22k instead of 32k. When i looked at my api stat, i'm only using 225 request out of 10k. Why can't i get the rest of the comments ? many thanks.

[@emmanuel.alcantara]

how can I extract for example the 967 comments to a txt file?

[@analyticswithadam]

Pull the dataframe to a csv using python.. Are you using Google Colab?

[@emmanuel.alcantara]

@@analyticswithadam yes

[@analyticswithadam]

@@emmanuel.alcantara First answer here. stackoverflow.com/questions/53898836/export-dataframe-as-csv-file-from-google-colab-to-google-drive

[@christinapachaki3554]

great work, can someone please share with me the video that demonstrates the API KEy?

[@analyticswithadam]

Have it in the follow up video too How to Extract all KZbin Comments and Comment Replies from a Playlist kzbin.info/www/bejne/d2KsenyVebaIaqs

[@zeha997]

Is there any way to extract comment after translating in english? I want to extract only english comments.

[@analyticswithadam]

I don’t think anything detects the language in the api but I’m sure Python has a package to classify the languages then you can filter

[@analyticswithadam]

Try this medium.com/swlh/language-classification-using-machine-learning-in-python-fa0768daea67

[@HauntedRabbit-x7j]

Very useful video. Thank you for sharing sharing knowledge.

[@BECEOOYaashuDave]

What if i want to extract the replies as well?What should we do?

[@analyticswithadam]

Have a video coming on Tuesday that includes the replies, here is the script …github.com/analyticswithadam/Python/blob/main/Pull_all_Comments_and_Replies_for_KZbin_Playlists.ipynb

[@julianebrito5788]

Hi from Brazil, I loved the video. I would really like to know how to extract it from the KZbin chat

[@robosergTV]

Why would you do while 1==1 and not simply while True? Just for jokes?

[@analyticswithadam]

Its a weird one. Its an infinite loop until the Try Except block breaks you out of it. So the while loop isn't evaluating anything. Nothing is ever going to be False.

[@KHAIRUNNISAKAMALARIFFIN]

how to get the data in excel/csv ?

[@analyticswithadam]

Add three lines of code at the end. Im assuming you are using Google Colab from google.colab import files df.to_csv('my_data.csv', index=False) files.download('my_data.csv')

[@joysarkar-hb4tj]

how i get or save file of these extract comments???

[@analyticswithadam]

You can pull the dataframe out of a notebook as a CSV

[@gsonbiswas9765]

Does it extract the nested comments?

[@analyticswithadam]

It does not

[@sarvarbekfozilov9829]

#question hi adam thank for tutorial. currently i need scrap comments from instagram and facebook how can i do that?

[@analyticswithadam]

No idea at the moment but I’ll have a look..

[@04mdsimps]

Really nice work. Good stuff

[@fuadanjasmara]

Good video but I have a problem in dev number. How you get that?

[@analyticswithadam]

Dev number?

[@fuadanjasmara]

@@analyticswithadam yes

[@analyticswithadam]

@@fuadanjasmara Sorry just looked at the code, Its a API Key you need for KZbin, its free but you need to go to Google Cloud to generate.Go to 1.50 of this video for a walkthrough on how to get it. kzbin.info/www/bejne/Z3zad3mYlNminLs

[@analyticswithadam]

@@fuadanjasmara Ive made a few videos on the API the first one had the instructions and I skipped on this one

[@fuadanjasmara]

@@analyticswithadam I see. Thanks, Adam.

[@eig-datos]

You are the best, thanks soo muuchhhh

[@analyticswithadam]

Glad it helped

[@zeha997]

how to extract these comments in excel?

[@analyticswithadam]

If you are using Google colab If the dataframe is named something other than df change all the 'df' below to whatever its called from google.colab import files df.to_csv('df.csv', index=False) files.download('df.csv')

[@zeha997]

@@analyticswithadam thanks

[@ResearchonSihr]

@@analyticswithadam Hi, thank you for your tutorial. I'm not familiar with Google colab so how to extract the comments in a txt or csv file ? I don't really understand what you said in reply to zeha997. Can you elaborate please? Thank you in advance

[@emaa786]

I want to ask something anyone here?

[@damochi100]

does exactly what it says

[@analyticswithadam]

Sure does

[@mrpepeluis950]

Chido one, great video!