Python Tutorial: Web Scraping with BeautifulSoup and Requests

[@sng8240]

*Great video! Even the IBM staff suggested watching this video to complete a part of a course offered by them. Did you know? Congratulations!*

[@coreyms]

I didn't know that. Very cool! Thanks for letting me know!

[@joaquinrequena3494]

Damn course, I'm getting whipped there

[@JoaoVitor-dq9sf]

Yes, it was Alex Aklson from IBM

[@vaibhavkrkm]

He deserves that 😀

[@amazing-graceolutomilayo5041]

I don't know why I'm really happy about this 😂. Thank you Mr Corey

[@Masonx42]

Hello,I'm a viewer from China, comparing with other youtube python tutors, your English speaking is very clear for me to understand. I just registered as a one dollar patron for a little support.If I can find a job as programer later, I will contribute more to help you continue making these high quality videos. Thank you Mr Corey!

[@coreyms]

Thanks!

[@abby4922]

@@coreyms hey you still using this acc for youtube?

[@abby4922]

hey Einsteiincy , are you still using this youtube acc, im also from China, would like some help with python

[@kyefondeme136]

@@abby4922 i think we in China knowledge seeker lol

[@abby4922]

@@kyefondeme136 hahaha broah lets link up,

[@samueltukua3061]

I started watching your videos years ago and I was afraid you wouldn't be around when I returned from my programming hiatus. Thank you for still being here!

[@imtiazshahed944]

Corey Schafer Thanks a million. best tutorial I have ever seen on Web Scrapping in Python, No one explain like you in a very single point of code. That's how we learn. great job and keep it up.

[@failuregreat9424]

you never fail to impress me with your teachings! I don't watch videos at all once, because I feel it's easier to learn watching by 'parts' - but again and again, I'm watching the videos and I feel 'wow.. this is really well done, I have to like this, it's the least I can do' and I realized I already liked before lol

[@cosmicdarkmatter1128]

@Failer Great...Agreed. Corey Schafer is very good at explaining things. A lot of professors can't do that....

[@proxyscrape]

5 years ago, this video was already a great tutorial. Now, it's even more valuable. Thanks for the great content!

[@f.2funky]

Hi Corey I'm a new python learner from China. Listen a class in English is still a little hard for me, not to say with the things I am completely unfamiliar with, like Python. But your English speaking and tutorial are very clear for me to follow! I can't believe that I have just finished this 45-minute-long video with Python in English. I have never thought the I could did this before. You videos helped me a lot in my Python learning. You are amazing. Your tutorials are amazing. Without them I probably have already gave up. You are the best Python teacher. Thank you so much! ^^

[@SableWinters]

The best and quickest tutorial for small to medium sized website data scraping I’ve encountered. You are clear, concise, and you voice is easy to listen to and follow void of irritation. I actually have confidence now to achieve my tasks after years of placing them on the back burner.👍🏼🤗

[@sairamsubramaniam8316]

You are the best programming teacher I ever had.

[@leslievanelsie7545]

I'm not really a video type of person, I learn faster with books but I always find myself coming to your channel to clarify things. You are the best Corey

[@ashishm8413]

It's nice how you break down the subject into small increments that others can understand and build upon easily. True quality of a great teacher. Thank you!

[@user-lh4hv3tx8b]

Every time I find myself struggling with python as a beginner and going through so many random sources, you always seem to be the clearest.

[@varunabhi]

No where I have seen advanced topics explained in an easy way like this. Even in paid courses THANKS Mr.Corey. THANKS A LOT! I'd learnt much from your videos in a easy way.

[@mohanlpuri1]

I have watched couple of videos on web scrapping with Python. This video is very clear, and pace of delivery and the example teaching of the author is excellent.

[@mankaransingh2599]

Most professional guy on youtube, who dosent boasts to be know stuff.

[@mohamedabdo-xg6tw]

I don't know why some people disliked this video? I paid a money to learn this and the content of the class was way bellow this level, thank you for the great Tutorial!

[@tntcaptain9]

This is how to teach in a proper way, in some videos a code is already given and they just read it out which makes it difficult to understand. Thanks Corey, keep up the great work.

[@Reese_414]

Tech with Tim is horrible at that... I'll click to watch a tutorial on how to do something and he pulls up a game that's already half way complete... And on top of using pre-written code, he also talks very fast... Salute to Corey for live coding while speaking clearly with good explanations!.!.!

[@tonyt1343]

True story: I haven't had to scrape anything for 5 years and have forgotten how to use most of the tools. I browsed youtube for a minute and started to get discouraged because most of the videos were bad. Then I thought, "I know my man Corey probably has a video. Let me go check out his site." Sure enough, high quality video. Thanks Corey!

[@samdavepollard]

45 minutes of pure, unadulterated, hardcore geek-porn. Absolutely loved it. Thanks, Corey. Even by your standards this one was pretty special.

[@coreyms]

Thanks!

[@pauloesperon7697]

I felt the same.

[@mateusribeiro9384]

I actually enjoy much more watching his videos than watching normal movies, series...

[@dope8452]

@@mateusribeiro9384 Nerrrrrd

[@shivanishimpi7018]

@@coreyms at the time 26.50, how do I scrape out both tags instead of just one?

[@ТимуркаДолгорукий]

I'm viewer from Russia, your English speaking is very clear. Thx.

[@eyoba6027]

My first video from your channel and it didn't took me long to press the subscribe button. You are just perfect, no second wasted. Every word is important and worth mentioning. Cheers!

[@upasanasingh7567]

Agree!

[@balasubramanian4314]

ONE OF THE BEST DEVELOPER I HAVE EVER SEEN in this planet! Hats Off to your skills!

[@gwanghyeongim768]

Thank you Corey. I can personally assure that your work doesn't go for nothing. I hope I can donate more to your patreon in the future.

[@amitanand9125]

Corey Schafer : I don't know if Python is easy or you made the Python Learning very easy.I did watch almost all your videos till date and refer my friend to do so. Thank you for making our live easier with such a ease of learning

[@alexpaes9161]

It could be possible to hit "like" twice for ur videos! Mate, ur a legend! Congrats!

[@prabav86]

Hello from Canada! I've been impressed with the way you teach Python and related subjects. So I was referring/recommending your youtube videos to my friends. Recently, I'm finishing up online course on Data Science from IBM, I was pleasantly surprised that course recommended your video on webscrapping! All your hard works helping a lot of folks out there. Simply outstanding! take care

[@hasanfares5774]

thanks was just taking an online class about it and couldn't understand until I got the notification you just uploaded wow you really are in sync with your followers

[@paulohsgoes1959]

I was quite sure you wouldn't let me down. You're the best, man, because you go straight to the point. Thanks a lot for sharing your knowledge.

[@baconsledge]

As always, Corey’s videos are top-notch. Thank you, Corey!

[@BoxingPills]

Im doing IBM data science certification. Understood nothing. After 2 minutes of this video I m already hooked! you re phenomenal!

[@manojuppala3941]

Corey is the world's best python teacher.

[@daringetae5136]

Best explanation I've found so far on how to scrape websites using soup. Worth watching yours.

[@allahakbar9580]

Such a beautiful and comprehensive explanation! I liked the idea that you started from the simplest things, like scraping a small HTML file, not an entire website! Because that what I could not find in the many videos I watched about BeautifulSoup, they jump into the deep straight ahead!

[@painpills66]

Dude, i'm from colombia and i'm tryng to lear how to do web scraping. The point is that I don't know much English but it seemed to me just by listening to you a little that you were giving important information, so I saw the entire video translating every word that I didn't know, and I learned much more than with the information I find in Spanish, huge thanks and new sub.

[@RadioactiveNinja]

One of the best tutorials Ive ever followed. Hopefully you help kickstart my career in data science!

[@oluwadamilaretijani1777]

Mr Corey, please wherever you are, keep safe. I really miss your content on KZbin. You're a legend when it comes to teaching.

[@OrientalMelodies2010]

An awesome video. Your way of moving from one idea to another is amazing. I have been searching KZbin for years for a good tutorial video explaining web scrapping with BeautifulSoup and finally found it. Thanks a million!

[@shariqahmed7319]

One of the finest web scraping with BS4 tutorials on KZbin! Thank you so much sir!

[@adityatripathi9865]

your understanding of python is very deep and your teaching method is excellent

[@iliveforthevibez6216]

I try to get into Data Science and I did a coursera online course to learn python which also contained lessons about html and web scraping with BeatifulSoup4. I did not understand everything that was explained in coursera but your tutorial right here summed everything up in 40 minutes and could have saved me some hours if I had watched it earlier. Instant subscription man, the way you teach (and also speak, I am from Germany so it is easy to understand) is amazing , keep blessing us with your videos !

[@KenPryor]

This was the first of your tutorials I've seen. I definitely subscribed after this great video. I've struggled with learning Python and this was very helpful.

[@FredMny]

Thank you!!! I love it how you are able to not over explain, but also without leaving out anything important.

[@geekyprogrammer4831]

Best Python WebScraping video on KZbin! Seriously brother you deserve money for this masterpiece! Keep it up :')

[@jw2965]

pls no charge. me no have money

[@rift8997]

@@jw2965 what the heck

[@jw2965]

@@rift8997 was trying to be funny, but jesss i failed hard.

[@rift8997]

@@jw2965 ye ok i was just confused. lmao

[@allistergraham9448]

Very informative video, you're an excellent teacher! While other KZbinrs ramble off topic, you get straight to the point, and I really appreciate that.

[@MrKashyani]

Man..The way you explains !!! It works like a charm. Be it Regex or web scarping, it's just awesome ! Love your posts.

[@leosimon1746]

I have to say you did a great job! You covered pretty much everything that a novice programmer needs to know about scraping. I immediately developed my own solution after watching the video. Thank you so much!!!

[@tarunsukhpalani9514]

Master Corey, you are a magician. I just mesmerized.

[@Reese_414]

Just wanted to share something I do to practice web scraping that others may find useful... Instead of running your request to a live url while working out your code and possibly overwhelming a website, I right click on the browser and select the "view source" option. Then select all and copy the source code and paste as an HTML file in the same local folder then scrape through it until I get it working the way I want... May not work with all sites, but good for simple sites that aren't JavaScript heavy... P.S. Excellent tutorial, thank you for being so clear and so thorough!.!.!

[@Nearco1000]

Just curious about those 34 slackers that did Thumbs down on this vid, seriously?. It's very clear and straight forward, plain English, highly understandable .......................Thumb up.... Very appreciated #Corey, I was looking for this, great vid, again and again tku for your contribution.

[@coreyms]

Thanks!

[@abdallahalaa492]

What a great teacher, I never wanted to watch any tutorials about scraping as it was explained in then most boring way by others, but when it comes to you , you did a great job (y)

[@qianlima2196]

I spent 1 hour to figure out why my outcome is different from yours, then I finally realized it's because your website content has changed!

[@iumair_mehmood]

I am glad you figured it out 😂lolx

[@i8nff225]

@@iumair_mehmood 😂😂

[@DJ-ct6so]

Excellent Corey. Very comprehensive, well explained, step by step, easy to follow. You're a very talented teacher imho.

[@MarkJay]

Another great video Corey! Keep them coming.

[@elliottcarter7700]

I can always count on clear and concise explanations from Corey Schafer! Thanks!

[@dpoznyak]

Corey your Python tutorials are great! You explain everything very well. It would be great if you could make a series of Django tutorials for beginners! Keep up the good work!

[@tobaobiwale735]

I bought this course on Udemy, and I decided to check youtube for a web-scraping when I was confused. Men, I fell in love with your teaching skills, so understandable in every step, I bet that the Udemy tutor had watched your video to create his own. Ain't rich for a subscription but I bet I will follow you and learn from you at all time, cheers mate!

[@18nikhilchauhan80]

thank you so much. i got my first client just because of you.

[@branderbuilt8908]

I was watching another video on web scraping, and then I read the comments, and someone told me to come here and watch your video first because you explained things more coherently. Keep producing great content. You've got a lot of KZbin guardian angels redirecting traffic to you, Corey.

[@kanika724]

Corey a Big Bow from India..... really you are too amazing

[@pkavenger9990]

Seriously your teaching style and explaining is amazing, you even explained what split does, most people just assume that you already know about it but even then you explained it and it refreshes everyone memories and helped those who forgot due to some reason.

[@saritaprabhu]

You're a legend! There's a lot to learn from you. 🙇

[@ikki411]

This tutorial was incredible. I've done 2 Python courses that touched the 'Web Scraping' subject, but I wasn't able to fully understand it. This video was one of the two videos that made me fully understand it, and I couldn't be more happy about it.

[@lawrencekrukrubo2640]

This is just amazing, your knowledge, no your practical knowledge is amazing

[@X19-x5f]

Excellent video Corey, thanks. I finally got to see it because we're all quarantined to our houses due to the virus. Videos like this make it bearable.

[@brucemurdock5358]

00:00 What is webscraping 00:30 Our end-goal 02:23 Installation of modules/libraries/packages 05:05 Primer on HTML and CSS 08:32 Example 19:27 Scraping the webpage

[@sudip7949]

Love you bruh😚

[@emileberhard8116]

Ty!

[@rajnigoyal6653]

Thank you Corey Schafer.. I really loved your teaching style and got every part of this. The best part is pointing out about the problem and give the solution.

[@rb8058]

Great introductory information! Would love a similar one for how to scrape javascript rendered websites!

[@clairein1433]

Thank you so much for making this video! I'm from China and I've searched many courses to learn Web Scraping. But it is you who make me successfully run my code.

[@hashcoder]

I've tried today! Successfully. Thank you, Corey!

[@adarshtiwari7395]

I truly appreciate your compassionate teaching and dedication to the tutorials. Thank you for all the teaching!!

[@khrenovina]

Thanks, man. You're great! Everything is clear for the beginner like me.

[@pydocs-pro]

Corey, this is just THE BEST explanation about web scrapping with python. Just notice that you could explore a little bit more about the page response with the get method. Sometimes I have trouble trying to access one specific site, and I have to manage it in different forms. And some more detailed explanations about when it's possible and not possible to scrape data from websites. Only this detail that I've missed. All the rest it's just perfect.

[@silverzero9524]

awesome was just surfing the web about bs4 and u uploaded this awesome awesome awesome thx

[@_boris]

Whoever gets weird symbols when opening the .csv in *Excel*, change this line: csv_file = open('cms_scrape.csv', 'w', *encoding='utf-8-sig'*) Thanks once again Corey for brilliant content 💎

[@wolfisraging]

Holy maaaaan, you are truly most amazing.... Big fan😊

[@MdBodruzzamanSIfat24]

Brother, your videos are always great. I follow them and learnt python very easily. This course worth money but you are serving free. Thanks a lot.

[@amysandstrum531]

Thanks! I have very limited knowledge of html but was able to scrape price information off of websites. Just for funsies :D Really, all i did was tweak it and instead of div they use span class, so i switched 'div' with 'span' and got the same result.

[@anurodhchoudhary1689]

This man is a legend, all your explanation is easy to understand

[@jakesnell7707]

Corey you have the coolest videos, you've taught me much my friend.

[@mangaart3366]

Love how you explain things so clearly. Corey you are the best!

[@maicomcoelholopes9032]

Corey muito obrigado eu sou brasileiro, seus tutoriais são excelentes e me ajudam muito, principalmente porque não tenho dinheiro para comprar cursos, muito obrigado

[@dhssb999]

This is definitely the best web scraping tutorial I've ever seen!

[@jackbird5839]

great solution and tips bro!! thank you for your video., I will definitely try to improve my scraping skills with your tutor. but as non-tech user Also I have found good alternative to scrape amazon reviews e-scraper maybe it helps to somebody too.

[@willjohn6807]

Thank you Jack, e-Scraper really help me with my web scraping needs for my store.

[@vskiy26]

Awesome tip Jack

[@jonatanlast3849]

@@willjohn6807 En este tutorial de programación de Python

[@jonatanlast3849]

@@willjohn6807 nasıl yapılacağını öğrenmek

[@jonatanlast3849]

@@vskiy26 BeautifulSoup o se meafaigaluega sili ona lelei mo le faʻavasegaina

[@arnelseco6429]

I watch the video till the end, the explanation is very clear and easy to follow. Thanks Sir for this educational video. GodBless

[@johnjacobs6234]

Just a note, BeautifulSoup doesn't work with dynamic content. If you see something like: ::before (html content) ::after That content is dynamically loaded by your browser from some sort of redirect or Javascript. The html inside it doesn't exist when BeautifulSoup parses everything.

[@pravinbagul5305]

Thanks Corey for making my python journey very simpler with your tutorials!!!

[@salah2fois]

Using Pyton 3, I added this part: csv_file = open('web_scraped.csv','w', newline = '') in order to avoid empty lines in the csv file. stackoverflow.com/questions/3348460/csv-file-written-with-python-has-blank-lines-between-each-row

[@hamidrezaghafori4200]

thanks bro i had the same problem

[@TimFromPerth]

Outstanding. Clear and simple. Thank you.

[@venkysmoments9155]

Corey please add tutorials related with multithreading and multiprocessing in python.

[@ApplySkills]

Venkatesh K this gets my nipples hard^

[@warofwars7608]

did u have nipple alert app installed

[@MidjourneyandB]

I just found your videos. I'm 5 months in to learning coding (toward data science). Your examples and explanations are extremely clear and helpful. I love how large the text is and how clear the narration is as well. Thanks. I will be watching tons more.

[@efqanmustafayev5189]

Thanks for video, sir. How we can scrape login required websites?

[@lucasmiguel2633]

Have you found out yet? I'm having a rough time trying to do this

[@RahmanVsa]

Thank you. Can't believe I have been building scrapers for months still got so much deep understanding from a single video.

[@mohitupadhayay1439]

This guy is a pure Genius!

[@robertlee9761]

Superb tutorial, by far one of the best explained BeautifulSoup tutorials to date. Thank you Corey.

[@centralscrutinizer5116]

You sir, by far, have the best structured videos for learning from. Great job there young feller!

[@yaboi2043]

This is the only video I could find which explains this topic in a simple, understandable way. Thank you!

[@albertwalker9396]

Excellent, I found this video after trying to follow some other vids with continual syntax error. I haven't touched Python for several years (retired) and found this video an excellent guide to getting reacquainted Thanks for your efforts. I have subscribed to your channel.

[@paulseldn]

The clearest explanation I have seen of webscraping, and at a pace we can actually follow. Many thanks Corey!

[@JenPurple2022]

Clear and easy to follow, best tutorials! If I learned from you at first, I would not quit due to mounted frustration and limited understanding of available tools.

[@oracle11iappsdba]

I watched this video just once but moved back the time line several times. Result I learned more than what I got in a stupid 1 day paid workshop. Thanks man.