Python Tutorial: Web Scraping with BeautifulSoup and Requests

In this Python Programming Tutorial, we will be learning how to scrape websites using the BeautifulSoup library. BeautifulSoup is an excellent tool for parsing HTML code and grabbing exactly the information you need. So whether you're pulling down headlines from news sites, scores from sports websites, or prices from an online store... BeautifulSoup and Python will help you get this done quickly and easily. Let's get started...
The code from this video can be found at:
github.com/CoreyMSchafer/code...
Difference Between Parsers: goo.gl/zdy9br
Python File Objects: • Python Tutorial: File ...
Python Strings: • Python Tutorial for Be...
Python Try/Except: • Python Tutorial: Using...
Python CSV Files: • Python Tutorial: CSV M...
✅ Support My Channel Through Patreon:
/ coreyms
✅ Become a Channel Member:
/ @coreyms
✅ One-Time Contribution Through PayPal:
goo.gl/649HFY
✅ Cryptocurrency Donations:
Bitcoin Wallet - 3MPH8oY2EAgbLVy7RBMinwcBntggi7qeG3
Ethereum Wallet - 0x151649418616068fB46C3598083817101d3bCD33
Litecoin Wallet - MPvEBY5fxGkmPQgocfJbxP6EmTo5UUXMot
✅ Corey's Public Amazon Wishlist
a.co/inIyro1
✅ Equipment I Use and Books I Recommend:
www.amazon.com/shop/coreyschafer
▶️ You Can Find Me On:
My Website - coreyms.com/
My Second Channel - / coreymschafer
Facebook - / coreymschafer
Twitter - / coreymschafer
Instagram - / coreymschafer
#Python

[@sng8240]

*Great video! Even the IBM staff suggested watching this video to complete a part of a course offered by them. Did you know? Congratulations!*

[@coreyms]

I didn't know that. Very cool! Thanks for letting me know!

[@joaquinrequena3494]

Damn course, I'm getting whipped there

[@JoaoVitor-dq9sf]

Yes, it was Alex Aklson from IBM

[@vaibhavkrkm]

He deserves that 😀

[@amazing-graceolutomilayo5041]

I don't know why I'm really happy about this 😂. Thank you Mr Corey

[@Masonx42]

Hello,I'm a viewer from China, comparing with other youtube python tutors, your English speaking is very clear for me to understand. I just registered as a one dollar patron for a little support.If I can find a job as programer later, I will contribute more to help you continue making these high quality videos. Thank you Mr Corey!

[@coreyms]

Thanks!

[@abby4922]

@@coreyms hey you still using this acc for youtube?

[@abby4922]

hey Einsteiincy , are you still using this youtube acc, im also from China, would like some help with python

[@kyefondeme136]

@@abby4922 i think we in China knowledge seeker lol

[@abby4922]

@@kyefondeme136 hahaha broah lets link up,

[@hasanfares5774]

thanks was just taking an online class about it and couldn't understand until I got the notification you just uploaded wow you really are in sync with your followers

[@baconsledge]

As always, Corey’s videos are top-notch. Thank you, Corey!

[@KenPryor]

This was the first of your tutorials I've seen. I definitely subscribed after this great video. I've struggled with learning Python and this was very helpful.

[@RadioactiveNinja]

One of the best tutorials Ive ever followed. Hopefully you help kickstart my career in data science!

[@OrientalMelodies2010]

An awesome video. Your way of moving from one idea to another is amazing. I have been searching KZbin for years for a good tutorial video explaining web scrapping with BeautifulSoup and finally found it. Thanks a million!

[@adarshtiwari7395]

I truly appreciate your compassionate teaching and dedication to the tutorials. Thank you for all the teaching!!

[@imtiazshahed944]

Corey Schafer Thanks a million. best tutorial I have ever seen on Web Scrapping in Python, No one explain like you in a very single point of code. That's how we learn. great job and keep it up.

[@alexpaes9161]

It could be possible to hit "like" twice for ur videos! Mate, ur a legend! Congrats!

[@ashishm8413]

It's nice how you break down the subject into small increments that others can understand and build upon easily. True quality of a great teacher. Thank you!

[@MarkJay]

Another great video Corey! Keep them coming.

[@allahakbar9580]

Such a beautiful and comprehensive explanation! I liked the idea that you started from the simplest things, like scraping a small HTML file, not an entire website! Because that what I could not find in the many videos I watched about BeautifulSoup, they jump into the deep straight ahead!

[@proxyscrape]

5 years ago, this video was already a great tutorial. Now, it's even more valuable. Thanks for the great content!

[@garydunken7934]

Thanks a lot Corey. It tremendously helped me to understand BeautifulSoup and quickly use it for a small web scrapper script that I needed. This 45min learning was so value adding for me.

[@sairamsubramaniam8316]

You are the best programming teacher I ever had.

[@user-lh4hv3tx8b]

Every time I find myself struggling with python as a beginner and going through so many random sources, you always seem to be the clearest.

[@hashcoder]

I've tried today! Successfully. Thank you, Corey!

[@JenPurple2022]

Clear and easy to follow, best tutorials! If I learned from you at first, I would not quit due to mounted frustration and limited understanding of available tools.

[@pkavenger9990]

Seriously your teaching style and explaining is amazing, you even explained what split does, most people just assume that you already know about it but even then you explained it and it refreshes everyone memories and helped those who forgot due to some reason.

[@samueltukua3061]

I started watching your videos years ago and I was afraid you wouldn't be around when I returned from my programming hiatus. Thank you for still being here!

[@failuregreat9424]

you never fail to impress me with your teachings! I don't watch videos at all once, because I feel it's easier to learn watching by 'parts' - but again and again, I'm watching the videos and I feel 'wow.. this is really well done, I have to like this, it's the least I can do' and I realized I already liked before lol

[@cosmicdarkmatter1128]

@Failer Great...Agreed. Corey Schafer is very good at explaining things. A lot of professors can't do that....

[@MidjourneyandB]

I just found your videos. I'm 5 months in to learning coding (toward data science). Your examples and explanations are extremely clear and helpful. I love how large the text is and how clear the narration is as well. Thanks. I will be watching tons more.

[@paulseldn]

The clearest explanation I have seen of webscraping, and at a pace we can actually follow. Many thanks Corey!

[@varunabhi]

No where I have seen advanced topics explained in an easy way like this. Even in paid courses THANKS Mr.Corey. THANKS A LOT! I'd learnt much from your videos in a easy way.

[@MrKashyani]

Man..The way you explains !!! It works like a charm. Be it Regex or web scarping, it's just awesome ! Love your posts.

[@leosimon1746]

I have to say you did a great job! You covered pretty much everything that a novice programmer needs to know about scraping. I immediately developed my own solution after watching the video. Thank you so much!!!

[@ikki411]

This tutorial was incredible. I've done 2 Python courses that touched the 'Web Scraping' subject, but I wasn't able to fully understand it. This video was one of the two videos that made me fully understand it, and I couldn't be more happy about it.

[@mohanlpuri1]

I have watched couple of videos on web scrapping with Python. This video is very clear, and pace of delivery and the example teaching of the author is excellent.

[@dpoznyak]

Corey your Python tutorials are great! You explain everything very well. It would be great if you could make a series of Django tutorials for beginners! Keep up the good work!

[@FredMny]

Thank you!!! I love it how you are able to not over explain, but also without leaving out anything important.

[@Muzahid19dec]

Thank you. Can't believe I have been building scrapers for months still got so much deep understanding from a single video.

[@gwanghyeongim768]

Thank you Corey. I can personally assure that your work doesn't go for nothing. I hope I can donate more to your patreon in the future.

[@eyoba6027]

My first video from your channel and it didn't took me long to press the subscribe button. You are just perfect, no second wasted. Every word is important and worth mentioning. Cheers!

[@upasanasingh7567]

Agree!

[@paulohsgoes1959]

I was quite sure you wouldn't let me down. You're the best, man, because you go straight to the point. Thanks a lot for sharing your knowledge.

[@SableWinters]

The best and quickest tutorial for small to medium sized website data scraping I’ve encountered. You are clear, concise, and you voice is easy to listen to and follow void of irritation. I actually have confidence now to achieve my tasks after years of placing them on the back burner.👍🏼🤗

[@alextakele9652]

Thank you, I am data science student from university of gondar,Ethiopia,I seriously admire you way speech,thank you for all you did!

[@adityatripathi9865]

your understanding of python is very deep and your teaching method is excellent

[@shariqahmed7319]

One of the finest web scraping with BS4 tutorials on KZbin! Thank you so much sir!

[@prabav86]

Hello from Canada! I've been impressed with the way you teach Python and related subjects. So I was referring/recommending your youtube videos to my friends. Recently, I'm finishing up online course on Data Science from IBM, I was pleasantly surprised that course recommended your video on webscrapping! All your hard works helping a lot of folks out there. Simply outstanding! take care

[@lawrencekrukrubo2640]

This is just amazing, your knowledge, no your practical knowledge is amazing

[@tntcaptain9]

This is how to teach in a proper way, in some videos a code is already given and they just read it out which makes it difficult to understand. Thanks Corey, keep up the great work.

[@Reese_414]

Tech with Tim is horrible at that... I'll click to watch a tutorial on how to do something and he pulls up a game that's already half way complete... And on top of using pre-written code, he also talks very fast... Salute to Corey for live coding while speaking clearly with good explanations!.!.!

[@elliottcarter7700]

I can always count on clear and concise explanations from Corey Schafer! Thanks!

[@yaboi2043]

This is the only video I could find which explains this topic in a simple, understandable way. Thank you!

[@f.2funky]

Hi Corey I'm a new python learner from China. Listen a class in English is still a little hard for me, not to say with the things I am completely unfamiliar with, like Python. But your English speaking and tutorial are very clear for me to follow! I can't believe that I have just finished this 45-minute-long video with Python in English. I have never thought the I could did this before. You videos helped me a lot in my Python learning. You are amazing. Your tutorials are amazing. Without them I probably have already gave up. You are the best Python teacher. Thank you so much! ^^

[@mankaransingh2599]

Most professional guy on youtube, who dosent boasts to be know stuff.

[@allistergraham9448]

Very informative video, you're an excellent teacher! While other KZbinrs ramble off topic, you get straight to the point, and I really appreciate that.

[@abdallahalaa492]

What a great teacher, I never wanted to watch any tutorials about scraping as it was explained in then most boring way by others, but when it comes to you , you did a great job (y)

[@tarunsukhpalani9514]

Master Corey, you are a magician. I just mesmerized.

[@samdavepollard]

45 minutes of pure, unadulterated, hardcore geek-porn. Absolutely loved it. Thanks, Corey. Even by your standards this one was pretty special.

[@coreyms]

Thanks!

[@pauloesperon7697]

I felt the same.

[@mateusribeiro9384]

I actually enjoy much more watching his videos than watching normal movies, series...

[@dope8452]

@@mateusribeiro9384 Nerrrrrd

[@shivanishimpi7018]

@@coreyms at the time 26.50, how do I scrape out both tags instead of just one?

[@leslievanelsie7545]

I'm not really a video type of person, I learn faster with books but I always find myself coming to your channel to clarify things. You are the best Corey

[@balasubramanian4314]

ONE OF THE BEST DEVELOPER I HAVE EVER SEEN in this planet! Hats Off to your skills!

[@silverzero9524]

awesome was just surfing the web about bs4 and u uploaded this awesome awesome awesome thx

[@user-ex6pk6fh4p]

Thanks, man. You're great! Everything is clear for the beginner like me.

[@BoxingPills]

Im doing IBM data science certification. Understood nothing. After 2 minutes of this video I m already hooked! you re phenomenal!

[@TheDevilOfJesters]

Thank you so much. You answered several of my questions and literally saved my ‘avoiding boredom’ project

[@geekyprogrammer4831]

Best Python WebScraping video on KZbin! Seriously brother you deserve money for this masterpiece! Keep it up :')

[@jw2965]

pls no charge. me no have money

[@rift8997]

@@jw2965 what the heck

[@jw2965]

@@rift8997 was trying to be funny, but jesss i failed hard.

[@rift8997]

@@jw2965 ye ok i was just confused. lmao

[@saritaprabhu]

You're a legend! There's a lot to learn from you. 🙇

[@amitanand9125]

Corey Schafer : I don't know if Python is easy or you made the Python Learning very easy.I did watch almost all your videos till date and refer my friend to do so. Thank you for making our live easier with such a ease of learning

[@rajnigoyal6653]

Thank you Corey Schafer.. I really loved your teaching style and got every part of this. The best part is pointing out about the problem and give the solution.

[@manojuppala3941]

Corey is the world's best python teacher.

[@18nikhilchauhan80]

thank you so much. i got my first client just because of you.

[@Josiasnyc]

All of your videos are just phenomenal. Well done for the amazing work you do! I'm glad we have people like you, very competent, professional, knowledgeable and straight to the point. 10/10. Have you considering writing? You should!

[@ktay895]

Made a convoluted topic crystal clear. Such a master teacher -- you're my go-to resource for coding!

[@wolfisraging]

Holy maaaaan, you are truly most amazing.... Big fan😊

[@rb8058]

Great introductory information! Would love a similar one for how to scrape javascript rendered websites!

[@josephsteen230]

Outstanding, Corey. I really enjoy and appreciate the clear information in your videos. Keep it up, and I am joining your patreon.

[@robertlee9761]

Superb tutorial, by far one of the best explained BeautifulSoup tutorials to date. Thank you Corey.

[@kanika724]

Corey a Big Bow from India..... really you are too amazing

[@qianlima2196]

I spent 1 hour to figure out why my outcome is different from yours, then I finally realized it's because your website content has changed!

[@iumair_mehmood]

I am glad you figured it out 😂lolx

[@i8nff225]

@@iumair_mehmood 😂😂

[@albertwalker9396]

Excellent, I found this video after trying to follow some other vids with continual syntax error. I haven't touched Python for several years (retired) and found this video an excellent guide to getting reacquainted Thanks for your efforts. I have subscribed to your channel.

[@clairein1433]

Thank you so much for making this video! I'm from China and I've searched many courses to learn Web Scraping. But it is you who make me successfully run my code.

[@Nearco1000]

Just curious about those 34 slackers that did Thumbs down on this vid, seriously?. It's very clear and straight forward, plain English, highly understandable .......................Thumb up.... Very appreciated #Corey, I was looking for this, great vid, again and again tku for your contribution.

[@coreyms]

Thanks!

[@maicomcoelholopes9032]

Corey muito obrigado eu sou brasileiro, seus tutoriais são excelentes e me ajudam muito, principalmente porque não tenho dinheiro para comprar cursos, muito obrigado

[@4122inc]

I know this is an older video but it's really great how you explain in detail exactly why you did everything! For a newbie like me, other videos left gaps.

[@nickgibra1861]

Thank you very much for this tutorial. Your explanations were better than some courses and books that I tried. Thank you again.

[@brucemurdock5358]

00:00 What is webscraping 00:30 Our end-goal 02:23 Installation of modules/libraries/packages 05:05 Primer on HTML and CSS 08:32 Example 19:27 Scraping the webpage

[@sudip7949]

Love you bruh😚

[@emileberhard8116]

Ty!

[@amysandstrum531]

Thanks! I have very limited knowledge of html but was able to scrape price information off of websites. Just for funsies :D Really, all i did was tweak it and instead of div they use span class, so i switched 'div' with 'span' and got the same result.

[@pravinbagul5305]

Thanks Corey for making my python journey very simpler with your tutorials!!!

[@iliveforthevibez6216]

I try to get into Data Science and I did a coursera online course to learn python which also contained lessons about html and web scraping with BeatifulSoup4. I did not understand everything that was explained in coursera but your tutorial right here summed everything up in 40 minutes and could have saved me some hours if I had watched it earlier. Instant subscription man, the way you teach (and also speak, I am from Germany so it is easy to understand) is amazing , keep blessing us with your videos !

[@jakesnell7707]

Corey you have the coolest videos, you've taught me much my friend.

[@johnjacobs6234]

Just a note, BeautifulSoup doesn't work with dynamic content. If you see something like: ::before (html content) ::after That content is dynamically loaded by your browser from some sort of redirect or Javascript. The html inside it doesn't exist when BeautifulSoup parses everything.

[@hajajmaor]

The best video I found on KZbin. Search for a long long time but this is simple one the most understood one and the best one thank you man

[@mangaart3366]

Love how you explain things so clearly. Corey you are the best!

[@salah2fois]

Using Pyton 3, I added this part: csv_file = open('web_scraped.csv','w', newline = '') in order to avoid empty lines in the csv file. stackoverflow.com/questions/3348460/csv-file-written-with-python-has-blank-lines-between-each-row

[@hamidrezaghafori4200]

thanks bro i had the same problem

[@jackbird5839]

great solution and tips bro!! thank you for your video., I will definitely try to improve my scraping skills with your tutor. but as non-tech user Also I have found good alternative to scrape amazon reviews e-scraper maybe it helps to somebody too.

[@willjohn6807]

Thank you Jack, e-Scraper really help me with my web scraping needs for my store.

[@vskiy26]

Awesome tip Jack

[@jonatanlast3849]

@@willjohn6807 En este tutorial de programación de Python

[@jonatanlast3849]

@@willjohn6807 nasıl yapılacağını öğrenmek

[@jonatanlast3849]

@@vskiy26 BeautifulSoup o se meafaigaluega sili ona lelei mo le faʻavasegaina

[@daringetae5136]

Best explanation I've found so far on how to scrape websites using soup. Worth watching yours.

[@venkysmoments9155]

Corey please add tutorials related with multithreading and multiprocessing in python.

[@ApplySkills]

Venkatesh K this gets my nipples hard^

[@warofwars7608]

did u have nipple alert app installed

[@efqanmustafayev5189]

Thanks for video, sir. How we can scrape login required websites?

[@lucasmiguel2633]

Have you found out yet? I'm having a rough time trying to do this

[@eizastanford8593]

Thank you so much for your video! I tried looking at tutorials online from numerous articles, but I couldn't truly wrap my head around BeautifulSoup; even after looking at my final pieces of code for long periods of time, I felt my way of scraping was very inefficient and ineffective. Now, thanks to your tutorial, which has *so* much common sense by the way, I can now write readable code to scrape ebay listings and results!

[@abdulmalikgiwa920]

This is a very good and explanatory video Corey, been having issues understanding beautifulsoup but this just broke it down perfectly. I was referred here from coursera, great one 👍🏽

[@DJ-ct6so]

Excellent Corey. Very comprehensive, well explained, step by step, easy to follow. You're a very talented teacher imho.

[@lucastoccheton989]

Absolutely the best. Didn't need anything else for my first web crawler!!! Thanks you!

[@TherandomclipTCP]

Thank you so much Corey for this beautiful video. You taught such a vast topic in a very very simple manner. It's really appreciable.

[@NeBueR4U]

I just loved the way you explained every bit about this topic. Crisp, clear and to the point. :) I have subscribed your channel for more content.

[@MalekBenkouider]

thank you so much, for me you're the BEST python teacher. I really enjoy watching your videos and learn complex staff in very easy way!

[@Ultrakicks]

This was a wonderful video that helped me understand the basic of web scrapping with Python. Thank for you so much for the in-depth tutorial!!

[@pydocs-pro]

Corey, this is just THE BEST explanation about web scrapping with python. Just notice that you could explore a little bit more about the page response with the get method. Sometimes I have trouble trying to access one specific site, and I have to manage it in different forms. And some more detailed explanations about when it's possible and not possible to scrape data from websites. Only this detail that I've missed. All the rest it's just perfect.

[@user-te7ei6yy6i]

I'm viewer from Russia, your English speaking is very clear. Thx.

[@painpills66]

Dude, i'm from colombia and i'm tryng to lear how to do web scraping. The point is that I don't know much English but it seemed to me just by listening to you a little that you were giving important information, so I saw the entire video translating every word that I didn't know, and I learned much more than with the information I find in Spanish, huge thanks and new sub.