Python Tutorial: Web Scraping with BeautifulSoup and Requests

[@sng8240]

*Great video! Even the IBM staff suggested watching this video to complete a part of a course offered by them. Did you know? Congratulations!*

[@coreyms]

I didn't know that. Very cool! Thanks for letting me know!

[@joaquinrequena3494]

Damn course, I'm getting whipped there

[@JoaoVitor-dq9sf]

Yes, it was Alex Aklson from IBM

[@vaibhavkrkm]

He deserves that 😀

[@amazing-graceolutomilayo5041]

I don't know why I'm really happy about this 😂. Thank you Mr Corey

[@Masonx42]

Hello,I'm a viewer from China, comparing with other youtube python tutors, your English speaking is very clear for me to understand. I just registered as a one dollar patron for a little support.If I can find a job as programer later, I will contribute more to help you continue making these high quality videos. Thank you Mr Corey!

[@coreyms]

Thanks!

[@abby4922]

@@coreyms hey you still using this acc for youtube?

[@abby4922]

hey Einsteiincy , are you still using this youtube acc, im also from China, would like some help with python

[@kyefondeme136]

@@abby4922 i think we in China knowledge seeker lol

[@abby4922]

@@kyefondeme136 hahaha broah lets link up,

[@samueltukua3061]

I started watching your videos years ago and I was afraid you wouldn't be around when I returned from my programming hiatus. Thank you for still being here!

[@failuregreat9424]

you never fail to impress me with your teachings! I don't watch videos at all once, because I feel it's easier to learn watching by 'parts' - but again and again, I'm watching the videos and I feel 'wow.. this is really well done, I have to like this, it's the least I can do' and I realized I already liked before lol

[@cosmicdarkmatter1128]

@Failer Great...Agreed. Corey Schafer is very good at explaining things. A lot of professors can't do that....

[@proxyscrape]

5 years ago, this video was already a great tutorial. Now, it's even more valuable. Thanks for the great content!

[@imtiazshahed944]

Corey Schafer Thanks a million. best tutorial I have ever seen on Web Scrapping in Python, No one explain like you in a very single point of code. That's how we learn. great job and keep it up.

[@f.2funky]

Hi Corey I'm a new python learner from China. Listen a class in English is still a little hard for me, not to say with the things I am completely unfamiliar with, like Python. But your English speaking and tutorial are very clear for me to follow! I can't believe that I have just finished this 45-minute-long video with Python in English. I have never thought the I could did this before. You videos helped me a lot in my Python learning. You are amazing. Your tutorials are amazing. Without them I probably have already gave up. You are the best Python teacher. Thank you so much! ^^

[@leslievanelsie7545]

I'm not really a video type of person, I learn faster with books but I always find myself coming to your channel to clarify things. You are the best Corey

[@varunabhi]

No where I have seen advanced topics explained in an easy way like this. Even in paid courses THANKS Mr.Corey. THANKS A LOT! I'd learnt much from your videos in a easy way.

[@SableWinters]

The best and quickest tutorial for small to medium sized website data scraping I’ve encountered. You are clear, concise, and you voice is easy to listen to and follow void of irritation. I actually have confidence now to achieve my tasks after years of placing them on the back burner.👍🏼🤗

[@user-lh4hv3tx8b]

Every time I find myself struggling with python as a beginner and going through so many random sources, you always seem to be the clearest.

[@oluwadamilaretijani1777]

Mr Corey, please wherever you are, keep safe. I really miss your content on KZbin. You're a legend when it comes to teaching.

[@mohanlpuri1]

I have watched couple of videos on web scrapping with Python. This video is very clear, and pace of delivery and the example teaching of the author is excellent.

[@painpills66]

Dude, i'm from colombia and i'm tryng to lear how to do web scraping. The point is that I don't know much English but it seemed to me just by listening to you a little that you were giving important information, so I saw the entire video translating every word that I didn't know, and I learned much more than with the information I find in Spanish, huge thanks and new sub.

[@tntcaptain9]

This is how to teach in a proper way, in some videos a code is already given and they just read it out which makes it difficult to understand. Thanks Corey, keep up the great work.

[@Reese_414]

Tech with Tim is horrible at that... I'll click to watch a tutorial on how to do something and he pulls up a game that's already half way complete... And on top of using pre-written code, he also talks very fast... Salute to Corey for live coding while speaking clearly with good explanations!.!.!

[@tonyt1343]

True story: I haven't had to scrape anything for 5 years and have forgotten how to use most of the tools. I browsed youtube for a minute and started to get discouraged because most of the videos were bad. Then I thought, "I know my man Corey probably has a video. Let me go check out his site." Sure enough, high quality video. Thanks Corey!

[@eyoba6027]

My first video from your channel and it didn't took me long to press the subscribe button. You are just perfect, no second wasted. Every word is important and worth mentioning. Cheers!

[@upasanasingh7567]

Agree!

[@daringetae5136]

Best explanation I've found so far on how to scrape websites using soup. Worth watching yours.

[@sairamsubramaniam8316]

You are the best programming teacher I ever had.

[@mohamedabdo-xg6tw]

I don't know why some people disliked this video? I paid a money to learn this and the content of the class was way bellow this level, thank you for the great Tutorial!

[@mankaransingh2599]

Most professional guy on youtube, who dosent boasts to be know stuff.

[@paulohsgoes1959]

I was quite sure you wouldn't let me down. You're the best, man, because you go straight to the point. Thanks a lot for sharing your knowledge.

[@ashishm8413]

It's nice how you break down the subject into small increments that others can understand and build upon easily. True quality of a great teacher. Thank you!

[@balasubramanian4314]

ONE OF THE BEST DEVELOPER I HAVE EVER SEEN in this planet! Hats Off to your skills!

[@gwanghyeongim768]

Thank you Corey. I can personally assure that your work doesn't go for nothing. I hope I can donate more to your patreon in the future.

[@amitanand9125]

Corey Schafer : I don't know if Python is easy or you made the Python Learning very easy.I did watch almost all your videos till date and refer my friend to do so. Thank you for making our live easier with such a ease of learning

[@allahakbar9580]

Such a beautiful and comprehensive explanation! I liked the idea that you started from the simplest things, like scraping a small HTML file, not an entire website! Because that what I could not find in the many videos I watched about BeautifulSoup, they jump into the deep straight ahead!

[@prabav86]

Hello from Canada! I've been impressed with the way you teach Python and related subjects. So I was referring/recommending your youtube videos to my friends. Recently, I'm finishing up online course on Data Science from IBM, I was pleasantly surprised that course recommended your video on webscrapping! All your hard works helping a lot of folks out there. Simply outstanding! take care

[@samdavepollard]

45 minutes of pure, unadulterated, hardcore geek-porn. Absolutely loved it. Thanks, Corey. Even by your standards this one was pretty special.

[@coreyms]

Thanks!

[@pauloesperon7697]

I felt the same.

[@mateusribeiro9384]

I actually enjoy much more watching his videos than watching normal movies, series...

[@dope8452]

@@mateusribeiro9384 Nerrrrrd

[@shivanishimpi7018]

@@coreyms at the time 26.50, how do I scrape out both tags instead of just one?

[@BoxingPills]

Im doing IBM data science certification. Understood nothing. After 2 minutes of this video I m already hooked! you re phenomenal!

[@OrientalMelodies2010]

An awesome video. Your way of moving from one idea to another is amazing. I have been searching KZbin for years for a good tutorial video explaining web scrapping with BeautifulSoup and finally found it. Thanks a million!

[@ТимуркаДолгорукий]

I'm viewer from Russia, your English speaking is very clear. Thx.

[@MrKashyani]

Man..The way you explains !!! It works like a charm. Be it Regex or web scarping, it's just awesome ! Love your posts.

[@iliveforthevibez6216]

I try to get into Data Science and I did a coursera online course to learn python which also contained lessons about html and web scraping with BeatifulSoup4. I did not understand everything that was explained in coursera but your tutorial right here summed everything up in 40 minutes and could have saved me some hours if I had watched it earlier. Instant subscription man, the way you teach (and also speak, I am from Germany so it is easy to understand) is amazing , keep blessing us with your videos !

[@hasanfares5774]

thanks was just taking an online class about it and couldn't understand until I got the notification you just uploaded wow you really are in sync with your followers

[@abdallahalaa492]

What a great teacher, I never wanted to watch any tutorials about scraping as it was explained in then most boring way by others, but when it comes to you , you did a great job (y)

[@manojuppala3941]

Corey is the world's best python teacher.

[@shariqahmed7319]

One of the finest web scraping with BS4 tutorials on KZbin! Thank you so much sir!

[@geekyprogrammer4831]

Best Python WebScraping video on KZbin! Seriously brother you deserve money for this masterpiece! Keep it up :')

[@jw2965]

pls no charge. me no have money

[@rift8997]

@@jw2965 what the heck

[@jw2965]

@@rift8997 was trying to be funny, but jesss i failed hard.

[@rift8997]

@@jw2965 ye ok i was just confused. lmao

[@FredMny]

Thank you!!! I love it how you are able to not over explain, but also without leaving out anything important.

[@RadioactiveNinja]

One of the best tutorials Ive ever followed. Hopefully you help kickstart my career in data science!

[@ikki411]

This tutorial was incredible. I've done 2 Python courses that touched the 'Web Scraping' subject, but I wasn't able to fully understand it. This video was one of the two videos that made me fully understand it, and I couldn't be more happy about it.

[@alexpaes9161]

It could be possible to hit "like" twice for ur videos! Mate, ur a legend! Congrats!

[@allistergraham9448]

Very informative video, you're an excellent teacher! While other KZbinrs ramble off topic, you get straight to the point, and I really appreciate that.

[@baconsledge]

As always, Corey’s videos are top-notch. Thank you, Corey!

[@tobaobiwale735]

I bought this course on Udemy, and I decided to check youtube for a web-scraping when I was confused. Men, I fell in love with your teaching skills, so understandable in every step, I bet that the Udemy tutor had watched your video to create his own. Ain't rich for a subscription but I bet I will follow you and learn from you at all time, cheers mate!

[@KenPryor]

This was the first of your tutorials I've seen. I definitely subscribed after this great video. I've struggled with learning Python and this was very helpful.

[@Reese_414]

Just wanted to share something I do to practice web scraping that others may find useful... Instead of running your request to a live url while working out your code and possibly overwhelming a website, I right click on the browser and select the "view source" option. Then select all and copy the source code and paste as an HTML file in the same local folder then scrape through it until I get it working the way I want... May not work with all sites, but good for simple sites that aren't JavaScript heavy... P.S. Excellent tutorial, thank you for being so clear and so thorough!.!.!

[@adityatripathi9865]

your understanding of python is very deep and your teaching method is excellent

[@branderbuilt8908]

I was watching another video on web scraping, and then I read the comments, and someone told me to come here and watch your video first because you explained things more coherently. Keep producing great content. You've got a lot of KZbin guardian angels redirecting traffic to you, Corey.

[@brucemurdock5358]

00:00 What is webscraping 00:30 Our end-goal 02:23 Installation of modules/libraries/packages 05:05 Primer on HTML and CSS 08:32 Example 19:27 Scraping the webpage

[@sudip7949]

Love you bruh😚

[@emileberhard8116]

Ty!

[@leosimon1746]

I have to say you did a great job! You covered pretty much everything that a novice programmer needs to know about scraping. I immediately developed my own solution after watching the video. Thank you so much!!!

[@qianlima2196]

I spent 1 hour to figure out why my outcome is different from yours, then I finally realized it's because your website content has changed!

[@iumair_mehmood]

I am glad you figured it out 😂lolx

[@i8nff225]

@@iumair_mehmood 😂😂

[@clairein1433]

Thank you so much for making this video! I'm from China and I've searched many courses to learn Web Scraping. But it is you who make me successfully run my code.

[@tarunsukhpalani9514]

Master Corey, you are a magician. I just mesmerized.

[@MdBodruzzamanSIfat24]

Brother, your videos are always great. I follow them and learnt python very easily. This course worth money but you are serving free. Thanks a lot.

[@Nearco1000]

Just curious about those 34 slackers that did Thumbs down on this vid, seriously?. It's very clear and straight forward, plain English, highly understandable .......................Thumb up.... Very appreciated #Corey, I was looking for this, great vid, again and again tku for your contribution.

[@coreyms]

Thanks!

[@X19-x5f]

Excellent video Corey, thanks. I finally got to see it because we're all quarantined to our houses due to the virus. Videos like this make it bearable.

[@MarkJay]

Another great video Corey! Keep them coming.

[@MidjourneyandB]

I just found your videos. I'm 5 months in to learning coding (toward data science). Your examples and explanations are extremely clear and helpful. I love how large the text is and how clear the narration is as well. Thanks. I will be watching tons more.

[@dpoznyak]

Corey your Python tutorials are great! You explain everything very well. It would be great if you could make a series of Django tutorials for beginners! Keep up the good work!

[@Edwin-ri8xu]

Wow....didn't even notice that the video was 45 minutes long. Thanks, Corey for the wonderful explanation!

[@kanika724]

Corey a Big Bow from India..... really you are too amazing

[@DJ-ct6so]

Excellent Corey. Very comprehensive, well explained, step by step, easy to follow. You're a very talented teacher imho.

[@18nikhilchauhan80]

thank you so much. i got my first client just because of you.

[@RahmanVsa]

Thank you. Can't believe I have been building scrapers for months still got so much deep understanding from a single video.

[@saritaprabhu]

You're a legend! There's a lot to learn from you. 🙇

[@oracle11iappsdba]

I watched this video just once but moved back the time line several times. Result I learned more than what I got in a stupid 1 day paid workshop. Thanks man.

[@rb8058]

Great introductory information! Would love a similar one for how to scrape javascript rendered websites!

[@rajnigoyal6653]

Thank you Corey Schafer.. I really loved your teaching style and got every part of this. The best part is pointing out about the problem and give the solution.

[@lawrencekrukrubo2640]

This is just amazing, your knowledge, no your practical knowledge is amazing

[@anurodhchoudhary1689]

This man is a legend, all your explanation is easy to understand

[@khrenovina]

Thanks, man. You're great! Everything is clear for the beginner like me.

[@albertwalker9396]

Excellent, I found this video after trying to follow some other vids with continual syntax error. I haven't touched Python for several years (retired) and found this video an excellent guide to getting reacquainted Thanks for your efforts. I have subscribed to your channel.

[@hashcoder]

I've tried today! Successfully. Thank you, Corey!

[@pydocs-pro]

Corey, this is just THE BEST explanation about web scrapping with python. Just notice that you could explore a little bit more about the page response with the get method. Sometimes I have trouble trying to access one specific site, and I have to manage it in different forms. And some more detailed explanations about when it's possible and not possible to scrape data from websites. Only this detail that I've missed. All the rest it's just perfect.

[@amysandstrum531]

Thanks! I have very limited knowledge of html but was able to scrape price information off of websites. Just for funsies :D Really, all i did was tweak it and instead of div they use span class, so i switched 'div' with 'span' and got the same result.

[@dhssb999]

This is definitely the best web scraping tutorial I've ever seen!

[@silverzero9524]

awesome was just surfing the web about bs4 and u uploaded this awesome awesome awesome thx

[@neighbourhoodcoder5988]

this 45_M & 47_S tutorial= 10 hours of other tutorials+plus some books+some paid courses. keep it up...respect & love

[@jakesnell7707]

Corey you have the coolest videos, you've taught me much my friend.

[@elliottcarter7700]

I can always count on clear and concise explanations from Corey Schafer! Thanks!

[@wolfisraging]

Holy maaaaan, you are truly most amazing.... Big fan😊

[@pkavenger9990]

Seriously your teaching style and explaining is amazing, you even explained what split does, most people just assume that you already know about it but even then you explained it and it refreshes everyone memories and helped those who forgot due to some reason.

[@jackbird5839]

great solution and tips bro!! thank you for your video., I will definitely try to improve my scraping skills with your tutor. but as non-tech user Also I have found good alternative to scrape amazon reviews e-scraper maybe it helps to somebody too.

[@willjohn6807]

Thank you Jack, e-Scraper really help me with my web scraping needs for my store.

[@vskiy26]

Awesome tip Jack

[@jonatanlast3849]

@@willjohn6807 En este tutorial de programación de Python

[@jonatanlast3849]

@@willjohn6807 nasıl yapılacağını öğrenmek

[@jonatanlast3849]

@@vskiy26 BeautifulSoup o se meafaigaluega sili ona lelei mo le faʻavasegaina

[@centralscrutinizer5116]

You sir, by far, have the best structured videos for learning from. Great job there young feller!

[@maicomcoelholopes9032]

Corey muito obrigado eu sou brasileiro, seus tutoriais são excelentes e me ajudam muito, principalmente porque não tenho dinheiro para comprar cursos, muito obrigado

[@hamzakhalid8668]

This guy is the best. Hands down. For python tutorials. My man slayin'

[@johnjacobs6234]

Just a note, BeautifulSoup doesn't work with dynamic content. If you see something like: ::before (html content) ::after That content is dynamically loaded by your browser from some sort of redirect or Javascript. The html inside it doesn't exist when BeautifulSoup parses everything.

[@eizastanford8593]

Thank you so much for your video! I tried looking at tutorials online from numerous articles, but I couldn't truly wrap my head around BeautifulSoup; even after looking at my final pieces of code for long periods of time, I felt my way of scraping was very inefficient and ineffective. Now, thanks to your tutorial, which has *so* much common sense by the way, I can now write readable code to scrape ebay listings and results!

[@efqanmustafayev5189]

Thanks for video, sir. How we can scrape login required websites?

[@lucasmiguel2633]

Have you found out yet? I'm having a rough time trying to do this

[@mangaart3366]

Love how you explain things so clearly. Corey you are the best!

[@salah2fois]

Using Pyton 3, I added this part: csv_file = open('web_scraped.csv','w', newline = '') in order to avoid empty lines in the csv file. stackoverflow.com/questions/3348460/csv-file-written-with-python-has-blank-lines-between-each-row

[@hamidrezaghafori4200]

thanks bro i had the same problem

[@timfulton6920]

I love this training. I am going to subscribe for sure. I pasted with a questions about an error I was getting. Then I clicked play to continue and realized you answered that question next. Great video. Thanks for the help this is my first endeavor into Python. Thanks

[@venkysmoments9155]

Corey please add tutorials related with multithreading and multiprocessing in python.

[@HouseofFPS]

Venkatesh K this gets my nipples hard^

[@warofwars7608]

did u have nipple alert app installed

[@poorvikjain5323]

why i didn't find you before sir i just wasted my time by seeing unhelpful tutorial of others you are one of the beat teacher of my journey in python 💓💓💓💓💓💓

[@davideschreiber2821]

Well done! I've duplicated on my computer what you're showing in your video, writing comments at every step. It will be my guide for any web scraping I do in the future. Thank you!

[@dp0813]

Holy bejebus, dude. This one video answered sooo many questions i had after trying to build a scraper & referencing 2 other tutorials. SMH! Can't wait to get home & fix my script based on this tutorial! Thanks!

[@Josiasnyc]

All of your videos are just phenomenal. Well done for the amazing work you do! I'm glad we have people like you, very competent, professional, knowledgeable and straight to the point. 10/10. Have you considering writing? You should!

[@_boris]

Whoever gets weird symbols when opening the .csv in *Excel*, change this line: csv_file = open('cms_scrape.csv', 'w', *encoding='utf-8-sig'*) Thanks once again Corey for brilliant content 💎