Web Scraping with Python and BeautifulSoup is THIS easy!

[@ammarsalmi]

Very helpful. Thank you for taking the time to make this.

[@TomsTechAcademy]

Thank you for your comment, glad my video was helpful! Happy coding 👨‍💻

[@ahassan7270]

Thank you so much for your valuable information and the easy-to-follow explanation style.👏👏

[@TomsTechAcademy]

You're very welcome Abdullah 😃 is there anything else you would like to learn?

[@WannaTryChannel]

Thank you for your great work! This is incredibly helpful for beginners who want to learn Python. Please keep it up -- you're making a big difference for learners like me!

[@TomsTechAcademy]

Thank you for your kind words, these kind of comments motivate me to keep creating content!

[@gonzalobarral2357]

Hi Thomas, thanks so much for sharing it! i was looking all day trying to find some good material and this by far saved the day!

[@TomsTechAcademy]

Thank you for your comment! Glad to hear my video has been helpful for you :) happy coding!

[@rkcorbes]

Thank you so much man. I'm totally beginner and it's very easy to follow.

[@TomsTechAcademy]

Thank you for the kind words! Happy coding 😀

[@ebinmolloy7596]

Great video, really very helpful, thanks!

[@TomsTechAcademy]

Thank you Ebin! 😀

[@FatimaMubeen-r9z]

This one tutorial is all we need to practice and understand the web scraping using BS! Thanks a lot!

[@TomsTechAcademy]

Glad it was helpful and thank you for your comment Fatima!

[@priyeshudawat]

Very good explaination.. Tyx.. Hope you make some more videos on data scraping using BeautifulSoup

[@TomsTechAcademy]

Thank you for the kind words, there’re more data scraping videos in the pipeline 😀

[@illuminaughty666]

Awesome tutorial 👍

[@TomsTechAcademy]

Thank you for the kind words! Happy coding :)

[@islamicdesiretv]

Hy, Thomas you've done a great job, I learned a lot of imp things from you by watching only this video. Plz make a playlist on web scraping from basic to advance so that a new person can understand better. Once again amazing video

[@TomsTechAcademy]

Thank you for your comment, expect more videos about web scraping soon! Cheers, Thomas

[@islamicdesiretv]

@@TomsTechAcademy Thomas, that's good for all new web scrapers bcz your style of conveying useful content is very simple thanks ☺☺☺

[@japhethmutuku8508]

hello, I can teach you web scraping from the basics to advanced

[@codecobber1107]

Good video. Many thanks.

[@TomsTechAcademy]

Thank you for your comment Code Cobber! Glad my video was useful for you 😀

[@mahatechtalks9496]

Great Video , Thanks

[@TomsTechAcademy]

Thank you, glad to hear my video is useful for you 😀

[@Test-ne6bt]

Thanks for the video! My first Python experience. I did get 'No module named 'openpyxl'' error when trying to export to excel, even though CSV worked. Although I tried to install via pip in VS. I ended up having to run 'py -m pip install --user xlsxwriter' in my windows terminal and then it worked. Thanks!

[@TomsTechAcademy]

Congrats with your first Python script! Here's to many more scripts ;)

[@bouncepen932]

Good tutorial.

[@TomsTechAcademy]

Thank you for your comment, glad my video is useful for you 😀

[@nmsv1]

Since the items are added to a set, how do you ensure that the columns always come back in the same order?

[@TomsTechAcademy]

Hi, the items in the dictionary (e.g. Title, Link, Price) will always be in the same order

[@SaifAliShugan]

very nice

[@TomsTechAcademy]

Thank you for your comment, glad the video was valuable for you 😀

[@alexanonym1584]

There could be f used instead of plus

[@TomsTechAcademy]

Thanks for your suggestion :)

[@drendelous]

ty please consider covering other frameworks or websites

[@TomsTechAcademy]

Hi, there're some other helpful scraping videos on my channel, such as this one: kzbin.info/www/bejne/amK4nmuXaZWLqKM

[@y.hidayat7391]

Nece tutorial, thanks

[@TomsTechAcademy]

You are welcome, glad my video was helpful for you!

[@abigailmapuladikobo9941]

I followed the same code for a different website except I left out the if and else codes because I wanted to scrape only the first page. Everything worked out except my output which came back in the same column, and it is very disorganized. How can I fix this in my code?

[@TomsTechAcademy]

Hi, best way to help you is if you share the website + your code on my Discord, then I can help you right away :) Join my Discord here: discord.gg/WHJWFNDXXX

[@jesusleguiza77]

Hello, very good. What about products that have variants that need click for show price? How are different prices handled, whether by color or other different features? called without ajax, in detail page

[@BhushanAhire-gg5wb]

thank you so much

[@TomsTechAcademy]

Thank you for your comment, glad my video is helpful for you :)

[@teclote]

Brilliant. Thank you.

[@TomsTechAcademy]

You're very welcome!

[@raagha7733]

Can i scrape the reviews from amazon through this same method? Cuz I'm unable to access the page through my requests

[@japhethmutuku8508]

are you new to web scraping?

[@mahdiosman7921]

Great video and flow. I did this for another website but when i run the code at the end the scraping of pages should be 235 but it doesn't stop it keeps scraping pages? How do i stop it at the amount of pages i would like to scrape from the website.

[@TomsTechAcademy]

Hi, there’s a link to my Discord server in the description of this video. If you share your script and the website you want to scrape there, I’m happy to help you with your script 😀

[@daviannajones5648]

Thanks for the video! The only issue I have is that it will not allow me to put the results into an excell file. It does for a csv, but not excell

[@japhethmutuku8508]

hello i can help you with this

[@TomsTechAcademy]

Hi, you mention "it will not allow" - which error are you getting exactly?

[@Blackops1990]

Hey I also had this problem. I fixed it by adding the openpyxl inside my python interpreter

[@abigailmapuladikobo9941]

Thank you for this video. What is the risk of web scraping a Google scholar page (only one page) with an unhidden IP address?

[@TomsTechAcademy]

If you only scrape 1 page and only do it one time (so not for example every hour) I would say the risk of your IP being banned is pretty low.. In case you have any doubt, just google "free proxy server" - there're a lot of proxy servers you can use for free. The challenge with free proxy servers is that they aren't reliable - but if you only scrape one page that's not a big deal.. Good luck with your project :)

[@abigailmapuladikobo9941]

@@TomsTechAcademyThank you 🙏🏽❤️

[@hotdoggyboomboom411]

does it work in similarweb

[@TomsTechAcademy]

I can't tell you whether this method works for specific websites. It should work for most websites, although in case you need to log in or the website uses lazy loading, you need a bit more work to get it up and running..

[@sleex_official]

Ich rieche deutsch

[@TomsTechAcademy]

You’re almost right, my native tone is Dutch :)

[@LastGamerName]

Hi! I'm practiced in C#, but new to Python. I watched your video all the way through before watching again to try myself. My requests.get() works off and on without changing any of the code. I keep getting 'getaddrinfo failed' error even though it just worked with no changes made. I didn't add a proxy yet since that's later in the tutorial. Would that cause this?

[@TomsTechAcademy]

Hi, this shouldn't be about the proxy. It could be either an issue with the website or the connection. Could you try: - Running the same script on a different network / PC - Try scraping from a different website (preferably a very simple one like the one I use in the demo - definitely not something behind Cloudflare) This will help you to figure out whether this issue is network/pc or website related. Good luck!

[@Mrshanmugham1]

Nice explanation, Very Good, I have a doubt , Kindly provide your email pls....

[@TomsTechAcademy]

Thank you for your comment, glad my video is useful for you 😀 you can ask any questions in the comments, this way others who have the same question can also benefit from the answer.