Web Scraping in Python using Beautiful Soup | Writing a Python program to Scrape IMDB website

Web Scraping in Python using Beautiful Soup module. In this video I will explain how to perform web scraping in Python using Beautiful Soup and Requests module. I will write a python program to scrape the IMDB website and then load the desired data into an excel file. I will write this program from scratch so you can follow along.
Web Scraping is the process of programmatically extracting some data from a website. In Python, we can easy write program to scrape website by using the Beautiful Soup and Requests module. Requests module can be used to access the desired website and Beautiful Soup module can be used to parse the HTML source code of the website. Beautiful soup makes it very easy to parse HTML content and then provides multiple methods which can be used to extract the data from any HTML tags. All the website are written in HTML language hence in order to perform web scrapping, our program needs to read the HTML content. This is were Beautiful Soup comes into picture to make it very easy to access the contents of HTML using very simple methods.
You don’t need to be an expert in HTML to learn python web scrapping. You just need to know the most basic HTML which is to know what is a tag and how to identify the attributes associated with a tag.
In this video, in order to explain the concept of web scraping using BeautifulSoup in Python, we will be writing a python program which will access the IMDB website and then fetch the top rated movies present in the IMDB website and then load this data into an excel file.
IMDB website contains movie ratings but in this program we are only interested in scraping the top rated movies of all time only.
In order to load data into an excel file, I will be using openpyxl module. Using openpyxl, it is very easy to create a new excel file and then rename the sheet name and then load data into the excel file. I won’t be explaining openpyxl in detail but will just cover enough to know how to create an excel file and then load data into an excel file.
We will be installing Requests module and Beautiful Soup module using the pip installer. I am using mac so the command in mac to be run from terminal is “pip3 install module_name”. If you are using windows then you need to run the pip install command from command prompt using the command as “pip install module_name”.
🔴 WATCH MORE VIDEOS HERE 👇
✅ SQL Tutorial - Basic concepts:
• SQL Tutorial - Basic c...
✅ SQL Tutorial - Intermediate concepts:
• SQL Tutorial - Interme...
✅ SQL Tutorial - Advance concepts:
• SQL Tutorial - Advance...
✅ Practice Solving Basic SQL Queries:
• Practice Solving BASIC...
✅ Practice Solving Intermediate SQL Queries:
• Practice Solving INTER...
✅ Practice Solving Complex SQL Queries:
• Practice Solving COMPL...
✅ Data Analytics Career guidance:
• Data Analytics career ...
✅ SQL Course, SQL Training Platform Recommendations:
• SQL Course / Training
✅ Python Tutorial:
• Python Tutorial
✅ Git and GitHub Tutorial:
• Git and GitHub
✅ Data Analytics Projects:
• Data Analytics Projects
THANK YOU,
Thoufiq

[@hxghrxnk5291]

I've watched so many Web Scraping tutorials out there and this one is the best by far.

[@dennisaxopoulos5421]

I love that you explain every line of code

[@daveg5857]

Very helpful. I'm not a programmer, but I want to use these techniques to streamline my job. All the other videos I watched just sailed over my head. This is a great explanation, and easy to follow, even for me! Thank you.

[@MdRizwan-uu9dv]

Not feeling any difficulties to follow along. which means that our trainer is maestro of explaining the things which seems to be difficult.

[@shanmukhch3763]

Damn I love the way you explain the things makes anyone understands without much effort. I've watched many videos on web scraping but really this video helped out a lot. You're amazing brother

[@TomMartinek]

This was SUPER helpful! I've watched a few other tutorials on this subject, but this one really helped me understand the whole process. Thank you!

[@alisonhenley2551]

Thank you so much Thoufiq. You have been my star instructor since the beginning of my data analyst journey. Much appreciation towards your explanations and many videos.

[@2380raj]

one of the best videos of such clarity and precisely what code does. Only a person with a deep understanding can do this. Thank you for sharing.

[@techTFQ]

Thank you so much ☺️ Glad you liked it

[@AbhishekSingh-jy5ot]

I am so happy i found this video. I have just started with my data science career n was looking to learn webscraping for my first office project, i had went across tons of webscraping videos but this video just made my day. thank you so much.

[@techTFQ]

I am glad to hear that .. good luck with your job

[@anoopbhagat13]

Excellent tutorial on web scraping. Loved the way you explained everything with so much ease & great details.

[@techTFQ]

Glad to hear that :)

[@valarietan]

Brilliant tutorial on web scraping and importing data into Excel. Two thumbs up! Thank you.

[@SalehGoodarzian]

I had been suffering so much from not understanding how things work in web scrapping. Your video gave me a clue and understanding on how approach the problem...I can't be more thankful. I really appreciate you and wish you all the best...

[@techTFQ]

Thank you so much .. glad you liked it

[@sudarkrish]

Beautiful indeed! I am going to try this myself!! Thank you for the detailed walk thru. You make great videos!!

[@techTFQ]

Thank you once again Sudarsan 🙏🏼 Really happy to see your feedback 🙂 Glad you liked this content ..

[@vishwanathmuthuraman5530]

absolutely loved it !! understood so much watching compared to other videos!!

[@grandparick3176]

You are amazing. This is indeed one of the best web scraping videos I have seen. The way you explain each and every step and what it does really cleared things up for me. Thank you so much.

[@kimvo4929]

Thanks for this helpful video. Although in 2023, the page sources are totally different, I can figure out the way based on your syntax. It seems not very hard to do. Your video is very detailed and straightforward for a beginner like me. Hope you can produce many good videos like this. Thanks!

[@AwanievaStudios]

How do i like this video 200 times. I have an assessment on web scraping due soon. I've watched numerous Python web scraping videos but i did not get the information i need. this was the best so far. Thank you so much. You got a new subscriber 😊

[@AYFITwins]

The best explanation ever! Thank you for your great help and efforts🙏

[@aniketsingh1879]

Thank you for explaining..the explanation is very good and the best part you have explained in a very simple and effective manner...I was struggling to understand the web scrapping through other videos....but now I've understand properly....Thanks to you 🙏

[@sadiarasoolbaksh2562]

After concluding my work successfully and I exactly download the excel sheet. This was my first web scrapping file. After this I hit the Subscribe Button

[@shankhaneeldutta4617]

Such a wonderful explanation with least effort...is very rare !!! Thank you so much Tohfiq 😘😘😘😘😘

[@Notunes]

Absolute best tutorial for beginners, Thank you ☺️

[@hakimdanyal2129]

Damn this was insanely good and easy. Amazing explanation and great job!

[@techTFQ]

Thank you Hakim

[@purushottamarasankula5059]

The best video i have found for web scrapping 🙏

[@BurnLai]

the clearest teaching to web scraping. Thankyou very much.

[@techTFQ]

Thank you Lai 🙏🏼 So glad you liked it..

[@theforlorned8293]

This was the most helpful educational webscrape video I’ve watched, thank you

[@techTFQ]

Thank you :)

[@RS-vu5um]

Excellent Tutorial. Your explanation generated interest in web scraping for me. I was finding it hard with other tutorials. You nailed it with simple and clear explanation.

[@techTFQ]

Thank you buddy… I am so glad to read this feedback 🙏🏼

[@joelngige5776]

Great!!! video on web scrapping I have watched other KZbin videos about the same subject but your made everything very easy to follow and understand.

[@akhtarattar2744]

Best video to learn webscrapping ,very easily taught

[@Sharmasurajlive]

Clear and crisp explanation !! Kudos. Keep up the good work👍

[@techTFQ]

Thank you Suraj 🙏🏼

[@ShubhamKumar-xt3vi]

Great work, finally understood the code for Web scrapping

[@pratiknale6993]

Aag laga diye bro!!!🔥❤‍🔥🔥

[@sadakagaja]

Thank you sir Using your video/content my first web scraping

[@johnfife1168]

Super helpful, especially writing out the results to Excel.

[@vijayaraghavanselvaraj2711]

Thank you so much, was searching this kind of simple explanation for so long.

[@techTFQ]

Your welcome 🙏🏼 Glad you liked it

[@dma6481]

Best and simplest tutorial so far. I strongly recommend it

[@techTFQ]

Thank you 😀

[@ayeshafathima5554]

This was an excellent tutorial with great explanation. Perfect for beginners.

[@saurabhgope]

Beautifully explain. Thanks and keep up the good work of helping others ☺️

[@arielwendichansky5265]

Genious! you really helped me out I understand this much better now!

[@abhikdebbarman]

Thank you for this video, very very helpful and detailed with well explanation of each step. Please continue to create such content.

[@techTFQ]

Thank you Abhik for the feedback 🙏🏼 Glad this helped you 🙂 Sure, will keep doing my best ..

[@AnilkumarPaturu]

very detailed explanation. Thanks Toufiq

[@TaLha-uc3eb]

One of the best video I ever seen on this topic

[@ahanaroshan7741]

Awesome this should reach many people , many views . Worth it. Came back after your Sql content

[@techTFQ]

Thank you so much for your kind words 🙏🏼 So glad you liked the video 🙏🏼☺️

[@sg8581]

BeautifulVideo. Very informative.

[@BabatundeRaheem]

Thank you . I wished i came across your channel this time last week, I would have gotten a DS job

[@kamilmohammed9722]

It's different than other videos and i learnt a lot. Thanks

[@studybuddies697]

Thank you so much , this video helped me understand web scraping in a better manner. i even did a project after learning from your video

[@abbastechtalks]

Thank you for this amazing video. I did my first web-scraping in python :)

[@techTFQ]

Awesome bro, Glad it helped

[@eshwarpalatli5143]

The way you explained is soo simple and easy to understand thank you soo much

[@nimnanilwala5245]

Excellent work sir. Thank you so mutch.

[@Hichamabid]

Excellent explanation! Thank you!

[@learnwitharbia3477]

Thank you so much! This is just amazing!

[@MalayKhandkar]

Thank you so much!! Enjoyed the tutorial a lot

[@vicentesoto1628]

Definetly excellent straight and concise application Thank you techTFQ

[@kevin3447]

Thank you so much for this tutorial. Really helpful one. God bless you!!😊

[@lakshminarayanan5912]

finally done .thanks a lot for sharing this video

[@soumyadip112]

Amezing ! I just wanted this . Thank you ❤️

[@techTFQ]

Thanks a lot for the feedback 🙏🏼 So glad you found it helpful ☺️

[@nikhilreddy4582]

How boss..., literally damn good explanation....💯💯💯

[@Zydres_Impaler]

thanks a lot, the best intro to web scraping.

[@27ayhel]

Thanks! this video is very useful. God bless

[@YoussefMrabetYMF68]

Awesome, thank you very much, for this excellent tutorial...!

[@neondemon3304]

thankyou so much sir............this tutorial really helped me in learning something new.

[@maxee5908]

Good web scrapping video

[@mirzaimadbaig]

thx for the tutorial it helped me immensely

[@deelovely7565]

This was a ton helpful. Thank you so much!

[@techTFQ]

Glad it helped

[@devbites77]

Beautifully explained! Next step, I am interested in scrapping dynamic websites.

[@techTFQ]

Thank you 🙏🏼 Glad you liked it 🙂

[@davidkiplangat2590]

Hey, you are just smart, i loved it.

[@anduamlaktadesse9284]

Well explained sir! Thank you so much sir!

[@ketankumar5689]

I like your way of explanation. Can you please create a bootcamp or course on data analysis with essential tools like Excel, SQL, Tableau/powerbi, python ? It would be great.

[@fmtgamers]

Superb Explanation!

[@shruthig8817]

Thank you for this amazing session!

[@techTFQ]

Glad you liked it

[@ahabweandrew]

Bro, you are amazing !!

[@sahilgogna3393]

Great video! I think I will become a python and SQL champ in a few weeks. I am going to watch all your videos! Thanks

[@techTFQ]

Thank you Sahil .. Definitely bro you can

[@amitsaini1109]

Thank you so much for explaining easily and this helps me

[@sateeshsunkara3]

Loved the way you explained everything !! One of the best tutorial on Web Scraping 🎉🎉🎉

[@techTFQ]

Glad it was helpful!

[@misskisulkens8075]

Thank you, friend!

[@basavarajnh9550]

Thank you so much man it helps me alot,

[@open-source243]

Thanks for your efforts ... It is a excellent tutorial :)

[@osoriomatucurane9511]

Awesome tutorial, crystal clear, the best one of its class by all standards.

[@techTFQ]

Glad you think so!

[@Sandunbalage]

Excellent. Thank you

[@mdomarfaruq8945]

Excellent tutorial. Explain everything very well. Thank you

[@techTFQ]

Thank you buddy

[@lingamarunachalam]

Beautiful session, was very useful to learning about data scraping.😊

[@techTFQ]

Glad you liked it ☺️

[@MrBonbatong]

Great explanation, thanks for sharing!

[@shubhamawale4881]

The explanation is bit by bit, Fabulous.

[@techTFQ]

Thank you 🙏🏼 Glad you liked it

[@limitless582]

spectacular video

[@aromals8564]

Thank you! Nicely explained! 👍

[@techTFQ]

Your welcome 🙏🏼 Glad this helped

[@charanvenugopal2197]

Awesome explanation, just subbed to the channel.... Great Content

[@theorganics5427]

Really interesting video I have learnt a lot about how things can be done in real world.

[@avinashshinde2659]

thanks for the video,easy to grasp the concept

[@Codewithteck223]

Easiest one, thanks❤

[@wellingtonsantana9470]

Amazing content. Thank you very much!

[@techTFQ]

Your welcome 🙏🏼 Glad you liked it

[@dimi_ga]

Great video, thanks a lot 🎉😊

[@yuyuchung]

Straightforward! thanks.

[@techTFQ]

Your welcome 🙏🏼 Glad you liked it..

[@aliboughazi7037]

You saved me bro, thanks!

[@techTFQ]

Glad this helped

[@suke933]

Very very informative... ❤❤

[@ai_anisha3118]

Thank you so much , I executed perfectly 🤩🤩 no error . Awesome, saved my day 😍😍

[@techTFQ]

Your welcome 🙏🏼 It’s nice to hear ☺️

[@sanjubell]

Hi I have an assignment to scrap a web page and save it to database and the script should run for every 15 min for latest data.. Can you help me please

[@drkrueger]

Crystal clear and excellent presentation. Thank you!

[@techTFQ]

Glad it was helpful!

[@drkrueger]

Can Beautiful Soup interact with websites? For example, can it enter something in a search box, press the search button, then retrieve the results as in your example?

[@techTFQ]

Nope beautiful soup cannot do it but you can use selenium for that

[@drkrueger]

@@techTFQ Thank you!

[@Pawan-tc2ih]

Great Tutorial with nice explanation!!🙌

[@techTFQ]

Thank you Pawanya 🙏🏼

[@vijaykumarkammar3300]

Best video , explained very well

[@mohammedshahil4898]

This was very interesting🤓 Really enjoyed understanding web scrapping👌👌

[@techTFQ]

Thanks a lot Shahil ❤️❤️ Glad you understood it..