Amazon Web Scraping Using Python | Data Analyst Portfolio Project

Take my Full Python Course Here: bit.ly/48O581R
Web Scraping isn't just for those fancy "programmers" and "software developers". Us analysts can use it too! In this project I walk through how to scrape data from Amazon using BeautifulSoup and Requests.
LINKS:
Code in GitHub: github.com/AlexTheAnalyst/Por...
Anaconda: www.anaconda.com/products/ind...
Find Your User-Agent: httpbin.org/get
____________________________________________
SUBSCRIBE!
Do you want to become a Data Analyst? That's what this channel is all about! My goal is to help you learn everything you need in order to start your career or even switch your career into Data Analytics. Be sure to subscribe to not miss out on any content!
____________________________________________
RESOURCES:
Coursera Courses:
Google Data Analyst Certification: coursera.pxf.io/5bBd62
Data Analysis with Python - coursera.pxf.io/BXY3Wy
IBM Data Analysis Specialization - coursera.pxf.io/AoYOdR
Tableau Data Visualization - coursera.pxf.io/MXYqaN
Udemy Courses:
Python for Data Analysis and Visualization- bit.ly/3hhX4LX
Statistics for Data Science - bit.ly/37jqDbq
SQL for Data Analysts (SSMS) - bit.ly/3fkqEij
Tableau A-Z - bit.ly/385lYvN
Please note I may earn a small commission for any purchase through these links - Thanks for supporting the channel!
____________________________________________
SUPPORT MY CHANNEL - PATREON/MERCH
Patreon Page - / alextheanalyst
Alex The Analyst Shop - teespring.com/stores/alex-the...
____________________________________________
Websites:
GitHub: github.com/AlexTheAnalyst
____________________________________________
All opinions or statements in this video are my own and do not reflect the opinion of the company I work for or have ever worked for

[@BummerSlug]

The real talk is nice. “It took ten hours over two weeks”. These are things people need to hear. Some people watch these videos on YT and think it is just that easy. This is why your channel is on my short list of channels I subscribed to. Thanks for all your time on these. Hey MS Excel - sponsor this channel!

[@AlexTheAnalyst]

I try to make it as realistic as possible - I used to think people could do this all off the top of their heads and I would get discouraged. Glad to hear that! :D

[@pkabir4625]

@@AlexTheAnalyst For the same product, I couldn't find the id for price...it shows div class...what to do?

[@ayoni02thebadgamer39]

This should work if you tweak it well enough

[@samosagaming9476]

@@pkabir4625 go little bit up you will find id but you have to use strip function and [1:4] or insert the values as per your requirement to get the exact vales. this worked for me

[@kuldeeprana7659]

it didnot work for me , not showing price id ,its in span tag

[@kishwaralamgir4468]

Alex is so honest and down to earth, he doesnt have that usual KZbinr vibe that we are accustomed to. Man we're so lucky to find you as a mentor.

[@AlexTheAnalyst]

That means a lot! Thanks for watching! :D

[@pulakkabir2276]

@@AlexTheAnalyst hi. how to find the code you showed right of the 't-shirt' web page?..you selected price...then the code for price got selected. how to do that?

[@abdouelkafrawy1726]

it's been a year on this project and despite me searching and watching other channels, I always come back to your channel ,you are simple the best person I have learned from . you are genuine and always able to get your point across .I hope you expand your "python for data analysis" series just like you did with SQL. Thank you so so much .

[@patcharaphanjearsavaswatta6149]

Thank you for this, Alex. I felt so happy when I finally could scrap the website I had been trying to scrap (I applied your teaching to another website). Really appreciate your work.

[@simoyland]

Always thank you for all your efforts and good work! I love watching your videos. Your positive attitude and way of expression make the lesson even more fun. I've seen a few people say the video is too long, but I think being able to walk through the lesson together rather than other videos that show written code is much better for learning. Thank you thank you thank you ☺

[@podawe8051]

So great Alex! I followed along with this entire project and added it to my portfolio! I'll be sure to give you credit in my README file. :)

[@enestemel9490]

Hey Alex quick tip: When you were working on spaces like 34:21 if you select everything that you wanna move and press tab this way everything you choose goes 1 tab right. Little things like that improve your quality of life sometimes. Thanks for the tutorial :)

[@reshuraj5005]

you're already doing a great job man. Thanks a ton, and hats off to you. But, We need that part 2. Please do it asap Alex.

[@skugga7]

This project gave me a taste of how challenging web scraping is. Great video that makes things look easy and less intimidating.

[@atanasprodatanasov9233]

The section where you speak about how you shouldn't know this by heart is so good. Honestly... I am learning SQL as per your recommendation, but in the back of my head I am scared as I think I should learn and memorize each single block of code... And this is awful... Thank you for being honest and clear on that!

[@BadWeatherfreak]

How is it going?

[@NathanEckelTV]

Thrilled to successfully get to the end of this @Alex - appreciate these real-world worked examples.

[@AlexHitchon]

Thanks Alex I am working on my own web scraping project for checking placements of searches and this video definitely helped

[@DustInTheAir]

Hey Alex! Thanks for this helpful video! The best part of this video is whenever you said 'I don't know what that is' (12:50) , instead of some difficult theory. You don't know that, I don't know neither, so it makes me feel less pressure on learning python...

[@fernandamello837]

Wow!!! This is awesome!!! You have so easy way to teach, I already have a base with Python but I’ve never made this before and you make this so smother and easy to do!!!! Thank you thank you ❤

[@sj1795]

Hi Alex, I really appreciate how you shared how long this project actually took you. It helps to know the difference between what we go through on your channel and the work/time it actually takes behind the scenes. AWESOME project! I learned tons and found all of it very useful/helpful. You are such an AMAZING teacher and resource! As always, THANK YOU!!

[@valadhruv6920]

Bro did you got nonetype error and how you solved it?

[@abdelrahmanmalekgenedy8751]

I have this error bro, and don't how to solve it @@valadhruv6920

[@amelieguzman9639]

You don't imagine how this tutorial has helped me in my new position. Thank you so much!!

[@AlexTheAnalyst]

So glad it was helpful! :D

[@adisonfryman144]

@@AlexTheAnalyst Did you ever make the second one? So many people want to see it? Please do send it out!

[@SayedKassem-pn7tr]

Your tutorials are so good. and i follow you on LinkedIn, your content is awesome. i love how you explain things in a clear way. keep up the great job!!

[@mercantilism954]

This is what I have been waiting for! Thank you

[@yashwanthjetty7845]

Thanks a lot for enlightening on Web Scraping. Came to know only after watching this video that such stuff can be done.

[@user-vy8kk9ob3s]

Hi Alex, I have learn a lot from 65 videos of the Bootcamp. God bless with everything. Thanks!!!

[@bernardnader]

Thanks so much Alex!! Just what I was looking for.

[@nevinphilbert]

man i've been battling with the bot blocker from amazon and also some scraping issues with price because the website display was changed a while after this video was uploaded, but I've managed to pull it off so i hope this might help those recent viewers who might be as confused as me when I started writing this code on my own. apparently you need to divide the second cell so you need to run the soup1 first before you run the soup2, then for the price you need to pull three part of span class=a-price-symbol, span class=a-price-whole, and span class=a-price-fraction and combine it together into 1 new variable (price), then you need to clean it using strip() and replace() to clean the whitespaces and 's. hope this helps!

[@deeplakshmiyadav]

Brother please elaborate it, I am stuck

[@ritiknegi8596]

Hey bro can you explain it or can you share your code, how you pulled three part together. I am stuck in this part

[@sdivi6881]

Could you please explain it, I am stuck in getting Title itself

[@ritiknegi8596]

@@sdivi6881 Hey i just solved it can you tell little more where you are getting stuck

[@ShortClipsPodcasts]

@@deeplakshmiyadav price_symbol = soup2.find(class_='a-price-symbol').get_text(strip=True) price_whole = soup2.find(class_='a-price-whole').get_text(strip=True) price_fraction = soup2.find(class_='a-price-fraction').get_text(strip=True) price = f'{price_symbol}{price_whole}{price_fraction}' print(price)

[@esonon5210]

Thanks for sharing! This is an awesome video. I'm not sure if you did this but I think it would be cool to learn how to scrape multiple pages then append the data in a def function.

[@thinkALF07]

Wow, this is EXACTLY what I have been looking for. Alex the GOAT in DA. :) You are 1000x Awesome!

[@AIdevel]

I am absolutely fascinated by your thorough explanation

[@Kaura_Victor]

Though the project was quite tricky, I got over it. Thank you so much Alex.

[@anthonyyawa2445]

Great video alex ... it was really helpful for a module in my course . Please i have been looking for the intermediate video you spoke about

[@demr5251]

Wonderful! I'll practice with this tonight!

[@michelchaghoury870]

Mannnn pleaseeee keep going we need your help you tuts are on a whole diff level I am able to learn and understand with ease tnx a lotttttt and once again keep going

[@pushpitjain7208]

Hey Alex! It was a super helpful video. Thank you so much for posting it. Have you uploaded the next part of this video. If yes, Please share the link.

[@neildelacruz6059]

Simply the best video, thanks Alex.

[@sunyoungjang7810]

I'd like to appreciate you for sharing this wonderful video! Thanks to you, I've just managed to make my own webscraper that helps me to save so much time. Otherwise, my coworker and I would have to spend more than 6hours per week😂

[@aadhithyanp6989]

Thanks man. You are helping a lot of people like me. Keep doing this portfolio videos!

[@ramakumarguntamadugu1299]

thanks Alex! really a great video... request you to kindly do a similar one on stocks realtime price capturing with time series and configure an email notification when the current price drops below say 50 day moving average .... I am looking forward for many videos like this...thank you!

[@sergioramos3437]

Love this .. I'm curious about the headers part I didn't know about that before

[@abhinandc5018]

The long awaited one ❤️💯

[@erkanlacin4729]

You are great, this is exactly what is am looking for...

[@shyamss2338]

Hi Alex, seems like this code is now not working. Would be grateful if you could do another web scraping project with EDA analysis? Love how instructive your videos are.

[@kendrasdustyroad]

Dude! I'm an amazon seller and this kind of work would come in super handy. Thank you. Did you ever get around to making the next video where you pull data from all the search results page? I'd be really interested to see that one.

[@horacezhen]

Thank you for demonstrating! I never thought that a simple project like this could use as a portfolio project. I just realized that I have what it takes to become a DA. Thank you for demonstrating projects!

[@dataisfun4964]

Thanks for this awesome video, this would help me in the nearest future.

[@laifookfatt3465]

This is super interesting, thank you very much!

[@bryan.canelas]

Thanks Alex! this was really useful. I am waiting for the second part with the pagination 😅😅

[@newyork397]

As others described, if you get an error when running the second cell it's probably due to a captcha issue where amazon thinks you are a bot. You can force it by pressing ctrl + enter again and again until you get an output. I'm sure there is a better way to get around this but that's the quickest semi solution I found.

[@newyork397]

The while loop definitely doesn't work now that Amazon updated their website with some sort of blocker for bots. It might work a few times but eventually stops running in background.

[@pomme_paille]

Thank you! Amazing. Waiting for the next video 😉

[@syaoransakura8839]

After a looooong time delay cause by many things, finally I can finish this portfolio.

[@user-gq1ij]

Sir, I am very near to get my first job through your project Thank you And this is also my first project

[@gabrielmorais7312]

Great video. Thanks Alex!

[@mitchellharding305]

Super early, love your stuff as always Alex!

[@AlexTheAnalyst]

You are very early! Lol Thanks for watching 😁

[@zebaszd]

man it was super easy to understand, you nailed it

[@AlexTheAnalyst]

So glad to hear it!

[@robertoenriquehiguera2648]

Thanks Alex. I’m a big fan.

[@gusinthecloud]

you saved me a lot of time. I really appreciate it.

[@amodeocarlo97]

Thank you so much Alex for your contents! Do you think you're going to do some projects regarding healthcare or covid 19 data? That'd be great!

[@SaffronKage]

I second that thought @Luis Ramos and honestly the video are really helping me better understand each topic in Data science with more depth and something more to try on myself piece of advice to fellow learners " Data science is not as simple as it was 2 years ago its been super competitive as there are 100 more individuals just to replace you so do expect hardships if you got the analytics skill keep polishing it "

[@jeisonsanchez4842]

29:30 - quick tip: select the file, hold shift and right-click to get “copy as path” in the context menu.

[@jiewang6449]

Thank you so much Alex!

[@andrespatricioespinozadelg4490]

It was a lot of fuuuuuun! Thank you Alex. Your channel has became one of my favorite about Python and SQL 🤓

[@AlexTheAnalyst]

So glad to hear that! :D

[@blackwhite2361]

I thought you were now only gonna only make videos on management and stuff. Glad you are still making tutorials

[@AlexTheAnalyst]

Nah, content really won't change much - I'll be doing Tableau tutorials very soon

[@skaigreen]

this was really helpful... I'm planning on transitioning careers to data analytics and would love to see how it goes.

[@Magmatic91]

Great stuff ! Please make the video where you show how to get for each item let's say the product description, reviews...etc.

[@yousseftaki1949]

What attracted me from your video hh is that you have 3 kids , this is a great man God bless your family

[@sidharthkoparde9183]

Really cool project with an email feature in the end! Thanks, Alex.

[@pfguo87]

By any chance was there a part 2 to this with the more advance scraping? Would love to see that :)

[@kyleluckritz2077]

Looking for the part 2 you mentioned in the vid!! Thanks

[@logenninefinger3420]

really cool, thank you for sharing!

[@MrAdhito]

I really like these long videos where you explain things like this instead short video, thanks for uploading Alex !

[@AlexTheAnalyst]

Glad to hear it! I try to change it up every so often :)

[@sharon_ng]

Hey Alex, thanks for the walkthrough. When is the next web scraping project coming? I'm so hyped.

[@aminmessioud9760]

Looking forward for that too

[@kamogelokhumalo4792]

Thanks a lot Alex, I have learned a lot from your channel. Please keep on posting

[@vintage-cartoons]

Really enjoyed this video! Any update on when the one for multiple pages would be ready? I didn't see it on your channel

[@thisissam504]

Hey Alex.....Thank You For teaching us ......

[@nidhigupta7606]

Thanks again for sharing this video.

[@ankitpandya2476]

nice Alex ,its Helpful Video ,Thank u So Much

[@apaumichael954]

Thanks for this Alex..

[@foodmaker5771]

Did you uploaded the second part. I loved this one. Please share second one

[@huangchou-ho5718]

Thank you so much Alex! I learned a lot from your videos and built my own portfolio. I decided to change my career to data analyst and start creating my resume. But I don't have data analyst experience, could you advise how to create a new resume with portfolio for someone wants to change career? should I focus on the projects?

[@IrfanKhan-eh7wu]

Alex Thanks a lot for this wonderful tutorial, I really learn things from it, can you mention in any of your videos how we can pick colors as well from any product details if it is available in more than one color. A prompt reply will be highly appreciated. Thanks

[@heartbrekid92]

nice video Alex ! Thank you

[@Mehendi_boutique96]

was waiting for this😍😍😍

[@patrickbarchok7741]

I am so grateful for finding you. Almost feels like I know you personally. I'm still very new to this whole Data Analytics but I'm learning a lot. A quick question: I'm on the Google analytics course by coursera and the language is R. Any ideas on where I can learn python- preferably in a structured way that is beginner friendly? Again thank you for the work. Truly amazing.

[@AlexTheAnalyst]

I’m so glad to hear that! I honestly will ups do some KZbin to just get the hang of it - then I would check out my Udemy Course recommendations in the description below - those are ones I’ve taken and loved. That would be my next step. Thanks for watching! 😁

[@danramirz]

Very nice 👍 that be good for checking the prices on udemy courses. 😅

[@reneepaz8077]

I am also taking the Google Analytics course, one question that I would like to ask is, how do you know or prevent bias from the data being collected?

[@nickmoritz1515]

One thing I'd like to point out here is you can easily switch from R to Python. There are plenty of courses out there like Alex had mentioned but the key take away from the course which I did finish and landed a data analysis job, is make sure when following Alex watch how he uses 'pandas' and other packages, which is essentially the same as the tidy-verse in R. Look at the packages and how he writes the code. I think that will help you out the most on top of taking courses.

[@ermano5586]

@@nickmoritz1515 Hey man! How is it going? Can you share some tips that helped you to land a Data Analyst job, maybe there is additional stuff, I've almost finished Alex's data analyst bootcamp and pre-graduate cource bachelor student. I would be grateful for sharing, please?

[@ramon421]

I liked that email thing. I've used Scrapy in the past, but beautiful soup seems waaay easier. I'll create a cool project to watch prices for me :D

[@krystlestevens2585]

Unfortunately it no longer works (due to Amazon website update I believe, as others have commented) :/ would love another scraping video so I can learn!! Love all the videos Alex and thanks so much!😊

[@krystlestevens2585]

@@nezzylearns happy to help

[@VishalSharmaOfficialVS]

Were you able to bypass the Amazon scraping detection? I am also receining NoneType error.

[@krystlestevens2585]

@@VishalSharmaOfficialVS I unfortunately wasn't able to figure it out :/ This is one of the harder projects (to me) so I was going to circle back after going through the rest of Alex's projects. If you figure out how to bypass it plz comment here with an update!

[@VishalSharmaOfficialVS]

@@krystlestevens2585 sure! I’m working on it. As soon as I have a concrete solution, I will post it here. Thanks for your reply.

[@adisonfryman144]

@@VishalSharmaOfficialVS did you ever figure this out?

[@henriquealmeida3045]

Hi Alex, great work as always! Are you still going to make the second Web Scraping video?

[@djmariohh]

Hi Alex, this was really great! Thanks for doing this video. Did you ever do the follow-up video that was mentioned at the end of this video?

[@sim9955]

i'm interested in that follow up as well!!

[@markchinwike6528]

Thank you so much, Alex! Your teaching style has made learning incredibly enjoyable and accessible. I've learned a lot in just one month and completed my portfolio projects, even though I skipped Excel and Power BI for now. Your anecdotes about your dog, family, and personal experiences have added a fun touch to the learning process. Your impact on learners like me is undeniable, and I'm looking forward to purchasing a course from your website soon. Keep up the fantastic work! 🥂🥂

[@SS-ie1od]

Thank you for all you do! Your videos are inspirational and extremely helpful. Will you be working on any Python interview example videos soon?

[@ArtAndTechWithNicolas]

Great video, thank you very very much.

[@aayanmalhotra1104]

One of the only channels with least haters ✨

[@AlexTheAnalyst]

I wish I had more so I could be cool

[@mdoz]

Great Video! really helpful. 👍

[@badissjabou252]

hello Alex? Thank you for this amazing tuto, it help me so much please did you do the video for all the website like how to scrape all the page thank you again

[@gilltim5711]

What browser are you using to pull up that Inspect/Elements method? I've tried it in Firefox and Explorer, and I don't get that window with either. I'm not getting the individual pieces on the page highlighting as I move my cursor over the code, but all that happens is the full page highlights, not the individual pieces. What am I doing wrong?

[@PratapO7O1]

Oh man I was thinking about a project related to amazon data scraping and here youtube suggested me B-)

[@AlexTheAnalyst]

Hope it helps!

[@PratapO7O1]

@@AlexTheAnalyst Yes it was, Thank You :-)

[@vish949]

this content is insane and im go glad i found your channel! I was wondering if you would like to create a video around how to login to a website with credentials (eg: netflix or youtube) and then extract any csv files from that site?

[@krahul456]

Wholesome Content !!! Thanks a Lot !!

[@HenryLinMusic]

Hey Alex , quick question - when you do a requests.get, why do we need to have the header? Thanks!

[@pocketsfullofdynamite]

Very helpful. Also a request, could you show a way where a login is needed to get to the contents. Maybe some API thing.

[@manishkumarsingh5982]

When will be the next part of web scraping? Thank you for posting this video!!!!!

[@AbdouChaabani]

Thanks bro great work Are there any further complex projects with pagination and those stuffs .

[@sunrajsharma736]

It was a fun project. Please drop the other version (the complete version) of this project @Alex The Analyst

[@okonkwolouis902]

Hello Alex, thank you so much for all you do. I am using this video now when Amazon doesn't include the word 'price'. While inspecting how do I go about that. I hope you reply because I am sure lot of new learners are having this issue.

[@pangtoubear6923]

Very nice tutorial! The amazon seems to change the code on the id = priceblock_ourprice part, could you update the code accordingly?

[@JorgeCortes-eh3qf]

thank you for everything!

[@AlexTheAnalyst]

You're most welcome Jorge!