Amazon Web Scraping Using Python | Data Analyst Portfolio Project

  Рет қаралды 284,631

Alex The Analyst

Alex The Analyst

Күн бұрын

Take my Full Python Course Here: bit.ly/48O581R
Web Scraping isn't just for those fancy "programmers" and "software developers". Us analysts can use it too! In this project I walk through how to scrape data from Amazon using BeautifulSoup and Requests.
LINKS:
Code in GitHub: github.com/Ale...
Anaconda: www.anaconda.c...
Find Your User-Agent: httpbin.org/get
____________________________________________
SUBSCRIBE!
Do you want to become a Data Analyst? That's what this channel is all about! My goal is to help you learn everything you need in order to start your career or even switch your career into Data Analytics. Be sure to subscribe to not miss out on any content!
____________________________________________
RESOURCES:
Coursera Courses:
Google Data Analyst Certification: coursera.pxf.i...
Data Analysis with Python - coursera.pxf.i...
IBM Data Analysis Specialization - coursera.pxf.i...
Tableau Data Visualization - coursera.pxf.i...
Udemy Courses:
Python for Data Analysis and Visualization- bit.ly/3hhX4LX
Statistics for Data Science - bit.ly/37jqDbq
SQL for Data Analysts (SSMS) - bit.ly/3fkqEij
Tableau A-Z - bit.ly/385lYvN
Please note I may earn a small commission for any purchase through these links - Thanks for supporting the channel!
____________________________________________
SUPPORT MY CHANNEL - PATREON/MERCH
Patreon Page - / alextheanalyst
Alex The Analyst Shop - teespring.com/...
____________________________________________
Websites:
GitHub: github.com/Ale...
____________________________________________
All opinions or statements in this video are my own and do not reflect the opinion of the company I work for or have ever worked for

Пікірлер: 661
@BummerSlug
@BummerSlug 3 жыл бұрын
The real talk is nice. “It took ten hours over two weeks”. These are things people need to hear. Some people watch these videos on YT and think it is just that easy. This is why your channel is on my short list of channels I subscribed to. Thanks for all your time on these. Hey MS Excel - sponsor this channel!
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
I try to make it as realistic as possible - I used to think people could do this all off the top of their heads and I would get discouraged. Glad to hear that! :D
@pkabir4625
@pkabir4625 Жыл бұрын
@@AlexTheAnalyst For the same product, I couldn't find the id for price...it shows div class...what to do?
@ayoni02
@ayoni02 Жыл бұрын
This should work if you tweak it well enough
@samosagaming9476
@samosagaming9476 Жыл бұрын
@@pkabir4625 go little bit up you will find id but you have to use strip function and [1:4] or insert the values as per your requirement to get the exact vales. this worked for me
@kuldeeprana7659
@kuldeeprana7659 Жыл бұрын
it didnot work for me , not showing price id ,its in span tag
@kishwaralamgir4468
@kishwaralamgir4468 2 жыл бұрын
Alex is so honest and down to earth, he doesnt have that usual KZbinr vibe that we are accustomed to. Man we're so lucky to find you as a mentor.
@AlexTheAnalyst
@AlexTheAnalyst 2 жыл бұрын
That means a lot! Thanks for watching! :D
@pulakkabir2276
@pulakkabir2276 Жыл бұрын
@@AlexTheAnalyst hi. how to find the code you showed right of the 't-shirt' web page?..you selected price...then the code for price got selected. how to do that?
@mulikinatisiddarthasiddu8245
@mulikinatisiddarthasiddu8245 2 ай бұрын
​@@pulakkabir2276 right click and click in inspect or use ctrl+shift+i
@nevinphilbert
@nevinphilbert 8 ай бұрын
man i've been battling with the bot blocker from amazon and also some scraping issues with price because the website display was changed a while after this video was uploaded, but I've managed to pull it off so i hope this might help those recent viewers who might be as confused as me when I started writing this code on my own. apparently you need to divide the second cell so you need to run the soup1 first before you run the soup2, then for the price you need to pull three part of span class=a-price-symbol, span class=a-price-whole, and span class=a-price-fraction and combine it together into 1 new variable (price), then you need to clean it using strip() and replace() to clean the whitespaces and 's. hope this helps!
@deeplakshmiyadav
@deeplakshmiyadav 8 ай бұрын
Brother please elaborate it, I am stuck
@ritiknegi8596
@ritiknegi8596 7 ай бұрын
Hey bro can you explain it or can you share your code, how you pulled three part together. I am stuck in this part
@sdivi6881
@sdivi6881 7 ай бұрын
Could you please explain it, I am stuck in getting Title itself
@ritiknegi8596
@ritiknegi8596 7 ай бұрын
@@sdivi6881 Hey i just solved it can you tell little more where you are getting stuck
@ShortClipsPodcasts
@ShortClipsPodcasts 7 ай бұрын
@@deeplakshmiyadav price_symbol = soup2.find(class_='a-price-symbol').get_text(strip=True) price_whole = soup2.find(class_='a-price-whole').get_text(strip=True) price_fraction = soup2.find(class_='a-price-fraction').get_text(strip=True) price = f'{price_symbol}{price_whole}{price_fraction}' print(price)
@abdouelkafrawy1726
@abdouelkafrawy1726 Жыл бұрын
it's been a year on this project and despite me searching and watching other channels, I always come back to your channel ,you are simple the best person I have learned from . you are genuine and always able to get your point across .I hope you expand your "python for data analysis" series just like you did with SQL. Thank you so so much .
@atanasprodatanasov9233
@atanasprodatanasov9233 3 жыл бұрын
The section where you speak about how you shouldn't know this by heart is so good. Honestly... I am learning SQL as per your recommendation, but in the back of my head I am scared as I think I should learn and memorize each single block of code... And this is awful... Thank you for being honest and clear on that!
@BadWeatherfreak
@BadWeatherfreak 2 жыл бұрын
How is it going?
@enestemel9490
@enestemel9490 3 жыл бұрын
Hey Alex quick tip: When you were working on spaces like 34:21 if you select everything that you wanna move and press tab this way everything you choose goes 1 tab right. Little things like that improve your quality of life sometimes. Thanks for the tutorial :)
@amelieguzman9639
@amelieguzman9639 Жыл бұрын
You don't imagine how this tutorial has helped me in my new position. Thank you so much!!
@AlexTheAnalyst
@AlexTheAnalyst Жыл бұрын
So glad it was helpful! :D
@adisonfryman144
@adisonfryman144 Жыл бұрын
@@AlexTheAnalyst Did you ever make the second one? So many people want to see it? Please do send it out!
@beyond1994
@beyond1994 7 ай бұрын
This project gave me a taste of how challenging web scraping is. Great video that makes things look easy and less intimidating.
@newyork397
@newyork397 2 жыл бұрын
As others described, if you get an error when running the second cell it's probably due to a captcha issue where amazon thinks you are a bot. You can force it by pressing ctrl + enter again and again until you get an output. I'm sure there is a better way to get around this but that's the quickest semi solution I found.
@newyork397
@newyork397 2 жыл бұрын
The while loop definitely doesn't work now that Amazon updated their website with some sort of blocker for bots. It might work a few times but eventually stops running in background.
@sj1795
@sj1795 10 ай бұрын
Hi Alex, I really appreciate how you shared how long this project actually took you. It helps to know the difference between what we go through on your channel and the work/time it actually takes behind the scenes. AWESOME project! I learned tons and found all of it very useful/helpful. You are such an AMAZING teacher and resource! As always, THANK YOU!!
@valadhruv6920
@valadhruv6920 8 ай бұрын
Bro did you got nonetype error and how you solved it?
@abdelrahmanmalekgenedy8751
@abdelrahmanmalekgenedy8751 8 ай бұрын
I have this error bro, and don't how to solve it @@valadhruv6920
@pradeep1900
@pradeep1900 3 ай бұрын
@@valadhruv6920 bro, did u found the solution for this? i can't figure it out
@DustInTheAir
@DustInTheAir 2 жыл бұрын
Hey Alex! Thanks for this helpful video! The best part of this video is whenever you said 'I don't know what that is' (12:50) , instead of some difficult theory. You don't know that, I don't know neither, so it makes me feel less pressure on learning python...
@shyamss2338
@shyamss2338 2 жыл бұрын
Hi Alex, seems like this code is now not working. Would be grateful if you could do another web scraping project with EDA analysis? Love how instructive your videos are.
@reshuraj5005
@reshuraj5005 Жыл бұрын
you're already doing a great job man. Thanks a ton, and hats off to you. But, We need that part 2. Please do it asap Alex.
@user-gq1ij
@user-gq1ij 2 жыл бұрын
Sir, I am very near to get my first job through your project Thank you And this is also my first project
@patcharaphanjearsavaswatta6149
@patcharaphanjearsavaswatta6149 Жыл бұрын
Thank you for this, Alex. I felt so happy when I finally could scrap the website I had been trying to scrap (I applied your teaching to another website). Really appreciate your work.
@podawe8051
@podawe8051 3 жыл бұрын
So great Alex! I followed along with this entire project and added it to my portfolio! I'll be sure to give you credit in my README file. :)
@yousseftaki1949
@yousseftaki1949 2 жыл бұрын
What attracted me from your video hh is that you have 3 kids , this is a great man God bless your family
@krystlestevens2585
@krystlestevens2585 Жыл бұрын
Unfortunately it no longer works (due to Amazon website update I believe, as others have commented) :/ would love another scraping video so I can learn!! Love all the videos Alex and thanks so much!😊
@krystlestevens2585
@krystlestevens2585 Жыл бұрын
@@nezzylearns happy to help
@VishalSharmaOfficialVS
@VishalSharmaOfficialVS Жыл бұрын
Were you able to bypass the Amazon scraping detection? I am also receining NoneType error.
@krystlestevens2585
@krystlestevens2585 Жыл бұрын
@@VishalSharmaOfficialVS I unfortunately wasn't able to figure it out :/ This is one of the harder projects (to me) so I was going to circle back after going through the rest of Alex's projects. If you figure out how to bypass it plz comment here with an update!
@VishalSharmaOfficialVS
@VishalSharmaOfficialVS Жыл бұрын
@@krystlestevens2585 sure! I’m working on it. As soon as I have a concrete solution, I will post it here. Thanks for your reply.
@adisonfryman144
@adisonfryman144 Жыл бұрын
@@VishalSharmaOfficialVS did you ever figure this out?
@simoyland
@simoyland Жыл бұрын
Always thank you for all your efforts and good work! I love watching your videos. Your positive attitude and way of expression make the lesson even more fun. I've seen a few people say the video is too long, but I think being able to walk through the lesson together rather than other videos that show written code is much better for learning. Thank you thank you thank you ☺
@shawonalam6780
@shawonalam6780 2 ай бұрын
16:30 Solution to get the price: price_symbol = soup2.find(class_='a-price-symbol').get_text(strip=True) price_whole = soup2.find(class_='a-price-whole').get_text(strip=True) price_fraction = soup2.find(class_='a-price-fraction').get_text(strip=True) price = f'{price_symbol}{price_whole}{price_fraction}' print(price)
@TSKYTOT
@TSKYTOT Ай бұрын
Thanks bro!
@garry6882
@garry6882 26 күн бұрын
finally seemes like they already changed the html. thanks bro
@AbhishekAbhishek-yn6tj
@AbhishekAbhishek-yn6tj 23 күн бұрын
but pricing in the website is $16.99 but after executing this code it is $12.97.
@antoninarudnytska596
@antoninarudnytska596 9 күн бұрын
Thank you a lot!!!)
@abhinandc5018
@abhinandc5018 3 жыл бұрын
The long awaited one ❤️💯
@pfguo87
@pfguo87 Жыл бұрын
By any chance was there a part 2 to this with the more advance scraping? Would love to see that :)
@kyleluckritz2077
@kyleluckritz2077 Жыл бұрын
Looking for the part 2 you mentioned in the vid!! Thanks
@jeisonsanchez4842
@jeisonsanchez4842 2 жыл бұрын
29:30 - quick tip: select the file, hold shift and right-click to get “copy as path” in the context menu.
@malishagavali9225
@malishagavali9225 Ай бұрын
OMG! i finally found this video after years, i recently opened this project i worked on a year ago and it popped up with some errors, now i can finally track back
@hafsahshazmeen1446
@hafsahshazmeen1446 Жыл бұрын
This man a God send gift to ALL the Broke data analyst students
@jachimmachristian457
@jachimmachristian457 Жыл бұрын
No kidding
@horacezhen
@horacezhen 3 жыл бұрын
Thank you for demonstrating! I never thought that a simple project like this could use as a portfolio project. I just realized that I have what it takes to become a DA. Thank you for demonstrating projects!
@mitchellharding305
@mitchellharding305 3 жыл бұрын
Super early, love your stuff as always Alex!
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
You are very early! Lol Thanks for watching 😁
@SayedKassem-pn7tr
@SayedKassem-pn7tr Жыл бұрын
Your tutorials are so good. and i follow you on LinkedIn, your content is awesome. i love how you explain things in a clear way. keep up the great job!!
@MrAdhito
@MrAdhito 3 жыл бұрын
I really like these long videos where you explain things like this instead short video, thanks for uploading Alex !
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
Glad to hear it! I try to change it up every so often :)
@kaleabgirma-x2b
@kaleabgirma-x2b 6 ай бұрын
Hi Alex, I have learn a lot from 65 videos of the Bootcamp. God bless with everything. Thanks!!!
@Philip_thinkData
@Philip_thinkData 5 ай бұрын
Wow, this is EXACTLY what I have been looking for. Alex the GOAT in DA. :) You are 1000x Awesome!
@fernandamello837
@fernandamello837 Жыл бұрын
Wow!!! This is awesome!!! You have so easy way to teach, I already have a base with Python but I’ve never made this before and you make this so smother and easy to do!!!! Thank you thank you ❤
@NathanEckelTV
@NathanEckelTV 2 жыл бұрын
Thrilled to successfully get to the end of this @Alex - appreciate these real-world worked examples.
@aayanmalhotra1104
@aayanmalhotra1104 3 жыл бұрын
One of the only channels with least haters ✨
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
I wish I had more so I could be cool
@tharindusamarakoon8831
@tharindusamarakoon8831 3 жыл бұрын
Can you please make a video on how to present these projects? I've seen your video about the portfolio website, but I don't have an idea on how to actually present the github.. And thank you very much. Your channel has been very inspirational to me through out my learning journey!
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
Good idea!
@tharindusamarakoon8831
@tharindusamarakoon8831 3 жыл бұрын
@@AlexTheAnalyst Hi Alex, to further add to my comment - I've taken a look at other "best example" portfolios online but comparing it to the Google data analytics portfolio guidelines, they are very different. Hence my conflict and lack of general understanding on how to present these projects in a website. Thank you.
@kendrasdustyroad
@kendrasdustyroad Жыл бұрын
Dude! I'm an amazon seller and this kind of work would come in super handy. Thank you. Did you ever get around to making the next video where you pull data from all the search results page? I'd be really interested to see that one.
@mercantilism954
@mercantilism954 3 жыл бұрын
This is what I have been waiting for! Thank you
@markchinwike6528
@markchinwike6528 7 ай бұрын
Thank you so much, Alex! Your teaching style has made learning incredibly enjoyable and accessible. I've learned a lot in just one month and completed my portfolio projects, even though I skipped Excel and Power BI for now. Your anecdotes about your dog, family, and personal experiences have added a fun touch to the learning process. Your impact on learners like me is undeniable, and I'm looking forward to purchasing a course from your website soon. Keep up the fantastic work! 🥂🥂
@Kaura_Victor
@Kaura_Victor 4 ай бұрын
Though the project was quite tricky, I got over it. Thank you so much Alex.
@Andrei_Neo
@Andrei_Neo Жыл бұрын
Hello Alex! One more step is done!!! It's so exciting, I got stuck at the stage where I had to get price data. I missed this metric to be scraped. Since the time you recorded this video some parts of html have been updated. So now price does not exist in the format of "ID=", it lives now as "div class=". So now it is challenging for to find out how to scrap the price though :))) will go deeper to the topic. Thanks much for your time and for sharing of your knowledge.
@AIdevel
@AIdevel Жыл бұрын
I am absolutely fascinated by your thorough explanation
@AlexHitchon
@AlexHitchon Жыл бұрын
Thanks Alex I am working on my own web scraping project for checking placements of searches and this video definitely helped
@pushpitjain7208
@pushpitjain7208 2 жыл бұрын
Hey Alex! It was a super helpful video. Thank you so much for posting it. Have you uploaded the next part of this video. If yes, Please share the link.
@aadhithyanp6989
@aadhithyanp6989 3 жыл бұрын
Thanks man. You are helping a lot of people like me. Keep doing this portfolio videos!
@esonon5210
@esonon5210 3 жыл бұрын
Thanks for sharing! This is an awesome video. I'm not sure if you did this but I think it would be cool to learn how to scrape multiple pages then append the data in a def function.
@anthonyyawa2445
@anthonyyawa2445 2 жыл бұрын
Great video alex ... it was really helpful for a module in my course . Please i have been looking for the intermediate video you spoke about
@patientson
@patientson 2 ай бұрын
You have been consuming alot of sugary, oily and salty meals in spring and summer when you embarked on this project. I have a living remedy for you. The treatment to what is causing you to clear your throat is lack of longsuffering walks and eating the right food at the wrong time and season, coupled with nothing doing the right thing with food in their seasons. In the world right now, only a handful of people know this knowledge and they are definitely not Europeans nor Americans.
@blackwhite2361
@blackwhite2361 3 жыл бұрын
I thought you were now only gonna only make videos on management and stuff. Glad you are still making tutorials
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
Nah, content really won't change much - I'll be doing Tableau tutorials very soon
@ramakumarguntamadugu1299
@ramakumarguntamadugu1299 2 жыл бұрын
thanks Alex! really a great video... request you to kindly do a similar one on stocks realtime price capturing with time series and configure an email notification when the current price drops below say 50 day moving average .... I am looking forward for many videos like this...thank you!
@lawrencelawliet5016
@lawrencelawliet5016 2 жыл бұрын
If anyone else has a problem like I did with getting a captcha output when printing soup2, I solved it by putting soup2 and the print statement in a different cell then run the first cell with soup1 then run the second cell with soup2 and the print statement separately.
@patrickbarchok7741
@patrickbarchok7741 3 жыл бұрын
I am so grateful for finding you. Almost feels like I know you personally. I'm still very new to this whole Data Analytics but I'm learning a lot. A quick question: I'm on the Google analytics course by coursera and the language is R. Any ideas on where I can learn python- preferably in a structured way that is beginner friendly? Again thank you for the work. Truly amazing.
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
I’m so glad to hear that! I honestly will ups do some KZbin to just get the hang of it - then I would check out my Udemy Course recommendations in the description below - those are ones I’ve taken and loved. That would be my next step. Thanks for watching! 😁
@danramirz
@danramirz 3 жыл бұрын
Very nice 👍 that be good for checking the prices on udemy courses. 😅
@reneepaz8077
@reneepaz8077 3 жыл бұрын
I am also taking the Google Analytics course, one question that I would like to ask is, how do you know or prevent bias from the data being collected?
@nickmoritz1515
@nickmoritz1515 3 жыл бұрын
One thing I'd like to point out here is you can easily switch from R to Python. There are plenty of courses out there like Alex had mentioned but the key take away from the course which I did finish and landed a data analysis job, is make sure when following Alex watch how he uses 'pandas' and other packages, which is essentially the same as the tidy-verse in R. Look at the packages and how he writes the code. I think that will help you out the most on top of taking courses.
@ermano5586
@ermano5586 Жыл бұрын
@@nickmoritz1515 Hey man! How is it going? Can you share some tips that helped you to land a Data Analyst job, maybe there is additional stuff, I've almost finished Alex's data analyst bootcamp and pre-graduate cource bachelor student. I would be grateful for sharing, please?
@asutoshnayak1391
@asutoshnayak1391 3 жыл бұрын
Alex please make a video upon how to create your own Data set for data analyst job ? Please make
@foodmaker5771
@foodmaker5771 Жыл бұрын
Did you uploaded the second part. I loved this one. Please share second one
@kevindeschepper8140
@kevindeschepper8140 4 ай бұрын
if you dont pull in the data due to the captcha, dont use the headers as second argument.
@okonkwolouis902
@okonkwolouis902 Жыл бұрын
Hello Alex, thank you so much for all you do. I am using this video now when Amazon doesn't include the word 'price'. While inspecting how do I go about that. I hope you reply because I am sure lot of new learners are having this issue.
@syaoransakura8839
@syaoransakura8839 3 жыл бұрын
After a looooong time delay cause by many things, finally I can finish this portfolio.
@michelchaghoury870
@michelchaghoury870 2 жыл бұрын
Mannnn pleaseeee keep going we need your help you tuts are on a whole diff level I am able to learn and understand with ease tnx a lotttttt and once again keep going
@Jj-th8vl
@Jj-th8vl Жыл бұрын
Hey Alex, first I want to thank you for this amazing series and everything you do to help the community. Second, I am working on this project and it seems Amazon implemented a CAPTCHA to prevent scrapping. Is there any way around this? Would love to know if this project is applicable and doable even 2 years later. Cheers!
@ChristinaRichardsonFitness
@ChristinaRichardsonFitness 11 ай бұрын
Yeah. I have been trying for 2 hours to get into Amazon. I think it is a bit more difficult now. Were you able to find a way?
@shriyakenkre1474
@shriyakenkre1474 10 ай бұрын
Having the same problem.
@MindOfAlpo
@MindOfAlpo 9 ай бұрын
Yeah, same here. There are ways to bypass but it looks like it might be borderline unethical. Zenscrape, Apify, or ScraperAPI give you the ability to fetch the data directly from the API instead of the HTML page(beautiful soup).
@abdallahibrahim3864
@abdallahibrahim3864 9 ай бұрын
if you are running into an issue with the header, try this: headers = {"User-Agent": ".......", "Accept-Encoding": ".....", "Accept": "......" } just put in whatever you get from the User-Agent link in the video description
@pangtoubear6923
@pangtoubear6923 Жыл бұрын
Very nice tutorial! The amazon seems to change the code on the id = priceblock_ourprice part, could you update the code accordingly?
@ayubindra3702
@ayubindra3702 2 жыл бұрын
Hello Alex, thanks for sharing, I have found the error for my code of this section title = soup2.find(id='productTitle').get_text() print(title) output: 'NoneType' object has no attribute 'get_text' Please I need your advise
@gorand8842
@gorand8842 2 жыл бұрын
same here
@cocojamborambo5435
@cocojamborambo5435 2 жыл бұрын
Hello! You can do this: page = requests.get(url,headers=headers) soup= BeautifulSoup(page.content, "html.parser") and then get your data: title = soup.find(id='productTitle').get_text().strip() price = soup.find('span',class_='a-offscreen').get_text().replace("$","") You don't need prettify anymore as your computer can easely read that
@janinepino5291
@janinepino5291 2 жыл бұрын
@@cocojamborambo5435 Weirdly enough, this works for one moment, and then it stops when I run it again.
@anonymous-3720
@anonymous-3720 Жыл бұрын
Same.
@barkamolurinboev6311
@barkamolurinboev6311 Жыл бұрын
You edit to headers and add 'Referer'
@RyanLoh
@RyanLoh 2 жыл бұрын
When I try to print the title im getting an error message "'NoneType' object has no attribute 'get_text''. What is the issue here?
@youthsrits9227
@youthsrits9227 2 жыл бұрын
Same
@sharon_ng
@sharon_ng 3 жыл бұрын
Hey Alex, thanks for the walkthrough. When is the next web scraping project coming? I'm so hyped.
@aminmessioud9760
@aminmessioud9760 2 жыл бұрын
Looking forward for that too
@nezerm2363
@nezerm2363 5 ай бұрын
"AttributeError: 'NoneType' object has no attribute 'get_text'" to solve this delete headers
@shubhamjha2017
@shubhamjha2017 3 ай бұрын
still showing the same error
@Kshitij-Yadav
@Kshitij-Yadav 2 ай бұрын
It worked
@mulikinatisiddarthasiddu8245
@mulikinatisiddarthasiddu8245 2 ай бұрын
​@@Kshitij-Yadav can you elaborate it bro
@Kshitij-Yadav
@Kshitij-Yadav 2 ай бұрын
@@mulikinatisiddarthasiddu8245 sure man If u get the above error Go to the place where u entered the URL Then headers delete the header So it'll be URL page = request.get(url) Soup1 This should fix it If u have any other issues lemme know I just finished this code so I went through everything I'll share it
@akshaybohra855
@akshaybohra855 2 ай бұрын
@@mulikinatisiddarthasiddu8245 price_symbol = soup2.find(class_='a-price-symbol').get_text(strip=True) price_whole = soup2.find(class_='a-price-whole').get_text(strip=True) price_fraction = soup2.find(class_='a-price-fraction').get_text(strip=True) price = f'{price_symbol}{price_whole}{price_fraction}' try this it will work if you are getting the error to get the price
@sunyoungjang7810
@sunyoungjang7810 2 жыл бұрын
I'd like to appreciate you for sharing this wonderful video! Thanks to you, I've just managed to make my own webscraper that helps me to save so much time. Otherwise, my coworker and I would have to spend more than 6hours per week😂
@bryan.canelas
@bryan.canelas 2 жыл бұрын
Thanks Alex! this was really useful. I am waiting for the second part with the pagination 😅😅
9 ай бұрын
Help im really having issues with "AttributeError: 'NoneType' object has no attribute 'get_text'" I have tried everything I could think of, how can I resolve this.
@qlintdwayne9044
@qlintdwayne9044 9 ай бұрын
experiecing the same. Used an if loop to determine whether the title exists, apparently it doesn't
@abrilgonzalez7892
@abrilgonzalez7892 7 ай бұрын
Same here !!! I'M STUCKED IN MINUTE 16:35, the code doesn't work :( ¿Could someone help us please? Thanks you!!
@yashwanthgunturi8762
@yashwanthgunturi8762 6 ай бұрын
did you find a fix? I am having the same problem.
@Reyhanemohamadinejad
@Reyhanemohamadinejad 6 ай бұрын
@@yashwanthgunturi8762 Also for 'title ', you can type the text below after soup's definition: title = soup.find(id='productTitle').get_text() print(title)
@varunkhandelwal623
@varunkhandelwal623 4 ай бұрын
@@yashwanthgunturi8762 yeah I found the error use this code title = soup2.find(class_='a-size-large product-title-word-break').get_text(strip=True)
@ZzZzZiggy
@ZzZzZiggy 7 ай бұрын
Hey, Alex. Did you ever go through with the more complicated follow-up project? Mutiple products listed on multiple pages? 🤔
@sunrajsharma736
@sunrajsharma736 Жыл бұрын
It was a fun project. Please drop the other version (the complete version) of this project @Alex The Analyst
@juandavidmoralesnavarrete3072
@juandavidmoralesnavarrete3072 Жыл бұрын
This seems like an amazing project, sadly something change in Amazon policy for scrapping their data and I couldn't access, if someone find a way to make it work I would love to hear it 😁I'll keep going with the other projects!!
@foodamormore
@foodamormore Жыл бұрын
so whoever is not able to find the id for the price and are getting tag 'span' and class on clicking on the price(mentioned on the website of the product) in inspect can follow this code price=soup2.find("span", attrs={'class':'a-price-whole'}).text.strip() print(price) replace 'a-price-whole' with whatever you are getting for the class
@ejidohugochi2858
@ejidohugochi2858 Жыл бұрын
Where you able to access the site at all? I did get an error 503 right from the beginning while trying to request the url. However I decided to use my selenium web drivers and it worked for me…. If you don’t know how to use that then I suggest you scrap another website. Amazon has gotten tighter.
@tevzuss4386
@tevzuss4386 Жыл бұрын
You can use Selenium to bypass all of their human checks. But it's a bit more advanced subject.
@yashwanthjetty7845
@yashwanthjetty7845 3 жыл бұрын
Thanks a lot for enlightening on Web Scraping. Came to know only after watching this video that such stuff can be done.
@sidharthkoparde9183
@sidharthkoparde9183 3 жыл бұрын
Really cool project with an email feature in the end! Thanks, Alex.
@andrespatricioespinozadelg4490
@andrespatricioespinozadelg4490 2 жыл бұрын
It was a lot of fuuuuuun! Thank you Alex. Your channel has became one of my favorite about Python and SQL 🤓
@AlexTheAnalyst
@AlexTheAnalyst 2 жыл бұрын
So glad to hear that! :D
@zebaszd
@zebaszd 3 жыл бұрын
man it was super easy to understand, you nailed it
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
So glad to hear it!
@neildelacruz6059
@neildelacruz6059 Жыл бұрын
Simply the best video, thanks Alex.
@manishkumarsingh5982
@manishkumarsingh5982 3 жыл бұрын
When will be the next part of web scraping? Thank you for posting this video!!!!!
@bernardnader
@bernardnader 9 ай бұрын
Thanks so much Alex!! Just what I was looking for.
@litoavila.
@litoavila. Жыл бұрын
You have 3 kids! and you are 28, omg you are back as my old wise sensei, master yoda.
@abdallahibrahim3864
@abdallahibrahim3864 9 ай бұрын
If anyone sees this, as of Jan 2024 this does not work because of Amazon. Sorry for everyone trying to do this project and for Alex's time going to waste
@bud384
@bud384 7 ай бұрын
The structure of a website may've changed over 2 years, so it doesn't mean that won't work anymore but you should find out the exact web element to use to extract the data
@Mehendi_boutique96
@Mehendi_boutique96 3 жыл бұрын
was waiting for this😍😍😍
@김제이-n3d
@김제이-n3d 2 жыл бұрын
This script is only giving me short html and I've got "NonType" value at the end.
@oriakuotito3985
@oriakuotito3985 Жыл бұрын
Hi Alex, still waiting for the other video. thank you
@djmariohh
@djmariohh 2 жыл бұрын
Hi Alex, this was really great! Thanks for doing this video. Did you ever do the follow-up video that was mentioned at the end of this video?
@sim9955
@sim9955 2 жыл бұрын
i'm interested in that follow up as well!!
@austenbach4987
@austenbach4987 8 ай бұрын
I'm watching this in 2024 working on learning all the tools to become a Data Analyst and just realized that the price of the shirt never went up since inflation. I might need to buy myself one now lol
@shashwatpaul3330
@shashwatpaul3330 2 жыл бұрын
Its a great video. You have mentioned in the beginning of this video that you will publish another video to web scrape all the products listed over 20 pages on Amazon. Have you published it yet? If yes, please provide me with the link. I could not find it in your channel.
@AlexTheAnalyst
@AlexTheAnalyst 2 жыл бұрын
Pushed it off for other content - I'll definitely get around to doing that one
@kr_eep
@kr_eep 2 жыл бұрын
@@AlexTheAnalyst Can't wait to see that video! Awesome content man, cheers!
@amodeocarlo97
@amodeocarlo97 3 жыл бұрын
Thank you so much Alex for your contents! Do you think you're going to do some projects regarding healthcare or covid 19 data? That'd be great!
@riteshkumar1497
@riteshkumar1497 Жыл бұрын
NoneType' object has no attribute 'get_text'. Iam facing this error
@robertoenriquehiguera2648
@robertoenriquehiguera2648 3 жыл бұрын
Thanks Alex. I’m a big fan.
@sergioramos3437
@sergioramos3437 2 жыл бұрын
Love this .. I'm curious about the headers part I didn't know about that before
@mukoko99
@mukoko99 Жыл бұрын
Can someone direct me to the other project that you discussed doing in this video. Where you build a scrawled that goes through each page’s content and gets their prices
@Magmatic91
@Magmatic91 2 жыл бұрын
Great stuff ! Please make the video where you show how to get for each item let's say the product description, reviews...etc.
@ShortClipsPodcasts
@ShortClipsPodcasts 7 ай бұрын
anyone stuck trying to get the price. price_symbol = soup2.find(class_='a-price-symbol').get_text(strip=True) price_whole = soup2.find(class_='a-price-whole').get_text(strip=True) price_fraction = soup2.find(class_='a-price-fraction').get_text(strip=True) price = f'{price_symbol}{price_whole}{price_fraction}' print(price)
@tranguyen4462
@tranguyen4462 7 ай бұрын
Hi. I am not even able to print(soup1) due to some sort of anti bot blocking from Amazon. do you have any idea how to solve that? Thank you
@Angelo_DS
@Angelo_DS 7 ай бұрын
Thank you!
@justinwelsh7376
@justinwelsh7376 5 ай бұрын
Thank you man, I realized something was different with the html but lack the coding skills to fix it in a timely manner.
@dragusinsilviu4862
@dragusinsilviu4862 5 ай бұрын
thumbs up, i've spend too much time looking for this comment, @Alex can you include that in the bio?
@onitolu9698
@onitolu9698 3 ай бұрын
Thank you so much for this
@dianasatapathy594
@dianasatapathy594 3 жыл бұрын
Hey Alex. When are you posting the next project?
@mphorasenyalo1230
@mphorasenyalo1230 2 жыл бұрын
from the first position, that's the word you were looking for.
@vintage-cartoons
@vintage-cartoons Жыл бұрын
Really enjoyed this video! Any update on when the one for multiple pages would be ready? I didn't see it on your channel
@badissjabou252
@badissjabou252 2 жыл бұрын
hello Alex? Thank you for this amazing tuto, it help me so much please did you do the video for all the website like how to scrape all the page thank you again
@demr5251
@demr5251 3 жыл бұрын
Wonderful! I'll practice with this tonight!
@PratapO7O1
@PratapO7O1 3 жыл бұрын
Oh man I was thinking about a project related to amazon data scraping and here youtube suggested me B-)
@AlexTheAnalyst
@AlexTheAnalyst 3 жыл бұрын
Hope it helps!
@PratapO7O1
@PratapO7O1 3 жыл бұрын
@@AlexTheAnalyst Yes it was, Thank You :-)
@skaigreen
@skaigreen 2 жыл бұрын
this was really helpful... I'm planning on transitioning careers to data analytics and would love to see how it goes.
@erkanlacin4729
@erkanlacin4729 3 жыл бұрын
You are great, this is exactly what is am looking for...
@jaimebaker1806
@jaimebaker1806 2 жыл бұрын
BTW some of the website code amazon has changed. New people will need to adjust accordingly.
Full Beginner Project in Tableau | Tableau Tutorials for Beginners
44:18
Alex The Analyst
Рет қаралды 299 М.
Exploratory Data Analysis with Pandas Python
40:22
Rob Mulla
Рет қаралды 493 М.
Family Love #funny #sigma
00:16
CRAZY GREAPA
Рет қаралды 37 МЛН
🕊️Valera🕊️
00:34
DO$HIK
Рет қаралды 20 МЛН
Trick-or-Treating in a Rush. Part 2
00:37
Daniel LaBelle
Рет қаралды 32 МЛН
The Biggest Issues I've Faced Web Scraping (and how to fix them)
15:03
Automating Crypto Website API Pull Using Python | Data Analyst Project
51:14
I Studied Data Job Trends for 24 Hours to Save Your Career! (ft Datalore)
13:07
Thu Vu data analytics
Рет қаралды 267 М.
How to use a Public API | Using a Public API with Python
14:01
Alex The Analyst
Рет қаралды 47 М.
Scrape Amazon Data using Python (Step by Step Guide)
24:14
Darshil Parmar
Рет қаралды 166 М.
Data Analyst Portfolio Project (Exploratory Data Analysis With Python Pandas)
57:55
Ryan & Matt Data Science
Рет қаралды 57 М.
PROJECTS that landed Data Jobs for my Subscribers
9:33
Luke Barousse
Рет қаралды 311 М.
Family Love #funny #sigma
00:16
CRAZY GREAPA
Рет қаралды 37 МЛН