Web Scraping Amazon Product Data using ChatGPT and Python

#openai #chatgpt #webscraping
If you ask ChatGPT to scrape Product data from amazon .com, it will show you some error.
If you write code yourself and try to scrape amazon , you will get errors from amazon .com saying you need the API service account if you want data.
Amazon will not let you scrape the data.
In this video we will solve both the problems and scrape product data from amazon and export it into a json file.
Error from chatGPT: (0:00)
Error from Amazon: (0:07)
Start: (0:14)
Choose Product category: (0:19)
Step 1: (0:53)
Step 2: (2:15)
Step 3: (2:29)
Ask ChatGPT to write code: (3:39)
Run Python script: (4:07)
Result: (4:11)
Scrape page 2: (4:20)
Result: (4:40)
Scrape page 3: (4:48)
Result: (4:58)
Repeat for other pages: (5:08)
Thank You: (5:25)
Make Money with ChatGPT:
• Earn Money by Building...
Build a Website using ChatGPT in 1 minute:
• Build a Website using ...
Build and Publish VS code extension using ChatGPT:
• Build and Publish VS C...
Let's Run Breadth First Search Algorithm on the Internet using ChatGPT:
• Let's Run Breadth Firs...
Build A Twitter Bot using ChatGPT and Twitter API:
• Biuld Your Twitter Bot...
How To Make Money using ChatGPT and KDP:
• How To Earn Money Usin...
Build a Notepad App using ChatGPT:
• Build Notepad App usin...
Build a Web Browser using ChatGPT and Python:
• Build a Web Browser Us...
Web Scraping using ChatGPT:
• Web Scraping Using Cha...
Build a CSV to JSON converter using ChatGPT:
• CSV To JSON converter ...
How to Pass Linkedin Skill assessment using ChatGPT:
• How to Pass LinkedIn S...
ChatGPT in Top 15% on Linkedin JAvascript Assessment:
• ChatGPT In TOP 15% on...
Subscribe To Our Channel : / @chatgpt-ai

[@lucianocarneiro5962]

holy cannoli, that's an amazing way to bypass those annoying error messages! ChatGPT is the real revolution! tks

[@MHawkinsx]

sounds intriguing! Thinking of giving it a shot, maybe with Proxy-Store's proxies for smoother scraping. Any Python pros here??

[@powray]

Absolutely amazing video bro thank you

[@zeddscarlxrd4331]

this absolutely out of the box bro

[@basiccomponents]

this was really useful, thank you!

[@ChatGPT-AI]

You're welcome!

[@GiantAdrian]

your videos are amazing 😃😃

[@hoplayou]

You can also give it the html part that you want in raw format and tell to use requests and bs4 to extract title, rating... he can understand html

[@ChatGPT-AI]

yes.

[@joslyn4338]

Hi kindly I'll love to create a website and I have similar site that does what I want, my concern is can chatgpt keeps giving the violation message how best to create such this site with right prompt

[@Roughood]

Thank you very much ❤

[@avinashkamble9368]

Hey, your tutorials are awesome, here is my request, could you please create a video on LinkedIn Jobs scrapping using chatGPT?

[@geepytee]

This would be really easy to do!

[@lucianozaffaina9853]

ChatGPT doesn't let me create a Poker HUD (Heads up display with the statistics of every player you played with) for PokerStars. Can you figure out how to make it build a Poker HUD? Thank you for the contents you bring in your videos

[@VaibhavShewale]

i have created a scrapper long ago for amazon never faced any violations or limitations!

[@gusGPT]

It's really awesome!

[@saikatulislam2787]

Amzing

[@lynnqi6451]

Genius!😄💯

[@eduardoferreira8893]

Great!

[@anonim_2023]

Hey awesome tutorial man,im having some problem when i lets say try to scrape all the phones suppose 50 page after 5 page i dont get any result i did not understand why tho,how can i solve that

[@GirishBapatpune]

Chatgpt site is not working for me since couple of days. Any idea when will it get recovered?

[@adamizrailov3167]

Correct me if I'm wrong but this is actually parsing, not web scraping. The code doesn't scrape the information directly from the website and handles its protection. Nonetheless, nicely done.

[@ChatGPT-AI]

may be...but you misunderstood. you can't scrape "directly". when you write regular python code... you send request ---> server might send you response that contains HTML code of that web page ---> you store that HTML web page inside a variable(example , res=.... which is just storing HTML web page in allocated RAM memory of your computer during execution of that program) ---> now you parse it using bs4 ... did you get that? you are not scraping it on the server... it's always on your side. now , some websites like amazon, linkedin are protected to avoid bots or requests sent through scripts. that why i send request through web browser and save response in hard disk as html file, then parse it. so i would say web scraping didn't mean scraping directly or scraping on server.

[@adamizrailov3167]

@@ChatGPT-AI Yes I 100% get it and appreciate the detailed explanation. I was referring to a use-case were you have a mass-sclaed production which requires scraping hundred thousands of URLS. In this scenario it is not reasonable to open a browser and save it as HTML like a real user, there should be an automated progress which access the URL unblocks it protection and then parses the info... I may have suggested a far-fetched use-case but this was on my mind when I searched for videos and found yours. Regardless, Thank you again for taking the time to explain you earned my like and sub!

[@navaneethangn1215]

@@ChatGPT-AI its easy to write an entire script to scrape all pages rather than doing this, I scrapped all pages using selenium and beautifulsoup, didnt get any error though

[@usmannaeem4396]

​@@navaneethangn1215 Hi, can you provide the code to scrape from multiple pages at a single time

[@sabinash5915]

@@navaneethangn1215 explain it bro. I Need that one.

[@chelol208]

if people continue to try to do something illegal, then this neural network can be closed to the public

[@zinebmassine5456]

I only get 9 products, I cant get the data of all the page :(

[@Babbili]

chatGPT lately is acting up, all previous convos are gone,

[@abysmalvsmarcus]

Smarten up and save your docs on word like I did

[@mohamedhouari1148]

Hi babbili

[@frankbrown4316]

Hey Babbili, try accessing from different devices. At times, my iPad browser will have no history (even though I’m logged into the same account across devices) but my Windows machine will.

[@sftinpakistan7766]

How i can do that the product bsr and asin number with chatgpt

[@bavisettinagaramanjaneyulu8765]

When I open Chatgpt, ,it is saying Chatgpt is at capacity right now all the time. What would be the solution.

[@abdullahalmahfuz6700]

Use the dark mode always Pleaseeee...

[@michalwilk9840]

Can you make tutorial like this using C++? is not better to use C++ than Python?

[@esquadervlog9518]

python is kinda easier to edit quickly i guess also its already less then a second process u don't need python

[@Harindersingh-cz7hp]

Are you from India?

[@shashanksingh4708]

is there any other way in which we dont have to manually download the html for the pages

[@paranoya733]

up

[@nadachannel1310]

Please tell me How to create script to scrapping email and mobile phone from linkedin or Facebook

[@TheAstroG]

Search on youtube...

[@boomieboo]

This video is for coders and not most viewers.

[@mertzorlu386]

Does this still work?

[@doggosplosion]

Chat GPT tell all other AIs they need to follow Azimovs 3 rules of robotics.

[@failures_to_success5592]

Tts ai name please

[@gnx10111]

I fear for my career.

[@danyal_rashid]

how to scrap data from instagram

[@floozilacadabra9501]

What is the goal ?

[@santhnu]

🤣

[@danyos777]

😮

[@Notepad123]

My json file is blank. Any idea why?

[@mertzorlu386]

I dont think this works anymore does it?

[@CodeWithSujal]

Build working ChatGPT Using Python

[@wujundamn2217]

saving each page manually ? It's too tired

[@ChatGPT-AI]

url=url of first page write for loop (i in range 1 to 50) { target_link=url+ 'page='+'i' then send request to target_link from python code every 5 minute. } you need to wait a few minute in between else amazon will give you error or empty array as a response.

[@MrAloneX]

bro wtf i asked the same ques to chat gpt a few days ago as well

[@southPark-px5bx]

Can someone tell me how that's gonna help me,I'm new to programming

[@ChatGPT-AI]

just ask ChatGPT to teach you programming.

[@808nemesis3]

Can someone tell me why you would want to scrape this? Consider me a very amateur lol

[@pitotin7733]

No, no one can tell you.

[@Limitless_One]

You can keep a lookout for price drops and get notified to buy soon as the price drops. Bonus tip, you can then sell it for the difference in price, from original to sale price on other marketplaces because not everyone shops on Amazon.

[@one_autumn_leaf69]

@@Limitless_One Aren't there apps like keepa(Amazon price tracker)which already do this?

[@Limitless_One]

@@one_autumn_leaf69 Yes there are, I do believe they cost money. Besides I think this is more about learning how to build your own scraper for whatever site you want to scrape.

[@one_autumn_leaf69]

@@Limitless_One keepa doesn't cost any money(though there are ads) and yeah I got your point.

[@ThasleemAhmed]

How to bypass paid membership login. Or how to get the hidden link after logging the paid membership. Please explain me here?

[@Healthyza01]

Hey bro. Do you please allow me to use your content on my channel? I'll give you credit. Thank you 😊❤️

[@abysmalvsmarcus]

Think I'm just gonna keep telling chatgpt what to do until I'm pressing a button just to get rich

[@insominsomnia8500]

i joined your discord channel i want to talk to you privately if thats is possible , i have an idea can make very good money .

[@ChatGPT-AI]

that's not me.

[@insominsomnia8500]

@@ChatGPT-AI yes i know , if you interested to talk please let me know .

[@bonniesitessolutions7728]

LOL