Web Scraping with ChatGPT Mentions is Mind Blowing!

[@ThePyCoach]

To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/ThePyCoach . You’ll also get 20% off an annual premium subscription.

[@xevenau]

Just watching your videos made me realize how complicated im making my life. Such a brilliant way to scrape.

[@slajoie0617]

Thank you can you please create a video detailing how to update the GPTs system instructions you plan to use to account for the new @mentions feature for seamless teamwork and collaboration between your team of GPTs

[@rayfellers]

The recommended scraper is unable to scrape pages requiring a log in and offers no way to log in prior to attempting a page scrape. It also has a relatively short timeout so slow responding or large pages can't be scraped.

[@nitrogamesstudios5276]

And also these features could be work with paid Gpt's

[@jatis2010]

Any plugin to overcome website that has security blocking ?

[@pooryagohargani8035]

Youre a legend

[@awakenwithoutcoffee]

hm sadly did not work as the scraper triggered anti-scraping policy of the websites I tried. This seems good for some minor quick scraping jobs but I wouldn't rely on it for actual production scraping.

[@danlee1027]

Yes you may need to utilize proxies when scraping

[@chrissilvestri9344]

Thank you! I’ll try it out 👌🏻

[@miketemple7686]

I went to Perplexity as to get some data on web scraping. It used your YT channel as one of its references. Welcome to the AI big time brother 😂.

[@luxipo9934]

The thing is that Perplexity sometimes is kind of “lazy” and that is due to the short token window it has. So for example if you ask for the updated market cap of 20 companies it will give you info that is not up to date for some of them.

[@atbapp]

Wow... Thats awesome! Thanks!

[@cirtey29]

Use gpt vision, more adoptive to html change

[@abdulmuizzikumapayi2811]

can i use this @ in my api call of openai to interact with the gpt like scraper and data analysis

[@haoshiangyu6906]

I've recently learn about ics. format file but I couldn't understand how to use it could you teach us how to use such file?

[@rahulrajeev9763]

Can we do this without having the paid version of gpt?

[@MuhammadEltiti]

Thanks for sharing! when I try to download I get this error message "Failed to get upload status for /mnt/data/worood_product_list.csv" I tried different browser and nothing change. any thoughts on what could be the issue?

[@ghada.b]

is there a way to do that with Gemini?

[@greenphones577]

Interesting. I have done some scraping before using Python to download date both in Excel and CSV format and insert to a MySQL database. In some of these examples I needed to use my username and password to access the data. Is this possible using Scraper?

[@saurabhprajapati4794]

Is there any method with open-source llms

[@fahimrafi9853]

Is there anything like these in hugging face or free as open source app? TIA.

[@PallaviChauhan91]

Hi, I can't see the plugins option under model dropdown like you have @3:36 . I have a chat gpt plus subscription. I have also activated the plugins under settings.

[@ThePyCoach]

Plugins have just been removed. You can either use the ScraperGPT or use an old chat where the Scraper plugin was activated (plugins still work in old chats)

[@hhmtp]

Hi, do we have to give all the pages that we want to scrap ? Is it possible to give only the root page? Great video by the way :)

[@ThePyCoach]

You'd need to give all the links, unless there's a pattern in the links that you can describe to ChatGPT

[@bjraval9818]

​@@ThePyCoach So how do you scrape all business locations with their name and latitude and longitude values from Google Maps when it shows different number of business with each zoom levels and hides the rest at another zoom levels? Conceptually the problem is similar to providing ChatGPT with all page numbers but here we have to deal with different zoom levels. Any thoughts?

[@sc20910]

Awesome!

[@muratkekec5994]

is this paid version of chat gpt?I dont see " explore gpt"?

[@nikobyu2778]

Awesome, but i rather manual way combine http header knowledge and linux, all command line

[@jd2161]

Good. You do that

[@travisross8704]

What about scraping data from private sites like LinkedIn? Is that also possible with a version of this method?

[@ThePyCoach]

If it requires a login then it’ll be impossible with this method. You can try with my other method that involves the code interpreter and an HTML file (the video is on the channel)

[@loudnlouize1870]

Sad part is most of the people don't have got 4 🙂

[@fahimrafi9853]

Yeah 😢 Is there anything like these open source/ hugging face?

[@jd2161]

Not sad

[@bjraval9818]

​ I added the very same comment to one of your replies below >>. @ThePyCoach So how do you scrape all business locations with their name and latitude and longitude values from Google Maps when it shows different number of business with each zoom levels and hides the rest at another zoom levels? Conceptually the problem is similar to providing ChatGPT with all page numbers but here we have to deal with different zoom levels. Any thoughts?

[@Its_me_bishal20]

Scrape skool group members name and linkedin