The Definitive 2024 Make.com Web Scraping Masterclass

[@nicksaraev]

My new step-by-step program to get you your first automation customer is now live! Results guaranteed. Apply fast: skool.com/makerschool/about. Price increases every 10 members 🙏😤

[@RocknRollCEO]

OK, I'm gonna need to take a day off just catch up and study the last few videos - iterator - aggregator (again), map - plus this one. And maybe Nick, you take a week off 😛 Cheers.

[@SamU221E]

Great guide, thanks Nick! 🙏 P.S. Here's the code for capturing the entire HTML of the page in your Apify scraper (47:21): const pageHtml = document.documentElement.outerHTML; Don’t forget to include pageHtml in your return object.

[@youriwatson]

I really appreciate that you take the time to also explain the concepts before diving into the practical details🙌

[@nicksaraev]

My pleasure Youri 🤝

[@ZachMeissner]

Nick DELIVERING AGAIN!!!! Thank you!!

[@nicksaraev]

Thanks again Zach 🙏

[@AmarHarolikar.]

Brilliant video. Like the HTTP and API masterclass. Power packed. Very valuable information. Thanks for sharing

[@iainhmunro]

Who ever corrected you on Regex, I would love to know how they say Registry. I say it the same as you - coming from the UK where we speak proper English.

[@futurizerush]

This is GEM. Thank you Nick.

[@doroga-peremen]

how do you make a record into a variable in a module? you wrote a split and turned it into a variable on the 38:01

[@andrejgazi3900]

Hi Nick, thanks for the video! question: I am curious if you have built something like a day and night duty planner for nurses, I tried to build it with GPT and failed XY times due to a complicated algorithm, any idea how to build such a thing?

[@igetobi3213]

I love you 😂😂...I was really hoping for this

[@JpresValknut]

A great video what I would actually be interested in is how to come up with ideas yourself for automation that one can propose to clients instead of reacting to their demand.

[@chimoji608]

Interresting stuff. How do you deal with a login wall like Linkedin for instance?

[@livingworkingoutsidebox]

This was to advanced for me. I just didn't understand the concept of scraping websites like this to sell as a service or use it for my business. 🤔 Feel like a 4th grader being given a geometry problem and told to use it to build a set of stairs.😮😢😂 I just don't have the experience yet. So will focus on leveling up. Then revisit.

[@rossjames7327]

Hey Nick, thanks for the videos. Ps. You mentioned in the video that you would upload the regex into the description but I can't seem to see it. Can you upload it please. Thanks

[@GarthB-uf6dr]

Great work thank you.

[@florakija]

Freaking amazing video. You da man. Thanks so much!!!!

[@ankitpurohit1293]

@nicksaraev - Thank you so much for the video. But when i use your method i only get 8 outputs per run, even though the page has like over 30 freelancers. Any idea why does it limit the output to just 8 data points? Thanks in advance.

[@ReyTamayoPhatInnovator]

Can you make a video on how to scrape KZbin to get thumbnails, author, channel name, transcripts, and video summary?

[@Opeyemi.sanusi]

This is where is feel make is lacking, it doesn't have a code piece that makes it difficult for a lot things to be done properly. Unfortunately this is not as easy as pulling the html data via http. I made a solution on activepieces and I also have a solution for transcripts on Make it's just a little dependent on other tools.

[@ilecto6531]

I can't even make a printscreen from a lot of website! (Secured URL's) so, how you can solve this issue?

[@radenfadil9256]

Thank you for the video nick, but i do have a question, is it possible to create http request on a web only accessible through vpn / certain ip / proxy? Is there a way to set this up?

[@dmytroocheretianyi7577]

Hello! I would like more information about using hidden API

[@SocialScheduler]

Help? Great video. need to find a tutorial where the data of a site is only available after having logged in with several pages of search results and that one that you'd have to click "next page" to see the next set of data. Help is appreciated.

[@julian-wohlers]

the thumbnail slaps

[@rtkkjson]

Thank you nick!

[@jordanmaxwell5548]

How do you scrape multiple pages? Not just one?

[@DeRiio]

Just yesterday I first time saw ur video and it was the same LOL, and now this video in recommendation. We live in matrix no way

[@businessresearch520]

Now that upwork ended access to the rss feed, I'm wondering if we can set up an automation using their hidden api to grab new jobs and filter them.... Hmmm... Anyone have thoughts on this?

[@andresdeves5079]

Is there a way to scrape a website, that has a form with captcha? I need to scrape the response from that form

[@MrCompetentG]

Try Apify

[@mshahmeerbasit6328]

nick is there a way to scrape instagram not just a profile and find multiple profiles falling in the same category

[@OmerNesher]

you're doing god's work. just saying. 😂 thanks man, that was really insightful.

[@therronhumphrey]

I have a question / idea. I’m beating my head against the wall. I made a knowledge based GPT and want to host it as a standalone function on a website but don’t know how. Any recommendations?

[@dmytroocheretianyi7577]

Hi! You can try to do this on the Flowise platform.

[@SaidThaher]

Just wondered a minute ago , strange Nick didn't uploaded a video today.. open YT .. and voala 😅😅

[@nicksaraev]

I scheduled this for yesterday but it was 12GB and my internet was not cooperating 😒

[@selcukaslantas]

You should be an angel

[@cezexcezex9888]

Try to scrap oficial make documentation api

[@hammers121]

What pattern for the regex if we want to get contact numbers please, emails is working well for me 🤩

[@songsbycoolkids]

any experience scraping tiktok?

[@Benfinecreates]

Nick slow down bro, I can't keep up haha. in all seriousness though thank you for the amazing content

[@lovetentacion5433]

Hi brother Nick , i wanted to join your skool community but there is only option for annual join. But I can only afford monthly subscription. can you do anything about that please

[@nicksaraev]

Hey man. I think you just signed up for annual and we tried processing you-could you check for an email from me? Cheers 🙏

[@lovetentacion5433]

@@nicksaraev I'm sorry brother i think that I'm signed up before reading whole email. And is there any way to join your skool community. Please give me one chance to show my hard work and i think you're best mentor for me. Thank you

[@Opeyemi.sanusi]

Make is very limited on what it can do. I have built similar tool more efficient tool using code integration on other platforms, also sending HTTP request would suck on websites that load dynamically, because it grabs the first "html" presented by the website. Using a code integration for example would save you cost on claude, save you cost on errors, cost on number of integrations to run and there are tons of library to integrate with to build cooler automations.

[@skylers1359]

What other platforms do you use with code integration

[@Opeyemi.sanusi]

@@skylers1359 Activepieces mostly and then latenode. The issue with these other choices is that they don't have as many integrations yet. Activepieces let's you code with Typescript and latenode doesn't only allow coding but you can deploy an actual scraper that launches a headless browser directly on the platform

[@Opeyemi.sanusi]

@@skylers1359 however there is one python integration tool on make. I just don't like it

[@devworx.studio]

Interested in what also you use?

[@patrick1992]

What tool

[@kobiomenaka4445]

You mean Sir Tim Berners Lee?

[@andrewgoodwin6864]

Yeah him, did some internet thingy

[@nicksaraev]

Yes! Daddy of the Internet 👑

[@michaellerch]

Who the hell says "REGG-EX" it's definitely "REGEX" sounds like [REJ-EX] and not [REGG-EX]. Sorry for the "AckTuaLy" nerds that shat on you last video. They're wrong. It could be one of those cultural differences like in the EU they say it that way. Midwest USA it's definitely REGEX. Never ever heard it as REGG-EX.

[@6sta3rs]

can you give regex to email

[@rthidden]

This is the Tim guy you are looking for, en.wikipedia.org/wiki/Tim_Berners-Lee

[@nicksaraev]

That's the one - thank you Ryan 🙏