Building a full stack WEB SCRAPING app with JAVASCRIPT tutorial

[@AaronJack]

If you're lost check out the description -- I've got a "web scraping made simple" video too Hope it was interesting.. remember to like it really helps :)

[@rosaliegriffith5468]

Really want

[@Scott-uz1cd]

Could you help me on the last tutorial please? When I type node scrapers.js nothing happens. And if I try to run active file it says Permission denied. I followed pretty much exactly, however at 2:15 on that video I typed the same touch scrapers.js but it didnt appear for me to open at the top, so I then manually created file called scrapers.js. Is this where I went wrong? Also not sure if related but in terminal it says Aaron then the file being worked on, mine is saying admins-MBP-7:~ admin$ node scrapers.js. Any help would be really appreciated!

[@AaronJack]

@@Scott-uz1cd do you have node.js installed?

[@Scott-uz1cd]

@@AaronJack yeah it's installed (it said to test if node was intalled to type node --help in terminal and it comes up). This is my error: bash: /Users/admin/Desktop/scrapers.js: Permission denied

[@AaronJack]

@@Scott-uz1cd I haven't encountered that particular error before, I would just try Googling "permission denied node js" and seek answers from there.

[@kaffnir]

pls keep up the good work, this channel gonna blow up real soon. you are my fav coding youtuber after coding train :P

[@TheKEEEEEeK]

Thanks man, I love your channel. Seems to me like you're gonna teach a lot of people a lot of useful stuff.

[@mikebovone5551]

Thanks! Your videos are, direct, detailed and insightful for a beginner like me to help conceptualize ecosystem + architectures and tools. Keep up the awesome work!

[@mayanksinghrana1878]

Awesome. It's the shortest and detailed explanation video on web scraping I found ever. loved it. Thanks Aaron.

[@AaronJack]

Thank you, my friend

[@UpTballA]

@Aaron Jack I remember I saw this video 7 months ago and thought, fuk web scraping, I have no use for that shi2t and forget about this video. Now I need to learn web scraping for my work and your video appears. I knew I would be solid after watching. Thanks a bunch bro

[@mustache2295]

Just want to remind you of your success. At the time of this video's creation you had 53k subscribers and now look at you. As of this comment you have more than doubled that amount to 114k subscribers. Congratulations and thanks for the video!

[@dr4kk0nnys6]

Boy do you deserve all the success in the world

[@bobDotJS]

I've been working with JavaScript full-time for the past 2 years and I've never seen someone use this method to create a sort of pseudo-framework. It makes perfect sense, if those are just best practices that I'm not aware of because I usually use a framework, well then it's still cool either way.

[@jer254]

I liked your videos when you were going KZbin part time but now that it's full-time I LOVE your channel ... You can definitely see the improvement ... good job man

[@kazdive]

I have just started learning web programming. I'm glad to see such a wonderful video.

[@EDC.EveryDayCode]

I'm a total beginner meaning self taught < 3 months but I really liked this I already have npm installed on my server. So I just got to do is install those other programs and I think I can start this project. Cool! 👍👍

[@danielriodejaneirobr]

You did it! Amazing work! Fast, quality, attractive. Lots of success to you!

[@jaimerojas6578]

Web scraping it's a lot of fun, this video gives me some project ideas thanks dude

[@MissPBxoxo]

very clear and concise! easy to follow! love it!

[@geebsayshi]

Hey, Although I it was hard for me to follow this video you have indeed opened my eyes to the beauty of web scrapping. Thank you.

[@eldiablorojo]

Normally I can watch videos in 1.5x or even higher. With this guy it’s more like 0.5 to see what’s going on 💨

[@scorpion32]

This is a brilliant idea, now I've got more ideas from this. Thank you

[@VeriousSmithIII]

Phenomenal teaching. Thanks man!

[@harshith_takkala]

that was awesome!

[@auhause]

Dang, this guy made the huge content with only 13 mins video. thumbs up

[@isagive]

Awesome tutorial Aaron, thank you very much 👍😎

[@AaronJack]

Appreciate your feedback, thanks :)

[@coubertin100]

Thank you for your effort and the great content!

[@Cindorqw]

what the "russian" folder on your desktop?

[@cristlui4]

pretty awesome! thanks for the walkthrough. This is pretty useful for a beginner like me.

[@CODTALES-KILLSTREAKS]

This is gonna be fun

[@kirankamath5891]

NICE STUFF BRO, THANKS FOR THE VIDEO

[@AjaySharma-jv6qn]

Excellent brother, Please keep posting

[@TheWorkingWeeb]

Subscribed. I hope I can learn more about full stack development from your videos, particularly the fundamentals.

[@jaisonbaby4031]

Awesome Content man. Please keep adding more such end to end apps. If you don't have a Patreon, please start one. I would love to support you.

[@christizzlee]

SO COOL! love your videos

[@pierrepucheu]

Great video man! Thanks!

[@muhammadfaateh6463]

thank you man i really love your stuff thanks for sharing

[@AniketMJ338]

Can't we load the data from the database without refreshing the browser??

[@GoncaloGuitarist]

Again, quite a helpful video! 😁 Thank you.

[@Deathfreeze14]

Thanks for the tutorial and helping us reach our goals

[@Sd-Bye]

I know the purpose of this video is to demonstrate web scraping but for building real website like this would it make more sense to use the KZbin Data API to get information about the channel rather than Puppeteer? I'd imagine making API calls to KZbin would be much faster than making a bunch of headless browsers for getting new channel data.

[@ΒασιλεύςἈλέξιοςΚομνηνὸς]

Yes but so what. It is a wonderful tutorial that someone like myself can use to extrapolate the idea out to other instances. If he just used the KZbin data API, then that would defeat the purpose of the video.

[@hellyski]

Great tutorial, thanks! 🙌🏼

[@vishalghag9607]

Thanks bro to putting it so simple

[@AaronJack]

🙏🏼

[@dean6046]

Awesome! Thank you man!

[@khalidelgazzar]

Very Kool and to the point

[@nemo1112]

Great Tutorial. Thank you.

[@ayorinde5782]

cannot find module 'typeorm' nestjs

[@theanonymoustalk]

if we didn't do the console.log at 12:05 could we have just not made a variable for await creatorRepo.save(creator)?

[@diggleboy]

Awesome tutorial videos. I want to do this with career websites and now I see how it is possible to do. Subbed! Liked! Smashed the bell! Keep up the great work and I really enjoyed hearing about your journey into a big tech company and out as a coder.

[@christianloizou4463]

You should make a 'web scraper' that gets the boilerplate code from expresses.com and writes it to a file for you, if you copy it every time

[@josephwong2832]

Thank you so much bro!!

[@sober90]

Can you explain why you are using a schema?

[@MaxOnMaxxer90]

You disable security rules for local development, but how would you be able to use the app in the real world? Thanks for the video! Hopefully someone can answer :)

[@himavad]

At 8:50, when trying to execute the scrapers.js code, I'm getting a (node:15308) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'getProperty' of undefined error. Anyone knows why? I checked the code, and even tried copying it from the github share exactly. Still unable to get past this error!

[@ΒασιλεύςἈλέξιοςΚομνηνὸς]

Same

[@tmiae0]

Awesome content dude! I wish I found it sooner. Btw, will you be back at Iron Republic Gym?

[@woodysvideos6756]

Great work! More tutorials!

[@AngeloIbarrola]

what extension was that preview feature called?

[@krishnakanani8227]

I watched all your Javascript videos and career videos i get more confidence from your story. After i learned Javascript, HTML, CSS , what should i have to learn next Please can you help me out? There's many framework like React, Angular, Node, which framework i have to learn?

[@MYmotivationn]

DAMN MAN, this is lit

[@zl7289]

Which one is better for web scraping, Python or JavaScript?

[@ashadnasim7952]

continue please, liking your content...

[@dimasmfth]

Thank you!

[@Redlaw]

But how will you run puppeteer on a live server/ actual site

[@RILCOMusic]

I cant get this to work, the first "attrs" with the funny 's', I cant get that to work on Brackets. Any help appreciated.

[@RILCOMusic]

I realize now I didnt install the prerequisite stuff. Sorry. totall noob

[@icodewrld9993]

Care to explain?

[@erichepperlewp]

Great tutorial on scraping! I had an #ERROR "Cannot GET /creators" when executing the curl statement. For anyone else running into this, I solved my issue by changing the port from 3000 to 3003. Why? It turned out that some other app was already using port 3000

[@frankzachary7230]

Hey there! I'm getting a TypeError referring to the name and avatarURL variables in the scrapeChannel function. They're both coming back as undefined. Any idea/ideas why?

[@fluffyblanket1398]

me too. i got the same error..

[@ΒασιλεύςἈλέξιοςΚομνηνὸς]

Same

[@hoangphamhuy9096]

i have a lot of Cannot read property 'getProperty' of undefined even try XPath or Full XPath :'(

[@ΒασιλεύςἈλέξιοςΚομνηνὸς]

Same

[@Evikeuklavier]

Why are you installing body parser? In latest Express versions express.json() is already your parser.

[@4541047]

Why the async in express routers if there is no await there?

[@proribrajokproribrajok7789]

Is puppeteer scraping is like python scrapy framework? which one better performe?

[@vonaudio5435]

Amazing tutorial

[@saileshsivakumar1404]

Can you post a link to your whiteboard?

[@larryboles5064]

What was the name of the VS Code addon?

[@alejorag]

HTML Preview

[@maxjohnson7623]

How to make a web scraper that would automatically go to webpages(e.g. channels) and grab the image, the title, and links of the videos for example? Coz then we can't solve the problem using XPath

[@mikemccann5923]

Getting this error.Anyone know what's going wrong? TypeError: Cannot read property 'getProperty' of undefined

[@ΒασιλεύςἈλέξιοςΚομνηνὸς]

Same

[@bmayer710]

Niceeee, thanks man, my wish became true. :D Cheeeers

[@ahmed.rosanally]

Thanks bro, appreciate the way you simplify a full stack web app into 4 layers. Wondering what are all the extensions you use on VS code?

[@sakoranj3944]

It’s my last year in high school and i kinda like computer science but know *NOTHING* about it , so i would appreciate some advise on which branch to choose , I don’t know , I’m just looking for some advise

[@Imthatguyx10]

I highly advise you look into starting with front end. Just google "how to learn front end" or "getting started with front end" or "most popular front end technologies" etc. You get the point. Also become a faster reader and improve your reading comprehension. B/c this path will involve a ton of reading. Hope that helps. Good luck!

[@zachwhite8054]

2015 Macbook Pro, or get a decent laptop and install Linux Ubuntu

[@Pumpamentals]

How's the Russian learning coming along? Unfortunately there's no npm package for that...

[@karimsz2009]

Coolest guy I've ever seen on youtube i am waiting for more advanced now i can store data in database can you explain how to make a code that auto scrap / skip pre loaded ones with schedule and panal to adjusting time also way to edit specific names for all at once like adding KZbinr: before names of each

[@wchorski]

Once we got into the Database section I feel like a step is missing. Do i also need to download an npm package of typeORM?

[@hawsh3066]

do more of these plz

[@justmoo9587]

Does someone receive the error code if you try to paste a link into the Url field ?: TypeError: Cannot read property 'getProperty' of undefined - For some Url it works , but not for all and I can’t see how to fix that problem…

[@abdelwahidbenzerrouk8271]

Great video ! Any way to skip js alerts?

[@BryceDoe]

I love you man

[@nagendrapp2213]

which theme u r using?

[@jpnote1774]

It's called Horizon

[@ipxify]

omg, this is crazily impressed, image what it could become when integrate AI into ....

[@declanstewart3570]

Thank you for another amazing video! Would you happen to know why the following error message appears: page.goto is not a function at scrapeProduct...

[@bigrtexas89]

whar program is he using when he is writing the code

[@abdelwahidbenzerrouk8271]

Awesome! please any way to skip Javascript Alerts ?

[@christianmarquez6365]

I don't know if I did something wrong but I was getting the CORS policy error even with the middleware. For anybody having this problem, this is what I used to get rid of it. expressjs.com/en/resources/middleware/cors.html#:~:text=header('Origin')%20%2C%20or,.com%E2%80%9D%20will%20be%20allowed. hope it helps!

[@kalianasu]

out of date Scraper does not working

[@gluharskixd3616]

Hey, can you share what is your vscode theme / font?

[@AaronJack]

It's Horizon theme and Dank mono font

[@brute505]

I keep getting an undefined error when trying to scrape the youtube pages

[@ΒασιλεύςἈλέξιοςΚομνηνὸς]

Same

[@andreallwood2127]

Great video !!!!! i wrote a test.js in node that works and now i want to press a button from index.html to get the same result but its not working. any ideas why?

[@devzorsmallz]

I get an empty string back for avatarURL despite the src very clearly not being empty. Anyone know how to solve this?

[@omarali8729]

its not working actually it was working until the final step it gives me an error in the terminal

[@chadtheabstract]

What font do you use?

[@FabianoQueirozMonte]

O melhor canal de 2020. Estou acompanhando por você usar coisas novas. Deixei em pt-br pra você precisar traduzir :-). Como você é nômade digital aparece aqui no Brasil.

[@nabil14409]

What's that VS Code theme's name?

[@michaelk9472]

Really want to know as well..

[@nabil14409]

@@michaelk9472 I think it's Horizon. I've found out about it later and been using it.

[@trolltothebank]

What do you do if its not a structured site like youtube? for example websites like Wikipedia have wildly different xpath's for their main picture depending on the article, I've been trying to find a fix for this but haven't yet.

[@livingstons6397]

what theme is that

[@michaelk9472]

Really want to know it as well..

[@arjay_2002]

Thank you

[@bripowered]

Thank you for the tutorial.Cool music as well.