Build a Web Scraper (super simple!)

⭐ Sign up for my Full Stack Developer Course: www.codewithan...
In this video I show you how to build a Web Scrapper in a super simple beginner friendly way using Node.js.
Web scraping refers to the extraction of data from a website quickly and accurately. Many people move onto selling their web scraping tools for money, either by building them as a chrome extension or API, or selling them to data capturing companies. So, the option to make money off this tool, is there for you too.
Common questions:
This tutorial assumes you have nodemon installed globally on your computer. If you do not, use command: npm i nodemon -g
Part 2 For Express Routing: • Get Data from Backend ...
Final code for Part 1 and Part 2: github.com/kub...
___
⭐ Use promo code ANIAKUBOW for 3 months free of WebStorm IDE here (I get no commission from this link, but am in a partnership): jb.gg/get_webs...
⭐ New to code and none of this is making sense? Watch my '12hr+ KZbin Coding Bootcamp' in which you will learn HTML, CSS and JavaScript Fundamentals completely from scratch. It's on my channel and its 100% free.
⭐ In most videos I use Tabnine as my A.I autocompletion tool. You can download it for free here (I get no commission from this link, but am in a partnership): bit.ly/tabnine-...
⭐ You can get a blockchain domain with my affiliate link here: bit.ly/get-a-cr...
⭐ If you would like to buy me a coffee, well thank you very much that is mega kind! : www.buymeacoff...
⭐ Sign up for weekly coding tips from my newsletter partnership: bit.ly/JS-tips
You can also find me on:
Twitter: / ania_kubow
Instagram: / aniakubow
#codingbootcamp​ #coding

[@josenardulli]

thanks Ania its working perfectly, may i ask why we cant use an arrow function inside the each on line 15 of the code when you call the cheerio ($) function i try it and i got all undefine but i cannot wrap my head around the why....

[@aniakubow]

I *think* it’s because cheerio is not configured to use arrow functions - but I can’t be sure - I haven’t looked into it enough :) I will have to when I’m back from holiday. I will pin this so others can see your great question 😄

[@aliabdelrhim]

@@austinps4026 This work for me: try { $('.fc-item__title').each(function () { const title = $(this).text const link = $(this).find('a').attr('href') articles.push({ title, link }) console.log(articles) }) }catch(err){ console.log(err) }

[@papajohnsuk5965]

because the arrow function scopes the function outside of 'this'

[@marcod.643]

You ca use an arrow function with the each method, but you must pass 2 parameters, an index and the element: $(".wathever", html).each((i, el) => { const title = $(el).attr("title") const image = $(el).find("img").attr("src") article.push({ titolo, image })

[@toddfisher8248]

Arrow functions use this from the calling context. Before arrow functions to do this you had to use the bind method when calling your function to switch the this context of your function. You could implement bind before it existed with apply when calling to have the function run with a different this… arrow functions just make what you usually want easier which is to carry the this context forward.. however many libraries exist that take advantage of the fact that the function you pass into another function can be called in away that changes the this…

[@revillsimon]

For anyone wondering… you’ll need to make sure the nodemon package is installed. You can run ‘npm i nodemon -g’ to use it globally on your machine. Or alternatively, you can run ‘npm i nodemon -D’ inside the project directory to use it as a development dependency while the project is running. Great video Ania, keep it up 👍

[@aniakubow]

Thanks for sharing this!!!! You are totally right I missed explaining this part- I will make sure to cover it in my next videos :)

[@alucard1532]

As a total newbie it saved me thanks.

[@SandraWantsCoke]

thx, but I like my demons

[@frogboy831]

thank u!! love when u have an issue that's way over your head and it's solved in the first comment u read

[@flexaeterna]

Ohhh that will do it. Thanks for this

[@adnanmadhoun8045]

This comment doesn't belong to this video, >> You saved me in my senior project last 6 months while I watched tutorial about web development ❤️❤️❤️❤️❤️

[@aniakubow]

Thanks so much for coding with me 💚

[@markharrisllb]

I am a boomer coding virgin except for a minute amount of C++ for my Arduino. No one has made sense to me before as there’s usually an assumption of a greater knowledge than I have. You are the first person who was coherent to me, every step explained and details of what is happening behind the scenes. Your beautiful diction helps a lot as well, thank you.

[@metatronicx]

Clear and precise english. This woman explains well all topic she talks about. Thank you for being a content creator. It fits you so well

[@fourlokouva]

Love how you just get into it without any sitcomish-intros, just straight to the point. Your videos have given me the confidence to finally start applying to dev jobs. Thank you so much, Ania!

[@artopaivinen3233]

Thanks Ania! Tried already two small projects of yours, and I must say, that you're a teacher, that speaks to novices as well. The samples you made are easy to follow, and simple enough to figure it out, how the packages work with each others. Of course that also shows a great understanding, how to produce working program, but having success with working sample encourages to do more. Thank you with appreciations!

[@aniakubow]

This is so lovely for you to say and has made my day :) 🥰 thank you so much Arto!

[@artopaivinen3233]

@@aniakubow ... Sorry - forgot to wish Merry Xmas! ... and now I do. Happy New Year goes with the same package ;-)

[@lilianamilano3707]

Thank you, Ania! It worked perfectly. I had no idea how to complete this task. You saved my day and gave me a lot of knowledge and fun too. I send you love from Venezuela, you are a genius! ♥

[@k-c]

Thanks!

[@aniakubow]

Thank you so so much Kaushik!!! It means the world!

[@KingstonFortune]

This was a very good intro video to web scraping, thanks! As a quick tip, when running the `npm init` command you can just append `-y` at the end so it becomes `npm init -y`, and then it will proceed to skip the checks and create the package.json file without you having to press enter several times.

[@koenslootweg5882]

Just had a web scraper project on work last week so used your idea. Thanx Ania. had some problems with nademon... But just installed nodemon wit npm i nodemon

[@aniakubow]

Amazing!!! I’m glad this video helped and you managed to get nodemon installed :) thanks for leaving your advice for others too!

[@jalapenogaming9740]

hey just here to say "thank you" !! you put a lot of effort into these videos i hope your channel grows fast and get to the top 🙌

[@noahmichael89]

Very cool simple project! One thing I noticed is that you must have nodemon installed globally (unless it's packaged with a version of Node I don't have?). For anyone that doesn't, the "npm start" script won't work. Of course you can just install it locally into this project with "npm i nodemon" in the terminal.

[@TheJosephTodd]

My man, thank you! Had this exact problem and you solved it. :)

[@authoritycamper]

Thanks man! This is the command for a Mac: sudo npm install -g nodemon

[@kenfoli330]

thanks alot man

[@maurov6861]

well done bro! installed on the project ;)

[@katykarry2495]

Works purrfectly and super simple code to use and follow. Can you do a part 2 where you learn us to crawl() with this scraping code? I mean the entire site for an example

[@emmabecker6460]

Yes please Khaleesi of codes do this!

[@hazemelbatawy1242]

thanks, Ania it is very good and working perfectly.

[@rinn1944]

Wow! You explained this so well and left nothing vague. A lot of other tutorials leave out so many parts, assuming everyone knows what they're telling. It's the first time I've come across your channel, and you just gained a subscriber. Thanks a lot!

[@glowiever]

I am assigned as data engineer and really need to scrape some data from a marketplace. this is a blessing!

[@RealElites]

Learning from Ania's style of teaching is easy and relieving. Gonna binge this stuff

[@galactic_4k]

Crazy, I just happened to need do something similar for a project and here you are uploading a video that helps a ton! Thanks!

[@Lucas_Gamaleri]

The mere fact that you have a couple longboards behind you convinced me to subscribe and like this video. You're a cool coder 🌝

[@elingrome5853]

🤮

[@thecoderabbi]

Subscribed! cause your contents are really great!

[@vinitmanerikar5444]

I have subscribed to many programming channels, however you are one of those who really add value..keep it up

[@theshazman]

Would love a follow up to this with more advanced scraping such as a page that is behind a login wall, or something that requires a query to be filled before getting the final web content. Thanks heaps!

[@jacksonaguilar1998]

I can listen to you all day. I hope you were my web dev lecturer.

[@suavebob]

Not sure why this channel/video showed up in my feed but most definitely this lady is a damn reasonable POA

[@emilyreese9551]

This is a wonderful explanation of every line of code. I've learned code through online resources, mentors and college. Some of this stuff I knew I had to do but I did not know why. Thank you!

[@Cyapow]

Damn. Wish there were videos to follow like this 20+ years ago when I was starting out lol. You make it very easy to understand

[@jeevanjotsinghnarula2453]

Seeing couple of software out there that helps people scrape data from the website. It's great to watch this video so I can be able to do the same by just learning programming from best people over on youtube.

[@AlcantaraMC]

sister used 70% of the vid to set up everything huhu. anyways, it works at the end, and its what we came here for. Thanks!

[@networkserpent5155]

I wanted to learn the power of web scraping. This video shows just that! Thank you Ania

[@DKSorg]

Awesome Video. Clear Precise and great Audio, easy to follow and listen to.

[@Shafty17]

Big thanks, Ania! With your help i managed to write a webscrapper to parse through all private repositories in my organization (had to add authorization as well), "read" it's package.json file, save it to a file, then run another conversion script that forms array of strings in certain way & then save it to .csv file, to be able to create a pivot table & analyze tech stack of our product :)

[@benimoza9746]

Omg, idk what words to say, i love you, this is so easy to follow

[@syntaxed4365]

This was a fantastic tutorial, and as someone that is a lover of JS, it's nice to see this approach. I just have to say, I know that they are no longer required, but my brain just gets a completely unnecessary comfort out of using semicolons 😅 even though I can definitely agree that it looks cleaner without. I still have the desire to go through and add them everywhere! haha

[@aniakubow]

Thank you so much!! I really appreciate your comment 💚. Haha yes , I’m team no semi colons for my projects on here haha. I used to use them at work, but even then an extension added them in for me 😛

[@BeepBlorp37]

hahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahah >=[

[@vskiy26]

Thank you for such awesome tutorial. Dziękuję Ci.

[@vancouverrrr]

You have helped changed the lives of many, thank you for sharing your priceless knowledge

[@romimaximus]

at 17:15 mins, i'm surprised, that you didnt get a "Cors" Error !!.... but great tutorial !!... thank you very much for sharing it !! 😃👍

[@lifeoffreelance9131]

Hi, this is my first time here, and i really love the way you teach. thank you for your very informative tutorial.

[@chrisjordan5849]

Thanks because of you i learned the basics of web scrapping 👍

[@edwarddejong8025]

Wonderfully clear exposition. A very effective teacher.

[@imagineaddis9050]

That is why learning docker was important for me. We can package our program to work on any machine without worrying about breaking changes from node or express. Thank you for the nvm tip!

[@havocblast]

yes docker is a life saver especially when you work on both mac and windows

[@maxmaksum4673]

Ania....it is just awesome...someday if you have holiday to Bali I will treat you and show you aound

[@CH3RRYxB0MBx]

I've been waiting for a tutorial like this! Thanks Ania!

[@ksivasuriyaprakash9976]

Thanks for update my knowledge on npm. video was short and sweet.

[@JoseHernandez-bg6zm]

Loving the bit more "fullstack-oriented" content! Actually have a project in mind where I could apply this perfectly. Thanks for the inspiration! ;)

[@Workshop385]

I don't know anything about programming, but I watched the clip to the end.

[@rukkychew]

Thanks for the intro. After that I was able to move around and play around to build a scraper for blogspot

[@nam-sdrawkcab]

YEAHH!!!! ILL BE SCRAPING ALL OF YOUR VIDEOS 😍

[@whalebird]

great video and nice accent! It was pleasant listening to this tutorial, I've subscribed : )

[@trafalgarlaw4778]

learning my first line of code because my teacher Ania is stop eating beautiful

[@TheDa6781]

I could listen to her say scraper all day long. The part of my brain that would enjoy it doesn't have that many lines of code either.

[@lifelessalarm]

Thanks for the tutorial =) As a few others have said, an alternative option might be to use puppeteer. It has a nice syntax and is very flexible. You can simulate natural browsing using simulated click events and run additional commands such as taking screenshots of html content based on css classes.

[@LetsScrapeData]

yes, I design a template in 4 minutes to scrape title and url using a tool based on puppeteer: kzbin.info/www/bejne/qHOYc3udZb2sgdU

[@RealJustinEstrada]

Your tutorials are very consice and to the point. Its greatly appreciated.

[@MartinJaszczuk]

Great video. I love the longboard on the wall. Thanks for posting!!!

[@crisolivares7847]

its 6 am and I was waiting for this I love your channel

[@aniakubow]

Thanks for joining Cris!!

[@brookestephen]

I built a hierarchical object model to hold the parsed HTML, made of nodes and parms, and a Trie to hold the key words for the parser. I wrote it in every language I've learned... Python, Java, C#, VB... You can create HTML with it too, and output to a string, so it's in every web server I build. There's several different search methods, so adding to the object model is fast and easy... no more counting

[@andrasbradacs6016]

I was looking for a completely different thing, but you gave a good idea with this video! 🤗 Thanx! 😁

[@MeditationRelaxSleeps]

Thank you very much

[@kuoyulu6714]

Thanks, this is really clear and simple to follow!

[@saudahmed2436]

Thanks Ania, had a little bit issues but after i got nodemon package installed and realized i was missing a . for calling the class it worked like a charm :). One thing to note i think is that windows doesnt have a NVM, so i had to als install one of those just so i could make sure i am using same version as you

[@KacperSieradziński]

This is great that you are going through the doc during video :-) It is great because you teach people how to read the doc... I do the same on my Python videos :-) Pozdrawiam! :)

[@lloyd27z]

Everything you explained is quite clear and simple .So very helpful to learn.Thank you 😘

[@aniakubow]

Thanks so much Sue!!! It’s great to hear this feedback as I’m always trying to improve 😄💚

[@lloyd27z]

@@aniakubow looking forward to your new tutorial!

[@hugodorian2165]

Omg 😱, waiting for this👩‍💻😊

[@kamertonaudiophileplayer847]

You provide an amazing technique for web scrapping. It would be good if you explain also how to manage sites using an anti scrapping technique, for example instead of just populate a text directly, they wrap every character in a different tag or even replace some digits with a similar text characters and so on.

[@javierfrancia1938]

i will build anything that beautiful Ania tells me, web scraper, 20 room house, spaceship, time portal, anything!!!

[@shahedtheboss]

Thank you so much i needed to learn how to build a web scrapper You are the best❤️

[@tarek_maza]

Congrats on the 200k subscribers

[@aniakubow]

Thank you Tarek!

[@luisamilcartavares26]

Interesting, whenever I think of scraper I always think of Python (beautifulsoup). This is really a very good perspective. thanks for sharing.

[@xmachina47x]

I think some of us, senior developers, will follow because we would like to see a surfing tutorial. .:D ..featuring the Lara Croft voice. Great coding! ;)

[@okidokiyowyow356]

Thank you so much. I always wanted to learn how to scrape the web I didn't know it's this easy.

[@FeedScrn]

Scrape it for what?.. if you don't mind me asking.

[@zaliblk_media]

Great tutorial. Very well spoken. Very well communicated. Great Job Ania.

[@matthewknobel6954]

Back in the day I had to build a scrapper by hand with only perl in a unix environment. It would start at a root, parse all the html, and then traverse to the sub pages. All as an intern at Xerox back in 94.

[@kar0ee0m]

I subscribed today because of my friend Hesham recommended you, so don't disappoint me

[@fiveminutesvideo]

Hi Ania, love the videos! Would love one on setting-up a web scraper to scrape every minute or so in a Litespeed server. Keep up the great work.

[@sadique_x_]

Ania makes programming very interesting!

[@SurfingTheMentawais]

Thanks for the video. You are obviously an expert.

[@calebcadainoo]

Amazing thanks for this tutorial😉

[@gabyph2013]

Excellent video Ania! your explication is very simple and easy to understand, thank you

[@Sloth740]

Thanks for the video! I had some issues with the "start" script on Windows but found that "node index.js" not "nodemon index.js" worked for me.

[@honeyverma2755]

I usually not prefer to watch foreigner channel videos bcz your accent sometimes bit difficult 😅to comprehend but for you I must say you have given a good explanation of web scraping with this simple example 👍🔥

[@federicomontesdeoca8977]

Hello Ania, thank you for the quality of your videos ! I have almost 4 years of experience as a js dev, but still learning with your content. Just one question, why you install express and turn the app into a server, since you only need the http client (axios) in order to scrap the page?

[@kuldeepsharma7499]

Same question

[@PraneethParsharouthu]

same ques

[@greyice73]

This soooooo cleaaaaan I like it ! nice Video Ms Kubow

[@aislinnsaunders-wallace9772]

Just created a stock data web scraping app thanks to this video!!! Thank you so much! I feel like a real pro lol :)

[@aniakubow]

That’s amazing! Thank you so much for sharing this with me 😍

[@legisam1754]

I like 😂😂😂. Just found your channel and I'm loving it. Excellent content. Keep it up and keep em coming 👍👍👍

[@torchgoat1707]

than you so awesome and well walked through. it allowed me to learn how to use replit and ect by forcing me to research lol i feel i leveled up

[@tancodes]

I have already creating a project with use of web scra and django. But this will be very helpful to learn from you 🔥

[@alexf1082]

Which Python package are you using? I don't like Selenium and I'm with requests-html but is no longer developed. The base requests package don't execute JavaScript.

[@hestJaevel]

Great video! Only feedback is: Use chapters so we can skip the steps to install node if we already have it for example :)

[@KDevGaming1921]

Thank you this is a great help, It was working. I just had an issue with nodemon and installed it in my machine. I am just encountering the PORT not showing up on my terminal. I have an older version of node js though. Haven't tried using nvm yet. Going to try and practice that tip you gave. This was really helpful.

[@mrCetus]

Amazing tutorial ! Keep up the great work ! The web scraper would now enable us to develop alternatives for all applications using APIs and this gives us great confidence.

[@EmersonButzen]

Great Ania!

[@aminagha3206]

I am really proud that you are my teacher, I learned a lot from you, what is really interesting is that you are beautiful in appearance and also you have a beautiful mind you can explain the information in the simplest ways. Eagerly waiting for your new one🌹

[@sabatauqeer1]

just what i needed i successfully got src but was stuck at chaining thank you

[@JordiVicensFarrus]

This is great Ania, thanks for your video! BTW just as an idea, would be great to have a second and a more advanced part, that shows how to do it in a site like Linkedin that requires Login or a site that has dynamic ids and classes. Cheerios!

[@marianneazzopardi974]

Love your content Ania! I'm currently doing a bootcamp and your videos are helping me through!! :D

[@PeterCooperUK]

There's an extension called "Selector Gadget" which lets you pick elements on a page and it'll work out the best CSS selector to reach those elements which is great for scraping like this.

[@marloonlemes]

My first time in your channel.. I thought you were Daenerys Targaryen from GOT 😄 Thanks for your content!

[@hursttrader4507]

You have the most mesmerizing eyes I have ever seen. Your tutorial is excellent as well, you certainly know your coding.

[@aldobangun223]

Thanks Ania, this is verry helpful

[@seniorqa_auto5367]

Great lesson! Special thanks for enlarging the screen with the code!

[@odeddugma9748]

Thanks Ania. You are a great teacher ☺️ You make it looks so easy.