Build a Web Scraper (super simple!)

[@josenardulli]

thanks Ania its working perfectly, may i ask why we cant use an arrow function inside the each on line 15 of the code when you call the cheerio ($) function i try it and i got all undefine but i cannot wrap my head around the why....

[@aniakubow]

I *think* it’s because cheerio is not configured to use arrow functions - but I can’t be sure - I haven’t looked into it enough :) I will have to when I’m back from holiday. I will pin this so others can see your great question 😄

[@aliabdelrhim]

@@austinps4026 This work for me: try { $('.fc-item__title').each(function () { const title = $(this).text const link = $(this).find('a').attr('href') articles.push({ title, link }) console.log(articles) }) }catch(err){ console.log(err) }

[@papajohnsuk5965]

because the arrow function scopes the function outside of 'this'

[@marcod.643]

You ca use an arrow function with the each method, but you must pass 2 parameters, an index and the element: $(".wathever", html).each((i, el) => { const title = $(el).attr("title") const image = $(el).find("img").attr("src") article.push({ titolo, image })

[@toddfisher8248]

Arrow functions use this from the calling context. Before arrow functions to do this you had to use the bind method when calling your function to switch the this context of your function. You could implement bind before it existed with apply when calling to have the function run with a different this… arrow functions just make what you usually want easier which is to carry the this context forward.. however many libraries exist that take advantage of the fact that the function you pass into another function can be called in away that changes the this…

[@revillsimon]

For anyone wondering… you’ll need to make sure the nodemon package is installed. You can run ‘npm i nodemon -g’ to use it globally on your machine. Or alternatively, you can run ‘npm i nodemon -D’ inside the project directory to use it as a development dependency while the project is running. Great video Ania, keep it up 👍

[@aniakubow]

Thanks for sharing this!!!! You are totally right I missed explaining this part- I will make sure to cover it in my next videos :)

[@alucard1532]

As a total newbie it saved me thanks.

[@SandraWantsCoke]

thx, but I like my demons

[@frogboy831]

thank u!! love when u have an issue that's way over your head and it's solved in the first comment u read

[@flexaeterna]

Ohhh that will do it. Thanks for this

[@adnanmadhoun8045]

This comment doesn't belong to this video, >> You saved me in my senior project last 6 months while I watched tutorial about web development ❤️❤️❤️❤️❤️

[@aniakubow]

Thanks so much for coding with me 💚

[@markharrisllb]

I am a boomer coding virgin except for a minute amount of C++ for my Arduino. No one has made sense to me before as there’s usually an assumption of a greater knowledge than I have. You are the first person who was coherent to me, every step explained and details of what is happening behind the scenes. Your beautiful diction helps a lot as well, thank you.

[@metatronicx]

Clear and precise english. This woman explains well all topic she talks about. Thank you for being a content creator. It fits you so well

[@k-c]

Thanks!

[@aniakubow]

Thank you so so much Kaushik!!! It means the world!

[@fourlokouva]

Love how you just get into it without any sitcomish-intros, just straight to the point. Your videos have given me the confidence to finally start applying to dev jobs. Thank you so much, Ania!

[@artopaivinen3233]

Thanks Ania! Tried already two small projects of yours, and I must say, that you're a teacher, that speaks to novices as well. The samples you made are easy to follow, and simple enough to figure it out, how the packages work with each others. Of course that also shows a great understanding, how to produce working program, but having success with working sample encourages to do more. Thank you with appreciations!

[@aniakubow]

This is so lovely for you to say and has made my day :) 🥰 thank you so much Arto!

[@artopaivinen3233]

@@aniakubow ... Sorry - forgot to wish Merry Xmas! ... and now I do. Happy New Year goes with the same package ;-)

[@koenslootweg5882]

Just had a web scraper project on work last week so used your idea. Thanx Ania. had some problems with nademon... But just installed nodemon wit npm i nodemon

[@aniakubow]

Amazing!!! I’m glad this video helped and you managed to get nodemon installed :) thanks for leaving your advice for others too!

[@lilianamilano3707]

Thank you, Ania! It worked perfectly. I had no idea how to complete this task. You saved my day and gave me a lot of knowledge and fun too. I send you love from Venezuela, you are a genius! ♥

[@KingstonFortune]

This was a very good intro video to web scraping, thanks! As a quick tip, when running the `npm init` command you can just append `-y` at the end so it becomes `npm init -y`, and then it will proceed to skip the checks and create the package.json file without you having to press enter several times.

[@jalapenogaming9740]

hey just here to say "thank you" !! you put a lot of effort into these videos i hope your channel grows fast and get to the top 🙌

[@katykarry2495]

Works purrfectly and super simple code to use and follow. Can you do a part 2 where you learn us to crawl() with this scraping code? I mean the entire site for an example

[@emmabecker6460]

Yes please Khaleesi of codes do this!

[@galactic_4k]

Crazy, I just happened to need do something similar for a project and here you are uploading a video that helps a ton! Thanks!

[@glowiever]

I am assigned as data engineer and really need to scrape some data from a marketplace. this is a blessing!

[@noahmichael89]

Very cool simple project! One thing I noticed is that you must have nodemon installed globally (unless it's packaged with a version of Node I don't have?). For anyone that doesn't, the "npm start" script won't work. Of course you can just install it locally into this project with "npm i nodemon" in the terminal.

[@TheJosephTodd]

My man, thank you! Had this exact problem and you solved it. :)

[@authoritycamper]

Thanks man! This is the command for a Mac: sudo npm install -g nodemon

[@kenfoli330]

thanks alot man

[@maurov6861]

well done bro! installed on the project ;)

[@theshazman]

Would love a follow up to this with more advanced scraping such as a page that is behind a login wall, or something that requires a query to be filled before getting the final web content. Thanks heaps!

[@rinn1944]

Wow! You explained this so well and left nothing vague. A lot of other tutorials leave out so many parts, assuming everyone knows what they're telling. It's the first time I've come across your channel, and you just gained a subscriber. Thanks a lot!

[@hazemelbatawy1242]

thanks, Ania it is very good and working perfectly.

[@emilyreese9551]

This is a wonderful explanation of every line of code. I've learned code through online resources, mentors and college. Some of this stuff I knew I had to do but I did not know why. Thank you!

[@vinitmanerikar5444]

I have subscribed to many programming channels, however you are one of those who really add value..keep it up

[@Cyapow]

Damn. Wish there were videos to follow like this 20+ years ago when I was starting out lol. You make it very easy to understand

[@thecoderabbi]

Subscribed! cause your contents are really great!

[@vskiy26]

Thank you for such awesome tutorial. Dziękuję Ci.

[@RealElites]

Learning from Ania's style of teaching is easy and relieving. Gonna binge this stuff

[@CH3RRYxB0MBx]

I've been waiting for a tutorial like this! Thanks Ania!

[@jacksonaguilar1998]

I can listen to you all day. I hope you were my web dev lecturer.

[@edwarddejong8025]

Wonderfully clear exposition. A very effective teacher.

[@DKSorg]

Awesome Video. Clear Precise and great Audio, easy to follow and listen to.

[@lifeoffreelance9131]

Hi, this is my first time here, and i really love the way you teach. thank you for your very informative tutorial.

[@RealJustinEstrada]

Your tutorials are very consice and to the point. Its greatly appreciated.

[@Lucas_Gamaleri]

The mere fact that you have a couple longboards behind you convinced me to subscribe and like this video. You're a cool coder 🌝

[@elingrome5853]

🤮

[@jeevanjotsinghnarula2453]

Seeing couple of software out there that helps people scrape data from the website. It's great to watch this video so I can be able to do the same by just learning programming from best people over on youtube.

[@networkserpent5155]

I wanted to learn the power of web scraping. This video shows just that! Thank you Ania

[@lifelessalarm]

Thanks for the tutorial =) As a few others have said, an alternative option might be to use puppeteer. It has a nice syntax and is very flexible. You can simulate natural browsing using simulated click events and run additional commands such as taking screenshots of html content based on css classes.

[@LetsScrapeData]

yes, I design a template in 4 minutes to scrape title and url using a tool based on puppeteer: kzbin.info/www/bejne/qHOYc3udZb2sgdU

[@benimoza9746]

Omg, idk what words to say, i love you, this is so easy to follow

[@MartinJaszczuk]

Great video. I love the longboard on the wall. Thanks for posting!!!

[@JoseHernandez-bg6zm]

Loving the bit more "fullstack-oriented" content! Actually have a project in mind where I could apply this perfectly. Thanks for the inspiration! ;)

[@suavebob]

Not sure why this channel/video showed up in my feed but most definitely this lady is a damn reasonable POA

[@syntaxed4365]

This was a fantastic tutorial, and as someone that is a lover of JS, it's nice to see this approach. I just have to say, I know that they are no longer required, but my brain just gets a completely unnecessary comfort out of using semicolons 😅 even though I can definitely agree that it looks cleaner without. I still have the desire to go through and add them everywhere! haha

[@aniakubow]

Thank you so much!! I really appreciate your comment 💚. Haha yes , I’m team no semi colons for my projects on here haha. I used to use them at work, but even then an extension added them in for me 😛

[@Ibloop]

hahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahah >=[

[@Shafty17]

Big thanks, Ania! With your help i managed to write a webscrapper to parse through all private repositories in my organization (had to add authorization as well), "read" it's package.json file, save it to a file, then run another conversion script that forms array of strings in certain way & then save it to .csv file, to be able to create a pivot table & analyze tech stack of our product :)

[@federicomontesdeoca8977]

Hello Ania, thank you for the quality of your videos ! I have almost 4 years of experience as a js dev, but still learning with your content. Just one question, why you install express and turn the app into a server, since you only need the http client (axios) in order to scrap the page?

[@kuldeepsharma7499]

Same question

[@PraneethParsharouthu]

same ques

[@zaliblk_media]

Great tutorial. Very well spoken. Very well communicated. Great Job Ania.

[@imagineaddis9050]

That is why learning docker was important for me. We can package our program to work on any machine without worrying about breaking changes from node or express. Thank you for the nvm tip!

[@havocblast]

yes docker is a life saver especially when you work on both mac and windows

[@kamertonaudiophileplayer847]

You provide an amazing technique for web scrapping. It would be good if you explain also how to manage sites using an anti scrapping technique, for example instead of just populate a text directly, they wrap every character in a different tag or even replace some digits with a similar text characters and so on.

[@KacperSieradziński]

This is great that you are going through the doc during video :-) It is great because you teach people how to read the doc... I do the same on my Python videos :-) Pozdrawiam! :)

[@vancouverrrr]

You have helped changed the lives of many, thank you for sharing your priceless knowledge

[@chrisjordan5849]

Thanks because of you i learned the basics of web scrapping 👍

[@maxmaksum4673]

Ania....it is just awesome...someday if you have holiday to Bali I will treat you and show you aound

[@ksivasuriyaprakash9976]

Thanks for update my knowledge on npm. video was short and sweet.

[@fiveminutesvideo]

Hi Ania, love the videos! Would love one on setting-up a web scraper to scrape every minute or so in a Litespeed server. Keep up the great work.

[@saudahmed2436]

Thanks Ania, had a little bit issues but after i got nodemon package installed and realized i was missing a . for calling the class it worked like a charm :). One thing to note i think is that windows doesnt have a NVM, so i had to als install one of those just so i could make sure i am using same version as you

[@sadique_x_]

Ania makes programming very interesting!

[@crisolivares7847]

its 6 am and I was waiting for this I love your channel

[@aniakubow]

Thanks for joining Cris!!

[@mrCetus]

Amazing tutorial ! Keep up the great work ! The web scraper would now enable us to develop alternatives for all applications using APIs and this gives us great confidence.

[@lloyd27z]

Everything you explained is quite clear and simple .So very helpful to learn.Thank you 😘

[@aniakubow]

Thanks so much Sue!!! It’s great to hear this feedback as I’m always trying to improve 😄💚

[@lloyd27z]

@@aniakubow looking forward to your new tutorial!

[@AlcantaraMC]

sister used 70% of the vid to set up everything huhu. anyways, it works at the end, and its what we came here for. Thanks!

[@whalebird]

great video and nice accent! It was pleasant listening to this tutorial, I've subscribed : )

[@gabyph2013]

Excellent video Ania! your explication is very simple and easy to understand, thank you

[@SurfingTheMentawais]

Thanks for the video. You are obviously an expert.

[@MeditationRelaxSleeps]

Thank you very much

[@nam-sdrawkcab]

YEAHH!!!! ILL BE SCRAPING ALL OF YOUR VIDEOS 😍

[@Sloth740]

Thanks for the video! I had some issues with the "start" script on Windows but found that "node index.js" not "nodemon index.js" worked for me.

[@kuoyulu6714]

Thanks, this is really clear and simple to follow!

[@shahedtheboss]

Thank you so much i needed to learn how to build a web scrapper You are the best❤️

[@andrasbradacs6016]

I was looking for a completely different thing, but you gave a good idea with this video! 🤗 Thanx! 😁

[@chikechris4411]

Great course. I enjoyed it.

[@KDevGaming1921]

Thank you this is a great help, It was working. I just had an issue with nodemon and installed it in my machine. I am just encountering the PORT not showing up on my terminal. I have an older version of node js though. Haven't tried using nvm yet. Going to try and practice that tip you gave. This was really helpful.

[@JordiVicensFarrus]

This is great Ania, thanks for your video! BTW just as an idea, would be great to have a second and a more advanced part, that shows how to do it in a site like Linkedin that requires Login or a site that has dynamic ids and classes. Cheerios!

[@calebcadainoo]

Amazing thanks for this tutorial😉

[@margin1636]

short, concise and useful. THANK YOU

[@im_cloudy]

I don't exactly understand why do we need to start an express server if we're trying to scrap an existing website. Is this some condition to run nodemon?

[@aniakubow]

It’s for part 2 :) we are going to show the results in the browser

[@ipunghendimunandar7827]

@@aniakubow so it can be called at endpoint right?

[@desarrollofacil411]

I was wondering the same, probably it will use a front end to get the url and display the results !

[@marianneazzopardi974]

Love your content Ania! I'm currently doing a bootcamp and your videos are helping me through!! :D

[@seniorqa_auto5367]

Great lesson! Special thanks for enlarging the screen with the code!

[@okidokiyowyow356]

Thank you so much. I always wanted to learn how to scrape the web I didn't know it's this easy.

[@FeedScrn]

Scrape it for what?.. if you don't mind me asking.

[@rukkychew]

Thanks for the intro. After that I was able to move around and play around to build a scraper for blogspot

[@odeddugma9748]

Thanks Ania. You are a great teacher ☺️ You make it looks so easy.

[@tarek_maza]

Congrats on the 200k subscribers

[@aniakubow]

Thank you Tarek!

[@sveinnarnarstefansson2720]

Well done, wonderful video, really makes it easy to start with scraping. Thanks Kubow!

[@hugodorian2165]

Omg 😱, waiting for this👩‍💻😊

[@hestJaevel]

Great video! Only feedback is: Use chapters so we can skip the steps to install node if we already have it for example :)

[@aashishsinghal7331]

Awesome Video. Please Do a Docker Begginer friendly project with react or MERN, I have been trying to learn that, With your videos it will be so much helpful.

[@aldobangun223]

Thanks Ania, this is verry helpful

[@csdevtauri]

thank you for your knowledge transmission Ms. Ania

[@Zo0m91]

Great tutorial! Love it! Thank you, Ania!

[@TheSilentBrush]

Outstanding! great project to try, thanks Ania

[@xmachina47x]

I think some of us, senior developers, will follow because we would like to see a surfing tutorial. .:D ..featuring the Lara Croft voice. Great coding! ;)

[@Kelz_369]

You should have a million subscribers... you're awesome

[@zippytyro]

Love it, so simple and unique way of teaching. Love from India >3000

[@vladimirchemir3737]

Amazing simple steps for extracting data! Maybe you ask about surfing web site structure with using some buttons or links clicks? Thanks!

[@greyice73]

This soooooo cleaaaaan I like it ! nice Video Ms Kubow

[@onurolce]

Really impressive tutorial. Thank you!:)

[@rahuldhiman6966]

Great tutorial as always ✌️❤️

[@vishaldhawan9236]

What a simple and superb video. Thanks !!

[@legisam1754]

I like 😂😂😂. Just found your channel and I'm loving it. Excellent content. Keep it up and keep em coming 👍👍👍

[@likewut007]

Very nice and understandable tutorial

[@trafalgarlaw4778]

learning my first line of code because my teacher Ania is stop eating beautiful

[@christianh.1160]

Thanks Ania for the video. You did a perfect job. All the best , stay healthy and have a wonderful week Cheers Christian

[@kimkroll4083]

you are amazing! love the way you pass the information (:(:

[@marcoantonioaguileratorres3754]

Really easy and helpful, I love the way you teaching ♥ ❤ I love your videos!!