Scrapy in 30 Minutes (start here.)

[@vouky8747]

Hey John, thank you! You actually helped me with a project I was stuck on. Great video!

[@elu1]

What a wonderful series, gentleman! love it ! Now I think I am more knowledgeable to be able to follow along with other stuff from this channel. Thank you very much!

[@JohnWatsonRooney]

Glad you enjoyed it!

[@faustozambrano4901]

Great stuff... Thank you John

[@アズネラ]

Great tutorial!, This series been pretty well structured and well explained!

[@TheBenchPressBoss]

Incredible scrapy video big JOhn thanks homie !

[@JohnWatsonRooney]

thanks!

[@DietervanderWesthuizen]

Excellent video once again. Thanks a million John.

[@bakasenpaidesu]

OP ❤. I thought scrapy will be overwhelming but it's great

[@JohnWatsonRooney]

Thanks 😅 also neovim config soon ;D

[@bakasenpaidesu]

@@JohnWatsonRooney I'll be waiting for that :)

[@tomlento6068]

great intro!

[@talhanadeem1720]

Amazing Man. I'm surprised! yoo Thanks

[@alan_tucker]

Another great video; keep up the great content.

[@JohnWatsonRooney]

Thanks mate appreciate it

[@michaelmuolokwu5039]

Amazing video! Are there any drawbacks in using crawlers instead of using normal spiders?

[@JohnWatsonRooney]

Thanks! No, they just have slightly different roles - use whichever suits your needs

[@surajsinghrajput5862]

Great tutorial John, thanks for sharing. I am trying to scrape a website that requires a login. Am able to do that by defining a function inside the class but not able to figure out how to scrawl from that point.

[@flaskwater44]

Great Tutorial!

[@oc3academy]

Great video. Could you please share your neovim configuration? Or what LSP are using?

[@loicleray]

Fuarrrk yeah. Thank you.

[@daved8698]

old man I just saw your videos 3yrs ago then hansome

[@Aidas_Li]

Love it

[@valuetraveler2026]

How about deployment? Any tips on which is best for custom deployment? A video perhaps?

[@ketankumar5689]

I am a newbie in scrapy. I am trying to access some info. in a job site(Monster) like jobtitle, company name, posting date etc.present in a job card through scrapy shell command but i am unable to do so and getting empty list even though i provide exact classname. What should i do or any video has been created to access such elements? Any help 🙏 P.S. - i tried and am able to access some elements in header and footer section but unable to access elements from cards which display info of each job.

[@mmcmobile4869]

🧡

[@julianvargas228]

great video!!

[@mecrayavcin]

🥰Excellent video. Thanks I'm confused about scraping Javascript based sites. Could you please make a fresh video about it. Thanks very much love you.

[@JohnVUceo]

open dev tools, go to network tab, press perserve logs. refresh page, click stuff on page and see which request has data u need. Or be a scrub and use browser 😛

[@SquirtleBaiano]

I had to use selenium to bypass this same problem easily, but I'm curious about it aswell

[@soul_maestro]

17:24 : "working just fine" while the price field isn't the price at all, but it look like the tilte or a description - even the one hoovered over after stopping it a couple seconds later...

[@JohnWatsonRooney]

I had the wrong selector which I fixed later in the video

[@alebeatz_5179]

Whats system John use ?

[@JohnWatsonRooney]

Linux, i3wm and neovim

[@phumudzomuvhango4193]

Thanks boss

[@BeyonderW]

Can you do a scrapy tutorial with C# ?

[@m.arehman8652]

Great

[@valuetraveler2026]

I found Scrapy to be much overkill but maybe they have improved it since?

[@JohnWatsonRooney]

depends on the use case but it shines for crawling and managing multi spiders in my opinion

[@valuetraveler2026]

yes I can imagine thats where it counts@@JohnWatsonRooney

[@valuetraveler2026]

Where do good scraper engineers advertise their services?

[@prohacker5086]

KZbin

[@franciscooteiza]

ebutuoY

[@valuetraveler2026]

You mean here? I dont see any.@@prohacker5086

[@sandunwijethunga6787]

Your the best John. thank you so much

[@daved8698]

ctrl +c ; ctrl +v

[@JohnVUceo]

Scrapy, more like Crapy 🙈

[@rastaricky9584]

Way too much jumping around in this video…from shell, n I’m, to documentation…also the dialogue is everywhere…but thanks for taking the time to