Python Scrapy Tutorial - 9 - Extracting data w/ CSS Selectors

[@Fmorrell100]

Really great tutorial. He goes through it step by step in order, so you have a clear understanding. That helps a lot

[@onksgk]

You just made my day! From last 2-3 days I am trying to learn web scraping but there are complicated videos on other channels .Today I watched your first 3 videos and then I got it you are going to kill it🔥🔥 and now suggesting that tool ...it became 💎💎. Thank you. You have one more subscriber.

[@Nirmal_rai]

bro really you are so underrated. bro you are teaching so well that i ,a mechanical engineering is doing this like its nothing. keep the hard work on . i love your videos and your teching style

[@devang1956]

You just gave me a breather with the Chrome Extension. Amazing video series! Keep up the good work. You earned a subscribe :)

[@echezonaazubike8054]

i have subscribed you nailed it bro i am Nigerian and we loved Indians

[@jerinfrancis4509]

oh my god ! how am I gonna pay you back. YOU JUST MADE MY DAY. Speechless. The chrome extension is damn good bro. Thank you so much for this particular video !!!!

[@buildwithpython]

Glad I could help.

[@MarsLanding91]

Wow. I was going to wait until the last video to comment but I had to do it now. THANK YOU for these videos! They are SUPER helpful.

[@NirmalSilwal]

your explanations are amazing, very engaging and interesting stuffs

[@debruppaul8239]

Bhiya I don't know how to thank you great job and thanks a lot,you just made selecting piece of cake,thanks again

[@rhn122]

Once again, great tutorial! Clear and straightforward!

[@truverol8205]

wow your tutorial is so great! good job

[@asifmohammed1270]

I wish I found this video much earlier. Just saved a lot of time and effort.

[@PMDJBMS]

I'm really finding your work helpful for a research project I'm on in the UK. A big thank you for your excellent videos

[@buildwithpython]

Glad I could help.

[@zeeshanshani8896]

Hey, great tutorial bhai ! What i get from it is that by using shell command on the terminal we can dynamically scrape data like we do with python request and beautiful soup. Thanks for uploading them.

[@5caioc]

Incredible series!!!! Thanks a lot!! The extension you recommended is extremely helpful

[@simasj1]

Nice! The video is so clear, I think you should consider a lecturer carrier! You have a gift to explain complicated things very simply.

[@simasj1]

P.S. www.buildwithpython.com does not work - it says "The account for this site no longer active. This content is not currently available."

[@buildwithpython]

Yeah it's not up. I didn't know people were even checking it out!

[@LV7agent]

a really good hands-on tutorial, 10x alot

[@umerimran3833]

Brother you're outstanding

[@sgerodes]

Your tutorial is pure magic. Thank you very much!

[@RobertRoman]

This video is Gold! I'm excited to learn web scraping now :D

[@buildwithpython]

Glad I could help.

[@web-dev-zargo]

OMG! IT IS WONDERFUL!

[@morganv3732]

Pure Gold. Thank you!

[@nathanheath3756]

Subscribed! Very helpful information ! definitely keep these videos coming!

[@VigneshSahoo]

Selector gadget is awesome. Thanks mate.

[@cstech2364]

Thank You So Much Sir 👍👍

[@sgerodes]

The selector tool is magic

[@linusjohansson3164]

How do you get the last command in pycharm? Up does not work here for me. I have to write response...etc all over again which is annoying.

[@alisiraydemir]

Just want to say Thank you!

[@md.mahabuburrahman8544]

Wow great tutorial

[@Abdullahkbc]

this extension is perfect. thank u so much.

[@hasnainahmed6706]

It is giving empty list on my pc at 11:12 please help me out.

[@whayAl]

many thanks for all your teachings

[@da_ta]

very exceptional excellent work thanks for doing this

[@redfeather22sa]

Your very Good !!!!

[@imaduddinsheikh3546]

Thank you so much for your Scrapy tutorials! However at 10:36, I tried running scrapy shell command on the Amazon website, and the response came back with a 503 code. How do I fix this? And, what's the issue behind it? I am running Windows 10.

[@imaduddinsheikh3546]

Nevermind, I fixed the issue. I reduced the concurrent requests in the settings.py file to 1(I also added a user agent for Chrome browser with the latest version in the same file)

[@ItsBen27]

@@imaduddinsheikh3546 THANK YOU!!! Your comment saved me from a lifetime of searching for the fix!

[@amitkumar-yu6yz]

great video man.ver very thanku

[@babuji010]

Nicely explained 👍. Thanks. Have a question. It looks like the "response" object under "Available Scrapy objects:" is responsible for response.css. is that right? There is no "response" object in the list for the web link I try to work on. Any suggestions? Ideas? Please.

[@Imperialcodex1]

Thanks man. 💪

[@souilahmaher7188]

You're great instructor!

[@buildwithpython]

thanks

[@167tejaswini]

I have tried below ..but still not displaying here anything >>> response.css(".a-color-base.a-text-normal").extract() [] >>> response.css(".a-text-normal::text").extract() [] >>> response.css("a-text-normal").extract() []

[@buildwithpython]

Did you try it on the example website I gave?

[@ReasonToKeepGoing]

Solved the issue in two different ways, response.css(".a-color-base.a-text-normal::text").getall() and response.css(".a-color-base.a-text-normal::text").extract()

[@mihirthakur917]

Facing the same. Worked with quotes to scrape but not with amazon. I tried it with Flipkart and it worked

[@buildwithpython]

@@mihirthakur917 hey I have a separate video for Amazon in the same playlist

[@CSSuccessGamer]

amazon must have found this video and decided to block scrapers...

[@giotsas]

Great video, just a small correction. In 09:00 you mention [1] is the first index of the list of authors. It's the second index.

[@anupamasingh1239]

Hey, I'm getting error 404 while scraping the amazon website which you gave. I tied finding solution but was not able to fix it. Can you please help me out on this?

[@amitjamwal1985]

Very helpful videos. thanks a lot :)

[@emm5138]

Great video! Thanks a lot!

[@jiyarahman2673]

Hi, I am following the code as you guide, but I am getting a Empty list for response.css even for previous video I got empty value can you explain me why?

[@andrejohnv]

Did you get it now? I'm getting empty list lol.

[@naveenkumardongre]

same I too getting empty list

[@bhavyajain2034]

will appreciate your help

[@CSSuccessGamer]

10:34 im getting 503 error from terminal for amazon

[@CSSuccessGamer]

it says forbidden by robots, what do i do?

[@aakashkhadka3835]

same here

[@web_devs]

Cant scrape amazon... returns empty list >>> response.css(".acs-product-block__product-title .a-truncate-cut::text").extract() [] any help..?

[@yudhvirsingh9909]

same here

[@CSSuccessGamer]

i have same error,

[@CSSuccessGamer]

did u find an alternate way?

[@juann9880]

What if response =403, I can't extract anything?

[@harshadmanglani1309]

The series is great, although there's something wrong with the quotestoscrape website, it gives me a twisted internet error, works for every other website though. Thanks.

[@jamezz2181]

how do you remove blank space like and spaces when it just has a bunch of them from it

[@ubaidmanzoorwani7491]

I am trying to scrap data from youtube but it is returning an empty list every time . please tell me what to do.

[@gurjeetkaur3626]

Helped a great....but after half of video .. view not clear

[@DarkScizor]

Hi there, I had a question. I wanted to parse the alt text off of an img. How would I go about this? I appreciate any help you can give!

[@pranaydas3320]

Use attrib("alt")

[@ThallaSampathKumar]

yes it is fine but it is not working for all websites returning me an empty list

[@Coney_island23]

great!

[@as-px2mv]

thanks a lot!

[@brendenandrews6965]

How to access the previous commands in the shell.. usually when I'm in the terminal I am able to access the previous command using the up button but in the shell I am not able to do the same as shown in the video.. can anyone help me with this..

[@Jgs8115]

Saved my neck thanks man

[@jabiraziz1219]

Just Wowww

[@shaikhanuman8012]

bro i am getting 503 error code how could i fix it please tell me brother

[@teo-medesi]

Go out back, find the biggest stick you can find, keep hitting your pc until it works. I hope this helped!

[@shaikhanuman8012]

@@teo-medesi I tried brother due to that I bought new pc (😄 I fixed the error)

[@shaikhanuman8012]

@@teo-medesithanks brother for providing valuable knowledge

[@teo-medesi]

@@shaikhanuman8012 Any time!

[@shaikhanuman8012]

@@teo-medesi tq sir

[@bhavyajain2034]

SIR, while running scrapy shell command, terminal is raising a ValueError : invalid hostname: 'http

[@jss2754]

i have some question of this video. As you know ,scrapy is have a two ways for xpath that is css and xpath. i wonder why are u using css on your video .

[@buildwithpython]

In the next video I use xpath. I just like CSS selectors

[@jss2754]

@@buildwithpython thank u for comment!!

[@anrm6]

You are god

[@TranLamYoutube]

Amazon's source could change, I can't crawl data, elements are render from script, not from sample Html

[@Pandazaar]

hey I was trying to follow along this video and I think you can no longer use response.css, because it was removed I guess, the error I get is: AttributeError: 'function' object has no attribute 'css'

[@buildwithpython]

Nope it's not removed. Don't think your scrap is installed properly.

[@Pandazaar]

@@buildwithpython oh, i did cd quotetutorial before opening shell,my bad

[@babuji010]

@@Pandazaar Hey, I am getting same error. Can you explain what went wrong? And solution pls. Thanks

[@Pandazaar]

@@babuji010 just type "cd .." and then open the shell

[@hau_hau_happu_singh]

I am continuously getting a null array, after using selector gadget.

[@danielamorariu6722]

you're probably getting a 503 error, which means service is unavailable. I solved this by specifying a user agent in settings.py and disabling the cookies, also in settings.py . User agent can be Mozzila 5.0 etc etc ( check explanations here: www.scrapehero.com/how-to-fake-and-rotate-user-agents-using-python-3/)

[@aakanshasingh9680]

scrapy crawl quotes. -> not returning anything. Nothing is displayed on the terminal

[@aakanshasingh9680]

Basically, the function parse is not getting executed. Anything else written outside parse but inside the class is getting executed.

[@doubled9645]

thx bro

[@buildwithpython]

No problem

[@SquaredbyX]

Should be called a css De-selector

[@Bihari_Chaman]

It is not a list its Array

[@Yuri-xx2gi]

It's a list, this is not C, in python they're called like that

[@vigneshsivasubramanian9193]

is it only me or the entire headphones shakes and trembles when he presses his keys think they are feared of him please dont overuse it and give some rest for both u and your keyboard

[@_thoneeer3220]

wow

[@danielhoyos6788]

Why am I getting an empty list when scraping Amazon?