How to build a web scraper for Zillow data - Python Tutorial

Follow along as I demonstrate how to extract the underlying data from a Google map, and use that knowledge to build a web scraper for Zillow data!
Link to Source Code and Import Requirements:
github.com/Jon...

[@saavedra29orig69]

NIce video and explaination focusing on the key points. But to be 100% correct and honest you have to say that we can't go beyond page 20! And this is very important because figuring out this makes all the difference between amateur and proffesional web scraping :)

[@tzhaki2354]

Do you know how I could do this to scrape info from a single listing?

[@user-vs3eg9yn2q]

I have a list of say 1,000 addresses. I don’t have ZPIDs. Is the someway to earth for the property details of each address, given I don’t have the ZPIDs?

[@nicolasvargas6846]

Nice video man, thank you very much

[@axel9546]

Hi, very nice video! Is it possible to webscrape some data and use that information to make a nice dashboard, tailored only to some data we are interested in? Topic: real estate auctions

[@Zaycancode]

Great tutorial!

[@eliesersantos4936]

Great. Unfortunately it is showing an error. (I think the tag that queries address/price and beds in zillow has been changed, and I can't find this part of the html) The error is: AttributeError: 'NoneType' object has no attribute 'find' when it tries to execute the line addresses = href.find('address') If anyone has an idea, it would be great to learn from you.

[@sharathpalthepu4408]

I am also having the same error

[@tzhaki2354]

How could I find the zestimate?

[@kanishk9593]

were you able to find out?

[@juannymoney9901]

not sure if zillow changed anything but when I checked the network settings in fetch/xhr and couldn’t find the get method

[@raajkiran1999]

yeah it did..

[@nolansaraceni2209]

So this is a very good step by step. But clearly a commonality is the tags have changed. list-card-link no longer exists. Idk about anyone else but form what I am seeing all i call see is property-card-link but even that doesn't seem to be giving me the correct output which are the links themselves. The df is outputting all nan values so i am assuming all the tags have been changed. I feel like the most challenging is getting those tag correct. Also, has anyone else been only get an X number of rows for their output? When i run df_lists or df_list.head(20) it only outputs max rows of 8 in grand total. Some assistance would be nice if anyone could help. Ill continue to update my findings, failures, and successes as i work through this. This is one of the better vids out there and it would be nice if the author could help as well.

[@nolansaraceni2209]

update - was able to capture the data by pinging the API through RapidApi. Looks like with them you can hit the API 30x /month with their free version. Saved me tons of time and heartache. As much as i would like to write all the code this was a quick and easy solution. Not going to provide link just look up Rapid API and sign up.

[@pablomartinez1504]

Hey I get an error while using the for loop to scrape the links. It says that the href variable has no attribute 'find'. Any clue as to what it could be?

[@SajjadAli-me8bm]

after defining href variable type if statement like(if href:) and thats it

[@asrarmostofa818]

I find this error AttributeError Traceback (most recent call last) ~\AppData\Local\Temp/ipykernel_456/2634912797.py in 20 for link in soup.find_all("article"): 21 href = link.find('a',class_="list-card-link") ---> 22 addresses = href.find('address') 23 addresses.extract() 24 urls.append(href) AttributeError: 'NoneType' object has no attribute 'find'

[@BAbout1t]

@@SajjadAli-me8bm This worked for me, thank you my friend!

[@riyavirani7774]

@@asrarmostofa818 were you able to solve this error? And @ravenrecords834 can you help if you have any idea how to resolve it?

[@jason19920816ify]

great content

[@MsAS092]

df_list[0] .For this line I am not getting any results.Just this column names with no records in it - prices address beds Can some one help me with it

[@kanishk9593]

same

[@Ahmed7255]

Thanks Raven, is it possible to do that in excel VBA? thanks

[@kitwesselhoeft8026]

LOL WHY just use python homie it might seem like a lot atm, but it will forsure be worth it