Have real estate data analytics questions? Join the private Facebook Group to ask questions and learn from peers - facebook.com/groups/techinrealestate
@joeyalfieri17213 жыл бұрын
I've been watching web scraping/api tutorials for a month now, this is by far the best channel ive come across. You will blow up with more subscribers soon!
@TechInRealEstate3 жыл бұрын
Thank you I really appreciate it, glad it’s helped!
@dandan141414 Жыл бұрын
This was my first experience with Google Colab and RapidAPIs. Excellent tutorial -- and I have to say, I'm really blow away by this script. Very cool. I got it working following your directions and by making a copy of your project in google colab.
@TechInRealEstate Жыл бұрын
Thanks for watching Dan, glad to hear you found the tutorial easy to follow 👍
@hansenmarc4 ай бұрын
Great tutorial. Thanks!
@Razolution Жыл бұрын
How up to date is the data that you say was scraped from Rapid? Btw, great video
@TechInRealEstate Жыл бұрын
Thanks for watching! It should be real-time as per their description - rapidapi.com/apimaker/api/zillow-com1
@zacharyknepp9272 жыл бұрын
Did not know about rapid api before this. Thank you
@TechInRealEstate2 жыл бұрын
Thanks for watching Zachary!
@jordans42343 жыл бұрын
Thank you for posting this. I learned a lot of new things that'll be so helpful. One thing I'd really like to find is a metric like ("number of contacts" or "number of applications") per "number of days posted as available for rent." I don't really see those fields as available options through the API, but with so much data there, maybe I'm just missing it. Again, thanks for the great video
@TechInRealEstate3 жыл бұрын
Hi Jordan, thanks for watching! I do not see an option to retrieve rental listings fields. However, you could try requesting it by contacting the API maker. Or if you need a web scraper built to retrieve it you can try using Apify - apify.com/industries/real-estate?fpr=cy7vq
@jordans42343 жыл бұрын
@@TechInRealEstate Thanks for responding Ariel. Maybe I'll do that eventually once I get more comfortable with the wealth of info you provided on this video. In fact, when I looked at the sample file your report output (after changing a few of the parameters since I wanted to look at "for rent" and not "for sale"), I found the "price history" field to have some pretty good info in there. That might be kind of an alternative for me. Piggybacking off your comment to Jacob about outputting more than just a page at a time with a for loop or do while loop - that sounds great. I'm kind of new to programming and very new to python. Would it be like creating a list and appending a page to the list for each iteration? Thanks again. Looking forward to your class!
@TechInRealEstate3 жыл бұрын
@@jordans4234 Glad the price history field can help! Yes, it would be using a for loop to get the response for each page. Then joining it back together. I created a quick tutorial on how to do so, hope this helps! - bit.ly/3Cbytmk
@jordans42343 жыл бұрын
@@TechInRealEstate That tutorial was awesome. Exactly what I needed. Thank you!
@chideraokafor40872 жыл бұрын
Hi, Love your videos. After following your steps, I only got 40 results for HTX. I want to use the data for a ds project, how can I get more than 40 rows. Ideally, I'll love 10k+ rows so I can show my skill working with big data? Thank you for your tutorials!
@TechInRealEstate2 жыл бұрын
Hi Chidera thanks for watching! The API returns 40 properties at a time via the /propertyExtendedSearch endpoint. To obtain all pages you need to do a FOR LOOP to iterate through all pages available. The number of pages can be found in the "totalPages" key from the response. You can set the page you want to get data for via the "page" parameter. The resolution can also be found in the "popular questions" section of the API - rapidapi.com/apimaker/api/zillow-com1/tutorials/popular-questions Hope this helps!
@chideraokafor40872 жыл бұрын
@@TechInRealEstate a channel owner that responds! Thank youuu that worked. One last question.. do you know if we can get info for multiple cities at once using the search string? Or it’s only for 1 city at a time?
@TechInRealEstate2 жыл бұрын
@@chideraokafor4087 Glad it worked! Only 1 city at a time
@aloginov Жыл бұрын
Thanks for the video, very well done! On zillow website they have "Est. payment" / "Estimated monthly cost", which includes mortgage, taxes, HOA, etc.. Do you know if this API has this data, or it's calculated dynamically on the website?
@TechInRealEstate Жыл бұрын
Hi Alex thanks for watching! Yes you can extract that information from the same API using the /property endpoint. This gives property details including mortgage, tax, HOA. My upcoming course details how to extract this information to calculate metrics like cash flow
@aloginov Жыл бұрын
@@TechInRealEstate Right, I see that endpoint has all the data that goes into the "Estimated monthly cost" that is shown on the website, but you need to do this calculation on your own, right? Doesn't look like they have the calculated monthly payment in the API response.
@TechInRealEstate Жыл бұрын
@@aloginov Yes that's correct it needs to be derived
@johnniewalkeriii34372 жыл бұрын
Thank you for the amazing video. I really appreciate the helpful information. I ran into a problem when I converted the data frame. It ran the cell, but it's saying that I am missing the required parameters.
@TechInRealEstate2 жыл бұрын
You're welcome! Could you send over the notebook you are running to arielherrera@analyticsariel.com? I can take a look
@jacobjacob34313 жыл бұрын
Hey Ariel thanks so much for this video, super helpful! I had two questions regarding the API. Have you noticed you're only getting 40 results with the response? I was wondering if you know how to increase the amount of properties the API extracts? I was also having issues with retrieving just "Apartments". I altered the "house_type" argument to "Apartments" but didn't have any luck with that approach. Curious if you know of a way to fix these issues I'm experiencing?
@TechInRealEstate3 жыл бұрын
Hi Jacob, thanks for watching! The propertyExtendedSearch returns a key called 'totalPages'. For the default example, it has 2 pages. I tested it and if you use the parameter "page" and set the value as 2, then it returns the additional listings. For apartments, it looks like it only works when 'status_type' is 'ForRent'.
@TechInRealEstate3 жыл бұрын
I also do coaching for Python + Real Estate Data in case you are interested - www.analyticsariel.com/
@jacobjacob34313 жыл бұрын
@@TechInRealEstate Thanks Ariel, this helped a ton! I was just curious if there's a way to set the page variable to all the pages available? I ended up running my notebook 25 times and changing the page variable value 25 times but I was wondering if there was a more efficient method to do this?
@TechInRealEstate3 жыл бұрын
@@jacobjacob3431 Great question, if you set "totalPages" to a variable and perform a , then you should be able to dynamically pull each one. If that does not work then you can add a statement to continue retrieving the next page until there is no next page. Here's a quick guide for that - www.freecodecamp.org/news/python-do-while-loop-example/
@skyejohnson24772 жыл бұрын
@@TechInRealEstate Hi Ariel, thanks for the great video. After looking through the message thread here I found where you showed how to iterate through the different pages if the Total Pages is more than 1. Thank you for that! I am curious the area I have been searching with the parameters is showing around 57 listings but my csv file will only output 10 at most. How do I output all the listed properties that are being found through this search?
@justinhewitt57252 жыл бұрын
Hi Ariel, thanks for this great video. When there are multiple pages of results and you're using a while loop to iterate through the pages, how do you compile the results (a list?) I'm struggling to convert the list to a dataframe
@TechInRealEstate2 жыл бұрын
Hey Justin thanks for watching! In the for loop you should be able to compile each response or DataFrame into a list and concatenate them using Pandas - pandas.pydata.org/docs/reference/api/pandas.concat.html. Feel free to reach out to me directly at arielherrera@analyticsariel.com or on the Facebook Group if you need help! 😊
@justinhewitt57252 жыл бұрын
@@TechInRealEstate Thank you!
@lisadevito49562 жыл бұрын
Really informative. Thank you! Is there a way to find properties a list of properities that were previously for sale, pending or under contract but not sold for a given geographic area, e.g., zip code? Thank you very much.
@TechInRealEstate2 жыл бұрын
Thanks for watching Lisa! I have not found a straight forward way to locate expired listings yet. You can take the approach of retrieving daily listings from a zip code via the API. Store the data in a table. Implement logic to determine if a property was on your list prior date vs. current date. For those properties that are no longer listed, cross-check via the "property-details" api to see if it was recently sold or not. If the property was not sold, then it should meet your criteria. Feel free to schedule time with me if you have any further questions via my website
@lisadevito49562 жыл бұрын
@@TechInRealEstate Thank you very much, Ariel! That makes sense as a workaround.
@deontebodrick8610 Жыл бұрын
hi i am trying to use this to integrate it into podio do you have a video on how to do this
@TechInRealEstate Жыл бұрын
Hi Deonte thanks for watching! I messaged you back on the most recent comment on steps to find integration help 👍
@haoxuesong18143 жыл бұрын
Super helpful! Can we call Zillow API directly rather than using RapidAPI to avoid the fee? If so, can you have a tutorial on that?
@TechInRealEstate3 жыл бұрын
Thanks for watching! Unfortunately the free API Zillow managed is no longer active. You will need to request an API key through Bridge Interactive. I believe there is a cost associated with them too
@sidverma18883 жыл бұрын
Awesome content, great delivery! Thank you!
@TechInRealEstate3 жыл бұрын
Thanks for watching!
@ryanmakipour91933 жыл бұрын
Hi Ariel, I've scraped my county website for the addresses and have them in a df. Anyway I can input those addresses into rapidapi and get a list of zpids?
@TechInRealEstate3 жыл бұрын
Hi Ryan, Zillow used to have an easy way to grab ZPID by entering an address, however they deprecated their public API as of Sept. I recently created a backdoor way of retrieving ZPID for a list of addresses by using the Google library in Python. It searches the address as a Google search then extracts the ZPID from the url. I’ll be posting a video on it soon
@ryanmakipour91933 жыл бұрын
@@TechInRealEstate Thanks!
@JorgeJaramillocom3 жыл бұрын
Great video, Ariel, thank u.
@TechInRealEstate3 жыл бұрын
Thanks for watching!
@sonjaburke90973 жыл бұрын
thank you so much for this!! this is so helpful!!
@TechInRealEstate3 жыл бұрын
Thanks for watching, glad it could help!
@Nearco10002 жыл бұрын
is it any tutorial when prices change in a particular property notify you with an email?.
@TechInRealEstate2 жыл бұрын
I can look to create a video on it. The solution would require to query the API repeatedly (I.e. every hour, once a day). If the property is new, then save the data to a database (Google Sheets or CSV file could work). If the property exists in the database already, but the price is different then log the change and send an email - either within the same code or with Zapier
@creneemugo942 жыл бұрын
Does the api give you more than just one picture of the property?
@TechInRealEstate2 жыл бұрын
Thanks for watching! Yes, if you pass in the ZPID of the property into the "GET /images (Property images and video)" endpoint, then you can get all of the images listed. You will see it on the left-hand panel on the API page - rapidapi.com/apimaker/api/zillow-com1/
@creneemugo942 жыл бұрын
@@TechInRealEstate Thanks!
@TheLegendOfNiko2 жыл бұрын
Recently was informed that the Zillow API is now deprecated...
@toberighteous2 жыл бұрын
Thanks a lot
@TechInRealEstate2 жыл бұрын
Thanks for watching!
@xairk_2 жыл бұрын
I see you ran into the XML error in your Colab notebook too. Now I can rest I know its not something on my end.
@TechInRealEstate2 жыл бұрын
Thanks for watching! You can check out my article on the medium for an up-to-date solution. Logic had to change due to the library no longer working to get ZPID - levelup.gitconnected.com/how-to-get-property-data-to-analyze-your-market-using-python-e9ef5336d141
@martinmorenocerda86872 жыл бұрын
Now you have to pay for Zillow API
@TechInRealEstate2 жыл бұрын
Thanks for watching Martin! Yes the data providers charge for larger volume of data requests. If you’d like to stick to a free option you can create your own web scraping bot for the website and run the code locally 👍