Scrape Sensor in Home Assistant - Extract Info From Websites

[@stephanieboyd4373]

Will, love what your doing and your videos always entertain me, you are really animated and upbeat whilst explaining things clearly. Your presentations always make me smile as you come across like a really excited puppy, full of excitement, wanting to share info and help us all along. Keep it up I love it, it’s so refreshing!

[@WillSurridgeTech]

haha, thank you very much!

[@ukaszZalewskiLooker]

Hi Will, now I found out a good few usages for HA. Thanks a lot!

[@WillSurridgeTech]

Happy to help!

[@dameck9570]

Wow! Your channel is amazing. Thanks man! Definetly a follow worth

[@WillSurridgeTech]

Awesome, thank you!

[@forschi]

For some reason my sensors for KZbin via Socialblade still shows unknown after adding the User-Agent The log reads "2021-04-25 10:34:26 ERROR (MainThread) [homeassistant.components.scrape.sensor] Unable to extract data from HTML for Forschi KZbin Views"

[@gonzalolaplata]

I was just looking that!! You are the best!!!

[@WillSurridgeTech]

Glad I could help!

[@nakiros24]

How do you set the interval between scrapes ?

[@nakiros24]

Nevermind. The answer can be: Use Multiscrape from HACS instead and set scan_interval. Also it lets you scrape multiple information in one go. Maybe a new/updated video?

[@matthewpetersen9791]

css selector tester seems to be gone now. any other suggestions?

[@giordanogobbo]

Hi will, clear and well explained tutorial as always! Thank you for the great resources you are uploading about home assistant!!

[@WillSurridgeTech]

My pleasure! Glad it helps :)

[@alcidesmarcano]

Great content and very well explained!!! thanks again Will for an awesome job to HA community... I appreciate that!!!

[@WillSurridgeTech]

Thank you, glad you like it!

[@Hashterix]

Annoyingly my council's bin collection site uses the same ID for two types of collections on the same page and no way to differentiate :( ugh!

[@Grimroper]

Will, yet another great video. I really appreciate you covering different subjects to other KZbinrs. Keep up the good work.

[@WillSurridgeTech]

Thank you! More to come!

[@thebaldfox]

Great video. I’m a little stuck. If I was looking for the average review or number of reviews of a product on Amazon, what CSS would you be using? I’m trying but getting nowhere. I’m wondering if it’s because Amazon is blocking (even with the Mozilla 5.0 header) of a, getting the CSS selection wrong....

[@nagi603]

I wish it had two extra functionalities: - scrape multiple data points from a single pull - set how often the data is pulled. (e.g.: zero reason to pull info every 30s that's only refreshed every week) Also, you can just r-click in the inspect "elements" on whatever you want and go copy -> copy selector

[@Videxthebest]

you can install multiscrub from hacs. Then you can selec freqency, and yes multiple data points.

[@hamhumtube]

thanks for this good vid. what is frequency sensor is fetching values?

[@hamhumtube]

scan_interval: 3600 www.home-assistant.io/integrations/scrape/#bom-weather

[@WillSurridgeTech]

Yeh the scan interval is how often (in seconds) it will scrape the website for an update

[@shane_taylor]

How do you determine the polling interval in HA?

[@tutorialz]

The Link for "CSS Selector Tester" is faulty.

[@CommittotheIndian]

I'm trying to scrape the weather alerts from the NWS alert page. Alerts show up as an #id with an h3. If there are multiple alerts there are multiple h3. Is there a way to scrape multiple ids? resource: url select: "#content h3"

[@photelegy]

How to add attributes to your scrape-sensor? I have a scrape-sensor which gets the water-temperature from a lido. So I want the coordinates as attributes. like: latitude: 47.3519 longitude: 8.5360 If possible I would even like to have an attribute with the website-link where I scraped the information. Are there some informations about how I can create such attributes or how they have to be named ("link:", "source:", "web:", ...)?

[@jno8301]

CSS selector tester is no longer available: pluginsaddonsextensions.com/google-chrome/css-selector-tester-chrome-extension_23245

[@DesertGardenPrepper]

I tried to use this to make a credit karma credit score sensor, but couldn't figure it out. thanks for the tutorial. I don't understand selectors or maybe it is not possible with that website.

[@WillSurridgeTech]

Its probably not possible I'm afraid - scrape sensors are very basic

[@Aletzi1]

Could you make video about Scrape sensor with login field ?

[@WillSurridgeTech]

I don't believe that's possible because of the limitations of the sensor

[@moyamiga]

thnaks a lot

[@ShivaShankars]

I have my data in cloud and by doing http get request I can get that sensor values is it possible to create a sensor with http get request if yes please share the tutorial this could help me lot waiting for your video or any response

[@davidparil8100]

Thank you

[@WillSurridgeTech]

No worries

[@TheHellis]

Could have used the KZbin API instead.

[@WillSurridgeTech]

This is true, but then I wouldn't have had that example!

[@rolidcz7954]

The over-the-top speech is completely unnecessary.