Guide to Grafana 101: Getting Started With Alerts

[@TimescaleDB]

Handy Bookmarks of the main parts of the session: 2:42 - Alerting Principles 5:26 - Alerting in Grafana 14:53 - Let's Code - Implementing 3 alerting rules and 3 notification channels 16:40 - Demo Alert 1: Alert with FOR sent via Slack 25:58 - Lifecycle of Alerts with FOR 30:10 - Demo Alert 2: Alert without FOR sent via PagerDuty 35:56 - Lifecycle of Alerts without FOR 39:43 - Demo Alert 3: Alert with No Data sent via OpsGenie 44:45 - Lifecycle of Alerts with No Data 48:07 - Resources and Next Steps

[@vanglequy7844]

Sorry if I missed it somewhere, but it is useful to point out the version of grafana and other things you are demoing. Thanks for very useful tutorials anyway.

[@sugardaddy8471]

Well done! Well explained! The notification channels are well known and common. :)

[@avthars]

Thank you! Really glad you found it useful!

[@wildflowers465]

Great video! It's very confusing that the alert frequency definition is above the condition; especially seeing as it's the opposite, spatially, in the alerts not created from graphs in newer versions of grafana.

[@victornoagbodji]

🙏 🙏 😄 another great session. thanks❕

[@avthars]

Thank you! Really appreciate it!

[@hamidja1537]

Thank you for sharing

[@avthars]

Glad you found it useful!

[@johannesschmidt8611]

Really well explained, thank you!

[@avthars]

Thank you!

[@shakibamoshiri2503]

the best one so far 👍

[@evanlee4364]

Is it free for creating alerts or paid in grafana

[@avthars]

The open source version of Grafana is free to use. So yes, creating alerts are free and you can create as many of them as you want. Of course, you may have to pay for hosted and managed Grafana, on platforms like Grafana Cloud or Timescale Cloud

[@MrSupermansmother]

great stuff

[@avthars]

Thank you Rico!

[@cuteangel1726]

For a circuit breaker alert what value should be put in FOR

[@Alicetian_Deutschland]

thanks for the video, now I understand why i can't create an alert based on table format. so how could we create an alert based on table format?

[@danielantunes3982]

Hi, very good session but I had a doubt. Is it possible that a created alarm system only triggers the alarm between two different times? For example, between 09.00am and 23.00pm?

[@avthars]

You could probably write that in a WHERE clause in your an SQL statement which defines the trigger conditions

[@dickymr1878]

How to show the red ticking background when alert is firing? My grafana is 9.1.7 but only show red hearth when firing

[@user-pg3pv8eh4e]

Thanks for the very informative video @TimescaleDB. Could you please help clarify this? Is it possible to create an alert for long running queries in Redshift? The notification message should contain the query_id and user_id. It could be either be by setting the query monitoring rule within Redshift and then fetching the log from table or by extracting results from table (both using the query) The goal is to send couple of columns from the result set of the query as the notification message on Slack. Looking forward to your reply. Tried every other possible solution but in vain. Really hoping there's a way around it using Grafana.

[@feynmaz]

thank you for the video

[@harsha20896]

Hi Avthar, If we want to create an alert for the entire dashboard, is there any feasibility for doing it.

[@avthars]

I think alerts are per panel in Grafana -- but you could try hack around that

[@davidj7607]

You created 1 alert for 1 server and 1 metric. How do you do this in large scale applications of 100 servers? If I want a separate alert for CPU utilization on 100 servers how do I set this up?

[@avthars]

You could aggregate all your metrics in one database table and write one query to alert on that table. This is something we see people use TimescaleDB for. Alternatively, you can use something like Prometheus alerts manager to programmatically alert.

[@socencounter]

Is it possible to configure an alert dashboard that simply lists the alerts as they come and go, without a notification channel, we simply want to manually view all alerts in one dashboard/list which refreshes every 5 minutes?

[@prawininstro]

Great session....while setting up email alert rule,alerting state provide message and metric name. But after it back to OK state the same message again triggering without metrics. How can we solve this.Ok state message bit mis leading🤔

[@AlgoTribes]

i am trying to include an alert image with the notification message..couldn't do it...i am receiving the alert message but without the image..can someone tell what needs to be done..i am trying to send to the slack using webhook

[@chavyakhanna8874]

Hello, First of all great video! However, there's no mention on how we can use "Tired" Notification system in order to send low severity alerts to one channel and high severity alerts to another, is there some reference that we can follow?

[@vijaysinghrana6027]

sir please help to find percentage or division in grafana please suggest any solution sir

[@gautamdikshit9888]

Hello, I'm using grafana 8 in a GCP VM. I'm monitoring CloudSQL metrics for a number of services which are also running in GCP VMs. My requirement is to monitor 1) Memory Utilization 2) CPU Utilization and 3) Server Uptime and send alerts to the concerned staff if the usage is more than 95%. I'm not using prometheus+cadvisor for this. I'm monitoring it directly from the GCP plugin in grafana. By default, the unit of y axis is in percentile. If I retain that unit, what number should I have to give in the condition such that I will receive alerts exactly at 95%? Also, even if I change the y axis unit from percentile to percentage and save the alert rule, the state immediately goes from OK to pending even if usage is less than 1%. I receive a lot of alerts unnecessarily. The condition is this - WHEN avg() OF query(A, 5min, now) IS ABOVE 95. What am I doing wrong here? How do I set the condition correctly? Honestly, I don't even understand the difference between percentage and percentile. Please help.

[@letbrick2286]

Is it possible to use any variables inside messages?

[@vijaysinghrana6027]

sir Kindly request please suggest me any workaround to find division or percentage in grafana or panel with elasticsearch please help sir

[@pramitshah931]

Hey, great video, I have a question...how does the percent_diff function work? I have set up alerting rule where I should get an alert if the percentage difference of the last 10 days is outside range (-10, 10) but instead of the last 10 days, it is considering all the days. I am using query(A, 240h, now) to mention the last 10 days

[@pramitshah931]

@@lorrainep876 hey thanks for the reply, I looked into it and I get the formula but my problem is that the series it takes for the calculation is the whole series instead of the subset of 10 days that I need! If you know anything about it that would be great else thanks for all the help!

[@Abhishekkumar-be9nn]

I am in the process of setting up a few alerts in Grafana for a couple different dashboard panels. I noticed that it shows vertical markers for each “Alerting” status and each “OK” status. That’s great to be able to see when/where it triggered, however, it dirties up the panel-graph and makes it look more confusing such that I see a bunch of extra vertical lines when trying to look at the graph. Is there a way to suppress these markers?

[@avthars]

I'd say reach out to Grafana / their forum to find out more

[@NiteshKumar-tc7jb]

Is Grafana Alert feature is available with open source licence or it only comes with enterprise licence?

[@TimescaleDB]

Hi Nitesh, Yes, it’s available in the open source version.

[@sateeshkumar1145]

Great video thank you I want to include time stamp in the email of resolved alert. How can I do that. Please your response will be greatly appreciated

[@manojgautam3559]

Does Grafana support Loki as datasource to create alert?

[@avthars]

Check their docs

[@BILLYBRAY-wb1tv]

10 minute talk drawn out to 52 min

[@chetanchetan-dn4pr]

on my grafana no alert icon show this is very confusing i have grafana/grafana-enterprise what is your virsion or it is free ???