Understanding Counter Rates and Increases in PromQL | Reset Handling, Extrapolation, Edge Cases

In this video, I explain the exact value calculation behaviors of the rate(), irate(), and increase() functions in PromQL for computing rates of increase for counter metrics, show how they are different, how they deal with counter resets, how and when they extrapolate values, and which one you should use.
Check out my Prometheus training courses if you want to learn Prometheus in a structured way from the ground up:
training.promlabs.com/
Chapters:
00:00 Introduction
00:45 Absolute Counter Values and Why We Want Rates
01:18 The Three Counter Increase Functions
02:36 Behavior of rate() and increase()
03:02 Handling Counter Resets
04:41 Calculating the rate() and increase() Slope
05:08 Extrapolating the Return Value for the increase() Function
06:42 Confusing Extrapolation for Slow-Moving Counters
07:37 Limiting Extrapolation for Series Starting or Ending Under the Window
08:26 Limiting Extrapolation to Zero Sample Values
08:52 The irate() Function
09:59 Which Function Should You Use?
10:30 Outro & PromLabs Trainings
---------------------------------------------------------------------------
CREDITS: "Subscribe Button" by MrNumber112 • Free Download: Subscri...

[@JesseGoodier]

Just a note to say thank you for this video! I was using the functions randomly until now :)

[@PromLabs]

Woohoo, hopefully now you can use them in a more confident and targeted way :)

[@marcosvm99]

What a great video, not only for my own education but also to show our internal clients.

[@DenysVitali]

I love the visualizations and forwarded this video to my colleagues. I knew the concepts behind everything you explained, but they're so beautifully explained and represented that I'll just forward your video from now instead of trying to explain it in my words 😁 I hope you'd create more content like this. Keep up the good work!

[@PromLabs]

Wow, thank you for the kind words! And yes, that's definitely encouraging me to create more :)

[@merkispasha]

Awesome explanation through visualization!

[@myfamily837]

Finally found a great video with visualization that helps understanding these 3 functions. Thank you so much!!

[@GarvitJain_1]

Thank You for sharing this in such an awesome manner

[@mikoajsienkiewicz4183]

It's great that you add some "good practices/know-how" when talking about the overall concepts. I find it very helpful to get a general grasp and feeling of what the authors of specific mechanisms had in mind when creating certain things.

[@haouser]

Another amazing video. Thanks for share it!!

[@y3boy]

Finally, I understand the difference 🥳Thanks !

[@edmundandbaldrick]

first class content julius, keep up the great work 😀

[@reshantchandra2301]

thank you , great video`

[@AndreyTulenev]

thank you very much!

[@mickiXXX]

Thanks for the great explanations, in particular the charts helped me a lot to understand what is happening. Now the values I see start to make sense ;-)

[@PromLabs]

Yay! Thanks :)

[@rohitmishra2296]

Wow! Just in time. I am writing my LF PCA certification next month and these tutorials are the perfect second shot I needed to circle-in my preparation. Thank you for these!

[@PromLabs]

Nice! Check out my full in-depth Prometheus trainings at training.promlabs.com as well - those are a pretty good preparation for the PCA :)

[@user-lx5ki6tu6x]

very nice

[@srikanth.chekuri92]

Greak work.

[@babakhabibnejad508]

turly amazing video! I have taken at least 3 courses on prometheus+grafana but non of them are properly detailed like your videos

[@aksheengalundia824]

This is great. Can you explain for the following scenario? I want to set up an alert for average increase between two time window, but also dependent of the total number of increase in the same window. For example, For 1st hour -> increase by 1,2,3 and for 2nd hour -> increase by 3,4,5,6 avg increase in 1 hour -> 6/3 = 2 and avg increase in 2 hour -> 18/4 = 4.5 Difference between the hour is 4,5 - 2 = 2.5 How can i set up such a metrics/alert?

[@driodeiros]

This is outstanding Julius. Great work. Prometheus deserved high quality visual materials. Not it has them. Keep up the good work. May I ask you what software you use to create the videos? Specially the charts, they are pretty cool. I also like the background you use and the sound and video are fantastic. Thank you.

[@driodeiros]

Also, did you put together the setup (both hardware and software) or you got some help from someone? Would you mind sharing your resources?

[@PromLabs]

@@driodeiros Thank you, and thanks for asking! I create and produce everything completely by myself. My setup is by no means perfect and I am always improving it, but right now this is it: - Everything is 100% recorded and produced/edited on Linux - Recording software: OBS Studio - Video editing software: DaVinci Resolve Studio - Chart animations: built on motioncanvas.io (basically reimplemented rate()/irate()/increase() and other stuff from previous videos in there, with animations) - Camera: Sony A6400 (connected via Elgato Cam Link to record 4K directly into OBS) - Mic: Deity VO-7U - Teleprompter: Desview T3 (this doesn't fit natively on my Sony cam lens, so I built a custom contraption to mount them together) - Key lights: 2x Elgato Key Light - Background: Walimex Pro Roll-Up Panel - Background lighting: 2x NEEWER 660 PRO RGB LED - Script writing: Google Docs - Teleprompter app: Nano Teleprompter (nice because you can sync it with your Google Docs for script writing!) - ffmpeg to convert the 30 FPS of PNGs produced by Motion Canvas into a video file - Thumbnails: Krita for cutting out shots of myself, a custom GatsbyJS + Puppeteer setup for composing all the thumbnails in HTML/CSS and then automatically screenshotting and uploading them to KZbin (so I can replace them all at once with a new design if I need to)

[@driodeiros]

@@PromLabs Thank you so much for the details. A follow up question: How do you solve the delay introduced by the elgato cam link)? I get a 300ms delay between the audio and the video. I know I can add a delay in OBS to compensate but that doesn't help, not sure why. I was wondering if you have some other technique you use to solve the problem.

[@PromLabs]

@@driodeiros I just do it exactly as you said in OBS, except that it works fine for me. I currently have a 200ms Sync Offset configured for the mic.

[@shiyan7412]

Thank you Julius, this is a brilliant video. Just wonder do you plan to produce more similar videos like this to embed into promlabs training courses?

[@PromLabs]

Thank you! I can't say for sure about the future, but so far I've intentionally kept video content out of the training courses. The reason is that I really want to keep the courses as up-to-date as possible over time, and that is not a realistic thing to do with video (given the immense amount of time it takes to produce or update even just a couple of minutes of video). It could still be that I'll add video content to the courses at some point, but then I'd want to make sure that the content is so general that it basically never needs to be changed.

[@shiyan7412]

​@@PromLabs Thanks for the reply. Now I understand why the training courses are mainly text based 🙂On the other hand, adding some videos or more pictures will make the learning easier, especially for the beginners.

[@PromLabs]

@@shiyan7412 The courses definitely contain a lot of pictures (screenshots and diagrams) :) But yes, video would also have its benefits especially for certain types of learners, but it's a cost/benefit tradeoff.

[@sylviohmartins]

Perfect! Do you have any PromQL examples that do what was demonstrated in: 03:02 Handling Counter Resets. Well, the 'rate' was used together with the 'increase', right?!

[@PromLabs]

Oh, it's just either using rate() or increase() directly on a counter metric - both functions do exactly the same kind of reset handling. So e.g.: rate(requests_total[5m]) or increase(requests_total[5m])

[@sagarhm2237]

How get average response time

[@PromLabs]

Take a look at this part of the metric types video where I explain how to calculate the average request latency from either a histogram or a summary metric: kzbin.info/www/bejne/nJnbYZiepdWAfa8