Influx vs Prometheus vs Timescale

Influxdb vs Prometheus vs Timescaledb. We'll cover timeseries databases (TSDBs) and metric collection, along with a quick intro to influx and telegraf.
A comparison of different approaches and what you can expect from each one. Stay tuned for part 2.
00:00 Intro
00:15 Definition of TSDBs
01:45 Push vs Pull
09:00 Counters
11:00 Datatypes & Pushgateway
15:15 CNCF vs Influxdata
16:13 Easiest to start
16:35 Additional Features we'll cover
17:30 TimescaleDB
19:49 Coming Soon

[@kieranthompson172]

Honestly quite helpful with the decision I was try to make. I'll be integrating Influx after reading some articles and enjoying this explanation.

[@pooyakoosha8516]

Great video! Please keep it going with a advance tutorial/dive into InfluxDB. Looking forward for it

[@Danielo515]

It's impressive how within the first 10 minutes you clarified all my doubts about influx vs prometheus

[@andydataguy]

So excited for the Flux videos!! 🙌🏾🙏🏾

[@reibax]

Very educational! Thank you very much for sharing your expertise!

[@banzooiebooie]

Great video, I hope you will do more of them!

[@felipef6300]

Really thanks for this master class! You've spoken for clear way

[@VaibhavPatil-rx7pc]

You are the champ, fantastic explanation!!, nice t-shirt by the way !!

[@nosh3019]

very useful and clearly explained! Thanks a bunch!

[@ZombiecowCast]

Fantastic video. Helped with my second guessing

[@aronkoffler]

very, very, very good content and clear explanation! Great selection of behavior/characteristics to emphasize as a mature professional should be able to advise! thank you so much! also: nice shirt!

[@Gabzim]

Thank you for taking the time and leaving a compliment! I’ll make more videos when I’m back from my trip

[@vinitrinh]

Appreciate the comparisons especially the section specifically on TimescaleDB. My feeling is that TimescaleDB is a lot easier to adopt when the company has a lot of old-school engineers who also want the relational data model that InfluxDB does not have.

[@Gabzim]

Yeah that’s precisely why I wanted to try it, it feels like the path of least resistance. Although, if I have to pick one as the one I feel has the most potential to grow and dominate, I think it’ll be influx, it’s just my hunch though.

[@marcosissler]

Nice explanation Gabriel. Thank you for your dedication. I saw some projects using the timescaleDB when fetching cryptocurrency data. I saw some comparison with influx, prometheus, benchmarks. I think is a good option!

[@Gabzim]

TimescaleDB is the one I have little experience with, though it’s the one I wanted to use, it’s a pity that managed Postgres in AWS doesn’t support it and my company is too large to move all the parts to sign a new contract.

[@Gabzim]

Having said that, my experience with influx so far is great. Nice performance, flux is easy once you get it. Curious to see what influx IOx can do.

[@mateusztenteges3244]

I would need those next videos :D

[@Climate_Action_Pakistan]

Looking forward to you flux videoes

[@andydataguy]

Great video!

[@pratyush__agarwal]

Crisp and precise

[@Caarve]

Great video

[@rohithsrivathsav6110]

Great video! Quick question - to query the blockchain periodically and get token balances, would you recommend timeseriesDB or influxDB? It's somewhat similar a stock price ticker data, but frequency would be once a day/week/month.

[@Gabzim]

I like influx a lot so I’m biased. In this case I think either one would work fine, I personally would go for influx. Even Prometheus gauges here would be fine since you are ok with just periodic data.

[@stevehan7734]

This is great and helpful. Have you by chance looked at the AWS time-series database, Timestream? How would you compare them?

[@Gabzim]

Hey thanks for the comment! I haven't had the chance yet. My guess is that with SQL query language it's going to be more accessible for developers. One of the biggest barriers I have in my company with influx or prometheus is explaining how they must create their schema to avoid cardinality in tags/labels and the query language (flux/promql)

[@LucasRibeiro]

Super content!

[@progressiveways917]

appreciate it..thank you :)

[@mlkvdev]

Thank you

[@marioclavijo1283]

thank you!

[@gabrielsilveyra7024]

Me gustó. Cuando puedas los TIPS de FLUX... Likes asegurados

[@Gabzim]

Les debo ese video hace mucho. Estoy cerrando una release y me pongo a grabar

[@GA-pk1jr]

Very good explanation! What software do you use to edit your video?

[@Gabzim]

Hey thanks! appreciate the compliment. I’m using Final Cut Pro, but it’s overkill for the kind of editing that I do. I should have gone for a simpler tool. I use like 0.5% of the features it has.

[@GA-pk1jr]

@@Gabzim i get you, but i think most tools out there are pretty overkills, although tbh Final Cut Pro is pretty simple and straightforward to use Keep the videos coming!

[@prakritidevverma4315]

Can you recommend which database should I use storing stock market data.?

[@Gabzim]

I’d use influx for that.

[@Gabzim]

Flux also has other features that will be very useful for analyzing that data. Stick around, I’ll make a video on that.

[@prakritidevverma4315]

@@Gabzim thanks man! Great video btw 👍🏿

[@giridhart]

@@Gabzim thank you, waiting for the next

[@javierenlanube]

I'd go with QuestDB, which was designed specifically for this use case. It is used now for every kind of time-series data, but fintech/trading/crypto are some of the more popular use cases amongst its users

[@MocroBorsato]

Hi man, what program did you use for drawing?

[@MocroBorsato]

Ah seems to be something of ios.. Anyways great explanation keep it up

[@Gabzim]

GoodNotes and I just record my screen

[@user-sv7bx6gn8n9]

Which TSDB is suitable for tick data of stock market?

[@Gabzim]

Of the three, influx is the one I’d personally recommend. You can’t push data to Prometheus so you’ll lose resolution.

[@Gabzim]

In addition, flux has multiple functions that are useful for processing moving averages, predictions etc

[@johndoes461]

omg, I'm trying to learn the basics of a tsd and there is so much awful content out there. Then I find this video and this guy should be a teacher/professor as he breaks it down and explains it like I'm a 5 year old which is exactly what I was looking for. Sampling isn't explained very well even in the influx and grafana docs. Thank you for this video! (I even bought a udemy course on Grafana and although it's good, he assumes you already know all of what you are covering in this video)

[@Gabzim]

They Tom thanks for the kind words and I’m glad it’s helping you. I had a very complete series planned for this topic using examples from my real job. Unfortunately I’m a bit of an introvert so it takes a lot of energy for me to record myself and talk to a camera, I’ll try to put a bit more effort in if people find these helpful

[@johndoes461]

@@Gabzim I think the use of Grafana for BI is gaining popularity and observability is the new trend right now. TSDBs can be very confusing and intimidating and this comes from a guy that has been doing rdbms for 20+ years. People really just want a BI/reporting/visualization tool that is dead simple and they can generate pretty graphs/dashboads in minutes and not have to steep learning curve such as power bi/tableau/looker. Looking forward to your next videos.

[@Gabzim]

@Tom N agree, I’ll try to make a couple this week. Thanks for the support.

[@andys-channel]

in your limitation example of prometheus, why wouldn't you store the filename as a label (or dimension depending on terminology)?

[@Gabzim]

Because then each file would create a new metric with a single data point, after a while this would really wreck your performance and you’d need to accommodate for this in your queries. Labels have to have a low cardinality

[@andys-channel]

@@Gabzim labels are used like that in loads of places though, and pretty common right? e.g. node exporter, so it is possible, but like with all prometheus metrics, you just have to be wary of cardinality - though it feels like the same could be said when comparing to some downsides of SQL based systems - just feels like in this case its maybe a bad example to give of something you can't do in prometheus, when you can? Side note - really like your drawings, it fits in really well with the video as you explain stuff.

[@ashishposte8192]

What about we losing counter because of the service which generate the counter get restarted . Its with respect to the pull model of Prometheus

[@Gabzim]

That's ok because, when you query the data, the rate operation in prometheus takes this into account. Let's say you have a counter that goes like: 1, 3, 5, 8, 10, 2. That last 2 is "wrong" in the sense that a counter should always increase, so rate will look at the data like: 1, 3, 5, 8, 10, 12 (roughly speaking) and will calculate the rate from there.

[@Scott_Stone]

Wow, man, where are you? This video is so good but you haven't posted anything after this one.

[@Gabzim]

Hey brother, sorry about that. My life changed radically and I’ve been very bad at heading a company plus being a father plus KZbin. I really should organize things and get this done.

[@Gabzim]

Really appreciate the kind words btw

[@devrub623]

You can also push data with Prometheus

[@Gabzim]

if you mean via the push gateway i’d say “kind of”. you can push it to the push gateway but then that gets scraped by prometheus. if you mean you can push granular data in between scraping intervals i haven’t seen that.

[@jerkerj]

What happened to the next video? =)

[@Gabzim]

Sorry, I switched jobs and started a side project and I couldn’t keep up with everything, I’m hoping to take some time off soon and make videos

[@ErikS-]

the titles should be "influx vs prometheus". Timescale isnt being discussed in a decent way here.

[@Gabzim]

Valid point, sorry about that.

[@larsvontrierpung9337]

graphite

[@ElectrickSoundz]

🎉😂❤🎉😮😂❤😊😊😊😮😂

[@HM-vf8vh]

pull not poll