I've been using Redis wrong this whole time...

  Рет қаралды 325,060

Dreams of Code

Dreams of Code

Күн бұрын

Seriously. I've been wasting it's talents.
Redis happens to be one of my more favorite databases out there, but I often relegate it to the caching layer in my application stack. In this video, I decided to look at how to use redis as a database, rather than as a cache, and how that would work.
Become a better developer in 4 minutes: bit.ly/45C7a29 👈
Join this channel to get access to perks:
/ @dreamsofcode
My socials:
Discord: / discord
Twitter: / dreamsofcode_io
00:00 - Intro
00:46 - How did we get here
01:23 - Persistence
04:14 - Data Structures
05:02 - Tables
07:29 - Basic Index
08:17 - Columns
10:43 - Ordering
13:43 - Filtering
14:59 - Unique Constraints
16:13 - Transactions
18:51 - Drawbacks

Пікірлер: 423
@DesertRat44
@DesertRat44 7 ай бұрын
I would truly appreciate a homelab setup from you, love your videos, God Speed!
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you so much!! I'm really looking forward to making the homelab content. Glad you are enjoying the channel as well!
@DesertRat44
@DesertRat44 7 ай бұрын
Absolutely, thank you for making the content. You get exactly what you deserve :)
@suvetar
@suvetar 2 ай бұрын
Oh yes please! Love this concept and the surprise factor!
@eduardornh
@eduardornh 7 ай бұрын
Be aware that in AOF (Append Only File) persistence, Redis saves logs every second by default, not every write.
@dreamsofcode
@dreamsofcode 7 ай бұрын
You are correct! I didn't explain this caveat well in the video.
7 ай бұрын
I thought so too, but the video made me doubt whether I was confusing it for something else. The reason it's called AOF and not WAL is because it's not written ahead of the change itself 😁
@gritcrit4385
@gritcrit4385 7 ай бұрын
You can make it log every write, but it's slow.
@mohammadshaqibsiddique9291
@mohammadshaqibsiddique9291 7 ай бұрын
If redish will do that it looses much of it speed at writing operation.
@charlesbcraig
@charlesbcraig 7 ай бұрын
I found Redis to be too slow, so I sped things up by no longer storing any data
@blank001
@blank001 7 ай бұрын
Perfect example for: Just because we can doesn't mean we should. But content wise this video is top notch
@profiluefter
@profiluefter 7 ай бұрын
@@mythbuster6126 I've used Supabase (which is a software stack that includes PostgREST and Auth, etc. using PG) for a few projects and it really speeds up development to not have to create tedious CRUD endpoints for everything. Database logic using PLpgSQL is nice but yeah, we also had the problem that changing something in the procedure code is annoying because you have to recreate the whole method in a migration.
@kebien6020
@kebien6020 7 ай бұрын
I see this more as for this kind of situation: "Redis would be perfect for this use-case, except that for this little bit of functionality we need to do a 'relational' operation" And instead of going "welp, then let's discard Redis", you can go "for that case we can just setup [workaround] and this design would still work"
@RoyBellingan
@RoyBellingan 7 ай бұрын
Same feeling, you have to Plan ahead and do so many workaround to replace what in sql Is basically free.
@JohnWillikers
@JohnWillikers 7 ай бұрын
​@@RoyBellingan Its more then that, fully replicated redis is so expensive. What would be ~$600 USD in SQL s like ~$1300 in redis.
@user-gi4qu9do2v
@user-gi4qu9do2v 7 ай бұрын
@@JohnWillikersmoney for infrastructure does not matter for small companies (in compression with SE salary). Worst thing happens when system has a lot of cache that can be removed with some tricks and better understanding of system as a whole.
@darthvader4899
@darthvader4899 7 ай бұрын
Dedicated HomeLab pls
@dreamsofcode
@dreamsofcode 7 ай бұрын
Absolutely!
@karasira2696
@karasira2696 7 ай бұрын
Let's gooo 🥳
@InterpretingYou
@InterpretingYou 7 ай бұрын
Lets goo!
@Kingromstar
@Kingromstar 7 ай бұрын
Yes, so we can figure out the networking and how to do it securely.
@karngyan
@karngyan 7 ай бұрын
yayyy
@IshanAditya
@IshanAditya 7 ай бұрын
Hey! Nice one! Online Engineering Lead at Ubisoft here... Sorted Sets in Redis are the go-to solution to do leaderboards in the game industry actually, but you're constantly optimizing to battle server costs since everything stays in memory, and larger leaderboards need machines with more memory, which happen to be more expensive. But honestly, as soon as you want persistence and you enable AOF for every write operation, you'll start to run into performance issues on Redis too. There's no magic bullet in engineering - weigh the pros and cons for each use-case and then choose the tool.
@vivekkaushik9508
@vivekkaushik9508 4 ай бұрын
Nice to see a fellow Indian here. Preach brother. ❤
@earthling_parth
@earthling_parth 4 ай бұрын
Hey! Awesome to hear from someone actually using Redis in this use case. If I may ask about your experience a bit more, how much memory was Redis utilizing per instance in a server or a pod in Kubernetes for a given X number of entries (X number of users most likely) in a leaderboard? I want to gauge if the in-memory tradeoff is worth it for the speedy performance compared to RDMS (I'm assuming Redis is much lightweight compared to those databases as it doesn't have a lot of those abstractions and advanced structures built-in). I'm very excited to hear about your experience on this.
@Peter-bg1ku
@Peter-bg1ku 3 ай бұрын
Why not use a relational database for the leaderboards?
@bepamungkas
@bepamungkas 3 ай бұрын
@@Peter-bg1ku write locks and index rebuilding would be a nightmare in this kind of scenario. The former is somewhat a solved problem (most storage engine have row-level lock) but the latter is detrimental to the underlying behavior of relational db.
@Peter-bg1ku
@Peter-bg1ku 3 ай бұрын
@@bepamungkas good point. Yeah, indexes would be a massive pita to manage. But what if you stored your relational DB in memory. Would it not be quicker?
@cat-.-
@cat-.- 7 ай бұрын
Next video idea: most people scoff at the idea of using filesystem as a database, but did you know we can recreate Postgres with fs in c++ and achieve similar functionality and safety
@dreamsofcode
@dreamsofcode 7 ай бұрын
This is a fun idea. Maybe even building a simple database from scratch!
@jamesm4957
@jamesm4957 7 ай бұрын
basically a sqlite
@RevHardt
@RevHardt 7 ай бұрын
I’ve used git as a versioned nosql fs db. Beware of inode exhaustion
@jtms1200
@jtms1200 7 ай бұрын
use pastebin instead of the local file system and boom, cloud database! give me 500 million in funding plz k thnx
@jonintc
@jonintc 7 ай бұрын
I used a file system as a key/value store db about 8 years ago and it was very simple and very fast! Granted looking back, it had many holes such as potential for race conditions as addressed in this video, it still did the job beautifully!
@advanceringnewholder
@advanceringnewholder 7 ай бұрын
This made me appreciate SQL more. Look what they need to mimic a fraction of SQL power
@Widiar0
@Widiar0 7 ай бұрын
Weird flex... and incorrect.
@dijoxx
@dijoxx 7 ай бұрын
Is this a joke comment?
@janjensen3220
@janjensen3220 7 ай бұрын
Spoken like someone who has no idea what things even mean, bravo. What Redis does is giving lookups in sub milliseconds, providing a significant amount of data types (hashsets, linked list) that can be used to do distributed operations, which are necessary in cloud native applications. They also provide very fast pub/sub and stream patterns that can be used for message busses, none of which SQL provides. Also distributed locking that can be used to synchronize microservices along with probabilistic structures and much more. Plainly said, the Redis use-case is entirely different from SQL database. Their overlap is minimal at best. If you use a database for all of the things I just listed you are braindead. If you use Redis for data operations such as (aggregations, grouping) you are braindead as well.
@nandesu
@nandesu 7 ай бұрын
Likewise. I'm not sure why they all fear using SQL so much. Truly a strange phobia.
@nagoranerides3150
@nagoranerides3150 7 ай бұрын
Broadly agree but if you have a very specific application with clear and (more or less) fixed data definitions you do pay at run time for SQL's flexibility. Almost all SQL systems can be replaced by faster solutions tailored for the specific context. SQL's real power is that it's "good enough" for a vast range of requirements.
@jtms1200
@jtms1200 7 ай бұрын
In my past experiments with using redis as a primary store it tended to not scale well once the project grew to a certain level of use and complexity. I often use it as a primary store for specific slices of a given application's data persistence strategy when and where it provides advantages over a traditional RDB. Inserts when using a RDB tend to be expensive (especially if the table has a bunch of indices) so Redis can really shine in very write heavy areas of your app like messaging, sensor data capture, etc.
@rslakinski
@rslakinski 7 ай бұрын
You missed a key point. During snapshot redis will fork the process and dump what is in that forked process to disk. This means you need 2x the ram to perform this action but also it’s a global db lock during the forking process. If you have a small dataset this generally fine but if you have a large db this process will cause large spikes in latency when the snapshot begins.
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you for covering it! Definitely worth understanding when considering Redis.
@rcnhsuailsnyfiue2
@rcnhsuailsnyfiue2 7 ай бұрын
Another aspect of this is the time it takes to reload large snapshots into RAM. After rebooting my server, I often see errors for the first 30-60s as my application tries to read/write from Redis, but Redis rejects the commands it’s still loading the snapshot into memory in its entirety.
@MadsterV
@MadsterV 7 ай бұрын
yikes! key indeed
@hansj4832
@hansj4832 7 ай бұрын
On Unix, forking is fast - it only copies the page tables and sets them all to copy-on-write - you'll only incur 2x memory costs and anything resembling "global db lock" if you're regularly writing to the whole database. This is the reason Redis chose to implement snapshotting via forking in the first place.
@horseradish843
@horseradish843 7 ай бұрын
Not entirely true. Forks, at least on linux, only marks the pages as copy on write, so they are only copied when one of the two processes write to them. It might be theoretically possible to end up with that happening, but very likely not. You are right that this will create some lag spikes when the page fault is triggered, but it does not need to copy all the memory.
@devopstoolbox
@devopstoolbox 7 ай бұрын
Amazing work! Not only the editing and presentation which is always so delightful, the idea itself of using Redis as a database and the persistency concerns are perfectly explained. THANK YOU!
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you very much!
@tim.martin
@tim.martin 7 ай бұрын
Can confirm: you were using redis the right way at the start. Enjoyed watching this but wowzer, I can't imagine inheriting a project setup this way...
@harleyspeedthrust4013
@harleyspeedthrust4013 7 ай бұрын
yeah this feels like the work of the smart quirky programmer who didn't want to use traditional databases because "i can do it myself with redis and it will be faster because redis" I don't think a serious software engineer would actually use redis like this for a work project
@osx86touch
@osx86touch 7 ай бұрын
Yep, cool video and nice production quality, but it’s pretty irresponsible to seriously recommend doing this without many massive disclaimers.. I pity the junior devs who are going to watch this and blindly implement it in important projects in place of an RDB.
@alx8439
@alx8439 7 ай бұрын
He is also missing another key point - if you give any traditional RDBMS (Postgres, Maria, whatever) enough memory for buffer cache, disable write-ahead logs and gonna be using index-organized tables, you will get what Redis gives you out of the box but with some free extra features.
@bennymountain1
@bennymountain1 5 ай бұрын
True. At some point technologies become a vocabulary. So when you say "Redis", others read it as "Cache, I know how to use it". This dude just words and made his Esperanto with them. That's like using SQL as a message queue (which I've also encountered and ran away immediately).
@Peter-bg1ku
@Peter-bg1ku 5 ай бұрын
@@harleyspeedthrust4013 without a database schema definition, how does one figure out what is going on in a redis database? With an RDBMS, you can inherit a database without docs and sort of figure out what's happening. Not sure if the same is true for Redis. Very good tutorial as I learnt some neat Redis tricks but I wouldn't recommend this to my worst enemy as the way to build apps with persistence.
@82TheKnocKY
@82TheKnocKY 7 ай бұрын
So instead of using postgres I can use redis and recreate most of the SQL functionality myself, with less safety and more complexity and space for error... I struggle to see why to use this in any real world setting. Even for side projects, if I wanna set something up quickly, I don't wanna have to have to create indexing etc myself before being able to develop the app. I'd also be interested how the performance actually plays out once you start parsing long JSONs, maybe it will still be faster, but it's something you didn't consider. Also if you eventually decide to switch to postgres that could be a really hard rewrite.
@ArcticPrimal
@ArcticPrimal 7 ай бұрын
For years now, once in a while some devs say redis can be used as more than a cache but they themselves never use it as the main db or even as a message broker(bc of its limitations), they only used it as a cache. If they use it as a main db thats only for a short time and end up switching back to relational dbs. So just because Redis can be use as more than a cache doesnt mean it should be. Redis is the ideal choice for caching similar to how SQL dbs are ideal in supporting multiple tables and joining those tables.
@Zexanima
@Zexanima 7 ай бұрын
I see this as more of an exercise in what a tool can do, less so a "you should absolutely do this". He even goes over at the end of when you wouldn't want to do this and when you "might" want to. I don't think he's trying to push Redis as a be-all-end-all database. He just really likes the tool and is going over the capabilities to get people excited about it.
@novadea1643
@novadea1643 7 ай бұрын
As always it depends on the workload etc., there isn't a single correct database for all projects. If you need an SQL database then use an SQL database. The example here of directly comparing it to SQL is a bit extreme and only to show that they can be done, not that they necessarily should. Now on the other hand if your workload just requires blazing fast lookups/inserts without complex relations, eg. maybe an auth, permission or chat service then using redis as the main database starts to make sense.
@ArcticPrimal
@ArcticPrimal 7 ай бұрын
​@@novadea1643 using redis for auth, permission etc is mostly to support the main db and not using it as the main db
@dreamsofcode
@dreamsofcode 7 ай бұрын
I agree with your points. Redis isn't a full on replacement for postgres (I'm a postgres maximalist personally). However, I do think for a lot of simpler applications, Redis is pretty viable as the starting data store before you even know what your data model may look like. It also helps to reduce complexity of your overall application in order to get deployed more quickly.
@TankorSmash
@TankorSmash 7 ай бұрын
Really enjoyed this. It was cool to see the reasoning behind certain redis features. Thanks!
@Bukosaure
@Bukosaure 7 ай бұрын
You are really great at this. Your explanations are concise and clear. The pacing is perfect. The editing is quite good.
@johnholland2575
@johnholland2575 7 ай бұрын
You've been doing great. The content is presented at a reasonable speed, well at least for me. And the content its self has been very interesting and informative. Personally, I'm really interested to hear about your home lab, so please do an overview of it and maybe down the line a deep dive ;-) Thanks again, bud!
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you! I'm glad you enjoyed it.
@AlbertoHernandez1
@AlbertoHernandez1 7 ай бұрын
@@dreamsofcode I am also interested in your homelab. Everything very well explained in this video, a pleasure to watch and learn
@ecam90
@ecam90 7 ай бұрын
Definitely interested in your setup :P - thanks for the hard work on this content!
@dreamsofcode
@dreamsofcode 7 ай бұрын
My pleasure! Glad you enjoyed it!
@jayjay7333
@jayjay7333 7 ай бұрын
Such high-quality content we need more content like this on yt
@hyperkiko
@hyperkiko 7 ай бұрын
I was already using redis as a primary database, so seeing this video cheered me up!
@SoreBrain
@SoreBrain 7 ай бұрын
Great insights, thank you!
@shradhankumar4109
@shradhankumar4109 7 ай бұрын
I just started working on backend and we have a fairly complex setup. This video dropped just in time. Not only this tells you what redis can do or can't but is a small primer to starting with redis. Top content.
@armynyus9123
@armynyus9123 7 ай бұрын
Wow. Just wow. You totally delivered, forwarded this to my colleagues.
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you so much!
@timkarsten8610
@timkarsten8610 7 ай бұрын
Dedicated HomeLab channel?!? Yes, please
@tjnaz
@tjnaz 7 ай бұрын
i second this so hard
@mahdisoussi4314
@mahdisoussi4314 21 күн бұрын
straightforward, well written, clear and simple videos, love them!
@michaelayoub2211
@michaelayoub2211 7 ай бұрын
I can't believe how quick you made a 21 minute video about Redis feel. Great video
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you!
@novadea1643
@novadea1643 7 ай бұрын
Totally agree with the video, the biggest drawback is the lack of higher level primitives but in some cases it's indeed worth implementing them yourself on top of redis. The company I work for had a chat system as part of our django + postgresql api that could handle around 1k messages per second before it started chocking up (can't remember the exact numbers, it's been years). Partly because of django websocket implementation and partly because of the database model designs, they could have been optimized further but the performance would have still been far from ideal. We reimplemented it as a nodejs microservice with redis as main database and based on tests a single instance could easily handle 10k+ messages per second, no specific optimizations just from switching language and database. After deploying to production it has never gotten high enough real world load to use more than a fraction of a single cpu core in the cluster.
@mormantu8561
@mormantu8561 7 ай бұрын
+1 For the homelab setup video!
@CarlM_UK
@CarlM_UK 7 ай бұрын
Beautiful editing
@tom_marsden
@tom_marsden 5 ай бұрын
I think you've convinced me to stick with RDBMS + SQL as the primary database for my relational data 😁
@RafaelQuintela
@RafaelQuintela 7 ай бұрын
Great! Would love one like it on redis-stack please!
@GiovanniCKC
@GiovanniCKC 7 ай бұрын
ayyeee congrats getting a home lab set up that's sick!!
@2Sor2Fig
@2Sor2Fig 3 ай бұрын
For me, I fell in love with Redis when I had to do work on a logistics management system. Out of the blue, just reading the documentation, I found out that it has in-built geo location functionality and can easily determine the straight line distance between two points, accounting for the curvature of the earth (correct me if I'm wrong, but I believe under the hood, it uses the Great Circle function). Thanks for the vid.
@TerenceKearns
@TerenceKearns Ай бұрын
Your videos are amazing. So clear and well-paced. This is the first one of yours that ive come across. Definitely subscribing. Do you use after effects for all your anims?
@dreamsofcode
@dreamsofcode Ай бұрын
Thank you! I'm transitioning more and more to Davinci Resolve (Fusion) entirely, but there's some things I still use After Effects for! Fusion is really powerful, but there's a lot less resources on how to use it.
@TerenceKearns
@TerenceKearns Ай бұрын
@@dreamsofcodeyeah I used to use AE a little. I use Resolve now but have been a bit slow to dive into Fusion. It's actually pretty good from what I can tell. Thanks for letting me know. I' always interested to see how people make their videos cool.
@jack171380
@jack171380 2 ай бұрын
You don’t need a cache if your database has enough memory. You don’t need a database if your cache has enough disk space. You don’t need any of them if you have no users.
@danailenei9959
@danailenei9959 5 ай бұрын
I see some usecases in which you could use redis as you said, but for me the main drawback is data consistency. Having a very well defined schema of your data and the responsability of shaping it right makes things way more maintanable. That s my personal problem in general with nosql
@cod3r1337
@cod3r1337 7 ай бұрын
So now I know that I *could* sort of replicate a relational-ish storage model in Redis by reimplementing abstractions that are already in place in any RDBMS by hand at a lower abstraction level. Which is indeed quite interesting. The question is: Why should I, instead of just using an RDBMS?
@jochinq3116
@jochinq3116 7 ай бұрын
Awesome work! Thanks! Please also do a seperate tutorial for "Python and Redis Queues"
@martinzen
@martinzen 7 ай бұрын
Definitely do a home lab setup tutorial!
@ChrisHalden007
@ChrisHalden007 7 ай бұрын
Great video. Thanks
@mxlje
@mxlje 7 ай бұрын
If I had to chose a favorite DB of any kind it would probably be redis. I also recommend anyone to build a simple crud app only using redis as the database and implementing all the operations with that. It’ll teach you so much about structuring data and how to query it.
@follantic
@follantic 7 ай бұрын
If Redis supported disk only keys, I'd consider this once in a while. It's really nice!
@bastiaanschaap
@bastiaanschaap 5 ай бұрын
Thank you for this video, you tought me some new things about Redis! Did you leave out Redis JSON on purpose, because it's an awesome feature; it allows for storing JSON documents as values, and creating indexes on the actual JSON content. We're using it in our CQRS setup, both as message store as well as the Query model.
@earthling_parth
@earthling_parth 4 ай бұрын
Really good introduction to Redis! Huge thank you! I see you're a man of culture as well with those DrafonBallZ and Naruto references ❤
@azim0ff
@azim0ff 7 ай бұрын
Was using redis as my primary database, but still learned a lot from watching this
@epicmap
@epicmap 7 ай бұрын
My main concern of using Redis as a main database is memory, not persistence. Most people know that Redis can persist data on disk. Memory is much more expensive and complicated to expend than simply adding storage. And the fact you need to think about scaling sooner, rather than later. Scaling now depends more on the amount of data you need to always keep in memory (even if you're not using it), not on the number of users(who typically generate profit). If only there was a way to offload unused data and only keep in memory frequently accessed entries? Oh right, caching! That's why I prefer to use it as a cache. Redis used to have Virtual Memory feature(to swap unused data to disk), which is now deprecated for some reason. One can use swap of the OS however, but it never explained if it's a good idea to do so. P.S. Scaling Redis seems to be easier, than scaling SQL databases, but this aspect never mentioned when talking about using Redis as a main database.
@dreamsofcode
@dreamsofcode 7 ай бұрын
Did you watch until the end?
@epicmap
@epicmap 7 ай бұрын
@@dreamsofcodenot yet
@SR-ti6jj
@SR-ti6jj 7 ай бұрын
Alright, I'll try it for the project I'm working. If this fails we'll be back with our pitchforks
@flamewave000
@flamewave000 7 ай бұрын
I've already been using it as a primary database. Our data is very small and most of it is non-persistent (status hash tables that expire after 30 days). But we do keep record of transactions, which is an integer timestamp and a simple 41 character id string. I also commit an apparent faux pas, and I use the Logical DB set to organize my data. Apparently it's an anti-pattern because some people may implicitly think they're actually separate systems when they're not. But that doesn't really matter since the way I use them as almost like folders to help organize my data.
@damianblejwas9349
@damianblejwas9349 7 ай бұрын
What's the autocomplete/hint plugin you are using? It seems great.
@nhattran536
@nhattran536 6 ай бұрын
High quality content thanks Dreams of Code.
@dreamsofcode
@dreamsofcode 6 ай бұрын
Glad you enjoyed it!
@hamidsafe438
@hamidsafe438 7 ай бұрын
Absolute great video, I can't thank enough 🙏 sir you are a master jedi in redis ...P/S your tom bombadil part caught my eyes :)
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you! I'm glad you noticed that!
@harleyspeedthrust4013
@harleyspeedthrust4013 7 ай бұрын
silmarillion moment
@Ch40zz
@Ch40zz 7 ай бұрын
It might be good to add that redis transactions are not nearly as powerful as SQL transactions. You can resort to using LUA scripts to fix that but this is really annoying to maintain and to write proper tests for.
@ballsofglass
@ballsofglass 7 ай бұрын
I wouldn't recommend redis as a sole prod DB because as you said volatile memory is expensive and you did not touch on redundancy. So in addition to dealing with structuring all the data, composing the indexes, and so on - which is the fun part - you will need to also learn redis sentinel and redis cluster - the not so fun devops piece - in case your redis master goes down. Slaves will have to be promoted and properly synced, and if that process is not well understood / configured correctly you are in for a world of hurt. Also having 2-3 slaves running for a single master will 2-3x your costs. However it may be possible to justify some of the costs by setting the slaves as read-only to aid in throughput.
@chukwuemekaajima8373
@chukwuemekaajima8373 3 ай бұрын
At this point, I would like to see a video on how you make your KZbin videos. Very clean and interesting. Please make a video on how you achieve this including all the fun and interesting animations. Thanks
@deathdefier45
@deathdefier45 7 ай бұрын
This 20 minute video taught me so much more than all the other redis videos on the internet, thanks ❤
@dovh49
@dovh49 7 ай бұрын
For personal projects I use IndexedDB and then I sync it to sqlite on the back end. With everything on the front end it makes it super simple to have things nice and speedy. Granted, I don't need to worry about live data this way. So, it isn't a good model for all apps.
@jtms1200
@jtms1200 7 ай бұрын
I love this concept, but that api on indexedDB though... ooof, its not great
@AlleBalle54
@AlleBalle54 7 ай бұрын
nice video and good explantion
@JamieBainbridge
@JamieBainbridge 6 ай бұрын
I always wondered what Redis was used for, this is a great video to go from zero to having some clue.
@dreamsofcode
@dreamsofcode 6 ай бұрын
Glad you enjoyed it!
@BLUYES422
@BLUYES422 7 ай бұрын
Im working as a developer and i knew this stuff before i got there and kept wondering why people arent just considering redis as the db instead of just a chaching layer. I actually didn't know it could be used as a cache aswell before then 😂
@COLAMAroro
@COLAMAroro 7 ай бұрын
I would like to see performance and ease of use compared to an in-memory SQLite database To see if the ACID Compliance and the SQL features comes at cost, for the dev or the runtime speed
@BadDogeU
@BadDogeU 3 ай бұрын
Our company uses Redis as a runtime database for our live chat system, and it saves off transcripts into a real RDB at various checkpoints. The footprint in redis for all of that is 400MB, and we saved an immense amount of maintenance/development cost doing it this way..
@BolverBlitz
@BolverBlitz 7 ай бұрын
I did read redis docs before but your video explained it so well i made parts of my software 30%ish faster and i take that!
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you so much! I'm really glad it helped!
@fedorh
@fedorh 7 ай бұрын
Would love to see the homelab setup
@TheShorterboy
@TheShorterboy 7 ай бұрын
Nice I like REDIS but there is a lot to manage and that is a lot of data sets, a normal DB lets me abstract this away with table index magic and in this case I might as well just use python/TCL/C++ with hahses, sets and arrays (though TCL will let me push raw commands in avoiding the need for an API) and write every transaction to a rotating log file with a threaded snapshot by time.
@AdityaDodda
@AdityaDodda 7 ай бұрын
DBZ fan! Nice video! Very well explained.
@sharathk1881
@sharathk1881 3 ай бұрын
Great content! Subbed.
@encoderencoder1031
@encoderencoder1031 7 ай бұрын
redis should sponsor you now
@MikeGaruccio
@MikeGaruccio 7 ай бұрын
AOF persistence may be is fine if your doing a read-heavy workload, but limits your write performance to no better than a database that’s writing to disk. On the other side of the equation, if your willing to dedicate enough memory to your primary DB to hold your entire dataset in-memory you can accomplish that with fairly simple cache settings. There are still some things that redis will do better, but it’s also going to come with a lot of trade offs. With a big one being that you can’t really use an ORM, and that a lot of really large-scale databasing strategies just don’t work well with redis (at least without a ton of work)
@raulgarcia3614
@raulgarcia3614 7 ай бұрын
Homelab video please. I have a homelab setup and would love to see yours!
@kokizzu
@kokizzu 7 ай бұрын
I use tarantool, can support SQL, have append only file also, can store more than ram if using vinyl engine (not the default memtx engine) the gotchas is that no date data type (so i use epoch/timestamp without timezone), and the table/column name all capitalized so if you create table with lowercase/mixed-case you must quote it on query so can overcome all redis limitation.
@kokizzu
@kokizzu 7 ай бұрын
for non OLTP use case i use clickhouse, so for log (compressed, searchable), metrics (using materialized view), and all events stored there, if i need to read it fast i update periodically i push it back to tarantool, so all use case can be fulfilled with combination of both oh also there's one more gotchas in tarantool, you cannot alter table to insert column in the middle (only at the end is possible), have to copy whole table, but since it's really2 fast 200k rps like with redis/aerospike, it's really easy to change schema with full copy
@Vijay-Yarramsetty
@Vijay-Yarramsetty 3 ай бұрын
this is informative, thanks a ton
@paulrobertomessina1813
@paulrobertomessina1813 5 ай бұрын
I am very interested in your updated Home Lab Setup, please do a video or post about it.😀
@dfparker2002
@dfparker2002 7 ай бұрын
Please talk about use cases for redis and memory constraints or expectations per case load
@dreamsofcode
@dreamsofcode 7 ай бұрын
Great idea!
@05xpeter
@05xpeter 6 ай бұрын
Good to know. Sometimes you just happen to have a redis instance and need a minimum of database functionality. In my former job I implimented a caching service, were the invalidation logic was very complex I ended up implimenting it in postgres, but we allready had a redis instance for caching auth sessions, so I could have done it there as well.
@jonintc
@jonintc 7 ай бұрын
Using Redis for a db is similar to using MongoDb or any key/value NoSql database. You outlined the cons quite well but addressed them with Redis datatypes and transactions. I think the big pro to using Redis as a NoSql database is the simplicity if one has a use case that doesn’t need complex querying. This is referred to as impedance mismatch I believe, where the dataset your UI + API match that of the database.
@CarnaticCorner
@CarnaticCorner Ай бұрын
Such a healing rendition .
@pradhyumnakandamuru
@pradhyumnakandamuru 7 ай бұрын
congratulations on your homelab
@Kabodanki
@Kabodanki 7 ай бұрын
We have switch from redis to keydb, pretty cool alternative
@dreamsofcode
@dreamsofcode 7 ай бұрын
I'm gonna add a video to the backlog to check it out!
@xeridea
@xeridea 7 ай бұрын
Can't imagine manually attempting joins, or tables with more than a couple few fields
@tanvirkazi6889
@tanvirkazi6889 7 ай бұрын
I’m reminded off the scene in Jurassic Park when Ian Malcolm says ‘Just because you can, doesn’t mean you should’. Or something like that.
@bradenborman4396
@bradenborman4396 4 ай бұрын
I use it to lock out a request across multiple instances of an app. Where money and transactions are completed, I couldn’t have a user start more than one thread across multiple apps, so it’s GREAT to lock out that process single file by user
@scaffus
@scaffus 7 ай бұрын
1:50 that would be great!
@rektleet
@rektleet Ай бұрын
this is a great video, i hope redis doesn't change its license to a non-open source license
@Serizon_
@Serizon_ Ай бұрын
they already did,.
@thedoctor5478
@thedoctor5478 4 ай бұрын
you should also cover the redis extensions. they're great
@dreamsofcode
@dreamsofcode 4 ай бұрын
This is a great idea! I shall add it to my backlog
@llxxqqdd
@llxxqqdd 7 ай бұрын
Video about your linux dist setup and terminal setup? And I wait also video about Rust Tauri and Rust Yew
@exapsy
@exapsy 7 ай бұрын
The awesome bomb you threw was *"dont use KEY" in production unless you're debugging, it's slow and blocking* . I seriously didn't know that. I searched it up and it is absolutely true.
@dreamsofcode
@dreamsofcode 7 ай бұрын
It's a rite of passage when debugging in prod at times!
@jancarius101
@jancarius101 7 ай бұрын
Great video. I'm convinced Redis is only good for the caching layer.
@frangrau3750
@frangrau3750 4 ай бұрын
Very good and informative video, thank you. As a user of Postgresql since 2001, you are missing the biggest and most important feature of RDBMS -> data and reference integrity. This is unachievable by redis or any other nosql solution. Of course, if you have complex data models and you have the need and skills to write good SQL, redis falls short big time. (CTEs, Window functions, triggers, aggregates, etc) But if you are building something really simple or without structure, redis is great.
@ordinarygg
@ordinarygg 7 ай бұрын
Postgres Transactional speed > Redis Transactional speed. Also Redis is single thread write/read so you need compex cluster and one command could lock everything. You used Redis correct, now you start using Redis not what is designed for. Seems like you doing bold claims without real usage in big systems those approaches.
@dreamsofcode
@dreamsofcode 7 ай бұрын
Yep, you definitely have to consider the use cases for each approach. That being said, we've got a decent cluster than does some real heavy lifting, but we are using a 5 node cluster to achieve it with sharding.
@MechMK1
@MechMK1 7 ай бұрын
What I would have really enjoyed would be comparisons to something like Postgres. When does Redis perform better, when Postgres?
@brangtoggez6363
@brangtoggez6363 7 ай бұрын
it's a completely different database from redis, how can we compare it though ? Redis is for fast queries, postgres is for saving data, two completely different things.
@julienbongars4287
@julienbongars4287 7 ай бұрын
I think if you need a very high throughput for read/write operations, sure. But making redis a drop in replacement for something like postgres or mongo db.. It's not the same tool, it wasn't really designed to do the things you are suggesting so I feel you are trying to force a square peg in a round hole.
@handleistakenbruh
@handleistakenbruh 3 ай бұрын
@Dreams of Code , How would you classify a data base to be small or big? what if my writes to database is for 20,000 users with around 3000 concurrent requests to DB, Will Redis be ok in this scenario or Postgres might work out better? I will definitely go for a regression testing in both as the project itself is small - but still would like to know everyone's perspective as well.
@ytfeelslikenorthkorea
@ytfeelslikenorthkorea 4 ай бұрын
5:04 I am really tempted to migrate one of my projecrts to redis. At the moment it's running on RaspberryPi as a MariaDB table. It's 22 fields wide, 400k of rows. Normal updates are simple, a piece of code grabs the data from API, processes it and updates the table. Then the data is represented through Metabase which really stresses Pi's I/O, especially when I do manual improvements of the data (e.g. update all records where a field matches a regex, that triggers massive updates). Having it in memory would make the experience so much nicer.
@anandsingh-pw6mm
@anandsingh-pw6mm 7 ай бұрын
how to do create presentation like this?
@Ch40zz
@Ch40zz 7 ай бұрын
Killing the pod will just send sigterm and the DB can actually catch that and save the data. I think it might be much more interesting to have a program running in the background spam writing and then sigkilling the DB to see how frequently it saves to disk, if any data corruptions arise and how it handles these. It surely doesnt write every single action to disk without any batching, that would be too slow
@dreamsofcode
@dreamsofcode 7 ай бұрын
You can configure the AOF, on my instance it was flushing the disk every 1 second. It does store every write operation and it does impact performance. If you want a fast cache and don't care about data loss then it's not worth enabling.
@revetastogne
@revetastogne Ай бұрын
Would like to see your video on your homelab. Was there any already? Or would be? Thanks!
@bororobo3805
@bororobo3805 3 ай бұрын
I only use it for sessions. Works well especially for storing OAuth flow acces tokens
@jorgeosorio1613
@jorgeosorio1613 7 ай бұрын
Make a video talking about your home lab, thank you very much.
@user-db4dd4sh2j
@user-db4dd4sh2j 7 ай бұрын
Great video 👍
@rizalbagus614
@rizalbagus614 2 ай бұрын
interesting with your terminal, can you share what theme did you used? 2:23
@abdellatifsraiti2032
@abdellatifsraiti2032 7 ай бұрын
amazing video
@dreamsofcode
@dreamsofcode 7 ай бұрын
Thank you!
@hex7329
@hex7329 7 ай бұрын
Has noone made something like an SQL to REDIS compatibility layer? So I have a "create table" command and it does the appropriate steps for that in redis for me?
Adding a cache is not as simple as it may seem...
13:29
Dreams of Code
Рет қаралды 102 М.
When RESTful architecture isn't enough...
21:02
Dreams of Code
Рет қаралды 238 М.
Kitten has a slime in her diaper?! 🙀 #cat #kitten #cute
00:28
How did CatNap end up in Luca cartoon?🙀
00:16
LOL
Рет қаралды 7 МЛН
ОДИН ДОМА #shorts
00:34
Паша Осадчий
Рет қаралды 6 МЛН
Learn Database Normalization - 1NF, 2NF, 3NF, 4NF, 5NF
28:34
Decomplexify
Рет қаралды 1,7 МЛН
Tmux has forever changed the way I write code.
13:30
Dreams of Code
Рет қаралды 885 М.
Redis vs MongoDB (in 60 seconds)
0:53
DragonflyDB
Рет қаралды 1,4 М.
How Senior Programmers ACTUALLY Write Code
13:37
Thriving Technologist
Рет қаралды 1,3 МЛН
Using docker in unusual ways
12:58
Dreams of Code
Рет қаралды 381 М.
The Absolute Best Intro to Monads For Software Engineers
15:12
Studying With Alex
Рет қаралды 548 М.
I never thought I'd use Socket.io ever again
14:02
Dreams of Code
Рет қаралды 84 М.
How does Caching on the Backend work? (System Design Fundamentals)
22:45
Software Developer Diaries
Рет қаралды 26 М.
Never write another loop again (maybe)
10:48
Dreams of Code
Рет қаралды 247 М.
На iPhone можно фоткать даже ночью😳
0:30
GStore Mobile
Рет қаралды 1,4 МЛН
Save Work Efficiently on Your Computer 18/05/2024
0:51
UNIQUE PHOTO EDITING
Рет қаралды 306 М.
Я Создал Новый Айфон!
0:59
FLV
Рет қаралды 3,4 МЛН