Redis Deep Dive w/ a Ex-Meta Senior Manager

[@eastsideGK]

I've been waiting for videos on deep dives. More deep dives please!! These rock

[@hello_interview]

🫡

[@1234fewgfwe]

1 more thing to add; sorted set, internally is implemented with skip lists and hashtables in Redis. Thats why adding value to sorted set is O(logN) on average and searching is O(logN + M). If you bring this up, congratulations you passed your interview

[@abhijit-sarkar]

On the contrary, if I brought up this statement in an interview after having seen it on KZbin, I just blew the interview. Any interviewer when faced with a candidate that is trying to impress by throwing out irrelevant buzzwords, will ask "oh yeah, tell me more about how to implement skip lists".

[@bhawanasahu3000]

@@abhijit-sarkar couldn't agree more. Mentioning such points on your own always brings a follow up question, now I have to also read how skip lists works.

[@Global_nomad_diaries]

Can't thank you enough. You made system design so much easier to understand.

[@troyfischer6786]

Echoing the existing sentiment in that this content is incredibly informative and well put together. Really appreciate your work

[@hello_interview]

Glad it's been helpful! More soon.

[@remipan4604]

Hi i just want to say your work is truly fantastic and you are an excellent educator! I started watching your videos to prepare for system design interviews and your explanations and examples are top notch. I'm motivated to continue watching not just to prep but also to expand my knowledge and improve my craft. Eagerly awaiting the great work you'll do next!

[@rahulshrivastava3040]

You are best channel for system design interview. Such a indepth technical understanding, I really appreciate you making these videos.

[@artemkobeliev6042]

10:00 didn't quite catch how adding random suffix would solve hot node problem

[@hello_interview]

Splits the hot shard into multiple, so you can distribute that load across nodes.

[@nez14526]

@@hello_interview sorry, what's the approach? I think somewhere else in the video talks about it, but I also somehow did not understand that. Is it roughly the following? Generate X random suffixes. Maintain a map on the (Redis') client side of . Whenever we want to do some sort of "write", we get a random suffix from our list and append it to our key in the Redis command. Whenever we want to do some sort of "read", we get _all_ random suffixes (X of them) from our set, make X reads from Redis, and aggregrate on the client side. ---- Or maybe it's sometimes the opposite way. For example, in a cache, I'm guessing we'd want to write our value to each and every one of the key+randomsuffix. Then for a read, we can (randomly) pick one of the suffixes. ---- Maybe the missing link (for me and others similarly confused) is how/where the random suffixes are managed. There's also the question of how we (automatically?) identify a hotkey, or develop an algorithm that generates (or increases) the number of suffixes depending on some sort of access rate.

[@jain007neeraj]

@@hello_interview but how will you query the key then, as at read time you don't know what random suffix got added

[@Xiao-p9m]

@@hello_interview What if the new key conflicts with any keys added later?

[@vedavyasa9542]

@@Xiao-p9m How will it conflit, we append the certain value every single time so there wont be a conflict

[@MQ.2000]

At 22:55 how does hashing tweet IDs help if we're looking for the most liked tweets overall? Or were you referring to getting top liked tweets for specific keywords when splitting that key across multiple nodes?

[@stefanmai9879]

The latter!

[@anipendakur]

Really cool deep dive, updated with time as well. Thanks a lot for making this. Looking forward to more!

[@SagarSood]

Loving the consistency of uploads! :) would love to see a deep dive on Kafka as well

[@hello_interview]

Kafka is in the works! Written resource will be posted to the website this week or early next!

[@nbx-bi1sk]

Very informative and helpful video as always. I would appreciate if the visualization of the internals could be a bit more detailed and in sync with what you describe, rather than just a couple of boxes. Regardless, this is a refreshing take to step away from the usual system design videos to dive into technologies that are useful in both interviews and real life. Hope to see more of this in the future.

[@hello_interview]

Good feedback, thank you! Will try to make the visuals more engaging in the future.

[@davidoh0905]

The problem of hot key is clear but solution of hashtag is very difficult to understand. do we have to handle that all in the client? like figure out what are the hot keys?? and what will adding hashtag do really?

[@riyaarora7517]

Just love the way you add practical aspects! Thankyou for the thoughtful, concise and articulate deep dive! Was definitely a high ROI on time :)

[@hello_interview]

So glad it was useful!

[@henryso2717]

Great content. This is a great resource for learning before i apply Redis to my own projects. One thing I was wondering was if your content would be able to be added into Spotify so I could listen along on the go?

[@haribhatt34]

Absolutely loved it !!! Can we have one deep dive on SQL vs NoSQL, a topic always asked in system design interview.

[@hello_interview]

We actually have an opinion on that very topic, check this out: www.hellointerview.com/learn/system-design/in-a-hurry/key-technologies#core-database

[@jordanhasnolife5163]

Couldn't focus, too handsome

[@hello_interview]

The GOAT has arrived and has something important to say!

[@jordanhasnolife5163]

@@hello_interview Gigachad onlyfans allocator (of) toes, that's very kind of you

[@coledenesik]

Hey Jordan looks like you are getting your life back.

[@jordanhasnolife5163]

@@coledenesik unclear at best

[@lesterdelacruz5088]

What is the difference between Redis Pub/Sub and Kafka? From the other video, what I understand is Kafka is queue and event driven as well.

[@hello_interview]

You can use Kafka as pub/sub! Differences in durability (Redis in memory, Kafka persists to disk), which then has consequences on other things like ease of disconnection, etc. But they can both be used.

[@hello_interview]

Redis pub/sub is really dumb, at-most-once delivery of messages. You can think of it as a hashmap from a topic (string) to a socket connection to a subscribing server with all the associated problems (messages might get lost?) and benefits (incredibly lightweight, low-latency etc.) Kafka is more full-featured. Kafka has message persistence, you can make guarantees on delivery, etc.

[@lesterdelacruz5088]

Thank you.

[@dashofdope]

so in terms of consumer groups subscribing to topics in a pub/sub system... can you summarize when to use Redis vs Kafka?

[@Keyboardscholar]

I really enjoy your content on the website, the videos as well as posts on reddit. Can you make another video on behavior/leadership principles interview next? I would like to learn how to tell a convincing story about my leadership potential.

[@Nnngao4231]

this video is awesome! exactly what I expected!

[@hello_interview]

So glad to hear it!

[@rahulbm]

Great content. Really liking it. Can you please do a deep dive on TiDB, Rocks DB and Cassandra too

[@hello_interview]

Cassandra is soon!

[@SrinivasMuthukrishnan]

Fantastic video, thank you!😊

[@Marcus-yc3ib]

Hi, you talked about append a random number to solve the hotkey issue. But when you want to retrieve the data, you won't know the key because you append a random number.

[@kiranpai8]

Enjoyed both your videos on Kafka and Redis. Can you please make a video on CQRS?

[@nishanthooda]

Great watch! A similar deep dive video for Kafka would be amazing 🙏

[@hello_interview]

Kafka is in the works! Written resource will be posted to the website this week or early next!

[@nishanthooda]

@@hello_interview amazing!

[@aanurraj]

Can we have a deep dive on Apache Flink please ?

[@hello_interview]

We'll put it on the list!

[@armanmalik]

Why would I want to use Redis stream over something like Kinesis or Kafka streams

[@hello_interview]

In practice? Substantially faster and cheaper if you can accept some of the (many) compromises. In an interview setting it can be helpful to have a few multifaceted tools vs having to know all about Kafka (video on that soon). If your business was entirely built around append-only logs I probably wouldn't take Redis as an acceptable substitute, though it works in a pinch.

[@adityakirankorlepara4500]

Excellent video. Please also make video on mock interviews

[@hello_interview]

Mock interviews are what we do! What is it you want to see?

[@SlimJones62]

Wait .... I thought I clicked on a Dua Lipa music video. 😀. Just kidding. Stefan I love this - clear, concise, articulate and I learned something. Keep 'em coming.

[@hello_interview]

Lol. The intro is a banger, no :)?

[@rogermarin1712]

Can i book sys design coaching sessions on hello interview or is it only for mock interviews?I'm a senior engineer and I've just started preparing do you recommend to jump straight into mocks?

[@hello_interview]

Send us an email at [e-mail] and we might be able to make something work. Have some products planned here for people in your shoes.

[@rogermarin1712]

@@hello_interview done

[@ItsMeIshir]

It's a great deep dive. I'd like to give a suggestion, Can you guys add chapters to the video, so that, if I want to rewatch again about certain portion of the video then that would be easy thing to do.

[@hello_interview]

You should have chapters on this one! If you look at the description "Key Moments" will take you to the relevant sections. LMK if we're not doing something else we should be doing!

[@ItsMeIshir]

@@hello_interview I see that, you missed to add "00:00" time stamp in your "Key moments", that's causing YT to not auto chaptering your video. I have checked couple of other videos in the channel, videos which are having 00:00 in their key moments, respective chapters are added.

[@russbaker6861]

Would a sorted set implementation be appropriate for a priority queue?

[@hello_interview]

For many, yes. Depends a bit on the scalability requirements.

[@russbaker6861]

@@hello_interview Awesome, thanks so much for the quick response. I really appreciate what you guys are doing, this is a great resource and y'all rock!

[@kunalsharmain]

Loved it! Want to see more on how communication between service on global scale like using kafka with async or consistent connection around this please, if you get the idea, Sorry i am bad at explaining

[@ganeshkudva817]

Can you please do deep dives on Kafka and Spark too ? I love your videos

[@hello_interview]

Written deep dive on Kafka coming in a couple days!!

[@hazemabdelalim5432]

But how will you handle the at most one delivery of redis in the case messaging?

[@hello_interview]

Check out our Whatsapp guide for one example of this: www.hellointerview.com/learn/system-design/answer-keys/whatsapp

[@nikolagrgic3773]

Thank you very much

[@hello_interview]

Re-upload, sorry! Stefan is a KZbin noob.

[@stefanmai9879]

Facts.

[@aanurraj]

🤣

[@刘天旻]

Thanks so much for the video. I would be really interested in deep dive on elastisearch/opensearch

[@hello_interview]

That one's next!

[@PrasannaRanganathan6078]

Nice! Can you also do Deep Dive on Graph Databases(eg, neo4j) as well. please!

[@hello_interview]

Will add it to the list! Typically, graph dbs are not as commonly used at scale.

[@maxvettel7337]

It would be nice to see a Deep Dive video about location database. Pretty hard to understand for me

[@hello_interview]

Which part is most confusing? Geohashing? Quad trees? Where to use them?

[@maxvettel7337]

@@hello_interview Geohashing algorithm is too complex. I just cant take in all this diagrams. I also dont understand how proximity systems can search something inside the circle using square blocks of geohashes

[@hello_interview]

@@maxvettel7337 Gotcha! I'll earmark this for follow-up. The nice thing is you don't necessarily need to know about the geohashing internals to use something like Redis' proximity search in practice!

[@alpacasCodebytes]

Amazing Content.. Can't thank you enough!! Any Plans for DSA Mock Interviews 👀

[@hello_interview]

What are you looking to see?

[@alpacasCodebytes]

@@hello_interview DSA Mock Interviews with FAANG Candidates..

[@VIPINKUMAR-dr7vu]

Please make a video on Elastic Search

[@implemented2]

One can imagine the implementation of the geo-spacial index using 2d indexing data structures - quad tree, R-tree and others.

[@databasemadness]

Solid value.

[@womanwithtoomanyhobbies]

Finding it difficult to understand stream and consumer group

[@flyingpiggy741]

Could Even cover Redis again?

[@hello_interview]

Nah, this explanation is killer. Couldn’t do it better if I tried! -Evan

[@flyingpiggy741]

@@hello_interviewPlease

[@flyingpiggy741]

@@hello_interview at least we can highlight the knowledge point . Have some summary on the diagram

[@flyingpiggy741]

14:40 the nice thing about this stream is if we have got something we need to make sure all of the items of the stream are processed, the redis give us a bunch of premitives to work with. What does this mean?

[@Richard-yw9if]

I agree with you . This is the first out of ten video I felt I learned nothing after watching in this channel

[@Richard-yw9if]

Why not talking about cache invalidation when using as a cache?

[@ediancomachio2783]

I have never clicked so fast!

[@guidoscalise]

Kafka next, please!

[@hello_interview]

This week!

[@joemiller1057]

For hot key you really need to have caching in the service that's hitting Redis, unless you have a ton of hot keys a small LRU cache should work. Although this trades off some consistency unless you have a real short TTL :) - Dynamically sharding keys adds complexity to the application layer - Dynamically scaling read replica nodes seems wasteful (you dont need to duplicate the whole node) All solutions seem non optimal.

[@hello_interview]

Yes! No free lunch.

[@anuragtiwari3032]

I see , I like.

[@adithyar3160]

this video is too high level..need more simpler deep dive on redis if possible

[@hello_interview]

Which part did you miss? Maybe we can point you in the right direction.

[@flyingpiggy741]

I missed the co founder now

[@randyorton06]

do a deepdive on ethereum latest version, l2 rollups

[@XxXx-sc3xu]

Format reminds me of acloudguru

[@ayushjindal4981]

I had to struggle with the accent. Can I pls request to speak a bit slower and maybe a bit more clear accent, if possible. Thanks!

[@dafivers4127]

brah why do you use so much big words. lmao slow down with the vocabs damn.