Caching Pitfalls Every Developer Should Know

Get a Free System Design PDF with 158 pages by subscribing to our weekly newsletter: bit.ly/bytebytegoytTopic
Animation tools: Adobe Illustrator and After Effects.
Checkout our bestselling System Design Interview books:
Volume 1: amzn.to/3Ou7gkd
Volume 2: amzn.to/3HqGozy
The digital version of System Design Interview books: bit.ly/3mlDSk9
ABOUT US:
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.

[@IceQub3]

Cache invalidation is the elephant in the room

[@dave6012]

All I know is there are 2 hard things about programming: - naming things - cache invalidation - off-by-one errors

[@ANONAAAAAAAAA]

The most important knowledge about caching is: don't cache unless it's absolutely necessary. Also, database in conjunction with read-replicas can be much more resilient and performant than your homebrew crappy caching mechanism. Caching is final resort, after you've tried everything you can do with database, for example, tuning queries, adding indexes etc.

[@build-your-own-x]

this should be the first lesson in every caching learning resource.

[@RodrigoVillarreal]

This only applies if we are doing simple selects / batch data. Read replicas don't solve issues like time consuming queries (expensive joins / aggregations / etc). Of course, you clearly specified: "Unless it's absolutely necessary" :)

[@Osono2diWorld]

You’re not building a scalable system without caching.

[@wilfredv1930]

there should be a pretty deep evaluation to understand if caching is necessary or not. failing in that would lead to disaster sooner or later

[@michaelkhalsa]

Caches work well in tandem with claims, that is some data should never be cached, but rather claimed by a user during the editing process, with ability for another user to revoke the claim. Saves always check claim status first. Multi level cache invalidating is important. For example, products returned for a catalog can use a product cache with a polled cache invalidating process. Yet, for display on a product detail page a quick check of a timestamp is done which is reset if any of the various product tables are updated by a trigger. Thus a quick scaler db query ensures the cache is valid, which also helps protect the cache itself from becoming stale. This can be taken even further with a dedicated timestamp table for complex product information. This approach dramatically improves performance, always guaranteeing valid data on a detail page, while keeping catalogs valid within the timespan of the cache manager for polling a cache invalidation table for changes.

[@Kingside88]

I really appreciate your video. They are so high quality with the best explanation. Can you please make a video of what is the best strategy for using a relational database like Microsoft SQL or what else together with elastic search. How to keep them synchronized? Thank you in advance

[@maxvaessen]

Very useful! Thank you ❤

[@rogers.1228]

Use jitter to the TTL to reduce cache avalanche and many related issues

[@BigHalfSteps]

I'm surprised that there is no mention of a easy solution (albeit there might still be an issue when starting from a cold cache) for a Avalanche/Stampede: Just use different caching times. That should somewhat alleviate when the database is hit with multiple requests. But in essence, only cache what is necessary.

[@mailbrn78]

Thanks for the insights on cache management. Could you plz suggest keys in cache need encryption? What should be the key flush time.

[@toxicitysocks]

Would love to see you tackle cache consistency too: what happens when the database write succeeds but the cache write fails? Or if the database is written concurrently to 2 different values but the last write to the database was value a, while the last write to the cache was value b? Now the cache is forever inconsistent.

[@wesamadel3612]

remove cache entry on every update

[@toxicitysocks]

@@wesamadel3612 sure, but what if there’s a network failure getting to the cache after you do the update in the db?

[@eNITH24a]

Search the dual write problem. Using some form of event driven system, like a CDC (change data capture) in your db to write to an apache kafka stream. That way, the log is persisted in the queue. Say, the cache goes down, the event will still be in the queue and can be read when the cache is back up. However, I'm not sure what you would use to source data from kafka to redis.

[@toxicitysocks]

@@eNITH24a yeah, that’s the best solution I’m aware of. I just wonder if there’s any other way besides CDC or two phase commit. As for getting data from Kafka to redis, you could write a simple consumer service that reads the events and writes to redis.

[@eNITH24a]

@@toxicitysocks I figured you could write a consumer service, but then doesn't that introduce another source of failure? Or I guess it's a lower source of failure if all it does is push to redis from Kafka.

[@simonbernard4216]

at 0:27 on the left diagram, shouldn't the process order be : 1. Data request 2. Data response (no cached data) 3. Read original 4. Copy cache

[@vladyslavlen9490]

What if we add some kind of jitter for the key TTL, so we minimize the probability of having them expired at the same time?

[@eduardokuroda8586]

to use a bloomfilter it sounds like easy, but it can't delete element yeah? will it need to refresh the bloomfilter after something to ignore deleted itens?

[@parthi2929]

If something creates traffic, have a traffic signal, ok.. a lock here... regulate.. If something creates traffic, have multiple systems (web server/cache server etc) to handle it.. If something can fail, have redundant backups of THAT.. this applies to anything.. also.. to know if cache could fail in DB caz relevant answer not in DB, note down that before in some way..

[@kazama81]

Question: What's the point of a cache server? Why the server itself / webserver is not doing the caching? If caching is supposed to be for fast retrieval, if we store it in a different server, won't the network call take more time than querying a db in the first place?

[@devid3085]

At 3:58 the "find key" arrow has a typo and it should be (3) instead of (4)

[@raj_kundalia]

thank you!

[@PranaySoniKumar]

Can't we use request collapsing to prevent stampede? As it maily due to expired cache entry and multiple requests are trying to access the same resource?

[@micahpezdirtz8196]

How can there be less than 1m subscribers to your channel? You have the best explanations

[@nixjavi7220]

Amazing vídeos

[@NemiroIlia]

2:33 a hidden smiling gem at the bottom right

[@user-je9fw2rl5s]

What use to for presentation and demonstration, please ?

[@ANONAAAAAAAAA]

The scariest story I've head about caching is, pages containing user's private info are cached on CDN, then they forgot to include session id in the cache keys...

[@StoCoBoLoMC]

What is CDN? Why can't info be cached on it?

[@venberd]

Content Delivery network - Vendor operated nodes geographically located closer to user. Very little isolation guaranteed. Don’t want to put your users data permanently there

[@chrishabgood8900]

Could the db invalidate the cache?

[@krazeemonkee]

“cache me outside, how bout dat”

[@rizthetechie]

Just wondering if proactively cache the pages on expiry again, why put expiration ?

[@TyllerJorEl]

data may change; also, if delete-key events randomly go missing (it could happen for a myriad of reasons), stale data could pile up forever and fill up the memory

[@titogorla]

love it

[@zixuanzhao6043]

what about consistency

[@MrSofazocker]

Hear me out. What if you don't ever hit the DB, but the Cache either halts or returns a null value for all requests, and singularly fetches the data from the DB. Then answers all suspended requests!?

[@gus473]

Comment for algorithm, thank you! 😎✌️

[@vadymk759]

There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton

[@uchuuowl7605]

GOLD

[@franklinoladipo2343]

Was thinking he would talk about Cache Invalidation

[@rishiraj2548]

👍

[@jerichaux9219]

Obligatory zeroth.

[@aruneshprabu7925]

❤❤❤❤

[@navinmittal4809]

In summary, I think he explained 3 scenarios when there's a huge load: cache miss on single key (stampede), non-existent key (penetration), bulk cache miss on multiple/all keys (avalanche).

[@chologhuribangladesh7792]

Most people do not care about caching. They cache whatever they want.

[@burhanudinyahya]

😂😂😂

[@atifadib]

Cache Crash

[@wishmeheaven]

Is this video sponsored by Redis?

[@ChuckNorris-lf6vo]

Hi bro can you explain complex solutions with a little bit more LIFE in the voice? Like give it MEANING give it energy. Solutions are important bro. Be more alive so it is crystal clear, why a certain solution is being done even if it is very complex. Thank you. And make the video longer if you have to but make it the best god damn explanation of the problem. Thanks.

[@Geza_Molnar_]

I'd like to watch short (beginner : 5-15 mins) - medium (advanced : 10-25 mins) - long (professional : 30-60 mins roughly) videos about the topics which are on the channel. I'm fine with the content, that's already enough 'energy' for me. I have my motivation to listen, to think about that, to hit the back button when I need repetition or more time for thinking and understanding (no need for more 'life in the voice' for me to push me).

[@BenjaminMeasures]

There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton