Choosing a Database for Systems Design: All you need to know in one video

Oh honorable mention for elastic search when you need an inverted index for full text search but you shouldn't be using that as a primary database.
Make sure to use HBase when you make your Pornhub clone!

[@pieter5466]

Two things make this video stand out for system architecture interviews: 1) general knowledge of the available options, with arguments for and against 2) enough in depth knowledge to go deep and impress

[@user-ue9ri9ip1v]

One of the best videos of its kind. small inaccuracy: Hbase being wide-column store actually store column families together, not individual columns.

[@jordanhasnolife5163]

Appreciate it!

[@ritwik121]

Glad you are back with system design videos😭😭

[@jordanhasnolife5163]

We'll see about that one buddy, these have been covered mostly

[@JeT2686]

I gotta say, this summary video is great! As much as you dread redundancy here, I at least got a ton of value of out of it. The material is fantastic for reviews Kudos, and great stuff!

[@rajrsa]

I've gone through all your concepts and interview video and this video did a great job of summarizing everything! Thanks for everything, giga chad! :P All the best, y'all! Let's get this bread! 🚀

[@cezary.stanislawski]

wow

[@mnchester]

Thanks for this video man! While I agree with you that it'd be better to watch your more in depth videos, this compilation video works great for a quick recap right before going into your System Design interviews

[@jordanhasnolife5163]

Glad to hear!

[@pl5778]

this is an awesome video! thanks for such a great summary

[@arshadhammad]

Thanks for the nice series. I really liked your videos

[@thktomska]

This is what i'm look for! Great quality - thank you very much!

[@prafullakh]

Absolutely great work. Someday you should talk about the interview questions that you asked candidates and any interesting approaches they took and also about some interview questions that zapped you. PS: Towards the start of this video you asked us to get lotion and paper. What gives?

[@jordanhasnolife5163]

1) I've never interviewed anybody, I'm a sham :) 2) you need the paper to take notes and the lotion to keep the pencil from sticking to your otherwise sweaty hands

[@FranckPachot]

You are sharing awesome content. Great to link to for short and acurate explanations. Would be great to see more on Distributed SQL (you did Spanner but there's also YugabyteDB, CockroachDB, TiDB, YDB). And on PostgreSQL compatible databases (you did Aurora but there's also AlloyDB, Neon, YugabyteDB)

[@jordanhasnolife5163]

Nice idea! And thank you!

[@akhilkhubchandani2632]

Great work and amazing video! Could you also make more low level design videos?

[@amospan14]

Really good one! Thank you Jordan! =)

[@raj_kundalia]

Thank you, Jordan!

[@luiscarlosricoalmada4296]

I finished the whole series :) , wish me luck on my System Design interview

[@jordanhasnolife5163]

You got this!

[@420_gunna]

16:30, I haven't heard of column compression being used for image data in the way that you describe here, any pointers on what you were talking about when you mentioned this?

[@jordanhasnolife5163]

Hey so I don't actually mean to compress the images with column compression: I just mean having a column containing multiple images means that you only have to fetch the images themselves as opposed to potentially a lot of metadata that may come with them (if you were to fetch a row at a time)

[@BenLernerOfficial]

​@@jordanhasnolife5163 I paused the video at this point in confusion as well, because I'm afraid the example doesn't make much sense. In the query you described, you only want to get the thumbnails associated with a specific video, so you would either implement that with a relational table (full_video_id | thumbnail_id, where one full_video id is associated with one or more thumbnail_ids) or you'd store a list of the thumbnail_ids (pointing to the actual image data in, say, s3) on a document representing the full video. The only situation in which you would possibly want to store images in a column is if you'd want to somehow query ALL thumbnails across ALL videos, but that is not the situation you described - you described getting the thumbnails of a SINGLE video. That would be OLTP/row-based, not OLAP/column-based. Also, columns typically contain primitives (so you could, for example, perform an average across a column of floats)

[@jordanhasnolife5163]

@@BenLernerOfficial Yes sorry, this is assuming that one video might have many thumbnails (e.g. to create one of those gifs that you see on KZbin now). Sorry this wasn't clear, everything that you've said is accurate.

[@jordanhasnolife5163]

Another common use case is to load all thumbnails for a user's channel, such as if you were to click my channel page.

[@roywastaken]

Thank you senpai 🙏🏽

[@chits006]

Great Video, One question, where can we learn about db schema design? Some basics and exercises would be good, any online course you recommend?

[@jordanhasnolife5163]

I'd just look at database docs and existing engineering blogs from reputable companies!

[@andreybraslavskiy522]

Thank you for the great content

[@sohansingh2022]

Thank you buddy!

[@hdrkn5247]

which database is of choice when you need SQL database but the dataset is too large and you need to shard the data or the database needs to be distributed?

[@jordanhasnolife5163]

A SQL database lol. You can still shard your data here, just be smart about how you do it.

[@sarfrazz34]

Thanks for the video man! it was informative could you please create a video if possible on scenario-based database usage I am really confused about where to properly use sql db and nosql db I am little clear that if we need ACID properties then best is sql. but I am not completely aware of different other scenarios on where to perfectly use sql and nosql dbs. if you also have any resources please share I am not able to find a good one

[@jordanhasnolife5163]

I think you basically just expressed it yourself - "if you need acid properties use sql" - if data integrity is the most important part of your application, SQL is the way to go. Otherwise, NoSQL can offer greater speed while sacrificing some of these requirements.

[@sarfrazz34]

​@@jordanhasnolife5163 Thanks Jordan I am thinking of a scenario in case of storing product related things I see nosql is best suited as different product could have different properties, but how about managing the inventory for the product? in this case since it requires acid props to manage the inventory count properly, should we maintain the inventory count details alone in sql DB?

[@zuowang5185]

why redis instead of just using the hashmap in your program? for cross process communication?

[@jordanhasnolife5163]

Well sometimes you want many servers, sometimes you want replication, sometimes you want a writeahead log, sometimes you want database partitioning

[@franklinyao7597]

What if you need a NoSQL store with strong consistency? You need Hbase or MongoDB. And if you need a db optimized for heavy reads, you may need MongoDb since it uses B tree.

[@jordanhasnolife5163]

Mongo might be better for reading sure, but I caution you from saying it and HBase are strongly consistent. Hadoop has some weird writing thing that kinda makes it strongly consistent, and maybe you can configure mongo to do so, but Hadoop writes aren't like actually achieving consensus (and afaik mongo isn't either), so it's kinda just not great for that haha

[@franklinyao7597]

@@jordanhasnolife5163 what is that weird writing thing?

[@jordanhasnolife5163]

@@franklinyao7597 You like write to multiple nodes at once and only get a success message if it's hit a certain amount of them, but the write still goes through on some of the nodes even if you don't meet the success threshold if I remember correctly

[@danielvega-myhre4201]

Are your slides available to view/download somewhere?

[@jordanhasnolife5163]

In my channel description

[@ravindranaths513]

Could you please make a video on Wide column vs column family vs columnar vs column oriented DBs with some examples

[@jordanhasnolife5163]

Hey! I think I probably mentioned this more in the 1.0 series but not sure that it deserves a full video, just look up images of the formats :)

[@ravindranathsirisala6408]

@@jordanhasnolife5163 , please give me link of that video

[@whirr9755]

huh, i subbed for day in the life vids 😒

[@jordanhasnolife5163]

I'll sell out soon I promise

[@tamarapensiero8048]

Hey Jordan, just started watching every video you've created. I love them. I'm wondering how I could get in contact with you as soon as possible. Id like a couple minutes of your time if possible. Thanks x

[@jordanhasnolife5163]

LinkedIn would probably be best, my name is Jordan Epstein

[@tamarapensiero8048]

@@jordanhasnolife5163 thank you, sent a msg ^_^

[@ashwint959]

What about distributed sql databases like spanner/cockrorachdb?

[@jordanhasnolife5163]

I think these are probably worth knowing about from a software engineering perspective but probably not worth using in a design for an interview. Spanner (can't speak for cockroach) is great, but I think it may be too niche to be fair game here (since it doesn't exactly have a "dedicated" use case).

[@luli829]

how do you gain some much knowledge in system design? really amazing!

[@jordanhasnolife5163]

I have no life! No but actually, I just have optimized my knowledge specifically for the interview haha - I'm sure you all are better software engineers than me

[@luli829]

@jordanhasnolife5163 lol no. I'm trying to learn from you and get better :)

[@chits006]

Why no honorable mention of Dynamo & BigTable ?😀

[@jordanhasnolife5163]

Mainly because bigTable = hbase and dynamo = Cassandra (it actually may not assuming you're talking about dynamodb but theres no docs on internal implementation afaik)

[@rydmerlin]

Are trees with more than two children for a given parent still considered binary trees?

[@jordanhasnolife5163]

Nope

[@piercef7343]

Kudos!

[@Stella-se1lg]

Salute😊

[@ameygoel1]

bro I watched your earlier videos in 1.25x speed and now your normal voice feels weird and slow. Nevertheless great and orderly content. Cheers! Would recommend others too :)

[@jordanhasnolife5163]

Damn bro 1.25? Gotta speed that up to 2

[@sachin_yt]

Finalyyyyyyyyyyy

[@prathamsinghal5261]

Scylla DB ??

[@jordanhasnolife5163]

I'd consider it a Cassandra clone

[@effy1219]

hahahah i just like how he call us , you lazy f**s and do it

[@ladyv890]

Yay for Women!

[@jordanhasnolife5163]

Just defended women against a mysognist on Xbox live the other day

[@ladyv890]

@@jordanhasnolife5163 Yay Jordan! 🤗 lol

[@mvp4gman]

No S3 🥲

[@jordanhasnolife5163]

Not a database - though technically some cloud native data warehouses are being built using s3 as the storage layer and parquet files

[@SreekantShenoy]

This guy stores! 🫣