System Design Interview - Notification Service

Please check out my other video courses here: www.systemdesignthinking.com
Topics mentioned in the video:
- Functional (create topic, publish message, subscribe to a topic) and non-functional (high scalability, high availability, high performance, durability) requirements.
- High-level architecture of a notification service.
- FrontEnd service host components (reverse proxy, local cache, logs and metrics agents).
- Metadata service, distributed cache, consistent hashing ring, gossip protocol.
- Storage for messages: SQL/NoSQL database, in-memory store, distributed message queue, stream-processing platform.
- Message sender service, thread pool, semaphore.
- Duplicate messages, retry policy, message order, security, monitoring.
Inspired by the following interview questions:
Amazon (www.careercup.com/question?id...)
Flipkart (www.careercup.com/question?id...)
Microsoft (www.careercup.com/question?id...)
Uber (www.careercup.com/question?id...)

[@riteshthakur9064]

finally, someone talking about system design who actually know how things work internally :-). Kudos to you brother!

[@ted2101977854]

This is somehow the best system design tutorial I've ever seen. Keep doing bro!

[@SystemDesignInterview]

This is somehow one of the best feedbacks I've ever seen. Keep sharing your thoughts with us all bro! )))

[@gxbambu]

2nd this. Not only for interviews, but also for day-to-day work! thanks.

[@xiaopeiyi]

It is.

[@mostinho7]

@@SystemDesignInterviewthank you for making these videos! What would you recommend to learn these topics in depth? What do you personally use? Any good books, courses etc?

[@SamWhitlock]

The pedagogical steps in all of these videos are perfect. So many other "courses" say "here's how to design whatsapp" don't really start from first principles like this! (leaving me scratching my head wondering why they did something!) I hope you find time to continue making these kinds of videos. I'd absolutely support via patreon or an online course if you made it. There's really nothing as well-presented as this out there!

[@sigorbor]

Absolutely the best System Design videos on the internet. I especially love that the videos are kept relatively short despite the huge amount of knowledge and explanation. Waiting for the new videos! Спасибо огромное, Михаил!

[@sunnyshang4350]

This is the most densely packed sys design video I’ve seen and it’s so full of good information. Really appreciate your work!

[@funnyhjk]

Dude your videos are so good! Best systems design videos I've seen, love how you've especially tailored it for interviews. Thank you!

[@saurabhchoudhary9260]

Best System design Video ever. No buzzwords and it also goes into depth of various components reasonably well. I'm waiting for more videos from you

[@SystemDesignInterview]

Thank you for the feedback, Saurabh!

[@vova_dev]

Mikhail made a course. More material there now)

[@SatyanarayanaBolenedi]

I feel, This channel is hidden Gem!!! Thank you so much for explaining things in great details!!

[@SystemDesignInterview]

Thank you for the feedback, Satyanarayana!

[@hadimajeed1078]

The best part of this channel is that Mikhail does not use ready-made solution available in the market. He shows us how to think simple and pushes for us to think what is under the hood. The thinking simple forces us to challenge our basics. Exactly what we need to prepare for System Design interview. Sadly, he has not been publishing any new videos for a long time now :-)

[@SystemDesignInterview]

Glad you liked the channel, Hadi! Thank you a lot for the feedback. I do plan to come back to KZbin with more regular video postings. Just need a bit more time to finish what I am working on currently. Stay tuned.

[@gemtyler8258]

@@SystemDesignInterview appreciated with all the videos!

[@jituborse2193]

These videos are really high quality and in depth. Haven't found any others diving deep into each individual component specially with options for each component and pros and cons for choices made.

[@SystemDesignInterview]

Thank you, Jitu! Glad you liked the content.

[@amitdubey9201]

After watching several courses and reading several blogs yours is truly top class and one should watch any of your video more than once to consume it thoroughly.

[@bankybaba]

Highly underrated content that is not just full of buzzwords. Please keep it up. Can't wait for more uploads.

[@vinibaggio]

I had an interview at A Very Big Company that asked me a similar question and this video was extremely helpful.

[@SystemDesignInterview]

Very glad to be helpful! Thanks for sharing.

[@NitinPatel-ld5qd]

Another fantastic addition to my system design favorites! Such a simple and easy to understand design of notification service. Thank you Mikhail and pls keep posting such a great videos :)

[@SystemDesignInterview]

Thank you, Nitin, for the feedback! Much appreciated.

[@salmanahmed3949]

Another excellent System Design video: extremely thorough, detail oriented, and nailing both Functional and Non-functional requirements.

[@SystemDesignInterview]

Appreciate the feedback, Salman! Thanks.

[@nilanjansarkar100]

I am so lucky I stumbled upon this channel. Amazing work, please keep em coming :)

[@abhishekaggarwal9774]

Thank you so much for awesome content. Also, I want to thank you for replying to most of the comments. I could learn a lot from your answers in comments. I would like to see more and more such high quality videos, but given these videos requires lot and lot of effort and as you mentioned in other comments that you are too busy in your work, we respect your time and choice.

[@SystemDesignInterview]

Thank you very much for the feedback, Abhishek! Please let me know if you have questions/concerns or looking for an advice on a technical problem. Will be glad to help.

[@hanhai8515]

I browsed through 4 system design channels, you are the best so far

[@athanasiosterzakis128]

Very helpful content, not just for interview preparation but for a deeper understanding of the concepts. Truly enjoyed all your series. I actually interviewed with one of the FAANG companies and got similar system to design.

[@SystemDesignInterview]

Thank you for the feedback, Athanasios! I hope your interview went well!

[@dmxrahul]

Damn bro. This was dope. Thank you for making such a detailed video. Deepest gratitude ever expressed.

[@SystemDesignInterview]

Deepest gratitude for the feedback, bro )) Appreciate it!

[@amanvidura8267]

Every second of this video is GOLD!

[@SystemDesignInterview]

Glad you liked the video, Aman! Thank you for the feedback.

[@deshengli]

Please Please keep doing this bro! This is the best system design tutorial I've ever watched.

[@SystemDesignInterview]

Thank you, Desheng, for sharing the feedback! Much appreciated.

[@nikhil_mehta]

Mikhail - Amazing videos as always. Really good content and explanation. Thanks for spending time creating such videos! One minor suggestion - it will be great if you can also explain the DB choices (SQL vs NoSQL) and which of the popular NoSQL DBs (Dynamo, Cassandra, MongoDB etc) is a good choice for the use case being discussed in all your videos.

[@SystemDesignInterview]

Hi N M! Thank you for the feedback! You are right, DB related topics deserve more attention. And I plan to close this gap. The recent video (kzbin.info/www/bejne/mIard5pueL95rdU) contains some information on this topic. More to come!

[@yueliang171]

Best system design interview resource online!!!! The only drawback would be the number of video :) keep up the great work!!

[@SystemDesignInterview]

Thanks Yue Liang! Working on more videos. The next one will come out in a week.

[@weilai1958]

Yue, got you here:)

[@tingzhou9552]

Thanks for your sharing. This is the best system design video I've ever seen and I will recommend this to others!

[@SystemDesignInterview]

That is everything I can ask for - share the knowledge with more people. Thank you for the feedback, Jeremy, and the kind words!

[@umber3117]

Excellent explanation!!!! You are teaching us how to think which is a very important part. Most of the youtube channels are just showing big and scary architectures of the companies which are not useful from the interview point of view or from the learning point of view.

[@SystemDesignInterview]

Glad you liked the videos, kry kwy. Thank you for the feedback!

[@partrivedi1122]

I have been watching Mikhail's videos for 2 years now, and they continue to be extremely valuable, and the best sys arch videos on youtube. If you can grok all of his concepts, you will be prepared. Still, it would be nice to see Mikhail cover some of the more advanced topics that the book "Designing Data Intensive Applications" discusses.

[@GaneshManika]

Awesome! every component, every process, and every possibilities are explained very well. Thank you!

[@durgaprasana5531]

Nice explanation. 👍👏 Often interviewers are looking for the entity models for the stored data. i.e. metadata for a subscription in this case. It would greatly help to include those aspects as well.

[@lijuanzhang78]

Thanks a lot! Really the best system design which has very good balance of high level logic and low level details. This is what I have been looking for!

[@SystemDesignInterview]

Thank you for the feedback!

[@pawantiwari_ultimate]

You are doing a really great work. Thanks for compiling so much knowledge here.

[@junminstorage]

Thumbs up here! I implemented a similar but more complicated system (with additional retry queue, message filtering or transformation, complicated UI with search capabilities) like you described in this video, it is a production platform service used by engineering teams in the company. Certainly you are a seasoned engineer and have hands-on some of the technologies you are talking about in this video.

[@SystemDesignInterview]

Thank you for the feedback, junminstorage! You totally deserve my praise for implementing a real production system like this. There are many small details that are tough to do right. Well done!

[@dipendrabhatt8789]

Where were you my entire life bro. This is so good.

[@SystemDesignInterview]

Glad we eventually found each other ))) Thank you for the feedback, Dipendra!

[@vivekgupta8580]

I guess best System Design Interview Resource available. Thanks for creating this channel

[@vadimkrivcov383]

Rarely leave comments to videos - but that one is amazing! Great job! Please post move videos to related design problems (e.g. monitoring/logging systems, Twitter/Instagram clones etc.)!

[@SystemDesignInterview]

Thank you Vadim, appreciate the feedback! There are plans to cover all the topics you mentioned. But cannot promise specific dates.

[@152jatin]

Amazing content !! Probably the best. Thanks for sharing your knowledge in such a presentable manner.

[@novadragonxx5]

I just found your videos to help me study, they are the best in depth explanations!

[@SystemDesignInterview]

Glad to be helpful, novaaaa. Welcome to the channel!

[@mangeshshikrodkar6192]

Your videos are highly informative. Please create more such videos. They are way better than other youtube videos on system design i have seen. The only thing that I found missing is scalability numbers and estimates like bandwidth calculations, number of servers needed, amount of storage needed and how they can go as system stress goes as per growing userbase. In some videos i noticed top down approach like they start with numbers first and break down the question in some feasible small system. Here I see a bottom up approach where we start with small system and grow it with scale. I believe starting with small system and growing it is better than starting with big system and breaking it down and playing with numbers.

[@SystemDesignInterview]

Hi Mangesh. Thanks a lot for the feedback! Let me add a topic to my TODO list on how to estimate capacity (number of servers, required network bandwidth, storage, etc.) for a distributed system. As an interviewee, I find its hard to start with numbers, unless problem domain is well-known. Without understanding the API (what data we send into the system and how data is retrieved) and at least high-level design (where and how we store this data), high chances that estimated numbers will be far off. And as an interviewer, I would also recommend to postpone numbers till the end of the interview. Or provide them during the interview, if requested. In many cases the problem domain is not known to us upfront and we need to start with something simple (similar to a brute force solution in a coding interview). What is really important for me as an interviewer, is that a candidate is able to identify units of scalability. E.g. for the notification service such units are: number of publishers, average number of topics per publisher, average number of messages per topic, average number of subscribers per topic, amount of time we store messages in the system (retention period), etc. And as long as we can discuss scalability issues on a t-shirt size (small, medium, high) level, it is usually enough to evaluate candidate's ability to think at scale. Numbers are crucial for real designs, though.

[@SP-yf1ib]

@@SystemDesignInterview I totally agree with you that it is hard to start with numbers. We should've some idea about the numbers to come up with all the design considerations, but it doesn't sound like a great idea to just crunch all the numbers and capacity estimation in the beginning like how many hosts are required, before even getting to the details of the design. So in my mind, we should've some idea about the scale we need to support, but we need not get into the estimate capacity initially. But unfortunately all other random youtube video channels started following this idea of capacity estimations and now it is becoming a norm in such a way that some inexperienced interviewer also think that it is normal to expect that and insist on it.

[@gajapathy5209]

Great content! One question I had is currently frontend server pushes message to temporary storage and sender component retrieves message from temporary component. Here, we may need to keep watching the temporary storage to start the sender process. Instead we can take a hybrid approach. 1. All the publish() calls will be transmitted to sender component after persisting data in temporary storage. Now sender system acts on the task . 2. In the case of message failures, we will write that into a re-try queue and a re-try component which handles the request

[@ignashi7plays401]

" we may need to keep watching the temporary storage to start the sender process". But isn't the Sender service(using threads pool) that handles if it wants to read the data? at 15:25.

[@nehanigam4997]

Best Design tutorials, really helpful, Thankyou for making!

[@SystemDesignInterview]

Thank you for providing a feedback on a regular basis!

[@gyhuj1235]

The videos on this channel are one of the best system design videos. I wonder why you stopped doing them? Would there be more videos coming any time soon?

[@reyazahmed9320]

This is a gem. Thank you so much!

[@vichetlay1646]

I just found your channel in 2022. You have made a lot of good system design videos. It’s sad that no more new video I hope that you can make more quality videos.

[@quantumlexa]

Thanks a lot again for such a great channel. That's awesome. I'm in the middle of system design interview preparation and your channel helps me a lot. few comments/question 1. as far as I understood, notification event that is handled by sender service gets endpoint information from metadata service. Let's consider a broadcast event (for example: sms storm warning, email from a system to all users about a new policy, etc) that has to be delivered to a number of endpoints > 100 000 or even more. If it is a 1 single event that is processed by 1 single node even in multi threaded environment, it would take way too long. I'd probably generate those events that contain both - event and endpoint beforehand and speedup actual delivery. In even could not event itself, but just pair of two id's. This idea is based on assumption, that delivery itself is a more time-consuming operation than enrichment of an event with an endpoint information. 2. Regarding a temporary storage, you asked a very good question. The best that comes into my mind is probably a hybrid approach, i.e. having some key-value no SQL DB for failed events (that I'd redeliver later on) and Redis cache for everything else.

[@SystemDesignInterview]

Hi quantumlexa. Apologies for the delayed response. And thank you for the feedback! Your idea of decoupling event generation and the actual delivery is a good one. Please take a look at this thread, where we discussed resembling ideas: kzbin.info/www/bejne/mHO3gY1vg8mGoZo&lc=Ugzg_JJd9yUMUX9ySwt4AaABAg P.S. Wish you all the luck on your interviews!

[@AbhinavSingh-rh5dw]

A great resource for in-depth understanding of System Design.

[@dkseo1992]

Thanks for the detailed explanation.

[@SystemDesignInterview]

Thank you for the feedback!

[@shobhitarya1637]

Best System Design channel. Very helpful. Please keep posting learning videos.

[@SystemDesignInterview]

Glad you liked the channel, Shobhit! Appreciate the feedback!

[@saranyaks6436]

Great video! Keep up the good work! :) Pros for using SQS: data size is small, no strict requirement around ordering for this problem, reliability. On the other hand, that raises several questions - when do we exactly erase the message from the queue. Only when all the tasks have picked up the message? What happens if one of the tasks fail.. etc. It might also be useful to discuss about WebSockets for notifying the clients. I'm assuming that's what the tasks would ultimately do?

[@SystemDesignInterview]

Thank you for the feedback, Saranya! Agree with you. I should have extended the video to cover details of pushing messages to end clients (subscribers). Specifically, talk about HTTP polling (long and short), websockets, server-sent events. Let me leave this topic for a separate discussion.

[@sharathchandrareddy8959]

Great step by step approach , It would be great if you can provide a link for the final design that viewers can keep it handy as a big picture while learning specifics/details of each sub-component . This recommendation/request applies to all of your system design videos

[@SystemDesignInterview]

Hi Sharath. Thank you for the feedback! Can you please clarify how you see it. Do you mean combine all components with their details on a single slide? Or you mean a text blog post version of the video? Or something else?

[@jamess5330]

Excellent video! Another super effective way to prepare system design interviews: Do mock interviews with FAANG engineers at Meetapro.

[@hemantsethi8467]

I was speechless after watching this video. There were so many details and though process being discussed in it, that I ended up taking more than an hour(while taking notes and capturing thought process). A great video covering breadth and depth. I know it would be extremely hard to make up this high-quality content, but I would love to see more of those.

[@SystemDesignInterview]

Thank you, Hemant, for the feedback. Really appreciate your kind words!

[@mohhit100]

these are the best videos I have seen and I could not have learned better without them. Why don't you publish them anymore? If you shifted your publishing channel, please post in the reply.

[@kanaiyapatel5691]

Probably your design are best and explanations are easy to understand. I encourage you to do more system design videos.

[@SystemDesignInterview]

Thank you for the feedback, Kanaiya! Appreciate it.

[@sergeyprytkov6850]

I'm glad that FPS Russia has found a new passion in systems design.

[@silviojr2424]

Your videos are so good! Thank you!

[@DheerajKumarBarnwal]

This is mind blowing. I didn't expected this kind of system design tutorial on KZbin. Your style to explain HLD and then deep dive into each component, Its awesome and hidden gem. Hats off to you. I know it takes lots of time to create such content but please try to upload more videos.

[@SystemDesignInterview]

Thank you for the feedback, Dheeraj! Glad you liked the video!

[@xiangjieli7391]

Wow, this video and the channel is soooo helpful to me! Thank you so much!

[@SystemDesignInterview]

Glad it was helpful! Thank you for the feedback, Xiangjie!

[@nimash1612]

Great Job! very clear and detailed explanation. Please make more videos on other system design problems and topics.

[@AndhraKitchenFoods]

Awesome system design tutorial.

[@rontman]

Still waiting on your monitoring video follow-up! Really good content.

[@diptikaushik8250]

Brilliant!

[@deathbombs]

I don't think people realize how much of the functional requirements tie in with knowledge of TOPIC BASED Pub-Sub. Quite in depth for something that's not explained. I find part at 2:34 under explained.

[@callmehims]

Simply the Best (Y)

[@HarshaVardhan-jf9sd]

very apt...keepup the good work..content always speaks

[@SystemDesignInterview]

Thank you for the feedback, Harsha.

[@anmoldua6574]

Absolutely Amazing ....I haven't found any other video from any other you tuber with such details. Thanks for the content. Are there any videos of your content which are not available in you tube,might be in udemy .I want to checkout out that as well. Kindly tell. Thanks

[@lihaopeng768]

I watched these six videos several times. They are really good. Will you consider a video about monitoring system?

[@SystemDesignInterview]

Thank you for the feedback, Li Haopeng! I have this topic in my short list. But cannot promise specific dates.

[@gct_101]

Отличные видео, большое спасибо

[@pranavsakulkar]

This is simply amazing. Thanks for making these videos. I think you should a write a book or create some paid tutorials. I would definitely pay for those. I am learning a lot through your videos already.

[@pushpendrasingh1819]

he has already a paid course bro.. buy it

[@pranavsakulkar]

@@pushpendrasingh1819 Thanks for letting me know. Seems to be a new creation. Definitely checking it out.

[@abhishekanand5362]

Nicely explained, thank you for this great video.

[@SystemDesignInterview]

Thank you, Abhishek, for the feedback! Much appreciated!

[@amans.4701]

Excellent explanation. Why did you stop uploading videos? Waiting for more such topics.

[@Andrey-ny2dv]

Большое спасибо! I was wondering about a few things. Does the metadata service(in memory datastore have to be strongly consistent)? Or are we using the idea that anything related to topic name/topic owner identifies the host which gets all the writes about that topic name/topic owner FIRST? That way we are going to have all the latest information, and we can simply use that information to send the message to the correct person. Do we need to asynchronously replicate it across the nodes then? If we are doing it asynchronously, don't we have a chance to lose the data as we acknowledge the write before replicating? Are these things worth thinking about or how should I approach this? Another thing I wanted to confirm is that the database has only information about topics and subscribers, not the messages. And last thing is the temporary storage. I am trying to understand a few things here: 1) is the retention policy something you setup within that storage or is it another service that cleans up? 2) so you store a message let's say along with all the subscribers at the moment the message was received. Is it a good idea to have something like status of delivery for specific recipient? So that if you succeeded from the first try, you mark it as completed and don't retry next time as it was already sent.

[@venkatthota8634]

Great Content and and awesome explanation. keep doing more videos bro

[@rowin1]

There is so much detailed and thoughtful information packed into every second of this video. Thank you so much 🙏

[@SystemDesignInterview]

Thank you, Christopher, for the feedback!

[@rk2119]

You are too nice it's fine I will listen to sub titles Thanks for the effort

[@SystemDesignInterview]

Glad you found a way out. Thanks for the feedback!

[@nitingupta1591]

@System Design Interview - We can use Redis for temporary storage as it is in memory, provides persistence and has ease of use ( get and put). Please let me know if there is a better option

[@usamakhalid1990]

Great Tutorial, thank you so much. I have a question in the case of publishing a message, does FE service connects to Metadata service, if yes then for what purpose?

[@WilberChao]

super knowledged video

[@mdfarooq7145]

Awesome tutorial

[@SystemDesignInterview]

Thanks!

[@fahadmudassar426]

It would have been super awesome if you had also included schema of what we are storing in database.

[@amirhasan5587]

Can't appreciate more 🙌 Kudos! Pls make videos for 1.Design KZbin/Netflix , 2. Design Twitter/Facebook, 3. Design Yelp etc..

[@brianjia1279]

good job. very clear explanation.

[@zhuoqianzhang4399]

I think there are few major areas I would like to see more details if I am interviewer: 1. What's stored in the tmp storage? what's the schema look like? Is it denormalized msg? say for subscriber a, b, c there are msg 1, 2, 3 not delivered? 2. How to handle large fan-out msg? Say Twitter used this notification system and Trump tweet... XD Is it a single node sender service to pick the tweet and try to notify all followers/subscribers? I guess that will take forever. I really like expanding #2 above, because large fan-out is a really hard problem to solve in real life.

[@syedhusain5465]

One more thing i want to ask. The fan out pattern you discussed here. Is it applicable for fanning out tweets to home timeline of followers as described in most of twitter talks. Like if someone tweets, then that tweet goes to temporary storage and then fanout to different follower home timeline(Stored in some cache) or do you think they might be using some better method to fanout tweets

[@pranjalabhishek7566]

Awesome way of explanation ...could you please start covering all system design problems asked in big4 and if u provide LLD also it would be awesome...

[@SystemDesignInterview]

Hi Pranjal. Sure, there are plans to cover these topics. Just need to find more free time (( Unfortunately, every video takes a lot of time to produce.

[@xinwang6876]

This is really great. The best system design video I found online. I am trying to find some information about : How to design a Google Calendar like system. Would you mind publishing a video on that topic ?

[@SystemDesignInterview]

Thank you for the feedback, Xin! Added your topic to the TODO list. Please do not expect a quick answer though, the list is already quite long ((

[@jeremyshi4082]

The approach used in this video seems a synchronous push for the notification. Does it scale well if the publish and subscribe qps are at hundred thousands or even million level?

[@arunvijay2279]

Good explanation

[@hanspeterpfister2253]

Hi, this is one of the best videos I have seen. Can you make similar videos for WhatsApp & Twitter as well? I have gone through all your videos and they are all amazing. But I see you haven't been able to upload any video since past some time. A sincere request (and I think I can say this for everyone here), please work on more such videos. You are doing an amazing work.

[@SystemDesignInterview]

Hi Hanspeter Pfister. Thank you for the feedback! I have both topics you mentioned in my TODO list. I am pretty busy at work these days. But I keep thinking about all my dear viewers and will come back with more videos.

[@s298uren]

@@SystemDesignInterview Hey There - Hope you are staying safe - did you get a chance to action the TODO list :) ?

[@SystemDesignInterview]

Hi Suren. Thank you for pushing me :)) Appreciate it :)) I am not yet working on topics mentioned in this thread. But I do work on more content. Let me not give you false promises and specify any dates. I do plan to create and publish videos more regularly. Waiting for these days to come...

[@user-hl8rx2qc5p]

Брат ты лучший!!!

[@atabhatti6010]

Great video so thank you! However, I'm a little confused. You list the Frontend service responsibilities (~5:24) including SSL termination, authorization, authentication, request dispatching, ... and then (~7:40) describe it as processing the message. Is that right? Initially I initially understood your definition of Frontend Service as an API Gateway. Later I understood it as a Message Processing Service. Are these one and the same thing?

[@ChintanShah22]

Thank you so much for the great content, big fan of your channel. I have a question about the order of load balancer and reverse proxy in the design. You have the load balancer in front followed by the reverse proxy which is part of the front end. Since reverse proxy is doing rate limiting, ssl termination,etc shouldn't it come before the load balancer ? Also since there is usually just one reverse proxy for multiple web servers, wouldn't a load balancer feeding just one reverse proxy be redundant ?

[@bluelamar809]

So example of this, is haproxy as load balancer(which it is really good at), passing message to nginx as reverse proxy. Nginx is good at layer 7 routing and can do auth checks among other things before passing the messages upstream to 1 or more origin servers(for example based on the url).

[@SystemDesignInterview]

Hi Chintan. Thank you very much for the feedback! And my apologies for the delayed response. You are right, there are many different variants how reverse proxy and load balancer can cooperate together. Plus, they both have a couple of overlapping functions, like TLS termination and load distribution, and this can make things even more confusing. In the video, I mostly wanted to demonstrate one particular aspect - TLS termination on the host. There are several ways how TLS termination can be done. It can be done on the load balancer level or a reverse proxy level, when both these components are standalone components sitting in front of the service. One of the problems with this approach, traffic goes unencrypted from the load balancer or reverse proxy to the actual service machine. One of the ways to solve this, is to have a proxy running on the machine together with the service. Traffic comes encrypted all the way to the service machine. And is decrypted on the machine. This requires some good diagrams to demonstrate a data flow in a simple manner. I should probably create a video to cover these details. Thank you for the question!

[@true_human_007]

WONDERFULL THX.

[@gxbambu]

Hi just have a question about Messages Retriever: it seems in video there is only one retriever, is this intentional ? what if we have a distributed retriever, and how to manage the overall retriever throttling? I think a tempStorePollingClient lib can be used in message retriever , and all the pollingClients can access some distributed cache (redis) to obtain something similar to semaphore? thanks for your help.

[@zxyviopond]

Very clear and concise presentation for a complex system. For the winner of Temporary storage I choose streaming solution like kafka, wanna know your lucky winners as well. Also I wonder whether the MessageRetriever's implementation actually varies based on the choice on temporary storage.

[@SystemDesignInterview]

Agree with you, Xiaoyun. Here is my take on this: kzbin.info/www/bejne/mHO3gY1vg8mGoZo&lc=UgwmaW3Ek0-XnkXb8KB4AaABAg.8tJRBPf4mun8tKuzz6sHVs As for MessageRetriever's implementation, you are right, it depends on temporary storage we use and some other factors. E.g. whether order of messages is important. If it does, we better stick to a single-threaded retriever. If not, we can use multi-threaded message retriever.

[@arunbit]

@@SystemDesignInterview I was curious too - but, that link is taking back to this same video

[@syedhusain5465]

I must say, this is amazing content(probably the best on internet) and you really are doing great service to job aspirants and distributed system enthusiast. I have one question, what would be the database schema for meta data database. I guess we are only storing topic and subscriber information in meta data database and in worst case(Cache miss) if we are calling database, we need to get all subscriber list for a topic really fast for sender service. Also in worst case, we need to do request validation(e.g. topic is present ) fast. What do you think should be the database schema for meta data service and also in what format we are going to store data in caching layer.

[@vidushidwivedi4956]

Any insights to this question?

[@rahulsharma5030]

This is awesome. I have some doubts at 19:29, there seems to be redundant components, When message retriever thread has got the message, then why to create tasks , it can directly send to the http,email etc microservices? What are we achieving by putting them in task creater and then running threads again in task executor which eventually calls other micro service. Seems over complicated.. One message retrieval thread will take message, and then it just send to the http or required endpoint.

[@24572884]

Thanks for such detailed informative videos, these vidoes are helpful. Could you please make a video on SQL vs NoSQL? Also please provide tradeoffs and few examples in real applications.

[@SystemDesignInterview]

Hi Chinmay. I plan a series of videos related to databases. Although, it will take some time to create. Stay tuned.

[@24572884]

@@SystemDesignInterview Also can you please make a video on types of connections (https, tcp, socket etc) that can be used to connect client and servers for system design? It would be helpful if you could explain with different use cases/scenarios. (eg tcp is duplex etc)

[@xlin7868]

Thank you for sharing! This is the most helpful material I have seen online... A question: how to scale senders? I understand you mention pool of threads, but that is for a single machine. Sorry I missed your instruction if you mentioned it for scaling out. A simple solution: use consistent hashing to assign the task to a ring of instants/machines, just like distributed cahce. Will this be a good solution? Thanks!

[@SystemDesignInterview]

Hi Hullo, Thank you for the question. Sender service is scaled both vertically (by having more threads in the pool of a single machine) and horizontally (by adding more Sender instances/machines). Consistent hashing idea you mentioned will work as well. But we do not actually need consistent hashing. A simple random hashing will work. Consistent hashing is usually used to make sure the same machine is chosen for the same key (message). In our case we can just pick a random Sender machine for processing the message. Random hashing is simpler and less prone to "hot" Sender issue. Feel free to ask any follow up questions. Will be glad to clarify it further.

[@tejvepa8521]

System Design Interview Can these multiple sender instances be picking tasks out of a distributed mq like sqs?

[@SystemDesignInterview]

Hi Tej. Why not? AWS SQS is one of the options for the Temporary Storage. We can use other message brokers as well.

[@theboo5857]

@@SystemDesignInterview Firstly, thanks a lot for the videos. Its less than an hour, but when I go over it, it has so much material packed into it. Amazing. I have one question about horizontal scaling senders. All the threads from all the senders will now fetch msgs for sending from a certain tmp storage. How could we sync between the senders? With in one sender, we can use locks, and indicate which msg has been taken for delivery. Then other threads would not process this msg. But I am not sure how we can prevent 2 threads on different senders from processing same msg simultaneously.

[@ashwinravichandran1701]

Great video! I have a question regarding message retriever component. How do we ensure that same messages are not being read from temporary storage by multiple threads/hosts?

[@SystemDesignInterview]

Hi Ashwin. Great question. It depends on what Storage we use. Let's take a look at different options. Message queue. There may be several different flavors. For example, in AWS SQS, when message is retrieved, it is marked as invisible for some period of time. This prevents other consumers from processing the message. More on this here: docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html In case of Kafka or AWS Kinesis there is this concept of a monotonically increasing sequence number. Consumers keep track of what message (number) they have processed. In case of a database, we need to implement our own logic. For example delete this message from the database when message is retrieved and store it back in case message delivery failed. Please, also remember that SQS/Kafka/Kinesis supports at-least-once semantics. Which means that the same message may be delivered more than one time to the consumer.

[@kkthespidy]

@@SystemDesignInterview Instead of deleting message from database, we can consider making it inactive using some flag, because if the host processing the messages is down, all those messages might be permanently lost.

[@wfan2844]

@@SystemDesignInterview With Kafka/Kinesis/Azure EventHub, isn't Kafka only allow one consumer per partition? If you have multiple threads, kafka will block other threads from concurrently access the same partition. So, in reality, the threads are reading the kafka message sequentially. We could of course have different threads read from different partition of the same topic, but still there is no chance of those threads re-read each others message, unless there is a some thread/consumer crashed before commiting the processed sequenceId. Then another consumer thread might pick it up and re-read the message. Re-read should be fine, since if we want at least once guarantee

[@OleksandrAndrushchenko]

Thank you