System Design Mock Interview: Design Facebook Messenger

[@tryexponent]

Don't leave your career to chance. Sign up for Exponent's system design interview course today: bit.ly/48OhSV7

[@emanik2]

What tool is it?

[@sbylk99]

Although I point out several places that I think not proper, this is the most helpful system design interview walking-through on whole KZbin. Why? Instead of laying out these components names and introduce their usage directly, you start from comparing connection protocols and pick the right one(web-socket), then build up the very basic backbone in very natural way. Then find bottle neck of single API server, introduce load balancer. Then pub/sub message queue. Then you analysis the high volume request and pick up NoSql. Then introduce cache, cdn and S3 to solve low latency bottleneck! THIS IS very helpful, amazing sharp. THANK YOU SO MUCH.

[@NambiarRam]

Looking at the database schema, it suits sql database but Cassandra being a no sql, I wanted to find out how the fields are defined. Also how does the message get distributed to the user from message queue is not very clear to me

[@1petriq]

Exaclty I was gonna to write the same. In no sql you wanna avoid the joins.

[@stealthrabbi9064]

1. How does a messaging Service (e.g. kafka) write to a DB directly? Wouldn't you have individual services writing to a DB? 2. If we're using a NoSQL DB, why are there ER diagrams in the Data Types section? 3. Does a conversation contain messages? Doesn't there need to be some linkage between the two entities?

[@lijlijlil1898]

These are exactly what just came to my mind

[@AniketSomwanshi-ll7mz]

1. Yes 2. - 3. each message has a coversation_id so they are linked

[@Shrtr123]

Also my question is how we are going to create topics for such huge load. Million of user are sending messages per second will single topic will be able to handle it, also if each service consumes millions of messages and filter out for one user . I don’t think it’s scalable approach

[@NickCooperWasHere]

I think this is a great example interview and the end design is spot on, one thing i'd simply say to anyone watching this is also ensure to check up on terms and technologies as they always change (this video is 2 years before this comment). In particular modern web connections would probably motivate long-polling as the ideal solution. This is because HTTP2 is universally supported now, and it supports streaming and multiple server responses for a single request - so you can treat it like a one-directional WebSocket - however with faster connection handshakes and better compression support. Although from a client API standpoint you would issue multiple 'requests' for sending message it would be on the singular TCP underlying connection.

[@tryexponent]

Hey NickCooperWasHere! Appreciate the kind words. Thanks for letting others know the importance of keeping their knowledge up-to-date before going for an interview!

[@juliami9239]

I believe a media attachment can be big enough, so we don't want to send it through our API server. Instead, client can request a pre-signed link to object storage from the server to upload the file using that link. These links are valid for some short amount of time for security.

[@ManishKumar-mo6gx]

Nice addition, S3 does support a feature called pre-signed URL.

[@老谈]

That's a great point!

[@ThyRiki]

A socket is defined by (source IP, source port, destination IP, destination port). Different sockets CANNOT have the same exact bindings, but you can create the socket as long as the port or client IP is different. So, you can handle 65536 (the number of technically possible port ids) concurrent connections PER client. Of course, if you use other ports in your server, then you have many more possible connections. Let me know if I'm missing something! Always up to learn something new :)

[@lijlijlil1898]

Yeah, that's exactly what I wanted to say right after I heard this guy said the server can have at most about 65000 connections.

[@jvm-tv]

This was a good demo of managing details i.e. staying at a consistent high level. It's easy to dive too deep into in one area while neglecting some major components of the system. You can then leave it open for the interviewer to choose one area to get deeper on.

[@vetiarvind]

Nice, i think this has the right amount of complexity without going into too much details like some others. Seems like a realistic amount of info to spit out in 30-45 minutes of an interview.

[@gregoryskoczek5357]

The statement at 5:20 is incorrect. A server can have up to 65k connections *for every client*. This comes from the fact that we are filling in the ports into a standard connection 4-tuple: (source ip, source port, destination ip, destination port). Different clients have different IPs, which allows the server to have another 16bits/65k connection per client. Otherwise a great video, thanks!

[@cozos]

As somebody else in the comments said, I don't think its correct to say that the server can only handle 65536 concurrent connections. The server websocket port (80) should be able to share multiple TCP/websocket connections. The limitations would probably come from your application code or hardware (i.e. CPU/memory/network bandwidth)

[@firatkucuk]

I agree server sockets are different than clent sockets.

[@kkud01]

yes, WhatsApp had an article about 2M tcp connections on a single machine

[@dustinkftw]

I mean even if that were the case, wouldn't the load balancer have that limitation too?

[@kkud01]

@@dustinkftw Put a L3 load-balancer that distributes IP packets based on source-IP-port hash to your WebSocket server farm. Since the L3 balancer maintains no state (using hashed source-IP-port) it will scale to wire speed on low-end hardware (say 10GbE). Since the distribution is deterministic (using hashed source-IP-port), it will work with TCP (and hence WebSocket).

[@slyncemployee9332]

Yeah, the 65536 is only a limit on the number of applications that can be listening for the initial connection on the API server. We technically only need 1 of those ports to be listening to new WS connections.

[@jiangmouren]

The 65536 Port limitations is actually for the client machines not for the servers. In fact, the server uses the same port for all sockts.

[@giridhar510]

On Server side, server creates socket with ephemeral port. 65K limit might be for ephemeral ports ?

[@songjo3506]

@@giridhar510 the limit is per client's IP. on server side, it look like this: ipA:80 ipB:80

[@Kriishna47]

​@@songjo3506 Websocket uses TCP protocol and so is HTTP. Server sees them same way. So when an request for new connection comes in, it is always ported to an ephemeral port.

[@songjo3506]

​@@Kriishna47 the ephemeral ports are assigned to each peer IP. Scenario where If users are 1K users, It means the connections the server can handle is 1K IP x # of ephemeral ports. Access Linux console and trigger ss -tlap to see how the connection is established.

[@jvm-tv]

So what would be the limit for number of web sockets on each server?

[@lahirua]

Great video. One quick thing to point out. Web Socket server mux's the connection and would listen on a single Port so not subject to the 65kish limitation. Its a limitation in the client-side typically and mostly becomes a burden if you are running a load test to a TCP server.

[@tryexponent]

Good point!

[@meow-mi333]

how does websocket looks up connection based on user-id? I wonder if there is a need to bookkeeping connection-user information at application level.

[@namanmishra08]

​@@meow-mi333 Yes, that needs to be done at the application level

[@aceshare]

On one side you talked about using Cassandra or HBase and on the other your ERD actually drove towards a relational design. I also think that the design related to Notification service was incomplete, because it assumes a connection possibility with the client. A key part of a system design may be a back of the envelope estimation, which I found missing in your presentation.

[@kaushaldawra3527]

Over requirement of Size/growth of the data was missing. 1) Notifications works in a different way, there need not be any persistent connection at all. You generally push notifications through 3rd parties in case of SMS. Or you can in house have an integration FCM like setup to do the same. 2) IMO the DB schema wasn't really ERD. It was simply actor ids and their corresponding values

[@meow-mi333]

it's ok to have different schemas for NoSQL as well. You can put them in the same table to avoid joins. There is a pattern called adjacency list for many-to-many or one-to-many relationships. I agree that the bridge table for conversation-id and user-id is not needed.

[@chaoyao7623]

We can't say that the limitation of connection for a server is 65535. limitation for a server doesn't depend on the number of ports. Instead, it depends on the maximum open files of the server process, memory, and connection tracking (conntrack) limitation, etc. Of course, here I'm talking about a Linux server.

[@tarankaranth8782]

this is good. one question why is database directly connected to message bus? shouldnt there be an application server which reads from the message bus and stores in the database?

[@yili9725]

Thanks for the video. Can you elaborate on messaging service that is support to deliver a message from user A connected to API server 1 to user B connected to API server 2 ? You said messaging service is build for that purpose. But when there are millions users connected to hundreds even thousands API server. What's the best messaging component? what would be the topic that API server 1 post so that API server 2 picks but not others?

[@DavenH]

I don't really get what there is to cache. Chat messages are all unique right, not public and getting heavy reads.

[@Qichar]

Memory access is exponentially faster than going to the database for a read. If I were designing this messaging service, I could cache conversations that were going on *right now* between two online users, and allow cached messages to automatically expire with time. Think about it. That messaging service in his architecture diagram is servicing *millions* of users, potentially at the same time. You do *not* want to bottleneck on the database, but at the same time you need a way to persist messages for when they need to be delivered later to offline users, so you can't do without it. Or maybe users want to be able to log into a different device and see the history of their conversations, so they would need to be downloaded from the server. His design offers a way to provide both without sacrificing too much with regards to latency.

[@alige2143]

With HTTP/2 you have PUSH. Then, even though the connection needs to be established beforehand, the server can push information which is indeed much more scalable than pulling

[@YiWang_YW]

Agree. Surprised about this too.

[@k.i.m.5506]

12:00 I believe you want to put cache between message service and database.

[@tharindueranga6407]

one question, you have added a load balancer between the APIs and the clients. so are the clients keep the WebSocket connection with the load balancer? if it is, then again the 65k limitation can occur?

[@KoboldAdvocate]

I would add a timestamp for each message. You'll need that to determine ordering when displaying messages.

[@irakligvazava9944]

this was most perfect PM mockup ever, other vids are like 40 mins long. can you please make more, pleaaaaasseeeeeee

[@smita30apr]

Hi! I had one confusion. You said we will be needing NoSQL database but then you are doing data model exactly like a SQL database and these tables would have to have a lot of joins to get data from. Could you please explain that?

[@priyenpatel3954]

Did you figure out answer to this? am curious also

[@elachichai]

This one and the instagram design is way better than the Facebook feed. But could have spanned to about 30 min of info in terms of coverage. When you mention tables, are you meaning SQL database having earlier mentioned NoSQL?

[@cristianopassos5257]

Using a CDN for a chat application might not be worth it, as most Media is consumed just by one user or users from a Group.

[@dhananjaysingh3925]

Might need it if some content got viral and everyone one is sharing and viewing it.

[@Criiz22]

Thanks so much for your video, it was really concise and explanatory. I'm studying for an Architecture and System Design interview and your video was the first I watched because if was the shortest one.

[@ankitagarwal4914]

Hi Jacob - This is very interesting video , I am trying to draw along with you and would like to ask you which platform are you using for flowcharts. You drew it very smoothly

[@tryexponent]

Hey @Ankit! I'm using a great tool called Whimsical (whimsical.com)

[@AdnaanAhmedZohran]

@@tryexponent scrolled down for this comment. 😅

[@jadeedstoresupport8916]

5:24 The TCP protocol does have 16 bits for the port number. The total number of Ports available to an IP address are 65,536 (from 0 to 65,535). The WebSocket Server uses one of the 65,535 Ports (in case of secure Sockets that would be Port # 443). The rest of the ports are managed by the OS for various other purposes (Well-known ports: 0 to 1023, Registered ports: 1024 to 49151 and Dynamic ports: 49152 to 65535.) The maximum number of connections a WebSocket server could have with clients would depend on (i) the specs of the server hardware, (ii) capabilities of WebSocket server (e.g. employing techniques like Connection Pooling) and the capabilities of employed WebSocket library and (iii) OS environment settings. Many credible sources on the Internet claim to easily achieve 1 Million or more simultaneous connections on WebSocket server. (Of course the actual number of simultaneous connections would depend on the use case and there would be all kinds of nuances).

[@hjjscofield]

a socket connection is (client ip:port, socket ip:port), client can be any different ip:port combination, so a server with one port can serve as many clients as needed

[@vivekjadon1198]

this is much better than the other mock system design interviews videos

[@frankharvey88]

These videos are so helpful in learning to work through these system design questions. THANK YOU.

[@rogerhom1512]

Great video! Question about the data model - the tables are defined with schemas that fit a relational model, e.g. the "user" columns in "messages" and "conversation_users" tables are foreign keys pointing to "users" table's records. However, if we use Cassandra as you mentioned, we can't do join-queries directly in the database. (I'm not familiar with HBase, so can't say about that.) Is your intention perhaps to use a hybrid approach, e.g. a relational db for users, conversations, and conversation_users, and nosql for message?

[@bengillman1123]

+1 If this was a real interview, the interviewer would dig into this and the candidate would have to explain how Cassandra works. Candidates can't just throw out the names of databases and not know how they work. Exponent should go into more depth there

[@BlackIceSpa]

​@@bengillman1123 I Don't think you need to do any kind of join here, you can just filter each conversation when fetching it (aka when the user clicks it), you shouldn't need to fetch all the conversations at once if the user is not reading them. Maybe you query like 5-6 conversations if the system gets extended to remember your last conversations. And anyways, even if you needed to query all the conversations of a user, it shouldn't be so many and you can do that by just filtering (kind of in predicate).

[@ameynaik2743]

This system design talking points will give a lot of hard hitting questions from the interviewer!

[@gtsmeg3474]

Great video I have a small question, what's the tool being used here to display the architecture design ?

[@rafaelJEC]

Whimsical

[@iliya24]

hi cool tutorial thx, small question how u loadbalnceing WebSocket's?

[@abcwang569]

should we use api server to interact with DB, not MQ?

[@theyzilay]

Limitations of 65k is per client. On the TCP level the tuple (source ip, source port, destination ip, destination port) must be unique for each simultaneous connection. That means a single client cannot open more than 65535 simultaneous connections to a single server. But a server can (theoretically) serve 65535 simultaneous connections per client

[@javidjamae]

I think you mean per IP, not per client.

[@sbylk99]

@8:14, You said "We know we gonna support large volume of request and save lots of data". That's the reason you pick nosql over sql. And cap theorem's universal tradeoffs only apply for nosql db. And in chat system, we prefer availability and partition tolerance.

[@niceperson6412]

Just curious why sql can't afford large volume of data. And besides, I believe consistency should be the priority, you just don't want to miss any messages.

[@nickplays2022]

@@niceperson6412 I don't think that consistency is about not loosing messages, it's about handling multi-step transactions. Which I can't imagine in a chat service.

[@s.matushonok]

CAP applies to any distributed database. Also, the author misunderstands the 'P' in CAP; it is not about sharding, but about network partitions (network failures). Since network failures are inevitable, the choice is only between consistency and availability

[@techy0716]

How about Scaling the solution. is cloud providers scaling solution like Autoscaling group(AWS) or scale set(Azure)after load balancer will be appropriate?

[@sachinpatl]

Which application are you using for desing on the go??

[@fahrankamili7931]

Thanks!

[@arunraj2527]

Does Cassandra support foreign keys? The DB schema looks more like a Relational one.

[@abubalo]

What is the name of tool he used to designed the diagram?

[@SeanNeilanCodes]

What about created at on the messages table? How will we know when messages were sent?

[@gautam-narula]

Quick question: in the video it talks about using a nosql database, but in creating the data model for messages, conversations, and users, the models are relational. How exactly will our messages be stored?

[@tryexponent]

Hi Gautam! We can model relational data in a NoSQL database. In a NoSQL database, instead of the related data being split into tables, they are nested within a single structure.

[@gautam-narula]

@@tryexponent thank you!

[@bbs32]

what's the whiteboarding tool you're using in the video? looks pretty nice and smooth. thx

[@roycrxtw]

Whimsical

[@bbs32]

@@roycrxtw thank you!

[@jugalparulekar661]

This is the absolute best video explanation that I have ever seen for the system. I follow Exponent channel on youtube but being a student and financing myself, I am not able to get the subscription. However, I would suggest, along with mock interviews that Exponent have been uploading on YT, these pattern following videos on System Design are better suited for a quick but detailed explanation of the system. Please upload more of such videos and if there is any discount for students please help me.

[@paneerlovr]

If we use a CDN and it sits side by side with the load balancer, would we say that objects stored in the CDN must be public objects not secured? For example, giphy type images or something ?

[@aadimanchekar4423]

what's the purpose of using the cache? like ik it's fast and it is used to reduce the load on Db but what's the point of using it?

[@BlackIceSpa]

I would argue it makes more sense when using groups, but for peer-to-peer communication I don't find it useful aswell (at least not as part of the initial design, it can be added afterwards if performance is lacking)

[@aadimanchekar4423]

@@BlackIceSpa Thanks! Can you tell me the list of topics for learning system design?

[@BlackIceSpa]

​@@aadimanchekar4423 Just look for courses and follow their index, I've passed FAANG interviews before (also failed), but I got most of my technical knowledge from working in [good] consultancy companies, I come to this videos just to learn methodology and such, I don't prepare the interviews a lot unless they are for Spain which is my preferred location, just like a few hours during a few days.

[@aadimanchekar4423]

@@BlackIceSpa Thank you for sharing your experience :)

[@gabrielxu2087]

still confuse what's the message service in the middle

[@dbenbasa]

For the messages table - shouldn't we consider adding a timestamp for when the message was sent?

[@markkentmonnin]

I learned a lot just listening to this. Thank you!

[@ryanl7487]

How would the Message Service map onto AWS ?

[@adithyaks8584]

Messaging service and how the server process these messages is unclear. Also when we write to db, what we write to db and how we use cache is not explained. Also we might have to maintain a sticky session with load balancer otherwise it's not gonna reach the same server instance.

[@George-nx8zu]

Is there a mobile app version of this system design mock interview?

[@minagadallah6142]

How are you using the messaging queue to serve data to the api service

[@muneebmohammedsaleem997]

Hey Jacob. Interesting video. I was wondering why can’t we use concept of SNS topic where user B will subscribe to a topic and as soon as user A sends message he will receives it. I’m new to backend programming so was curious ?

[@DiwasTimilsina]

what's this whiteboard app they are using?

[@EricStratton79]

Does Facebook Messenger still use WebRTC (in lieu of websockets)?

[@sitianlu610]

Could someone help me understand why we need the message queue service and how that will work in detail? It is going to be something like SQS and SNS setup?

[@aniyt10]

You mentioned using a system that is both available and partition tolerant but ended up choosing hbase which is a CP. Wondering the reason behind it.

[@yanciliang7735]

Thanks for sharing this video! It provide me with a good example of how to design a system. In fact, I'm trying to break down an open source project using this method, as I want to gain a better understanding of it. After that, I hope to submit a pull request.

[@firmanjamal2871]

whats the difference between the message service and the api?

[@Dan-tg3gs]

Why didn’t you talk about consistency for this problem. I felt it was quite important

[@rajbhandari9605]

what diagramming tool are you using in these videos?

[@kevinsu2219]

What editor your use to design systems?

[@chrisy.703]

without any component design.... what's the point to sign up ur course? drawing blocks is too easy for a system interview...

[@balqaasem]

Great video, what software are you using for drawing the charts please?

[@tryexponent]

Hey alfellati, the whiteboard being used here is called "Whimsical"!

[@ralphez]

how do you invalidate or update the cache?

[@kp8607]

May I ask what tool you use for white boarding?

[@tryexponent]

Whimsical!

[@dr.debbiewilliams]

I don't have Facebook Messenger. I only had it for a few weeks in the beginning of 2016. I didn't use it prior to that time, now have I used it since then.

[@focusedbytes]

What applications do they use to create these designs in?

[@d123-y5i]

Do we need a timestamp for "messages" table?

[@zubin221]

Which whiteboarding app are you using?

[@MaxCoplan]

What tool are you using to draw this?

[@anubhavgautam5104]

Whimsical

[@thanga511]

Do we really need a cache for the chat application? What would be the purpose of it

[@tryexponent]

Hey Thanga! Caching messages on the client side reduces data transfer required between the client and the server, resulting in a snappier load time!

[@thanga511]

@@tryexponent makes sense. thank you

[@Quassar18]

Which whiteboard are you using?

[@tryexponent]

Hi DroneBusters! The whiteboard being used here is “Whimsical”. They have a free and paid version so do check them out if you are interested!

[@jimwichen7978]

hi, great video. what the software you use to draw diagrams

[@tryexponent]

Hi Jimwi! The software being used here is “Whimsical”. They have a free and paid version so do check them out if you are interested!

[@jimwichen7978]

@@tryexponent thank you

[@yog2915]

Bro whats the app you using for this designing ?

[@linc008]

Would be better if how&why the read-through cache is used wax explained. Messages sent between users should not have hotspot problem and I don't see why a cache is needed.

[@teenhon]

it's not a cache it's just precomputed feed data

[@asakhala]

What’s the need for cache?

[@vermaankit2005]

What is the software used for creating the system design diagram? @Anyone

[@AyrtonRicardo]

From comments above and other videos: Whimsical!

[@spsolanki]

What software used to make block diagram ?

[@maximkrouk3756]

What is the name of the design tool? 🙂

[@somcho]

i believe your Hbase suggestion sacrifices Availability instead of the Consistency you are aiming to sacrifice under CAP

[@tryexponent]

Hey @Chisomo! Good correction. Both HBase and Cassandra have been used by Facebook Messenger but you're right that their consistency models are not as similar as suggested here - we'll try to fix this in a future video!

[@thepinkbismuth]

@@tryexponent also, I'm new to Cassandra, but you're showing relational tables, and Cassandra isn't relational, right? So it should all be denormalized.

[@jigsaw6991]

Why can't api server talk to each other using rest instead of using queues?

[@Anonymous-sv4wd]

I am confused why you need a message queue. What do the servers need to talk about? Isn't each server just a replication of the others, so they can handle the requests independently. If they get a request to send "hi" to client B, it can just handle that . Why does it need to let the other servers know?

[@thebrucecyou]

Contrary to traditional API servers, web socket servers need to push back messages to the clients. Because servers are replicated, Server A is not connected with all the users using the chat application. Client A who is connected to Server A would then have no way to communicate to Client B on Server B.

[@Anonymous-sv4wd]

@@thebrucecyou Thank you for the clarification! I actually had my interview last week and was pressed on this point - but next time I'll know what to say!

[@maazhasan-mi8le]

I think you need to cover about having a full duplex connection like websockets which will provide chat functionality with very low latency.

[@tryexponent]

Hi Maaz! Thanks for watching and leaving your feedback. Appreciate it!

[@santosh_bhat]

Which whiteboard software is being used here ?

[@tryexponent]

Hey Santosh! The whiteboard being used here is called "Whimsical"

[@brentfernandez2907]

What whiteboarding tool is he using?

[@ravindrabhatt]

In the conversation channel can we add messageid and userid as FK?

[@littleleaves9886]

No I think. There should be another table containing conversation id and message id. And every message has user id in the schema.

[@sbylk99]

@4:20 You said "Once the data is sent back, we immediately request a new connection and keep it open until data is available". Is that real? Doesn't long polling always keep the open connection and that's the benefit? Why request a new connection after receiving data?

[@arunsp767]

To fetch the next message!

[@sbylk99]

@@arunsp767 Wow, thanks a lot!

[@jrapp654]

Shouldn't the API actually talk to the notification service if the user is not online?

[@deekshamishra3369]

Great content in limited time

[@didigamnithin]

which tool are you using to make the diagrams?

[@tryexponent]

Hey didigamnithin, the tool is "Whimsical"!

[@ahmedmkadem7854]

I liked the video, however the statements made regarding the database are not so accurate. In texting application (or messaging in general), consistency is key. The value returned from the databases (a message in this case) should be accurate. Later on, you mentioned Hbase and Cassandra as example of database that would fit the requirement you set (highly available and uptime). Fine for Cassandra, but Hbase is a highly consistent database that is not known for its availability so Cassandra and Hbase cannot be options for the same requirement as they satisfy two largly different requirements/use cases!

[@the10xdev]

What’s the tool he’s using for the whiteboard?

[@tryexponent]

Hey the10xdev! The whiteboard is called "Whimsical".

[@mirttawu7002]

Thanks! May i ask what is the platform you are using now to display those notes when recording the videos?

[@JettZheng]

whimsical

[@老谈]

I think the server -> message service -> database logic is wrong. Servers should write to database directly.