Cracking the Dropbox / Google Drive System Design Interview Part 1 of 2

[@ralucaioan7609]

This is simply awesome. Thanks a lot!

[@RakeshYadav-jc2nr]

Thanks a lot! This was simple, comprehensive and most helpful, as compared to others. The ref links are awesome too!

[@yuhengcai5600]

why do we need a block service to chunk file when there is already a chunker in client side?

[@rishisimply]

This was immensely useful. Good that you took ideas from Stanford presentation.

[@totsubo2000]

Some questions/feedback: 1. In the diagram that shows the meta server communicating with the not server using a queue, the queue has out-arrows to both the not server *and* Metada DB. I'm pretty sure you don't want two systems processing off the same queue. When one systems pops an item of the queue, it will no longer be available for the other system to process. 2. Curious why the not server uses HTTP long polling instead of Web Sockets. I would think web sockets would be more resource efficient than long polling.

[@Kamrun4U]

Thanks for sharing this video.

[@pawankishorsingh]

I feel at 25:17 timestamp and an many other places you are displaying "Client" inside a cloud kind of logo and server in a square box. I am ok with servers but "client" should not be drawn like this. Clients are generally user devices and hence that cloud thingy around it is not right.

[@_jatin_mittal]

amazing video! 😇

[@nehasht2]

It was really good video 😊 learnt alot

[@miriyalajeevankumar5449]

Great work

[@8888vampire8888]

did we forget an arrow, for feeding information, from the S3 cluster to the Queuing service? How will a new client, that just logged in, will get the actual modified chunk locally on his machine?

[@bhavyabansal1143]

Thanks for the amazing video, I have a question if you can please clarify. When user lets say create a new file, I have this workflow in mind (please correct if wrong): 1. Chunker create chunks 2. Watcher notices and updates indexer 3. Indexer updates DB on client and notify synchronizer 4. Synchronizer informs block server to upload chunks to S3 5. block server post upload informs meta server to update meta DB. Given this, looks like a client will never interact with meta server directly but in digram i see that connection. Am I missing anything here?

[@yuhengcai5600]

1. client will need to interact with metadata server to create/update metadata, with the status of file being "syncing" 2. then it will get a token to interact with block server, to upload file chunk by chunk 3. once that is done, block server will update metadata db. Status will be "synced", and we should also have the reference of the chunk passed into the db

[@wuaaron662]

thank you so much. The best tutorial i can find for designing dropbox The solution is logically correct while other video only gives you a general intro. But if you pay attention, their solutions is not even workable, due to missing pieces. One KZbinr says there should be request queue and response queue, blablabla. Compared with your solution, that one is a piece of 💩. A queue cannot send msg to client, without a consumer

[@pawankishorsingh]

At timestamp 40:07- why is there a need for block server to connect to metadata database? Isn't metaserver is a facade to this DB and all requests should go via this facade?

[@varshard0]

Answer to your question is kzbin.info/www/bejne/hnaXmKqpqbybo8U.

[@varshard0]

This diagram is based on their design back in 2008. They had to make the block server call Meta DB directly to reduce the number of round trip, because the block server and S3 are in a different region. They even had to copy a lot of APIs from the Meta Server into the Block Server.

[@sivakumartm]

nice. thanks for sharing this.. when is the part-2 video will be published?

[@durgadeep4988]

Lots of very good information !. thank you!.

[@AM-uc1sw]

Thanks a lot, it was great!

[@dnavas7719]

that primary key change was genius

[@pawankishorsingh]

At 42:52- why is there a need to close the connection & then reestablish? Why can't stay open till client is connected (like in case of websockets)?

[@varshard0]

That's because Dropbox use long polling, that's how long polling work. I wonder why they use long polling over websocket also. Probably for simplicity and the websocket doesn't reestablish convection automatically.