System Design Interview: Design Dropbox or Google Drive w/ a Ex-Meta Staff Engineer

  Рет қаралды 71,892

Hello Interview - SWE Interview Preparation

Hello Interview - SWE Interview Preparation

Күн бұрын

Пікірлер: 244
@abhijeet8710
@abhijeet8710 5 ай бұрын
"Have you done any System Design course ? How are you so good with this subject ?" - These were the word of my interviewer. I had a High Level + Low Level system design with a start-up recently. Surprisingly the question was to design a file sharing system such as Google Drive as described in this video with some additional features. I explained the HLD with the diagram as I had learned from the the concepts of this video. After the HLD was over, the interviewer told me that I have created a very robust & elegant system. He further said, he was so satisfied with the HLD, that he no longer wants to go into the LLD. Folks, these videos are the absolutely anything that you will ever require to ace a system design interview. Do remember to learn the fundamentals used in the system. A huge thanks to #Hello Interview for putting out the best content out there.
@JohnVandivier
@JohnVandivier 5 ай бұрын
"he was so satisfied with the HLD, that he no longer wants to go into the LLD. " GOALS! kudos and congrats
@hello_interview
@hello_interview 5 ай бұрын
This is epic!
@charan775
@charan775 Ай бұрын
which startup bro?
@KiritiSai93
@KiritiSai93 Ай бұрын
You guys remind me of the "Acquired" podcast hosts. No click-baits or cringe posts, just sheer passion about the subject and high-quality in-depth analysis of things. Kudos and hope you continue the great work!
@hello_interview
@hello_interview Ай бұрын
That’s the idea. Pure value no BS 🫡
@draugno7
@draugno7 8 күн бұрын
I also loved the jokes and an occasional reassurance in the Uber video, looking forward to more! Ddinngdding (that driver's phone after Taylor Swift concert in a badly designed system). This channel is simply amazing because it ties together all of the concepts I learned and even elaborates on different DSs and DBs. Someone said 'no shade to other youtubers' but I say 'yes shade' because they usually confuse and frustrate people who watch with incomplete diagrams and explanations.
@YeetYeetYe
@YeetYeetYe 3 ай бұрын
Simply amazing. I don't mean to throw shade to other channels, but this is by FAR the best system design interview prep. So many other channels are just people with a couple of months of experience at FAANG and it really shows the difference between junior FAANG engineers and Staff FAANG engineers. Extremely high quality work.
@hello_interview
@hello_interview 3 ай бұрын
So glad you like them!
@EamonLinskey
@EamonLinskey 6 ай бұрын
These are the best System Design videos I have found. Great framework for approaching problems, clear explanations, helpful diagrams. And I really appreciate the notes about how insight’s different seniority levels might approach specific parts
@Wololowizz
@Wololowizz 3 ай бұрын
I must say that this is the best system design video I've seen so far. You covered the problem and solution step-by-step while other videos just throws a bunch of ideas right away. Sometimes I feel overwhelmed watching other videos thinking that's impossible to know all of that, but watching this video we can know what's the expectation for each level and the most important thought: you don't need to know everything. And that's gold
@hello_interview
@hello_interview 3 ай бұрын
Glad you liked it! Check out our others if you haven’t already. Same format :)
@andjelaarsic9217
@andjelaarsic9217 5 ай бұрын
My mind is absolutely blown by how beautifully everything is explained. I love how you understand what would be possible questions/confusions from people watching and you address them by explaining pros and cons. Thank you so much for the content! Your walkthroughs are by far the most useful and interesting.
@hello_interview
@hello_interview 5 ай бұрын
High praise! Appreciate you taking the time to share this 😊
@madhurnsit
@madhurnsit 3 ай бұрын
This is the best content I have come across on System Design interviews. Wish I had landed here this sooner. Thank you so much!
@GauravGupta-op8ol
@GauravGupta-op8ol 7 ай бұрын
With my systems design interview coming up, I was looking forward to your video. It's great as always.
@prasidmitra6859
@prasidmitra6859 5 ай бұрын
These are like gift from God. The best SD resources I've found in the last 3 years.
@aldogutierrezalcala3047
@aldogutierrezalcala3047 2 ай бұрын
Bro, again me, just had a system design interview using your framework, still don't have the result but definitely this framework is basically pure gold to lead a conversation that i would keep using even in a daily job.
@hello_interview
@hello_interview 2 ай бұрын
Hell yes!! So glad it went well 💪
@cidwiththreeeyes
@cidwiththreeeyes 22 күн бұрын
Thank you for another great video! Honestly, I don’t have any constructive criticism, it’s pretty much a perfect format for these videos-practical, concise, insightful. Other creators’ videos like this are good, but they feel like they’re just going through memorized recipes. Your videos are actually teaching system design theory. Really hope you have more of these as I make my way through your catalog.
@yankomirov4290
@yankomirov4290 27 күн бұрын
You added systematic (pardon the pun) approach to such an open-ended nature of an interview. This was a game change for me! I really appreciate it, I went ahead and bought the Guided Practice which is also amazing and is my main practicing tool. Thank you so much!
@levimatheri7682
@levimatheri7682 4 ай бұрын
Wow, by far the best system design videos anywhere. I love how simple you make it, and the invaluable tips!
@tushargoyal554
@tushargoyal554 2 ай бұрын
This is the best channel for learning system design. I've gone through a lot of explanations but found them talking things in isolation making it very hard to connect to get a full picture. The popular system design interview book also doesn't help much due to very discrete and sometimes inconsistent sharing of knowledge.
@anmolgangwal9236
@anmolgangwal9236 Ай бұрын
bro we are ready to pay just enable the join icon in your channel, this content is too good to be free
@jagrit07
@jagrit07 Ай бұрын
Watched 20 minutes of the video so far and This is the 3rd resource I am watching regarding Dropbox design, I have read Alex's book, read Grokking book and now watching this just for fun and I think Evan King is actually the King lol. Amazing video, Please keep on adding more content. Yesterday, I commented on Tinder's Design video and now here. I think I might have to comment on all the videos once I watch those because this is really good stuff and we viewers should appreciate it and hence I will keep adding comments lol :D
@Gamble396
@Gamble396 Ай бұрын
One of the best System Design channels. Please keep uploading.
@batusun717
@batusun717 2 ай бұрын
please upload more stuff like this. This is literally the BEST on KZbin. Very much appreciate all the great efforts!
@alexandergordon9286
@alexandergordon9286 6 ай бұрын
It's pure gold! specially the parts where you are stopping the debates abouts what db to choose or if the calculations are needed. The deep dives are the best part.. no one goes that deep and thats actually what matters in an interview
@lorddel
@lorddel 7 ай бұрын
One more comment on this: comparing this to the written content on hellointerview, this one seems more round and well-thought (mainly regarding using S3 notif. on chunk upload completion, which wont work). Would be cool to see it reflected there on the platform! Good job
@hello_interview
@hello_interview 7 ай бұрын
Good feedback! I'll try to get that updated, particularly by adding sync which I just last minute decided to throw into the video.
@chongxiaocao5737
@chongxiaocao5737 5 ай бұрын
one of the best system design preparation video I have seen online.
@md_dm490
@md_dm490 6 ай бұрын
This channel has the best system design content on youtube. Keep up the good work.
@adeeshacharya7520
@adeeshacharya7520 5 ай бұрын
This is really good, irrespective of whether we are taking interview or not, any person looking at this level of explanation and detail would try to picture software differnetly. Thanks for making such videos, would love to see some more
@yourssachin
@yourssachin 6 ай бұрын
Love the content and explanation. I watched hundreds of videos on system design from last 4-5 years and also have paid subscription from few. I don't have any doubt that, your channel can become premier system design platform in no time if you keep the content quality high ( just like last 3 videos). Next video, I'd recommend to talk about messaging platform like WhatsApp or FB messenger. There are so many videos on this topic but didn't find any which explain the details and really help in the interview.
@AncientArtist7
@AncientArtist7 Ай бұрын
Your content is great and really easy to follow through each step of the process. Please continue to make more system design videos. It is extremely helpful !
@anuragtiwari3032
@anuragtiwari3032 6 ай бұрын
i dont comment much, but for this kind of explanation i gotta give it u. Hands down the best explanation on youtube . pls continue making these kind of videos . This channel will blow up
@hello_interview
@hello_interview 6 ай бұрын
♥️
@JShaker
@JShaker 4 ай бұрын
I'm so grateful for all of your videos. I've been practicing using the Hello Interview AI interviews, booked one mock with one of your interviewers, watched all the videos. The quality is so far beyond any other content out there, and I've successfully passed 5 system design interviews. Keep up the good content, your KZbin channel deserves to blow up and your website too #wouldinvest
@noobu
@noobu 7 ай бұрын
Great stuff again! Not only good for interview but also for daily work 1) Clear and concise structure 2) Weigh trade off rigorously and explain the final decision clearly. Every single component is well though out with real world considerations
@mehdisaffar
@mehdisaffar 5 ай бұрын
I love the content. It has been frustrating to watch some other system design videos where they just brush off over important details and act like everything is straightforward and easy, and just make 10s of services and never really explain the nitty-gritty details of how those things would work and IF they would actually work/be efficient etc. Thank you!
@mehdisaffar
@mehdisaffar 5 ай бұрын
I wish you had mentioned the challenges of 2-way syncing in this context. Because this is akin to master-master replication, in case of network partition (for example user makes changes to remote, hops on another offline device, makes changes, then comes back online) there is a chance of inconsistencies (user makes different changes on device 1 vs device 2). There would probably need to be a way to offer merging changes together or have the user choose between version 1 or 2.
@mehdisaffar
@mehdisaffar 5 ай бұрын
I think I talked too fast! You did mention reconciliation
@allenputich4192
@allenputich4192 2 ай бұрын
You do an amazing job of explaining the thought process, technical details, and growth opportunities!
@groovymidnight
@groovymidnight 6 ай бұрын
I really like the 5-step structure, it's the best I've seen and it effectively helps me think through the designs in a methodical way.
@hello_interview
@hello_interview 6 ай бұрын
Right on! So glad it’s useful
@galashrenik3404
@galashrenik3404 Ай бұрын
One suggestion I have is that when designing APIs, your videos often highlight the importance of handling partial data, which is typically expected of senior or staff engineers. In my view, API versioning carries a similar level of significance.
@3rd_iimpact
@3rd_iimpact 7 ай бұрын
I just finished reading the article on this lol. I’ll check out the video as well.
@pragatimodi950
@pragatimodi950 4 ай бұрын
Hi Evan, this is my first time giving system design interviews. Really glad I found this channel to learn from. Most of my prior feedback from mocks and system design have been framework related for when I explain my design. This really helps with that and I think even at work, this is a really good approach to follow for. most things. Awesome content, thanks a lot!!!
@viveksharma-tt5nj
@viveksharma-tt5nj 19 күн бұрын
Simply amazing !! Thanks a lot for such clear and concise explanation !
@JyotiKundani05
@JyotiKundani05 4 күн бұрын
This video was really helpful. Amazing work of putting this together and your explanation was on point. Much appreciated!
@hello_interview
@hello_interview 4 күн бұрын
Glad you liked it! 🙂
@vaibhavsharma1653
@vaibhavsharma1653 5 ай бұрын
Amazing. Some Notes: DeepDive: Chunking CDNs Adaptive Polling with only updated chunks Compression.
@jimitshah7636
@jimitshah7636 5 ай бұрын
Great video for system design preparation. Methodology, the way he approached the question was good. 5 steps. Pretty good
@Marcus-yc3ib
@Marcus-yc3ib Ай бұрын
Please keep upload these kind of videos. Thank you very much.
@crackITTechieTalks
@crackITTechieTalks 5 ай бұрын
This is the best system design video, I have watched!! Specially the deep dives, You nailed it !! Looking forward to watch your videos.
@phavelar
@phavelar 6 ай бұрын
one can argue that "supporting 50gb upload file size" is a functional requirement (you placed it under non-functional requirement) - just a call out. great video!
@ashutoshrana9998
@ashutoshrana9998 5 ай бұрын
Will be the best system design interview channel for sure. Neat content. Keep up with the quality Man!
@venkatamunnangi1287
@venkatamunnangi1287 7 ай бұрын
Thanks for the effort and videos. Easily one of the best in business for mocks and educational material.
@smalladi78
@smalladi78 5 ай бұрын
Thanks for posting these! Great interview as always! I am learning a lot from these interviews. I found it interesting that you jumped ahead in order for the non-functional requirements since you knew the large file upload requirement would impact the design enough that doing the other ones first was not beneficial since they would become irrelevant. Obviously, this comes with actual experience of working on the job. May I suggest doing a follow up that uses the final design from this interview and consider how it may change if you piled on a more advanced feature like syncing only a partial set of folders or sharing folders with other people.
@indreshgahoi7103
@indreshgahoi7103 7 ай бұрын
Hey Evan , thank you so much for providing the great content. I really live the way you organize and put content across the board. ❤
@satyajeetkumar2588
@satyajeetkumar2588 Ай бұрын
Awesome , so simple and elegant . It would have been great if you would have mentioned about checksum implementation to maintain data integrity as you have mentioned in the non functional requirements just to mention not the actual implementation.
@god_of_blunder
@god_of_blunder 3 ай бұрын
these are the best Design videos i ever found, Thanks and Kudos.
@hello_interview
@hello_interview 3 ай бұрын
❤️
@AlbaraaAlHiyari
@AlbaraaAlHiyari 6 ай бұрын
I truly appreciate all the effort you've put into making these amazing videos. Please keep them coming. One insignificant (not important) nitpick. 50 GB @ 100Mbps = ~ 1hr 7min. I think you just forgot to convert the decimal to minutes. You have it correct in the write up, as in 1.11 hours (0.11 * 60 = 6.6 minutes).
@hello_interview
@hello_interview 6 ай бұрын
Mental math is hard 😛
@AlbaraaAlHiyari
@AlbaraaAlHiyari 6 ай бұрын
@@hello_interview tell me about it... Also not fun under the pressure of an interview 🤣
@jeremyklein953
@jeremyklein953 6 ай бұрын
Really good approach. I love how you build up to the full solution. It makes a lot of sense to me and helps me reason these complex systems as well
@jherreria
@jherreria 5 ай бұрын
I really appreciate your help in this topic. I'm learning a lot! Keep the videos coming!
@ahmedkhan25
@ahmedkhan25 5 ай бұрын
Excellent sys design interviews - I like the informative tone and clear approach - thanks
@xparkyolo
@xparkyolo 21 күн бұрын
You don't need paid courses when the quality of content for system design on youtube is so high. Amazing explanation clear to the point and tackles
@suri4Musiq
@suri4Musiq 6 ай бұрын
Loved this resouce, thank you so much! But I just wanted to point out that in my interview I was asked about sharing files with other users and I feel like this design concentrated more on just syncing files across multiple devices. In the former, I think we can talk a little more about CDN/other approaches which were hand waved here.
@hello_interview
@hello_interview 6 ай бұрын
Checkout the write up I linked! I go into sharing there.
@DMA-I
@DMA-I 13 күн бұрын
I believe there is a slight flaw for the sync files from remote server feature (24:16). I believe we need to keep records in db which device which client has synced to date what updated time/what version or the get changes will loop endlessly (getchange will always get files needs to be updated, but they might just have been updated)
@surojitsantra7627
@surojitsantra7627 6 ай бұрын
One of the best and detailed explanation. Thank you so much for this content. Please upload more such videos.
@hello_interview
@hello_interview 6 ай бұрын
New one later today!
@vijaykhurana8766
@vijaykhurana8766 6 ай бұрын
Great content. Thank you for posting. One of the best system design video I have come across for this design.
@MrSnackysmorez
@MrSnackysmorez 3 ай бұрын
I love the videos and these are some of the best explanations. I love the flow and how everything builds on each other. It makes it much more manageable to do these problems. However you are driving and dictating this and this is so much harder to do when the interviewer wants to constantly interrupt and ask questions while you are doing these steps without first letting you explain what you are doing. I have this happen pretty often. How can you tell them to just chill and let you proceed? Appreciate these videos!
@KITTU1623
@KITTU1623 6 ай бұрын
Thank you very much for the videos. One small nit pick. DynamoDB supports a maximum of 400KB per item and if we are storing all the chunk metadata in the item, for a 50GB file with 5 MB chunk size, assuming we need 100Bytes per chunk metadata, our item size would be around 1MB.
@hello_interview
@hello_interview 6 ай бұрын
Good catch! True
@evangeloskostopoulos8173
@evangeloskostopoulos8173 6 ай бұрын
This is really awesome, thank you. Please keep them coming!
@jmms49
@jmms49 6 ай бұрын
great videos, thanks for uploading these. Easily the best content about system design interviews I've found. I would probably suggest to use merkle trees for the sync functionality, seems like a natual way to diff and sync large file systems
@BlunderMunchkin
@BlunderMunchkin 3 ай бұрын
Huh. I would have prioritized consistency over availability. So much so, in fact, that I didn't even think it was a question. Some of the biggest headaches I've experienced as a developer have been caused by having an out-of-date file. I would much rather be temporarily unable to retrieve a file than to be fooled into thinking that the file I retrieved is the correct version.
@deathbombs
@deathbombs 6 ай бұрын
Voted on your website for payment system! Banks love these
@IshaZaka
@IshaZaka 6 ай бұрын
Hi Evan, Thankyou so much for providing this type of content. plz make a system design video on payment system
@B-Billy
@B-Billy Ай бұрын
Pure Gold content!!! Thanks you so much.
@ndubuezeprecious391
@ndubuezeprecious391 Ай бұрын
Great stuff. This is the best I’ve seen so far. Can I know the app you are using for the white boarding, it looks really sleek
@hello_interview
@hello_interview Ай бұрын
Excalidraw
@krishnabirla16
@krishnabirla16 Ай бұрын
You did not talk about version inconsistency? If two clients keep changing their local folders, they will be in a loop of pushing their own sync and pulling the other client's sync. There has to be a timestamp/version based conflict resolution. Maybe a follow up please?
@pujamishra1475
@pujamishra1475 6 ай бұрын
I have a product architecture interview coming up. I was really looking for some good product architecture/design examples and then came across this. This is very helpful because you talk about the client, user experience, malicious users and relate it to the design decisions made. Thank you! One question, for a product architecture interview - should we go into more details about the APIs like explicitly write out requests, response, failure/success codes or the amount of discussion you did on APis is enough for senior level? Can you also tell me what topics/ points would you add over the discussion in this video if this was asked in a product architecture design round. Thanks again!
@puppy851226
@puppy851226 2 ай бұрын
Amazing content! Thank you hello interview!
@dark-knight494
@dark-knight494 6 ай бұрын
Big fan of this channel and Evan. Please solve whatsapp/messenger type chat system next if you get some time.
@depengluan7222
@depengluan7222 4 ай бұрын
Love the content! Thanks for putting the efforts! nit, fingerprint probably does not fit as a good id for chunk, as hash value can change over time when the content changes.
@danielkling4647
@danielkling4647 2 ай бұрын
First I would like to say that this content is excellent. Why though would you implement chunking yourself instead of using S3's multipart upload?
@ramannanda
@ramannanda 5 ай бұрын
For the delta sync bit, probably should go a bit deeper into rechunking for an existing file, to perform the delta sync.
@adityaagarwal5348
@adityaagarwal5348 Ай бұрын
At 50:08, the delta sync approach might work in case of downloading updated chunk from s3 using range-bytes query and then updating file on the local system but it won't work other way around specifically because of s3. S3 objects are immutable so there will never be a case where a chunk will be updated. So if this questions come up in the interview, should we just mention that we won't sync files > some GBs or we should further divide the storage into blob and file-system (s3 and EFS) based on file size and handle the complexity on server?
@Anonymous-ym6st
@Anonymous-ym6st 7 ай бұрын
Thanks for the great content! Your video really help me understand the "flow" about a good system design interview (which I do feel very important for staff+ engineers to direct the interview!) a few general questions: 1. Is it beneficial to mention the selection of specific DB (like dynamo), and mention to use Kafka/Spark for microbatching snapshot update etc. after evaluating the QPS / replication etc.? or keep it at abstract level would give equivalent signals for interviewers? 2. what signals / thinking might make you feel it even beyond staff (just out of curiosity, as feeling it has already been very perfect from the staff requirement)?
@hello_interview
@hello_interview 7 ай бұрын
1. You'll need to show technical excellence somewhere, these could be two good places. Ideally, you go deep in the places you know well and have hands on experience. If this is where that is, then go for it. If you keep it abstract, then the depth needs to come from somewhere else. 2. Hard to say, Staff candidates can usually teach me something, which is a key sign. They know some part of a system better from work experience then I do, so we can go deep there and I end up learning. It's abstract, but this is usually the best sign that a candidate is staff+.
@nobodyknows228
@nobodyknows228 4 ай бұрын
1. How can we handle write conflicts when we have a folder which is supposed to be consistent across multiple devices? 2. Also when two devices are disconnected from the internet and if users updates some files how does the sync happens when they come back online and when both tries to write the changes at the same time at a same file path? I am not sure if these solutions work but I think 1. We can use a Redis lock for writes with TTL same as the timeout or a little more of the pre-signed url. If connection fails in between we can just resume the upload when connected back. But this might be a problem when a user is trying to upload big files with large timeout durations since other users might have to wait till the user uploading currently is done. 2. When the user comes back online we should probably first fetch all the changes that are executed on the device and raise conflicts with the user asking what action to perform(similar to git) and acquire lock to write if required.
@ediancomachio2783
@ediancomachio2783 6 ай бұрын
this is pure gold thank you so much
@VyasaVaniGranth
@VyasaVaniGranth 4 ай бұрын
First - please continue making and sharing these videos, this is incredible. Very few high quality sources available out there and this is probably the best one in my eyes. Second - how realistic is it that the download and upload happen directly b/w client and S3? Are there security concerns with this approach that should be considered? For reference, there's a Dropbox engineer's talk where uploads go through an intermediate service - this does mean additional copies of the data meaning more memory / compute but seems more realistic. In general, for any design that has media upload (eg. newsfeed), would you recommend direct upload to S3?
@hello_interview
@hello_interview 4 ай бұрын
yah its a good point, most major systems don't do this for a number of reasons. While is largely academically correct and optimal, at youtube/dropbox/etc scale, they prefer more control so they're rolling their own systems here.
@TatianaRacheva
@TatianaRacheva 3 ай бұрын
IIRC, low latency was specifically low priority for Dropbox because they (like email) rely on the client syncing the data and user accessing the local copy when it is ready. Also, I question whether consistency is less important than availability. I don’t know, but I’m curious how the answer would be different if latency could be high and consistency had to be strong.
@amitb2921
@amitb2921 5 ай бұрын
Thanks for a great content, especially the Deep Dive part, which generally people do not discuss about. I have one question around storing the chunks as list in the DB. For 50 GB file and 5 MB chunks there will be 10K chunks created. So the chunks list will have 10K entries. Now updating one chunk list column for every chuck status change could be quite challenging. Would it be better if we have a separate table for chunks instead. Also while you do the matching of chunks with the fingerprint, You need to check 10K entries from Local DB(with separate table and indexed) vs 10K entries in the chunk list (in single table column), where former is more efficient. Kindly let me know what are your thoughts on above points ?
@hello_interview
@hello_interview 4 ай бұрын
Sounds reasonable to me! Good call out
@amitb2921
@amitb2921 4 ай бұрын
@@hello_interview Thanks a ton for the response. I have modified my comment above to be bit more clear.
@dannyryngler6425
@dannyryngler6425 4 ай бұрын
Question - what should the file id be? It can't be based on the file name, as names can change. It also couldn't be a hash of the whole file, as the file itself can obviously change. Amazing content, thank you!!
@hello_interview
@hello_interview 4 ай бұрын
Depends on if you want versioning or not. Can be the fingerprint or a random uuid, depends on requirements
@fragrancias972
@fragrancias972 2 ай бұрын
Excellent content. Please tell me if I’m mistaken, but I believe GET /files/:fileid would return a list of chunk s3 links, not the file itself. Also, I don’t think merely filtering chunks by update time would work for syncing. You would need a tombstone for when chunks are removed. You didn’t quite specify how “polling the DB”/ update time filtering works with delta sync. Merkle trees could be used to optimize the reconciliation you mentioned, right?
@tvmanikandan835
@tvmanikandan835 6 ай бұрын
the content is good, keep up the good work. expecting more SD videos in more details
@truthSaty
@truthSaty 6 ай бұрын
Your videos are v. good but your mock interview is costly for certain markets (BFS can give you more return). Wish to get interviewed soon !
@coderzlife
@coderzlife 6 ай бұрын
Please make a video around Designing a distributed login
@mindrust203
@mindrust203 7 ай бұрын
Hey Evan, this content is fantastic, thank you! I have a question regarding your solution to chunking around the 39 minute mark When we ask S3 to fetch us a pre-signed URL, do we do that for all our chunks as well? Does this happen on initial request to upload the file (metadata)? The way the File Metadata entity schema is described, it looks like we have a top-level S3Link, but also chunk-level S3 links embedded in the file metadata, so the upload flow is a little unclear to me
@hello_interview
@hello_interview 7 ай бұрын
Good question, you're right to be a little confused here. So as I alluded to S3 offers and API called multi-part upload. For this, it requires just 1 presigned url, but, multi-part upload re-stitches the chunks back into a single file in s3, so this does not allow us to send over chunk deltas for syncing. As a result, we have to upload as chunks manually without relying on multi-part upload. So, long answer, but yes, you'd actually need to request a presigned url for each chunk, I should have made that clearer but tbh was not sure in the moment if multi-part upload could be configured to not re-stitch the file, so I omitted :)
@SunilKumar-jl6dl
@SunilKumar-jl6dl 3 ай бұрын
Hey there, I have some questions. Would be great to get your thoughts: 1. S3 supports multipart upload and all the chunks would get reconstructed into a single file at S3. Isn't this correct? If yes, then having file chunks in the database would be redundant right? Or would S3 have the chunks always and give access to the download at the client end? 2. At the client end should we know how the updated/deleted chunks of a previously uploaded file be stitched back together? 3. Would folder sharing with other users be a possible follow up question? Like what Google drive offers.
@adityaagarwal5348
@adityaagarwal5348 Ай бұрын
At 27:24 For determining which files are already available on the local system, can we store a client to files mapping on the server based on client id and then getChanges API uses that data + file metadata to calculate which files needs to be transferred to the client? I know there can be issues when there is a sync gap b/w local and remote like file is deleted on the local but anyway system is eventual consistent. Keeping lots of data on the client will grow the app size.
@castulo
@castulo 6 ай бұрын
👏Bravo, on point as always. Thanks Evan, keep up the good work man!
@HandsomeLancerYouTube
@HandsomeLancerYouTube 2 ай бұрын
Amazing stuff!! Side note: please use dark mode.
@hello_interview
@hello_interview 2 ай бұрын
🫢
@HiraMalik-r3i
@HiraMalik-r3i 3 ай бұрын
Thanks for such a detailed video. Query: If File service is pushing the change events to Event Bus and also updating Db someway, wouldn't this lead to dual write problem? I do not see the same event being consumed by both( no message queues in that flow in the diagram). Shouldn't we instead use CDC or some other solution for this problem? What are your thoughts on that?
@VahidOnTheMove
@VahidOnTheMove 5 ай бұрын
Thanks for the videos. 47:45 I would like to know your opinion on push approach? By push approach I meant when the File service knows there is a change in a chunk, Sync service will let the client know. And, then the client will send a request to sync/download the chunk.
@haixiongwang4608
@haixiongwang4608 12 күн бұрын
Will the version management of files be out of scope for 35-45 mins interview discussion. Just want to get high level understanding the scope of current SD. Thanks
@bqrkhn
@bqrkhn 2 ай бұрын
Very nice video. A question: You added a updatedAt at each chunk. But chunks are identified with their ID which is calculated from a finger print. When the file changes, the finger print changes, how do we update the updatedAt? Possible Answer: From client we send both old and new chunk IDs and then update both id and updatedAt. Is this the correct strategy?
@fragrancias972
@fragrancias972 2 ай бұрын
Same question here.
@bqrkhn
@bqrkhn 2 ай бұрын
@@fragrancias972 what do you think about my possible answer ?
@insofcury
@insofcury 7 күн бұрын
@@bqrkhn +1 I think this definitely solves the problem.
@Ynno2
@Ynno2 7 ай бұрын
Do you suggest a different delivery framework for system design interviews which aren't necessarily "product"?
@hello_interview
@hello_interview 7 ай бұрын
Topical! Was chatting about updating the site with that soon. I’d recommend very similar, but core entities and api are what may change as they could be less relevant. Instead I’d frame it as focusing on the inputs and outputs of the system more generally. And then still thinking about the data persisted
@hello_interview
@hello_interview 7 ай бұрын
I’ll do a pure infra question next
@aslgomes
@aslgomes 4 ай бұрын
Hey Stefan, awesome video, congrats! I've got a quick question though. Around the 49:46 mark, you mention adding an "updatedAt" to a chunk at a specific id/fingerprint. If a chunk changes, its fingerprint/hash/checksum would change too, right? So that id wouldn't really match the changed chunk anymore, would it? Doesn't that mean the old chunk gets "invalidated" and a new chunk id appears? Sorry if I'm missing something obvious here.
@hello_interview
@hello_interview 4 ай бұрын
No this is spot on, good call out. I was loose here. If the fingerprint is the ID, then an updatedAt does not make sense. If the fingerprint is not the ID, then it of course does. Trade off here of whether you want to keep old chunks around for versioning.
@theoshow5426
@theoshow5426 3 ай бұрын
Keep going man! This is great!
@charan775
@charan775 Ай бұрын
how do you handle nested folders in your schema? also chunks could kept as separate table at user id level, so that we can reuse chunks of different files..
@caesar5555
@caesar5555 14 күн бұрын
Thank you! This is awesome! In Meta is your interviewer going to be at the hiring committee or will just send your and their notes?
@hello_interview
@hello_interview 14 күн бұрын
Depends on level and situation. most likely they won't be there
@caesar5555
@caesar5555 13 күн бұрын
Thank you for the answer. @@hello_interview Staff+ . So the interviewer is just a medium for intaking information....
@hello_interview
@hello_interview 13 күн бұрын
@@caesar5555 In some sense. They provide a judgement and the loop (the collection of interviewers, together with the recruiter) make a call on whether to put the package forward to the hiring committee. That group (usually a couple directors) does not have enough time to review every detail of the notes, so they use some heuristics to see where the major risks are and decide whether to move forward.
@robert23kim
@robert23kim 3 ай бұрын
Thanks. Great learning.
@omprakashpatel2079
@omprakashpatel2079 2 ай бұрын
great explaination
System Design Interview Walkthrough: Design Twitter
23:04
Hello Interview - SWE Interview Preparation
Рет қаралды 63 М.
Каха и лужа  #непосредственнокаха
00:15
This Game Is Wild...
00:19
MrBeast
Рет қаралды 133 МЛН
System Design Interview: Design Uber w/ a Ex-Meta Staff Engineer
1:03:05
Hello Interview - SWE Interview Preparation
Рет қаралды 114 М.
Google system design interview: Design Spotify (with ex-Google EM)
42:13
IGotAnOffer: Engineering
Рет қаралды 1,2 МЛН
Meta Technical Interviews in 2024: What You Need to Know Now!
14:13
Kafka Deep Dive w/ a Ex-Meta Staff Engineer
43:31
Hello Interview - SWE Interview Preparation
Рет қаралды 64 М.
System Design Interview: Design an Ad Click Aggregator w/ a Ex-Meta Staff Engineer
1:02:22
Hello Interview - SWE Interview Preparation
Рет қаралды 55 М.
Software Engineering Job Interview - Full Mock Interview
1:14:29
freeCodeCamp.org
Рет қаралды 1,5 МЛН
Basic System Design for Uber or Lyft | System Design Interview Prep
16:18
Каха и лужа  #непосредственнокаха
00:15