12: Design Google Docs/Real Time Text Editor | Systems Design Interview Questions With Ex-Google SWE

I swear Kate Upton and Megan Fox wrote I was handsome and sexy, you guys just didn't use two phase commit for your document snapshots and version vectors so you never received those writes on your local copy (since your version vector was more up to date than the document snapshot)!

[@AP-eh6gr]

this is production level detail - definitely requires a second sweep to memorize better!

[@jiangnan1909]

Hey Jordan, just wanted to drop a huge thank you for your system design videos! They were crucial in helping me land an E4 offer at Facebook Singapore (I did product architecture instead of system design). Really appreciate the knowledge and insights you've shared. Cheers!

[@jordanhasnolife5163]

That's amazing man! Congrats, your hard work paid off!

[@hl4113]

Are you able to give me some guidance on what to expect and the aspects that you felt was important to cover for the Product Architecture interview? I have one coming up and I'm at a complete lost as to where they'll steer the conversations.

[@jiangnan1909]

​@@hl4113 1. Contact your recruiter for a detailed outline of the interview's structure, focusing on timelines and key areas. 2. Use Jordan's channel and Grokking the API design course for preparation All the best!

[@venkatadriganesan475]

Excellent detailed coverage of online text editor. And you made it easy to understand the concepts.

[@renzheng7845]

dang this guy is really good! Thanks for making the video!

[@user-vz3zp2qg9q]

Thank you for this video! Pretty cool

[@user-wj1wy6ph5q]

🙇 interesting concepts covered. Thank you

[@DevGP]

Jordan ! Great video as always 🎉. I have a question , have you considered expanding into maybe dissecting an open source product in a video explaining why certain design decisions were made & discuss maybe how you would alternatively try to solve them ? Once again love all the work you put in, this is GOLD. Thanks !

[@jordanhasnolife5163]

That's an interesting idea! To tell you the truth, while I'm curious about doing this, the truth is that the amount of time that I'd probably have to put into looking into those codebases would be pretty wild haha. Not to mention that the guys working on open source software are a lot more talented than me!

[@Crunchymg]

Huge help in landing L4 at Netflix. Much thanks!

[@jordanhasnolife5163]

Legend!! Congrats :)

[@nowonderwhy200]

Got an offer from LinkedIn. Your videos were great help in system design interview ❤.

[@jordanhasnolife5163]

Legend!! Congratulations!

[@fluffymattress5242]

The level of detail in this video makes me want to burn all those stupid superficial bs i have been reading all these years. Imma name my 3rd kid after your channel dude ;).... the 2nd one is gotta be martin tho

[@jordanhasnolife5163]

"Jordan has no life" is a great name for a kid, I endorse

[@sarvagya5817]

Thank you amazing video 🎉

[@LeiGao-im7ii]

Beautiful！！！

[@gangsterism]

writing an ot has operationally transformed my free time into wasted free time

[@jordanhasnolife5163]

Sounds like a solid operation to me!

[@priteshacharya]

Great video Jordan. Two questions on final design screen: 1. Write DB sharding: What is the difference between sharding by DocId vs DocId+ServerId? 2. Document Snapshot DB: We are sharding by docID and indexing by docId+character position, is this correct?

[@jordanhasnolife5163]

1) We now become bottlenecked by a single database node. If we shard by doc and server id each server can write to a close by database. 2) Yep!

[@khushalsingh576]

great video and the information at 07:54 (Fortunatly there are engineers who has no life ... 😂😂) made the practicle touch

[@hl4113]

Hey Jordan, is there anyway you can make some content regarding how to tackle product architecture interview? I have one from meta coming up and couldnt find many sources of examples for content more focused on API design, client server interactions, extendibility, etc...? There are no examples I can find related to this on youtube. Thank you for all your content!

[@jordanhasnolife5163]

Hey! I've never done this interview myself so perhaps I'm not the most qualified. But considering that I've had multiple people on here say that they've passed meta interviews, I imagine it's pretty similar to systems design.

[@user-id1sf2ib3s]

Hi Jordan! Just watching the CRDT part of the video where you mention giving fractional ids to the characters, between 0 and 1. I was wondering how/at what point these ids are assigned. For instance, if you create a blank document and start typing, what would it look like? And if you then add a few paragraphs at the end, how would these new indexes be assigned? The example you gave (and that I've seen in other places) treat it as an already existing document with already assigned indexes and you just inserting stuff in between. I was thinking it might be a session thing - i.e. the first user that opens a connection to the file gets these assigned and stores in memory or something, but I watched another video where you mention it being indexed in a database. I'd love to know!

[@user-id1sf2ib3s]

I think I understood in the end, maybe? indexes 0 and 1 don't actually exist - your first character will be around 0.5, second character around 0.75, and so on... you're only going to get indexes < 0.4 if you go back on the text and add characters before the first character you added. If you write without stopping or going back, you'll get 0.5, 0.75, 0.875, 0.9365 and so on?

[@jordanhasnolife5163]

Hey! I think this is probably implementation dependent, but I imagine the idea here is that there's some frontend logic to batch quick keystrokes together so that they're all assigned similar indices as opposed to constantly bisecting the outer characters (see the BIRD and CAT) example.

[@asyavorobyova2960]

Hey Jordan, first of all, thnx for the great video! I have a question: can we use event-sourcing design approach instead of CDC? Meaning that using Kafka topics as the main source of truth instead of the writes' DB. We can consume from Kafka and build snapshots DB, and also users can consume from the needed Kafka partition to get the latest document changes. Thus we automatically get an order for writes inside any single partition and have persistence for writes. WDYT?

[@jordanhasnolife5163]

Absolutely! Keep in mind though that this implies that the single kafka queue becomes a point that all writes need to go through, which we want to avoid. If we do event sourcing with multiple kafka queues and assign ids to each write based on the queue id and the position in the queue, then use the vector resolution logic that I discuss, I think that this would be better!

[@asyavorobyova2960]

Thnx, of course I have in mind using separated Kafka partitions for each document (or set of documents), and using topic's offsets to store for using with snapshots. I'm not sure although if we can use the only one topic with multiple partitions for all writes, because if we have too many partitions for one topic it can increase latency. Maybe it's better to somehow split the incoming data and use many topics, to avoid this problem.@@jordanhasnolife5163

[@antonvityazev4823]

Hey Jordan! You did a great job with this one, thanks for you hard work! After watching this video and looking at the final design I didn't quite get to which place would a reader connect to receive updates about new changes in the document? I see that there are arrows to cache, to vectors db, to snapshots db and to write db, but don't see any WS server or something Could you clarify please?

[@jordanhasnolife5163]

Leader first gets a snapshot with a version vector from the vectors and snapshot db, and from there subscribes to changes on document servers, applying any subsequent updates

[@antonvityazev4823]

Much appreciate it

[@firezdog]

I’m 30 minutes in. Got the sense each client just gets all these messages from other clients and applies them using some merge function that guarantees the result of applying messages in the order received makes sense - with a little bit greater consistency (via version vectors) for writes from the same client. But I’m wondering - is there any sync point at which all users are guaranteed to see the same version of the document? Because if not clients could just diverge more and more over time…

[@jordanhasnolife5163]

Yep - no there is not any sync point. If we wanted to, we could occasionally poll the db on an interval to ensure we don't get too out of wack.

[@levyshi]

Great video! just curious what might be different if this was for a google sheets like product, rather than a document.

[@jordanhasnolife5163]

Frankly I think you'd have less collisions which probably means you can get away using a single leader node and not be that bottlenecked. If for some reason you did need to do this, you'd basically need a way of combining writes to the same cells, which doesn't really make much sense intuitively. I'd say if you want to do multi leader you should probably at least incorporate a distributed lock so that if two people decide to edit cells at the same time, we conclusively know which one came first.

[@levyshi]

@@jordanhasnolife5163 Was thinking the same thing, have them write to the same leader, and let the leader's own concurrent write detection decide.

[@RobinHistoryMystery]

Dayum boi

[@joshg7097]

I wonder why you would use another db plus two phase commit for the version vector table, instead of using the same db and use transactions instead.

[@jordanhasnolife5163]

If I have to partition the data for a big document over multiple tables I need a distributed transaction

[@jordanhasnolife5163]

If we assume all documents can fit on a single database totally agree that's a much better approach

[@joshg7097]

The version vector for a document can exist on the same partition as the documents partition. If we assume a document can only reach megabytes and not gigabytes it's safe to assume a single document can exist on a single partition. Even if a single document has to be chunked, then we can still colocate the version vector for that chunk.

[@jordanhasnolife5163]

@@joshg7097 Hey Josh, you can co-locate it, but it still becomes a distributed transaction which needs to use 2pc. Also, ideally, we don't have to be too rack aware in our write storing I feel like, because if we were to use something like AWS we don't necessarily have those controls. I agree with your point though, probably 99.9% of the time a document won't span multiple partitions and in such an event you should store the version vector local to its partition and don't need 2pc.

[@joshg7097]

@@jordanhasnolife5163 I accepted an L5 meta offer a few months, I watched every single one of your videos, huge thanks to the gigachad 😁

[@jasdn93bsad992]

19:15 the result of interleaving of "cat" and "bird" should be "bciartd", right?

[@jordanhasnolife5163]

Ah yeah maybe a typo on my end

[@jasdn93bsad992]

@@jordanhasnolife5163 Yeh right. No worries. Great video, thanks man

[@ShreeharshaV]

Thanks for the video Jordan. At kzbin.info/www/bejne/j3TNh3x6q5Wth7s How does the new client that has no content fetched so far get the content from Snapshot DB directly? What does it ask the Write DB or Document DB at this point?