System Design: URL Shortener like TinyURL (with FAANG Senior Engineer)

System design for a URL shortener by a FAANG Senior Engineer that has reviewed over 100 design documents 📚
Common names for this system design interview question:
URL shortener
Link shortener
Examples of real world implementations of this:
TinyURL
bit.ly ("bitly")
Request other problems for me to cover here:
forms.gle/afTK...
Here's a link to our discord channel where I organize the live discussions every weekend:
/ discord
Originally recorded - Session 1 9/3/2022
Some of my other system design videos that you might enjoy:
Hotel Booking: • System Design: Hotel B...
Web Crawler: • System Design: Web Cra...
Ad Click Counter and Aggregation: • Meta Interview Questio...
URL Shortener like TinyURL: • System Design: URL Sho...
Payment Gateway: • System Design: Payment...
KZbin: • System Design: KZbin...
Coding Competition Platform like Leetcode: • Meta Interview Questio...
Job Scheduler: • System Design: Job Sch...
Tinder: • System Design: Tinder ...
Chat App like WhatsApp or Facebook (FB) Messenger: • System Design: Faceboo...
#systemdesign #programming #technology

[@sophiemcgarity5458]

Really useful seeing you step through from the very beginning and hearing all your thoughts. Thank you!

[@paulatreides3400]

A great design discussion! Thanks! Some questions: * why do we need Kafka here; what does it solve? The URI management API seems to be perfect working synchronously. * btw, DynamoDB is serverless... your DB machine numbers are probably for the Cassandra case.

[@slover4384]

agree on both

[@fatehsinghparab1778]

Nice one. Keep uploading such informational videos.

[@shivujagga]

Running task runners just to store to a persistent data store & cache? I feel the link writing service should be doing that itself To me, it seems like a major overkill - and i haven't heard of it being done anywhere? - this would result in high costs and also maintainence of multiple microservices. Happy to hear anyone's thoughts on this

[@khushalidesai8753]

I agree, this is for design when we have lots of requests coming in and we use Kafka to handle the scale without any latency hit. In this scenario, he used task runner to take things off the kafka and store in DS and Cache.

[@sfontesv]

Totally agree

[@slover4384]

100% agree The design in video could be considered "unnecessarily complex" by interviewers

[@igorrybalka2611]

With Kafka buffer between datastore and link generation service, is there even a way to ensure there are no concurrency issues e.g. different short link generated for the same long URI? Or do we not care?

[@hamzafarhan348]

Thank you for sharing such helpful content, particularly in regards to calculation and estimation! I have a quick note regarding incorporating Kafka into the solution. I believe that processing the url-shortening POST request asynchronously may have a disadvantage, as the user who initiated the request would need to be notified somehow. However, based on the current design, there is no mandatory user metadata associated with the pub/sub mechanism to notify the user about the generated URL. This would result in unnecessary system overhead without a visible benefit, particularly in the event of a database failure or outage. What other potential solutions might be considered in this scenario?

[@diegoperez6575]

Are the sorther url generated by "link shortened service" or by "consumer runners that writes into the databases (dynamodb / redis)? I suppose that the shorter url are generated in the link shortened service, and the job of kafka here is liberate this service to handle the writing of data, because is faster to publish a message than write in the two database system. I suppose that rabbitmq or sns/sqs may be a valid solution too.

[@diegoperez6575]

Great job. I have a question: Calling to "unique id provider service" will not introduce a delay /lag in the petitions from our clientes too?. I think that if our main service "links shortening service" is able to build the unique Id using a library or helper for this, we can offer more req/res per second. SRP complaint is more important than the performance in this case? What is your opinion?