URL shortener system design | tinyurl system design | bitly system design

[@hnasr]

Big respect for your knowledge and how you unselfishly share this. There is no one on KZbin that does this in such details. Keep doing what you are doing man.

[@ujjwalbansal1070]

Ya, true man, @gauravSen, and this guy are providing this valuable information. If you know anyone like these guys on youtube reply back

[@karimkhan9713]

Hussein my man you also share you knowledge Love you

[@savithak.6516]

Nice work... any thoughts on this review ? medium.com/double-pointer/review-of-grokking-the-system-design-course-c8613c28f3a1

[@TaherAhmed16]

@@ujjwalbansal1070 Check out Hussein Nasser's videos, they are very informative and fun to watch.

[@Taming0fTheShru]

Oh my god!! Life has come full circle for me. Finding a Hussein Nasser comment on Narendra L's video. You two are my idols!!

[@conorlieu4635]

For anyone wondering, apparently a "lakh" or "lac" as he was saying at 29:57 "is a unit in the Indian numbering system equal to one hundred thousand" - Wikipedia

[@eeyore345]

Thank you! I was scratching my head about what that means... :)

[@nagu890000]

Yeah, indian numbering doesnt go up in terms of 3 digits like (thousand, million, billion, etc..) It goes up in 3, 2, 2, ... eg . 1,000 -> Thousand 1,00,000 -> Lakh 1,00,00,000 -> crore ...

[@0anant0]

@ 20:10, he represents 1 million as 10,00,000 as 10 lakhs (see position of commas)

[@snader128]

You should either stick to the metric system or use a global system that doesn't rely on localized counting systems. This would be a red flag for me.

[@RobertPodosek]

Thank you!!

[@tianchengxie1365]

Software Engineer in Google, I barely write any comments, but you really did good job by making those impressive videos. Really appreciate your work, inspired me a lot and I learned tons of knowledge from those.

[@mannion1985]

An additional strategy: have one or more background worker processes generating the base 62 random strings, checking they don't collide and adding them to a stack, creating a buffer of safe random values the app servers can just pop and use. Fantastic video, thank you.

[@youngcitybandit]

Whoa im completely new to soft dev at this level but that seems like a really good idea to me. If someone disagrees please lmk why

[@dElectroBuddha]

This is basically the same as using the zookeper pregenerated 'ranges of keys' presented in the video.

[@mr.mystiks9968]

Sounds like a bizarre strategy since you are constantly generating the random strings, and as you generate more and more you have to store them somewhere? You also have to generate and check that they are unique fast enough to keep up with all the user traffic. Lastly you need those unique urls being constantly generated to constantly replicate and manage which are being popped.

[@gleventhal]

You have the best Systems Design videos on youtube as far as I've seen. Well done! Subscribed.

[@sitanshushrestha2629]

This is way more comprehensive than any other video or article. Even the article in educative doesn’t guarantees collision avoidance. THANKS A LOT.

[@devendrawangikar2890]

THE BEST SYSTEM DESIGN VIDEO SERIES. I've seen other channels, I must say they are nowhere in the competition, because all they do is talk, and but You are doing fantastic work - You have your own study, references, alternative solutions to every system design problem and the way you deliver is fabulous. Simply loving this series. Thank you for everything - Keep up the best work !

[@TechDummiesNarendraL]

Thanks a lot :)

[@himanshusingh-yq3wi]

Excellent... from then i opened account 12 years ago.. i had not commented this world.. hats off bro to the capacity of sharing knowledge... only a few are there.. you are one among

[@ragingpahadi]

This was more clear than Ganga water in Gangotri : ] Thanks again for awesome explanation!

[@AmarjeetAnandsingh]

😂

[@RohitSharma-ku9lt]

i don't what i need to say . i understand everything this is my first system design video of my life . even in my dream i can explain this to any one . thanks sir .

[@petar55555]

Great video, maybe you can start with the API calls, Functional and Non Functional req to make it clear where are you going which will help you define your writes and reads and you could split the app servers to optimize your design. Also, I would like to know more about your approach on the reassigning or cleaning of ranges as those could go fast if you have such a limited number of characters for the short URL and reusability would be a factor as your number will be used completely within the first two years according to the analysis you had.

[@kumarabhishek7877]

The way you explains it seems like I can go back to my workstation and do it now. Hats off to your explanation.

[@marinoduran8077]

Great video! Some helpful feedback so that viewers also know this. Your base62 function has a typo. Instead of doing the regular division of deci and 62 ( deci /= 62 ), it needs to be an integer division so that you are not infinitely dividing because you will continue to output a value as you approach zero. The function will not exit until you are so close to zero that python will basically consider it zero and exit the while loop. So it should read deci //= 62 instead. Thanks again, cheers.

[@jimhiggins5027]

The single slash means integer division in a lot of languages.

[@nskeip]

it's actually not a typo, because it's in Python 2 (see the print operator below) :)

[@DariusAlixWilliams]

I am new to system design and find your explanations clear, thorough and insightful. Thanks!

[@piyushgupta7582]

Very well explained with all pros and cons way of implementations and finally landed with a robust and scalable solution. URL shortener looks simple but very hard and I rarely found anyone else reaches close to your solution and way of explanation. just advice will be better if you explained with UML and ER diagram representations as well.

[@hedonist_traveller]

This is brilliant .. I have seen ppl leave out the distributed part in system design but you nailed it bud 👍

[@cowscanclimbtoo]

The student teacher relationship analogy was pretty clutch. Thank you!

[@aniwigiastuti4323]

oh oh 🙏

[@abc1993able]

For the first time, I don't quite understand, and I watched twice, and now I fully understand, nice explanation.

[@junhuagu3812]

I passed the interview with similar approach two years ago :)

[@sundayezeilo192]

After 5 years, I still found it very helpful. It took me less time to understand.

[@sondn87]

Thanks for your work! One thing I noticed is that if you use the counter method, it may generate multiple new 7-character strings for the same long URL. A malicious client may keep sending requests for the same long URL and the system may run out of keys.

[@MegaKorth]

@Abhishek ChandelRate limiting can stop user for certain period of time only.

[@saikishorer]

Great job Narendra. The way you explained is "very clear ,easy to understand and provided in-depth in short time". Thanks a lot.

[@gauravsaraf38]

Thanks for this video. I really liked how in-depth you went in this one. I know it’s been a while since you created this video, but I would like to ask couple of questions: 1) We could just use the counter as the unique id for the short URL. This has the drawback of not generating enough combinations as each character only supports 9 combinations. 2) To solve the above problem, we are using Base62 encoding on the counter, which will always be unique because the counter will be unique. The question is how do you limit the encoded string to only 7-characters when the counter reaches a certain limit such that the encoding produces more than 7 characters.

[@PatrickRutherford2938]

Like you said we will use base 62 on the counter. Because we're using base62 we will actually have 62 different combinations for each character. Because counting in base 62 would include all these characters [0-9 A-Z a-z]. instead of just [0-9] 2) If you go to the highest number in base 62 which would look like: zzzzzzz this will equal a little over 3.5 trillion. Which is an extremely high number of unique IDs and that would take a significant amount of time to reach. But, after reaching that number then I assume they would probably clear out old data that aren't being used anymore and restart the counter... but if for whatever reason they could not keep up then I think, and I could be wrong, the solution would be to increase from 7 character limit to an 8. Which would make the number of ID's insanely high, over 200 trillion.

[@nicholaslorio2985]

@@PatrickRutherford2938 I think Sarafs 2nd question was more in line with how do we use the base 62 encoding to generate a 7 (or a different specific number) char encoding? Do we simply take the first 7 chars of the encoding we get by converting the long URL to base 62 using the auto incremented id? Wouldn't this result in the same potentially non-unique issue as the MD5 encoding approach? This is something I am confused about as well.

[@digvijaysingh521]

Your way of explanation is simply great. Since we are using counter to generate the hash key. For same long URL requests, you will store different hash keys which will unnecessary increase DB size.

[@FrequencyModulator]

Amazing video! But had to use the captions to understand that the used word is "lakh" and means 100000 :))

[@jessicest]

ah, thank you! I was re-watching minutes trying to figure out what that was!

[@prat534]

Thank you so much for the detailed explanation. Don't need to refer any other sources once you go through this material.

[@DumanIshanov]

How does a counter guarantees uniqueness? So if we convert 2 different long url strings to base62 it may give the same output, but if we convert 2 different counters, then there is no duplicate? how it works exactly?

[@financetruffle]

its similar to binary conversion. likewise when you convert 3 it is 11. when it is going to 4 it is 100. like this the value of 62 is z, if the chars are defined in the sequence of 0-9, A-Z and a-z. Then for 63 it will have two chars. and going on.

[@SundaramANI]

Simply amazing! The level of detail is beyond incredible! Your passion for both technology and teaching comes through so clearly in your videos.

[@kingroger007]

Thanks for the excellent video. learned a lot. A follow up question : if we set expiration date for short URL, how can we collect or recycle the numbers used for those expired short URLs? Thanks.

[@jain007neeraj]

So there are a couple of approaches as explained in Grokking the System Design interview and I am quoting them here : "If we chose to actively search for expired links to remove them, it would put a lot of pressure on our database. Instead, we can slowly remove expired links and do a lazy cleanup. Our service will make sure that only expired links will be deleted, although some expired links can live longer but will never be returned to users. Whenever a user tries to access an expired link, we can delete the link and return an error to the user. A separate Cleanup service can run periodically to remove expired links from our storage and cache. This service should be very lightweight and can be scheduled to run only when the user traffic is expected to be low. We can have a default expiration time for each link (e.g., two years). After removing an expired link, we can put the key back in the key-DB to be reused. Should we remove links that haven’t been visited in some length of time, say six months? This could be tricky. Since storage is getting cheap, we can decide to keep links forever."

[@sombudhya]

Hi, If I reuse one of the expired link again then is it possible that a customer tries to access a very old URL and gets redirected to a wrong URL ( recently used post clean up) . This might cause an issue if , a system needs to provide short URLs and keep the mapping for 10 years . Any solution for this?

[@abdullahahmed7781]

@@sombudhya That is genuinley good question and to be honest I think you may be right... according to this design I cant see imagine thaa a mapping will guarantee no repetiion of url "keys". But if you look at it pragmatically, to run out of 62^7 addresses your service needs to have to go through 3.5 TRILLION Writes which is alot 😂 but.... a busy service *could* use that up very quick..... a simple solution to that problem could be to transition to 8 character spaces on top of precious 7 character, that is over 22 TRILLION URLS! and Im pretty sure for any system to loop back from that counter, so much time would have passed that the distributor of old links would have realised themselves....

[@rishabhjain2404]

Good solution! I just want to add that if the application server goes down, the URL that was getting processed will be lost. To save us from that, we have to implement a temporary storage(with checkpointing) for a single app server or a distributed message queue for multiple app servers.

[@pawel93pe]

3:00 - isn't it enough to keep 7 characters for each shortened url? The base of the url is going to be the same for all shortened so at this scale it would be quite significant waste of space to repeat the same thinh so many times if I'm correct.

[@dilipkumar2k6]

yaa, there are many flaws in the soln discussed. I shared a list of problems in my comment.

[@yasserakbbach7342]

absolutely right we only need to store the unique generated string 👌

[@shreysom2060]

@@yasserakbbach7342 You can have the primary the table link value as primary and unique in that case it won't happen voila

[@RandomShowerThoughts]

this is a legit gold mine in terms of information. Really informative and you get to understand the details

[@grandstarchief3226]

I have been asked this question yesterday. Now I find the answer.

[@ssssahil]

In btech level interview?

[@grandstarchief3226]

@@ssssahil yes....so it's very very easy, isn't it?

[@ganzee6928]

Excellent video Narendra. Well structured, easy to understand, breaking down complex concepts in a crystal clear manner - superb!

[@kckalavacharla]

One quick question. I did write the implementation of base62 encoder and converted a simple text to Base62. The number of characters in the output is proportional to the length of the input. As the input increases, the encoded string length increases. In that case, how can u control the code to generate only 7 or 8 characters? taking the first 7 chars might not be a good idea. What is the solution?

[@RandomShowerThoughts]

good question, I think the cut off might be the only solution, or we have to randomize until we reach 6 characters

[@MySWESpace]

This was a really good video. I read the solution in a different resource but it wasn't as clear and concise as this. Thanks for sharing your knowledge.

[@whatdoyouwant1234]

you rock! a pleasure to watch!

[@abhaysoni8631]

you are great narendra, you explain everything so clearly and easily

[@wishniks]

Nicely covered in a simple, lucid manner. Easy to understand. Thanks for the same. Just one thing. How will the last solution handle the repeated shortening of URLs? Meaning, if the same URL is given twice for shortening, although counter will generate a new short URL, guess the system won't check in DB if the shortened URL exists or not for the same long URL. This can lead to lot of duplication, I suppose.

[@madeofhuman]

From what I understand of URL shorteners, what you described is actually a desired feature, especially in order to provide accurate metrics for the user (eg. if two users shorten the same long URL and want to know which shortened URL had the most visits, each shortened URL must be unique, not the long URL).

[@tushargoyaliit]

@@madeofhuman thanks

[@ChitrankDixit]

This is a very nice explanation to the question that is frequently asked in interviews. cheers.

[@sukanyasukku8459]

thank you, preparing for my aws system design interview :)

[@dayvince3117]

Did you get the offer?

[@happyfreeky]

Excellent tutorial, great presentation. Your confidence and knowledge of the material goes a long way to making this a very informative lesson.

[@Atpugtihsrah]

Just wanted to point out that the output of MD5 hash is 32characters long to be specific.

[@andylinkOFFICIAL]

at that point you can truncate the hash to 7 characters

[@vm1662]

You are amazing! Thank you so much for putting so much effort into these videos. I am loving the system design series right now! :)

[@a1988ditya]

Can you do a video on Zookeeper architecture and applications? Thanks

[@TechDummiesNarendraL]

Sure, when time permits!! :)

[@mohitdiwan22]

@@TechDummiesNarendraL What happens when Zookeeper service goes down, doesn't that introduce a single point of failure or should we think of a backup?

[@IC-kf4mz]

@@mohitdiwan22 What happens when a server goes down? Same thing. Multiple copies.

[@nikhilmishra7572]

No blabbering around. Straight to the point and high-quality content.

[@rustamhussain4139]

@Tech Dummies How will it insure that tiny url length is 7 char? Since we are pasing an integer and what if we passed big integer and that return more than 7 char from base62 method.

[@devpriya4693]

I think this can be controlled in the code of generating the unique code. generateUniqueCode(deci) { const a = [0...9a...zA...Z]; while(deci>0 && len

[@ankitagarwal4914]

It is very intutive and facinating to learn how these systems works behind the scene, i am watching this series as i if i am watching discovery channel.Thanks for sharing

[@ABHISHEKRAI-td9pf]

The explanation is awesome. However, I have a query: if "A" generates "XX" as hash value (common first 7 chars) and "B" generates "XX" as hash value(common first 7 chars) then why can't counter value C1 -> generate "YY" (common first 7 chars) and C2 -> generate "YY" (common first 7 chars)? If yes, then there can't be a "No collision guarantee" + here insertion is done without checking presence in DB.

[@Privacysavingname]

It's been 3 years and no reply... The counter solution does not use a hash. It's just a number represented in base 62 so there won't be a collision if the same counter values are not used

[@mohitpatidar8880]

Best system design playlist and that too in free, hats off...!!!

[@franciscoadan2921]

Great video, thanks for the explanation. One question though. How do we make sure the short URL is always 7 chars long regardless the counter number (1 - 3.5 trillion)?

[@_ityadi]

we can base64 encode the counter numbers and as explained in the video all the numbers between 0 to 62^7 can be encoded to create a string of length

[@srinivaslingamgunta4056]

very clear and great explanation. I especially liked the MD5 hashing part, you clarified it very well for me. A big thank you.

[@vyomrastogi7308]

For the counter approach at 18:48 you said that counter will give unique B62 value and we don't need to check in DB. I agree that counter will give unique B62 value, however we are only picking first 7 characters from encoded string, which might still result in collision. Is that right?

[@netaneldavidlevi]

I was also wandering the same thing until I look again at the code and played with and I got the answer. Notice that the base62 code that he showed is oblivious to the origin url, it is only output the counter in base 62. In this approach we ignore the original url while generating the key and only map counter in base62 as key to the original url

[@lostgoat]

​@@netaneldavidlevi Does that mean our Counter would need to start at a number high enough to generate 7 characters on its own since 1 for example = n which would not work in this case. Also what if the 7 digit prefix of 2 numbers is the same would this cause a collision?

[@anumonto]

great explanation.. u gave all answers to questions which came to mind while watching the video.. superb!! looks like u were reading the mind

[@SanketBarapatre]

Hey Hi, Thank you for sharing your knowledge. Just one question, what if I use the same url to shorten it twice..it will generate 2 short urls..is that breaking a test case..or fine. I think we may need to keep a track of all the long urls also in a set so as to return same shorten url.

[@saikishorer]

Same URl wont be converted to Short URL again. It returns the existing one from DB. If the URL doesn't exists in DB, Then it generates a new SHORT url.

[@karthikmucheli7930]

@@saikishorer So does it have to check from database?

[@SuhailKhan-lo4kd]

@@saikishorer But what if two user try to shoten the same url at the same time , then in that case there will be two entry of the same URL , how can we avoid that.

[@nickang6647]

In depth and clear explanation. Thanks Narendra!

[@adamhughes9938]

FWIW this exact question was asked of me at an amazon interview as well.

[@gsb22]

Did u get in?

[@dyms11]

Your video is so good, you have a natural gift for teaching others. I liked how you explained Zookeeper, like a class with teacher and students.

[@thebackendzoo]

Thanks for sharing such wonderful knowledge. I have a doubt here - Why cant we use timestamp number with prefix of server id and pass it to Base62 function to generate the short URL

[@beyourbestwithdr.prithikac4770]

When you perform Base62 on the Timestamp and prefix of server ID, the resulting string will contain more than 7 characters. Since the tinyURL cannot have more than 7 , you will forced to choose only first 7 or last 7 (as per your choice). This will create collisions.

[@Sevkai]

thank you - that was a great explanation! Imho, the only thing missing, worth mentioning is that app should have thundering herd protection - so e.g. if tweet gets viral we should avoid cache misses

[@andrewpryshchepa6703]

Is zookeeper become single point of failure in your solution ?

[@TechDummiesNarendraL]

Zookeper is distributed system. If one fails you will have one more node to take care.

[@adamhughes9938]

Zookeepr is deployed as a cluster - typically you would have 3, 5, 7 or more zookeeper nodes managing many instances of an app (say 20-200 instances). Zookeeper knows how to synchronize itself so if app instance 5 went down, all zookeeper nodes will eventually (and rather quickly) know about it.

[@anastasianaumko923]

@@TechDummiesNarendraL Thank you for the videos! Amazing job! One question - if we need to restart all Zookeeper nodes in the cluster, how Zookeeper would know the actual count of the counter ranges when it goes up? Do we need to store it somewhere?

[@aditijalkote6356]

Thank you very much for the video. You have explained all the pieces very nicely. You made it quite easy to understand the design.

[@CengizAkarsu]

Thanks for this top-quality content and video. Is that Zookeeper has the potential of a single point of failure here?

[@travellingsLenses]

Even i am thinking of same.

[@neel3297]

zookeeper can be configured as single server or in quorum mode with multiple servers

[@michaelamiro6076]

Great great work here. One thing I kept on wondering about is the size of the data in the database, more so for 'duplicate' entries, i.e for the same url entered multiple times. The database would grow exponentially, getting the same long url but multiple different short urls generated. I guess the fix for this would be purging unused short urls using the expired flag in the data design. Great work!! Great Help!

[@justSomeUserOnYT]

i think the solution here is to provide the already inserted short url? If it's the same long url, we wouldn't insert anything that already exists.

[@SchoolScience]

@@justSomeUserOnYT With the counter approach, we just get the counter and generate the short URL. We cannot check if the long URL is already available as it is costly. My approach to work around this is to allow having multiple short URLs for same long URL. But cache the long URL to Short URL pair. So that if there are multiple requests for the same URL, it will be rendered from cache. At least we can restrict the same user misuse the API. Use LRU cache eviction so that the most frrequently used long urls will not be duplicated. On top of this, there should be a rate limiter to avoid DDos.

[@candysnatcher6916]

Hey I really like your content, loads of respect for you. I'd like to add to this that you can very easily use a combination of unique values and dropDups at schema level in mongoDB (NoSQL) to tackle the duplicate issue like this: new Schema({ shortID : { type : String , unique : true, required : true, dropDups: true } }); Unique alone is very powerful but in very rare cases it might not work if index is not refreshed, for those situation dropDups comes to the rescue. dropDups means that MongoDB will "drop" any queries which try to create a record with a schema value that already exists in the database. For ex: if you already have a record with name : bob and you try to insert another record with name : bob then this insertion will fail due to dropDups.

[@KrishnaPatel-qu6nh]

i like this information...thank you for sharing it

[@PritamKumar-jh4te]

Your explanation of system design is very succinct and easy to grasp. Thank You.

[@varunmadhavan3227]

Thanks a lot for this! A few interesting questions: 1. How are the counters/addresses recovered after a 5 year cycle or when all of the 3.5 Trillion hashes are exhausted? 2. Does the zookeeper come with redundancy assurances? Or is it a bottleneck for the entire service?

[@vippan]

I believe we can remove the entries which are used or expired.

[@arthamsreenivas8858]

thanks a ton Narendra, this looks a bit long video but it is very clear after watching the whole video.

[@keatmin]

Thanks for the great tutorial, one qs I am wondering how to ensure we always generate 7 base62 characters with random integer/long as input. I tried with the lib I have, the number of characters is not quaranteed.

[@DjVilez]

Im also wondering about this

[@kentonWang0826]

how about taking it as a digit system with 62 characters, that meas we have abc....xyzABC...XYZ012...789 corresponding to 0...61, so an input 0 will have output aaaaaaa, and input 1 will be aaaaaab, so aaaaaba will be 62. For numbers from 0 ~ 62^7-1, we can have exactly 7 characters.

[@AI6NET_link_shortener]

Great analysis of the system design of a URL Shortener, we hit several of the caveats here especially when building the defender application, however; our job is far from done.

[@chopradimpy]

WIh final solution, I understand that if 2 requests come with same long URL then there will be 2 different tinyUrls assigned to each request?

[@Chi-HacksAndDIYs]

It is OK to have the scenario you described. You could have two different shortURL point to the same longURL, but not the other way around. In the example he provided, two different longURL (said longURLA and longURLB) resulted in the same shortURL (us.com/GA29x51). As a result, when shortURL (us.com/GA29x51) was referred and translated back to its corresponding longURL, redirection could go either longURLA or longURLB, which is not deterministic.

[@IC-kf4mz]

Yes. Does it matter to have different short URLs? Depends on the requirements.

[@NiloGarciaSilveira]

Thank you very much!!! I had this question in my Interview and this vídeo was a great help!!!!

[@davudsafarli5356]

There is one thing that bothers me is we solve problems by adding new technology (ZooKeeper in this example) and saying that it's highly available and never thinking about the case where the ZooKeeper fails :/

[@MohammedDawoodAnsari]

Correct! for that we need to design a system failover, and the same is also applicable on DB unavailability.

[@srabontibandyopadhyay5052]

Thank you so much for explaining so well a seemingly simple system. It would good to speak a bit about data purging of url, and the business requirements of how long the tiny url is valid.

[@neelabhsingh1986]

Thanks for the URL shortener system design Tutorial. As you mentioned to avoid collision we can concatenate the counter value range [1 - the size of data type] if we consider counter as Integer then it will be 4 bytes, So it will increase the key size which is currently 7 characters long. Suppose if we append an increasing counter to each input URL. I think still it does not work because we are taking only 7 characters after converting from base64 or MD5. I didn't understand your point. Could you explain more it. Thanks

[@studiousguy8138]

*Did he ask to concatenate? I think you can just store mapping of input long url to counter value in your DB that should be good enough and will guarantee zero collisions.*

[@rishabhjain2404]

@@studiousguy8138 but how will you map the hash in the URL with the counter. When the user uses the short URL, how would map it to the counter in the DB

[@shreysom2060]

@@rishabhjain2404 Hi! Rishabh it will not map to counter it will take it from database you can make the unique id as primary index and then while fetching it will be faster. SO when user click the shorten URL it will take the unique id fetch from database via primary index and as long url will be stored with short url you will get it.

[@hoyiya]

@@shreysom2060 I think the question is, multiple integers can be mapped to the same hash using Base62. How to retrieve the long URL now if you only have the hash, as it may be mapped to multiple long URLs?

[@shreysom2060]

@@hoyiya i think he mentioned that every shortened url will be unique right,in that case it can act as primary index in rdbms and user can fetch it right? I will watch the video once again though

[@abhishekgupta4360]

You made it so simple to understand. Great work

[@messerschnitt7943]

It seems to me that you didn't solve anything by adding the Zookeeper... Then the Zookeeper becomes the single point of failure, right? I.e. what if the Zookeeper is down? Also, the counter will give a bit strange short URLs, in particular they will be successive. Which could perhaps even be a security issue in some circumstances (it will be possible to guess valid short URLs).

[@nickitataranov7497]

We can consider ZK as a single service, but it is not a single server. It will remain available as long as majority of the under the hood machines stay alive. I'm pretty agree with you about "non randomness" of the resulting urls is a problem. Because your service is pretty DDoS-friendly :). But you can made this approach better almost for free. Say, you were given the range [2^16, 2^24]. You can generate some random number M from [2^16, 2^24]. And let your counter currently having value C. Then you can use (C xor M) instead of just C. This makes your urls not so much consecutive (you should not forget about the case when C equal to M :) ) Obviously, you can use more sophisticated hash functions as long as they are injective.

[@keshavrastogi5005]

Very clear and detailed explanation. Thanks a lot paaji!

[@kaushalgour]

Amazing explanation. Sir could you please add IRCTC System design.

[@TechDummiesNarendraL]

Sure, Some time

[@banerjeesaikat6]

IRCTC please

[@aloksharma3246]

@@TechDummiesNarendraL irctc on demand

[@vedant9173]

This channel is like a gold mine

[@bdc225]

Awesome video! Something that is still unclear that every 'Build TinyURL' video/article doesn't explain is an example of actually generating the url. So we agree that we use the distributed counter and then base62 encode it. However when you encode a somewhat large number (much less than 3.5 trillion), you get a string that is longer than 7 characters! Ex. 1234567 ==[Base62]=> MTIzNDU2Nw That number is not even 1.5 million and we're way over the 7 character limit. If we truncate that and only take the first 7 characters, then we have the exact same problem as truncating the MD5 hash: collisions! Any help would be appreciated thanks!

[@amanand727]

probably your conversion is incorrect, in my code b62(3000000000000) = QODkgKc which is in seven characters.

[@Atharva555]

Great video man! loved it! your way of teaching was phenomenal keep up the good work man

[@mariakhan628]

How are we making sure that base62 conversions gives a 7 length unique number

[@fb_a]

Wow...!This kind of explanation with such good analysis is really awesome!

[@nilanjanmhatre1960]

hell lot of metrics. is it expected in an interview?

[@Shogoeu]

Yes, but once you start talking and they see you know your stuff, they will interrupt you.

[@charan775]

Not for entry level positions. For positions like sde2 , sde3 these are the deciding factors

[@harrishm5841]

Well explained. Thanks for making these system design videos! These are dope af.

[@harishaseri]

Hi Naren , I really liked you videos . such a detailed and simple explanation for every system design video. Thanks for this selfless effort man

[@dhanyaravindranathan931]

The best explanation on Bitly..Thanks much!

[@shawnchen8963]

Amazing that I met this question for amazon on site interview just after I watched this on the same day.

[@adamhughes9938]

How'd it turn out? Offer?

[@nagamallareddy]

This is an awesome video bro, I love the way you explained the approach, design & conclusion.

[@Anilche]

Very good explanation and succinct way of putting the knowledge across. Thanks for sharing your knowledge!

[@ajoloteenvaso]

increible!!! gracias por compartir hermano!!! mis respetos, saludos desde Mexico.

[@gyanasahu1006]

One approach to collisions can be to use a counter along with a special symbol to act as counter. 1. In that case you have to make sure that the hash doesn't use the special symbol. 2. The hash portion of the url will serve as partition key and counter will serve as clustering key (Cassandra terminologies).

[@pinakadhara7650]

Can you explain a bit more about this?

[@avtaar3041]

we should use queue

[@arshdeepkumar2586]

Great, I didn't know about url shotner could be this complex, Thanks for Video Sir

[@kumarc4853]

one of the best videos for generating uniqueIDs, thank you

[@大盗江南]

Thank you Naren！😌 amazing video as usual

[@UmangMundhara]

This was wayyyy tooo good. Fully detailed and amazing explaination. God bless you sir.

[@vm1662]

Great video! Thanks for all the work you put in each of these video!

[@mdnayab5839]

Best explanation on internet, Thanks a lot 🙂

[@hvp9111]

This is nice informative video. I had implemented Scheduler Server in past where zookeeper was in BETA on AWS cloudformation. Which worked pretty much but it is nice to use available services since it is such a pain to maintain separate code base.