5. Scale from ZERO to MILLION Users in Detailed | System design interview: Scale to 1million users

In this Video, i talked about scalability, and how to grew a website to handle 1 million users. I cover load balancing, content delivery networks, database master/slave replication, horizontal/vertical sharding, and more.
To appreciate the work, pls join:
/ @conceptandcoding
#cdn #sharding #hld

[@irbansin]

Beautiful explanation

[@anojk1]

Great content! It would be helpful if you could include any references you used, such as articles, blogs, or books in descriptions

[@vishalwagh478]

I seen LLD & HLD in language but couldn't grab it.... because there directly they were discussing on the solution. But the way you went from 0 -> N that was amazing. I worked in monolith application but couldn't understand how to approach to the best change..... Hats of to you....your teaching method.... More Power to you !!!

[@jackeysharath7077]

Thank you for your service. Great job.

[@ConceptandCoding]

Thank you

[@harshgupta2710]

in love with all the videos !! curiously waiting for the consistent hashing topic , please drop it soon 🙂

[@ConceptandCoding]

:) sure thanks

[@sohansingh2022]

Beautiful explanation. Thank you!

[@ConceptandCoding]

Thanks

[@renuroy4826]

This stuff is really really cool, and Thank you so so much for sharing your knowledge. Appreciated. :)

[@ConceptandCoding]

Thanks

[@itsds.]

Thanks for the great video, whats difference between Horizontal Sharding and Indexing?

[@tech_knocking8440]

One of the best video on step-wise scaling!

[@ConceptandCoding]

Thanks

[@anonymous34565]

Liked the step-by-step scaling explanation. Pls make more such videos so that we can find them all here.

[@ConceptandCoding]

thank you

[@PadamAgrawal]

Maja aa gya , Love this video (y). Thank you so much

[@ConceptandCoding]

Glad to know that

[@prashantgupta387]

Great content, keep it up You have earned a new subscriber

[@ConceptandCoding]

thank you

[@anjumanislamiachhatarpur7093]

That is really cool to see such incremental approach. Appreciate this video. Could you please also make an extension of this video mentioning, Elastic Search Engine.

[@ConceptandCoding]

Noted

[@mahesh_kndpl]

Amazing content + highly recommended to watch it.

[@ConceptandCoding]

Thanks

[@afeeramehnaz1852]

Amazing content☺ on LLD and HLD. I was finding difficulty in answering about the design patterns in the interview, but the roadmap of LLD and HLD explained here really helped.

[@ConceptandCoding]

thanks

[@bittu225024]

Very nice tutorial. Provided detailed information in very short time. thank you

[@ConceptandCoding]

thank you

[@shivanshugupta5702]

Best video I have ever seen. Thanks so much

[@ConceptandCoding]

Thanks

[@willtank1635]

So clean and sophisticated explaination

[@ConceptandCoding]

Thanks

[@omkardeshpande7482]

Honestly, this is one of the fine grained system design lectures I have ever seen! I paid for udemy courses, looked for freecodecamp videos, medium blogs, but this is the finest of all gems! Thanks a lot! You deserve a lot lot lot!

[@ConceptandCoding]

thanks

[@harishaseri]

Hey , Thanks for a great content men. I have one doubt . I have few doubt thought. 1 - Scaling of db and scaling of app server is same right ? . I mean for both we use Horizontal scaling and Vertical scaling right ? only difference is implantation wise . 2 - Do you think Master slave is kind of horizontal scaling only bcz there we have replication of data(Slaves have same data as Master) and in case of app server we have replication of code on server and multiple servers are running them ? 3 - If we have master slave then what is the need of the Sharding (as it is also kind of horizontal scaling only) 4 - Why don't we have Load balancers in front of the Slaves in master slaves like we have in front of the App server ?

[@shobhitarya1637]

It's a great learning video. Thank you. I have seen in interviews and in other HLDs that they add queue to store requests when load increased too much, so that any request do not get missed. Could you please make a video on this concept explaining how this is done (whether the request thread wait on queue) and how response is send back in detail.

[@ConceptandCoding]

noted Shobhit

[@shiprakesarwani]

Thank you, really helpful video.

[@ConceptandCoding]

thanks Shipra, pls do share it with your connections

[@jspnser]

Really good explanation. Thank you

[@ConceptandCoding]

Thanks

[@MrBaaz-do9ny]

crystal clear explanation

[@ConceptandCoding]

thank you

[@SanjeevKumar-iz4rf]

You are the best Tech Proffesional.

[@ConceptandCoding]

Thanks

[@goyaldeekshant]

I was from an electrical branch, SO I didn't do that project, nor having learned of any computer science project. But now I am gaining it

[@ConceptandCoding]

Thank you

[@chirag_ccp]

Nice Video to get a summary of all concepts

[@ConceptandCoding]

Thanks

[@ravitandon9351]

Keep up the great work:)

[@ConceptandCoding]

thank you

[@sharkx0183]

Question: Suppose if there is huge data in DB and there needs to be divided into two shards, Will both the shards have their dedicated node, or both the shards will live in single node ? Also can a single node/vm contains only one shard or it can contain multiple shards

[@ehtashammazhar3518]

Bhai you are working great. I must tell you. I always recommnd your channel to the people who wants to learn Lld and DP. One note : when you are recording DP comes up with real time use cases . And where can we implement these DP concept in the hld questions asked in interview. Like can we implement any dp in parking lot.

[@ConceptandCoding]

Got it thanks for the feedback. I will do that .

[@chineseboycott829]

I think what you mentioned about sharding here is not actually sharding its partitioning, multiple partitions can reside in the same shard also. Sharding is always on db and partitioning is on data content (rows and columns). Different instances of db deployed are called shards. CMIIW.

[@NeerajSharma-eq8ch]

Hey @ConceptandCoding, How consistent hashing will help to reduce the tree structured sharding, I mean every shard has different data, redirecting the request using consistent hashing to a wrong shard won't help right?

[@amituchat4865]

Question - Step 4, How does the requests from App servers farmed out to multiple dbs, do we need internal load balancer between App and db servers? Appreciate it if you could add some explanation notes. Great videos, love it.

[@ConceptandCoding]

I will cover this in my next HLD video, it gonna be very interesting video

[@lokeshmehta9591]

agar hum sharding primary key ke base pr kre to jab bhi tree sharding problem hat jaegi ?

[@chetanm898]

Super

[@user-uc7zm3qn3p]

Summary: 04:25 Scaling from 0 to 1 million users 08:50 Implementing Load Balancer 10:00 Database Replication to handle system failures 13:15 Using cache and CDN to improve performance and reduce database calls 17:40 CDN is a solution to reduce latency for global users 22:05 CDN enables advanced technology with features to optimize load on DB and increase security. 26:30 Messaging queue helps in asynchronous processing of heavy load operations. 30:55 Fanout, Topic, and Messaging Queue are important concepts in message exchange 35:14 Horizontal and vertical sharding are methods of dividing data in a scalable system.

[@engineer_jaatni]

Can you please make a separate detailed video on Messaging Queue? It will be really helpful.

[@ConceptandCoding]

It's in my bucket list

[@MohdZaid-gi2mo]

Beautifully explained words aren't enough for your effort taken, thanks! Also i have doubt pls reply, i wanted to know how long does it take for a single software engineer to implement HLD all these mentioned concepts in real time, how long does it take to scale an app? Lets assume it's a food delivery app

[@ConceptandCoding]

Sorry I did not get the question, could you please elaborate when you say "how long it takes to implement"...

[@MohdZaid-gi2mo]

@@ConceptandCoding I'm curious about the time required for a lone software engineer to implement a high-level system design project, incorporating all the mentioned concepts from the video, in real-time, utilizing AWS. Additionally, I'm interested in understanding the timeframe needed to scale an app, specifically considering a food delivery app as an example." Do you also take classes? Webinar/course for the High level system?

[@rahulnaqvi994]

good job young man

[@ConceptandCoding]

Thank you

[@piyushmathpal4244]

Hello Question: How do data centre communicate with each other and avoid consistency in a system? Lets say at time t1 a write req goes to India DC(data centre), it then goes to US DC for replication. If this is a sync communication, user's write will get latency delay as sending ack back to user will take a while. To avoid latency, we choose an async comm btw data centres and series of events happened: 1. Since using async medium, Queue has data for US DC to be replicated. 2. Ind DC goes down 3. At time t2, t2 > t1, read req(read1) comes for USDC. At time t2, data is not replicated to US DC yet. 4. Say (read1) need to serve a heavy query result and supposedly we will have a cache miss. How to handle such case? PS: We can have multiple DATA CENTREs, but the crux here is read comes before replication to other data servers and primary/master(the server with latest write) goes down.

[@ConceptandCoding]

Thanks Piyush, for this detailed explanation of the usecase. Let me read out the paragraph from "Design Data Intensive Application" book "Often, leader-based replication is configured to be completely asynchronous. In this case, if the leader fails and is not recoverable, any writes that have not yet been replicated to followers are lost. This means that a write is not guaranteed to be durable, even if it has been confirmed to the client. However, a fully asynchronous configuration has the advantage that the leader can continue processing writes, even if all of its followers have fallen behind. Weakening durability may sound like a bad trade-off, but asynchronous replication is nevertheless widely used, especially if there are many followers or if they are geographically distributed. We will return to this issue in “Problems with Replication Lag”. " So this is single datacenter with Single Leader and multiple follower setup problem, but same applies to Multi data centre with multiple leader too, So its a trade off, and it does exists :) Hope this helps.

[@tusharkuwar4]

Very Nice !!

[@ConceptandCoding]

thanks

[@yashpalsingh3710]

Great content 👍👍

[@ConceptandCoding]

Thank you 🙌

[@RishabAggarwal-ss7ry]

I am watching this from New York but There are so many ads after every few minutes. Its so so frustrating. idk its by the author or location but it needs to be fixed. Content is good as always.

[@gunjanroy8876]

It totally depends on the region You can use brave browser or add blocker to bypass adds

[@ishaankaustav727]

why there is a need of join query in sharding ? we have same columns in every tables , so why we have to do join ? we can simply make query on respective shards ?

[@rambeersharma2091]

Great

[@ConceptandCoding]

Thank you

[@ANSHULGUPTA880]

1)Who will take care of making slave DB master if primary fails..? Is that db cluster itself does the job? I believe replication is also taken care by db cluster by itself. 2)In general we give 1 url in datasource bean of springboot property file for a relational db. What would be URL in springboot application if there are multiple replicas dbs and how can we change primary in that case?

[@ANSHULGUPTA880]

2nd was asked in an interview recently

[@waytolivelife]

1.) when db are run in docker conatiner we give these configuration in files.. exapmle is stated below MongoDB shell version v5.0.8 > db = (new Mongo('localhost:27017')).getDB('test') test > config = { "_id" : "my-mongo-set", "members" : [ { "_id" : 0, "host" : "mongo1:27017" }, { "_id" : 1, "host" : "mongo2:27017" }, { "_id" : 2, "host" : "mongo3:27017" } ] } 2. for the second question ..we can create these files in spring also..in the application.yml files we can give . spring: devtools: restart: enabled: false livereload: enabled: false data: cassandra: contactPoints: localhost protocolVersion: V4 compression: LZ4 keyspaceName: xyz jest: uri: localhost:9200 *jest is elastic search client and db here is cassandra

[@ANSHULGUPTA880]

Nice work buddy. Can you help me with these doubts (as soon as you can, I have an interview scheduled, thanks in advance)::::> Suppose If I have a trading application, and want to reduce the latency for geographical users, lets say a user is in US and other is in Singapore(APAC), and third in london(EMEA), so I scale my appservers, make application replicas and place them in different geographies, so 1) where will I place the gateway? Gateway is generally 1 right? 2) For applications to interact with DB, where to place the db (will it be better to make distributed db replicas placed across geographies )? 3) what type of db should I use who can sync ups b/w replicas? (looks like consistency will take a hit since db sync up will take time) 4) How to handle concurrent orders( trade order may have 1 unit left, and all users are trying to book that 1 unit, db isolation may not work here bcos they are reading different replicas OR use active passive strategy where write is always on primary db, in that case where to put primary db as far uaway users will again face latency in write operatons)

[@ConceptandCoding]

1. Instead of relying on centralized gateway. We can use Geographically distribute gateways like anycast gateways, CDN. 2. Yes, we need to make Distributed DB replicas placed across geographies. 3. Its a common scenario that distributed DB can sync up with each other, for example ORACLE uses OGG (Oracle golden gate) to transfer the committed data to distributed db. But its true, as even oracle mentioned in their OGG document that, it might take max 15 mins in the worst scenario. 4. Active-Passive is one solution and yes for far away users, latency issue will be present. But Active-Active also can be used with "Distributed Locking mechanism" (as part of this mechanism, lock on same db instance put on all active nodes)

[@ANSHULGUPTA880]

@@ConceptandCoding Thanks for such a quick and perfect response. I will be asking many more questions this weekend :) Also, Do you suggest any cloud certification and cloud related courses which can make us pro on these topics?

[@yashsingh3819]

Hello Sir, I want to present my idea in Tata Imagination Challenge. Can you please guide me on the same? If possible, please make a video so that it will be helpful for all of us. Thankyou.

[@adityasadhukhan8438]

I had a question, at your level like sde3 do you also have to work on frontend side also or more on backend architecture only

[@ConceptandCoding]

I am a backend engineer, never worked on frontend.

[@adityasadhukhan8438]

@@ConceptandCoding you are really lucky that you don't get frontend tasks, whenever I joined dev job I was expected to work on frontend which I really don't understand, I knew nodejs that time but no such frontend expertise I had, what should I do so that I can get only backend related jobs even after I crack any interview

[@ritwikgupta7540]

Whats the use of vertical sharding, it is just splitting the table on the columns, but the number of rows remain same, which kind of means we are creating multiple tables from one table, doesn't that make the design even more complex?

[@Satishsingh-yc9bs]

A user profile can have say 20 fields, out of this say only 5 are most sought after/used frequently. Now making 2 tables, one with 5 column and another with 15 will make the first DB lighter and faster in response.

[@akshatshah6413]

do you refer to alex xu?

[@nareshsaharan2498]

There is one concept of Service Discovery also. This should have also been added into the diagrams.

[@ConceptandCoding]

Noted

[@arghyadeepmandal4458]

Great video 😇.Hope there would have been some predefined steps to reach 1 million followers/subscribers 🥺

[@ConceptandCoding]

Yes first step is to share it with your connections :)

[@bhumithakur9967]

Please make a separate video on database scaling .

[@ConceptandCoding]

noted

[@shubhamjain1153]

Heyy, can you please explain how data will be replicated between databases situated in different geo. location; like what is the connection type between them.

[@ConceptandCoding]

Ack

[@anonymous34565]

Pls also start videos of spring boot framework.

[@ConceptandCoding]

noted Alok

[@lokeshmehta9591]

so you are saying first step should be replication than vertical scaling .......... but I thinks it should be reverse.......

[@vinoths7140]

Bro, when can we expect English subtitles to this video?

[@ajeetchouksey1167]

How to achieve Master slave replication in real time?

[@mahendratonape27]

Consecutive write and read operation for same data will result in failure if read is from replica even if replication gap is 3-5milli second

[@ConceptandCoding]

:) it will not fail, generally the data ia also cached. if weite and read happens almost instantly (in a gap of 3-5mill sec) it will mostly get served from Cache. So no issues

[@DeepakLalchandaniProfile]

Where to learn LLD HLD from ? and how to scale it to millions of users

[@ConceptandCoding]

pls check the video Deepak.

[@DeepakLalchandaniProfile]

@@ConceptandCoding im struggling with a coderanch problem . Can i emal it to you ?

[@shankysays]

Started HLD today. Hope to finish hld and lld in a week. Wish me luck

[@ConceptandCoding]

good luck buddy

[@AkshaySharma-bg3oj]

Hi can you pls share these kind of notes :) (onenote with us)

[@ConceptandCoding]

Sure, i will share from future, as i haven't saved earlier ones

[@arundoss6785]

Please post videos in English

[@karthikp1089]

English would be very helpful

[@ConceptandCoding]

Yes all latest videos are in English only

[@vinoths7140]

Can you please make this video in English if you have free time?

[@ConceptandCoding]

Sure i will try

[@vinoths7140]

@@ConceptandCoding Thanks a lot.

[@radhianand5647]

A humble request to speak in English

[@ConceptandCoding]

Hi Radhi, yes i have switched to English for all my latest videos

[@ajaysabarish9645]

Please speak in English or at least add english subtitles

[@shubhamrao5952]

Hindi seekh lo

[@ManishKumar-zt5sk]

Does CDN have server instance or only static data ? where does the client request goes first ? - CDN or actual server

[@ayushkaushik9637]

Manish suthar?