What is CAP Theorem? Degrees of CAP theorem, use cases | System Design Tutorials | Lecture 20 | 2020

[@Learnfundamentals-lg3dc]

One important thing here to avoid confusion - the CAP theorem is stated for Distributed Data Stores. Ant why we can't sacrifice 'P' because no distributed system is safe from network failures, so we need a way to ensure the system continues to operate i.e the system has to be Partition tolerant

[@saumitrasaxena8470]

you rock !!! I am looking for Architect roles and as a Lead/Developer was not aware about Architect.You gave me confidence..Thanks a lot !!!

[@aprajitakasyhap8430]

Very well explained with examples. Easy to understand these concepts. One of the best content on System Design i have came across. As a beginner in System Design it really helping me to connect with day to day challenges in software system.

[@sumeetgaur]

You are doing great help to job seekers. Keep up the good work!

[@akashagarwal6390]

Although i m pretty new to CAP theorem, but the introduction given during the first few minutes or so by you, goes to show simply how greatly you know about it. As i have been reading about it of late & i was like ok, this is another debatable topic, as ppl are fighting over getting their concepts rightly understood. But i think this video nails it...

[@sudocode]

Thanks a lot Akash. Much appreciated. All the best for your system design prep 🙂

[@usavedileb2201]

Very good attempt to explain the concept. Was very clear, thank you. At 11:50 you mention that in order to achieve some level of availability, B can continue to write even when the communication is Broken. But wouldn't it lead to loss of Consistency because after T2, a read from A will still return x=1 and not x=2 because that update has not been received by A?

[@RandomNagarsoge]

That is what she said. We cannot have both together if there is a partition.

[@akshayshah8264]

Looks like the example was not accurate. Instead of allowing B to only accept writes..you can let it do both as before. However, chances are fraction of requests will get accurate value if they are redirected to the same instance. But once the partition is recovered, we can have eventual consistency. Note that there is 100% availability even though there is lesser degree of consistency.

[@kunaldas9284]

@@akshayshah8264 you're correct...the example is not correct because if you're favouring 100% consistency but allowing B to take write requests, then any read requests served by A will return stale data i.e. not consistent. Therefore we could rather say that allowing B to take write requests and also serve requests (A is already serving both read and write), makes the cluster Available but NOT Consistent.

[@KeshariPiyush24]

I think there is no such thing as some degree of consistency as the technical definition also says that all the reads after the latest write should give out the same result. You either have full consistency or none.

[@siddhantjaiswal4231]

At 11:50 how will all the reads serve consistent data? 'A' wont be aware of the writes on 'B' bcoz of partitioning.

[@saanikagupta1508]

Example at 11.57 - It won't be consistent right even if the system is taking all the 100 writes. Because until they reconcile, node A might have incorrect data, because what if the latest write happened in node B and not node A. I didn't understand why we considered read to be only taken by one node. It's anyway inconsistent (eventual consistency though). So Node A would also give wrong data, and Node B will also give wrong data.

[@sudocode]

Eventual consistency is a type of consistency.

[@architsaxena6741]

Great explanation of CAP. Made it so easy to understand.

[@parveensishodiya5768]

Today i understood CAP 😌 Clear explaination with real world examples Thanks for your efforts to making things simpler

[@tanu6011]

By far the best explanation on cap theorem available. While doing a project on distributed systems I came across many explanations of CAP theorem, none of them explained it as effectively as you did.

[@sudocode]

Thanks Mrityunjay. We appreciate your letting us know of our quality very much ☺️

[@kapilrules]

Best description of difference between consistency and availability. I can not thank u enough Yogita

[@TheMickeyFats]

This is the best, most thorough explanation I have seen. Thank you!

[@surbhitamrakar1638]

This is really the best explanation on CAP theorem!

[@shubhamjain6471]

Hi Yogita, thanks for amazing videos. I had a doubt in degree of availability at 10:55 - if A gets x=3 at t3 and B gets x=2 at t2 since both of them can perform write operations then wouldn't the system become inconsistent as A never had the updated value i.e. x=2? So both availability and consistency are compromised. Please let me know if I'm not thinking correctly.

[@medharai5914]

Both A & B can perform write operations but since only A could perform read operations so i'll always get latest value of x it wrote hence we can say that system is consistent with certain degree of availability in regards to write operations. Consider an example that you are using a shared google doc with one of your friend. Now both of you are updating on the doc & your wifi goes down but the app instead of crashing (ie no availabilty) still allows you to write and whenever wifi goes up the doc is synced with your friend.

[@Ajay-zmngn]

@@medharai5914 it'll not always be latest value of x, although the system will still be called "consistent" as only A performs the read operation and is the sole node responsible for responding with the read requests.

[@akashagarwal6390]

Also, it is really good to see mam that you provide links to references

[@sudocode]

Glad you find it useful Akash!

[@parvezmulla3324]

Clean and Concise. Also, the Links are helpful too!

[@suhelalam2686]

In the degree (CP) and node B is writing and eventually read happens in node a it will return the wrong data right, how it is consistent than

[@gurupreetsingh8347]

hello sudoCode, can you explain how distributed transaction works and 2PC and 3PC in terms of Distributed Transactions ?

[@talesbytejaswini]

When there is network problem and both dB can't communicate and we are allowing reads on one(b) and reads and writes on other(a) , we don't get latest data from A right? Until and unless reconciliation happens? So this will also lead to inconsistency?

[@vipsclassicalboy]

@sudoCode #sudoCode, kindly confirm on this doubt. If this is the case, we can allow read from B also, or there is some policy to choose which writes can be allowed on B.

[@Kavishkhullar]

Hi yogita, you mentioned if we allow writes on the node b we will have comparatively high availability than completely turning it off. what if latest write is on node b and later read is happening on node a? in that case read value is wrong. so is it fair to say in case of consistency we can have wrong value as long as it's consistent in all our reads?

[@shyirtt]

I had same question as well.

[@ayushjindal4981]

Same question.

[@nimishgupta8357]

Same question

[@anuraggoyar]

Man I had the exact same question and I came to comment section to ask but you already did.

[@manishdadheech8440]

How do we come to know which node giving us correct result, and How can we decide at run time which node should accept write also ,in case of CA?

[@gauravgupta5530]

In morning, I saw your CAP video not the theorem ;) , and thought that when the next video will come. and see here I am watching your second video. Thank you for such amazing content.

[@sudocode]

Cool, thanks

[@hermayneMUNSTER]

Thank you for the explanation! Would be nice to have more graphics displayed like in the beginning of the video vs the white board. The markers seemed dried out and hard to take notes due to rewinding and squinting eyes to see what values you're updating.

[@indrajeet5000]

Great explanation as always. At 2:51 there's a mistake in reading about availability defn.

[@vinodkumargornalm.6647]

You mentioned that if I choose to compromise on availability, then the system can be consistent even when the two nodes are partitioned, one of them is accepting only reads, and both are accepting write requests. But if the system is partitioned and the other node has received last write, how can this node answer a read request while being consistent as the other node could have the most recent write.

[@sudocode]

The other node cannot answer correctly and that is what makes it not available.

[@vinodkumargornalm.6647]

That also makes it inconsistent, right? Because it gave the incorrect response as it couldn't communicate with the other node that had the recent update

[@rv6569]

This video s little confusing. But she has attempted sincerely.

[@ankitpahwa8571]

Yeah , same doubt. Even if Availability is compromised , I can’t understand how Consistency is maintained .

[@vinodgornal6159]

Learned that by giving an error, the system is still consistent. But, my first question is still valid. If the two nodes are partitioned, then both cannot accept writes, right? The possible case, per my understanding, is a scenario where Node1 is master and accepts both reads and writes. Node2 is slave and accepts only reads. If the system gets partitioned, slave stops responding to reads. Master remains consistent only it can writes.

[@java4raju]

Very well explained I watch all three video but still not understand .. please answer .. can we have system which is highly available and highly consistance without being Replicated database that is Partition tollerant ?

[@siddharthmagadum16]

did you get your answer, pls share if yes....

[@phanuwatwattanapraya7998]

This is very useful video, you have the explanation in very detail Thank you so much

[@rishabhjain8319]

So if system is P is broken as C will not be there at all, it so interconnected. Can system will be CA if P is compromised. CA is myth if P is compromised.

[@PradeepKumar-db7pe]

In case, system to be consistent. And partition happens, then how come the 2nd system ll come to know, it has not the latest value?

[@PradeepKumar-db7pe]

Ohh ok, May be its tracking about partitioning. If someone reads and there is partition then return 'value many be incorrect, i ll not return in this case'. i.e Avail forfeits. I.e can't have a system CAP.

[@PradeepKumar-db7pe]

CAP- Can't provide 100% consistency and 100% availablity and system is 100%fault tolerant

[@sudocode]

For a distributed system.

[@adityasingh2632]

great work. i have one small suggestion: please use a dark marker.

[@AbheyBhatiaCloud]

Kindly explain in terms of different Dbs available today, how they follow CAP theorem and which ones falls in CP, CA or AP categories.

[@amruthlakkavaram5218]

Wonderful video, really enjoyed it and understood well

[@dianemergui]

Girl - great video, super clear and concise!!!

[@sudocode]

Thanks girl 👧

[@UECAshutoshKumar]

Thank you

[@avinashb4485]

Can you please explain what happens if we can have different modes of communication between servers for replication/consistency? As in that case, even if there's one network failure between nodes, other modes can be up and running and thus facilitating consistency. Basically, it's expanding on "Availability" of communication methods between nodes.

[@avinashb4485]

Damn it! You have explained this one at the end. I commented without watching full video 😅 I'm not gonna delete this comment as this would be a lesson for others as well to finish the video first and only then comment the doubts 😄

[@sudocode]

😅😅

[@harshitpandey7521]

If the partition network breaks down, then the system would be available but we would not be able to maintain consistency as writes/updates couldn't be updated due to lack of communication channel, right? Then here we are not able to maintain two(C and P) of the three features. How is CAP theorem applicable here then?

[@sudocode]

Because you are able to maintain C and P. That's what CAP theorem says! At most 2 out of 3 properties can be achieved.

[@devpriya4693]

@@sudocode But in the case @harshit pandey , C & P wont be maintained.

[@vinodkumargornalm.6647]

If the system partitioned, you can compromise on consistency by providing the lastest update that you have not caring about other nodes, or you can compromise on availability by simply waiting for connection recovery and then responding after ensuring that you have the latest update received by the system. Right?

[@BhaktiSong1987]

if partition tolerance not occur then till that moment all cap are there but as per definition only 2

[@ritugupta3367]

Could you please create a playlist for LLDs

[@eddanapudir]

Thanks a lot. Concept is very well explained.

[@sonisaloni2794]

Some quality content right there. Great job!

[@jinalkumarpatel5138]

Good going. In distributed system some of the problem solve by redundant component and it is upto the business case to tolerate that cost or less in terms of overall system.

[@sudocode]

Absolutely!

[@saurabh5981]

Appears, you started just post 7.51..... thanks for educating

[@dhanushkajayathilaka5711]

please explain what is exactly meaning of partitioning

[@sudocode]

Two machines connected by network get partitioned if network is broken.

[@GiriGL]

Please add some more details on partition in this video.

[@colinmaharaj]

2:50 You made a little error in your reading of availability my dear.

[@deepenpatel5403]

MAJOR DOUBT: Lets say we want 100% consistency and partial availability and consider the value of x = 1. Now in case of partition if we allow B to serve writes, my write request could go to B, so it will update its local value to 2. And immediately if I send read request which is supposed to be served by A will return 1. So in this case, in trying to provide partial availability we are voiding our 100% consistency requirement.

[@kunaldas9284]

you're correct...the example is not correct because if you're favouring 100% consistency but allowing B to take write requests, then any read requests served by A will return stale data i.e. not consistent. Therefore we could rather say that allowing B to take write requests and also serve requests (A is already serving both read and write), makes the cluster Available but NOT Consistent.

[@goelshubham46]

it's some degree of both at same time and we shouldn't forget that it's only for the time when network is disrupted, once it resumes your data will be consistent again..Time in which network resumes is crucial in this. And again as explained in last section, we will only do this, when being consistent 100% all time is not our priority, if it is then the other node should become unavailable.

[@girishanker3796]

So this is a master - master architecture what you are saying. So here if you want Strong consistency then you have to introduce a third node which will help to maintain the state of the other two nodes. If I am correct, this problem is called a split brain problem. Whenever a write happens Node A will update it's state to Node c(our newly introduced third node) which will in turn update Node B.

[@vijayvaswani9717]

awesome explanation in both videos.

[@zeyadali3379]

Your explanation is great, keep the great work.

[@sudocode]

Thanks Zeyad

[@sakshamsrivastava6280]

It's really clear explanation!

[@sudocode]

Glad you think so!

[@ChandraShekhar-by3cd]

Thanks a lot for such an amazing and detailed explanation. Will there be any full system design video on system such as facebook, whatspp or uber? That will be really quite informational. Thanks

[@sudocode]

Thanks Chandra. That's in the pipeline. We will release such videos

[@MilindKTechnology]

Thanks for describing it so nicely

[@kunalthorat3290]

Amazing video ! while watching this video In-between i also got a thought that what if we can have backups of these network like what we have in data that is data backup right. and in the end of the video got my answer. Thankyou and KEEP UPLOADING videos.

[@sudocode]

That's amazing Kunal 👍

[@trinathtalluri]

Good Explanation!! Thanks

[@sudocode]

Glad it was helpful!

[@mayurdugar03]

Nicely explained 😀

[@anuradha8700]

Excellent explanation

[@sudocode]

Thanks Anuradha

[@suyogkulkarni6321]

thanks for explaining

[@manthan340]

Hi Yogita, Thank you for explaining in details. I want to confirm something. In the case of going for eventual consistency, we are choosing 'AP' right? I mean you mentioned that one node is giving consistent output - which I think, is not the case. As there is something written in Node B, which is not transferred to Node A until it is reconciled. Resulting in inconsitent behavior.

[@azadsingh8466]

loved this video

[@saurabh5981]

Disclaimer - I am "0" in technicals and a Sales guy, just got interested accidently watching your 3rd video yesterday. Here is my question on time stamp 7.51 ..... if B has not acknowledged then how come "A" registered the latest write............ in case, this happens, then CAP is absolutely right, if not then CAP is wrong.... Kindly excuse my ignorance

[@devprakash5320]

wonderful explanation !

[@ashishsinghchauhan6304]

you are very knowledgeable.

[@parijatbandyopadhyay8057]

Respect!

[@nageshmh]

Awesome 😍

[@DevOL-001]

Thank you 🌹

[@sahajjain395]

Hey, Thanks a lot for the awesome video. I have one small doubt. In the section on degrees of availability, you said that we can have writes on system B as well while restricting reads on A only, which will improve the availability. But what would happen if we read a value from A after we have updated its value on system B? won't this affect the consistency?

[@sudocode]

Yes, correct it will impact consistency but we sacrifice some degree of consistency over a high degree of availability. Some reads will get lates value, some will not but no read would fail because both nodes are "available". Definition of availability is giving a non error response with no guarantee of returning latest write. That's where eventual consistency comes into play!

[@sahajjain395]

@@sudocode ok thank you very much for clarifying my doubt :)

[@shankarnarayanb]

@@sudocode Quick question for you, in response to your answer, if we have decided to sacrifice consistency altogether and why limit "B" to write's alone? Why not allow the user to read from node "B" as well? Thanks, Shankar

[@ZulfiqarTariqBurmi]

nice and informative

[@shivamgupta5476]

Great Explanation ....

[@sudocode]

Glad it was helpful!

[@subee128]

Thank you very much

[@sumonmal009]

full till 4:30 kzbin.info/www/bejne/oaimd3uCl66bl80

[@shyirtt]

Great video Yogita. Thank you for the free content :) . Let's say we construct 3 DB instead of 2 DB in your example, is it possible to achieve all 3 in CAP theoram. node A, node B, node C - Any Write will write on all 3 nodes to achieve consistency. (write could take bit longer). Reads will be consistent. If there is a partition, between A & B, B will stop serving traffic but A & C are there for availability. In addition to this, we can configure multiple networks between each node to avoid partition as you mentioned in the last part of video.

[@parvgour5284]

What if node A fails?

[@stormShadow64]

You are wonderful

[@prithvirajan105]

Thank you :)

[@pratikpattanayak6518]

please use a darker pen if you can

[@koteshmeesala1630]

Awesome

[@DeepakSah3.0]

thanks

[@jasper5016]

I believe your Partition Tolerance explanation is not correct. Partition means if your node is disconnected from other nodes due to network issues. Partition tolerance means the ability to recover all the data that was lost during the connection/network issue.

[@ankurbhanot496]

first time saw someone so obsessed with black color as me :P

[@RamSingh-zp1nl]

Seems to be LEVI'S is coming for brand endorsement🤔

[@sudocode]

I am sorry that this was your key take away from a lecture on CAP theorem. I will try to do better next time ;)

[@myvinbarboza3038]