CAP | Consistency, Availability and Partitioning | System Design Tutorials | Lecture 19 | 2020

[@sudocode]

Correction at 11:42: When one partner stops taking order, the system is consistent but not available because one of the nodes is not available. For a system to have property of availability, all the nodes(number of partners here) of the system should be available.

[@laterlater8348]

this is wrong. When one node is not present for consistency no request is honoured. But for availability even if one node is available then it is available but the serving may not be consistent. As per your example if one partner does not take order then by virtue of CAP that order is served by the other person , end result is the client will not get something but may not be what he ordered. So for property of availability even if one node is present then it is available but may not be consistent , here we are comprimising consistency. For consistency all nodes must be present , that is the request is honoured only when all nodes are available, so even if one node is not present the request is not honoured there by we are compromising on availability.

[@datta3268]

@@laterlater8348 Exactly! Availability means at least one node should be available to process the request, whereas Consistency requires all the nodes to be available.

[@girishanker3796]

But if one of the nodes goes down, the system can still be consistent right. Now instead of N nodes we have an N-1 node which is serving the requests and serving the same data throughout. And a system is said to be highly available when we have multiple systems spread across which helps in serving more requests. I think what she said is right here

[@msrabea]

CAP is not possible, CA is RDBMS, AP such as Cascanda or CouchDB, CP such as MongoDB, HBase, BigTable

[@alishachhabra7995]

Having high availabilty and high consistency at the same time might be possible or not. But in real life scenarios I think, partition tolerance cannot be ignored. Hence, we have to choose either between the availability or consistency with partition tolerance, i.e., AP OR CP.

[@gomzysharma]

That's correct

[@prathameshlakhpati9970]

The story telling is really fantastic. I was able to recreate the same situation while being interviewed. Thanks alot

[@NeerajPahuja]

Network partitions are unavoidable . I don’t think you have a choice of not having partition . The only choice you can make is between consistency and availability, that is, CP or AP

[@sudocode]

That's almost correct. More details in next video!

[@kartikayagrawal5825]

The example for explaining CAP was too good. I think I should use this example to explain it to others. By the way one can only achieve two properties at a time in CAP. All the three can't be achieved at the same time.

[@princechhabra9342]

Thankyou Sister for making such wonderful playlist.. Your videos were so helpful in preparing intrws.. I think Interviewer asked everything from this playlist, at one point I thought he had also watched your videos.. lol He asked CAP theorm and I vent out everything that I learnt here and at the end he said”very good”.. that was such a lovely feeling

[@santa676neot8]

Lovely playlist ❤❤

[@Manojshankaraj]

This is the best video on CAP theorem in KZbin. Thanks for the clear explanation. Looking forward to more of your videos on System Design!

[@sudocode]

More to come!

[@laterlater8348]

lol

[@ShubhamSharma-ii9jn]

Great that someone is not just click-baiting and teaching something that's useful. Keep up the good work.

[@sudocode]

I appreciate your honest comment. This is one of the qualities we are focusing on for our channel. Thanks for appreciating.

[@thecloudbaba8668]

Partition P is must in real world for sure... now you have trade off between A & C .. Example of of AP - Social Media like Instagram, Facebook, LinkedIn etc. Where Consistence can be given off for eventual consistence or delayed consistency but Availability and Partiotion are preferred. for example, its okay to wait to read the comments or post r like with some eventual consistency. Example of CP - Banking, Finance where transaction needs consistence following the ACID properties. Where A transaction like sending and receiving funds needs constancy. hence C is must else your core service will have no purpose... now you have trade off between P and A but P is must... so most financial sectors follows PC CAP is itself a huge topic

[@shameerbasha-vo8gw]

Hello mam , u said that cap theorem do not support all 3 attributes when we look up on to our modern applications like FB , YT , In shorts where we can see consistency - up to date information , Availability we can use 24/7 , Partition tolerance - Even any one of our node crash it shows us the response due to its replication ..( My question was ) : if any one of the database like cassandra , mongodb , sql , orcale ,do not support how the Application work ??? it will helpful for my Btech project Thank you ...

[@kapilrules]

Hi Yogita thank you for the video. In case of active active we have high availability but it may not be consistent and in case of active stand by it will be consistent but not HA even though active can failover to standby any case of disaster. By consistency do we mean when A receives data copy of It is sent to B and then acknowledgement is sent back to client? I feel consistency and partioning are mutually exclusive Please correct me if I am wrong

[@ishankagarwal7798]

All your videos are great Yogita. Thanks for sharing your knowledge.

[@sudocode]

Thanks Ishank.

[@namahshrestha3226]

Understood the concepts properly 1. System needs to be Consistent as well as Available. Partitions cannot be tolerated because that would make the system inconsistent. Availability does not get affected in this case. This case we have CA. 2. System needs to Tolerate Partition and be Consistent. Then sacrifice one of the partitions by shutting that partition down. The system is Consistent as we chose one of the two partitions. No communication errors will occur. Also the system has just tolerated paritition. But it is not Available since we just shut down an entire partition. This case we have PC. 3. The system needs to Tolerate Paritition and be Available. Do not shut down one of the paritions. This means we do not shut down any partition and keep being available to the customer. But now Consistency is an issue since the partitions have a broken communication channel between them. This case we have PA. So you can have either: CA, PC, PA

[@alivation3409]

having partitioning tolerance should always be assumed, so really the only combos would be CP or AP, because partitions will always happen at some point, and having a point of failure due to them is a flaw in design imo

[@bilaladoui5866]

I would say, it can be implemented but it's not guarantee , there will be always a possibility for a failure.

[@aditatu4762]

Another brilliant video!!! - Thanks for making and sharing them.

[@SujeetYadav-pu9nb]

same example is share by SCALER Academy by Ansuman..

[@kartiksood5508]

Having consistency without partition tolerance is not possible because if there is no partition tolerance, that means there is no communication between nodes, there is no way for system to be consistent.

[@pathakutkarsh05]

A superb example that makes good sense. Kudos

[@jaytube277]

Excellent, Great explanation. Thank you very much Yogita

[@sureshiva4605]

HI...Why don't we hash the CustomerphoneNO or CustomerId while receiving order and send it to specific service for that hash...SO that data don't have to be consistent while taking orders...We can sync data in parallel if we need to be...

[@sureshiva4605]

We have to start with more services...We would be estimating in advance how many customers we would be serving(based on survery's) ....While customer's about to increase...we replicate data to more services with hashfunction( parllely with original system) a month in advance to more machines...and reboot Loadbalancer with new services+old service info...

[@Abcnews18]

Nice examples, theories are available on internet but the examples you provided for CAP & MQs are awesome, that's the quality of goood teacher. Thank you.

[@sudocode]

Glad you like them!

[@deadpool4098]

Awesome. Very clean video with clear understanding

[@RKARAN-zs5zn]

ONe of the Best explanation

[@pawankrishna8918]

possibility is CAP, CP, AP is it true??

[@eddanapudir]

Very well explained. 🙏

[@subee128]

Thank you very much

[@yogeshpathak]

after watching more then 7-8 videos on CAP theorem found this video easy to understand. Thanks for the Food and Bar example

[@buzzword4433]

We can achieve only two of them at a time.

[@santoshr4212]

Why we need partition?

[@laterlater8348]

Writing this for those who are new to this topic and the video is a bit misleading. When one node is not present for consistency no request is honoured. But for availability even if one node is available then it available but the serving may not be consistent. As per your example if one partner does not take order then by virtue of CAP that order is served by the other person , end result is the client will get something but may not be what he ordered. So for property of availability even if one node is present then it is available but may not be consistent , here we are comprimising consistency (AP). For consistency all nodes must be present , that is the request is honoured only when all nodes are available, so even if one node is not present the request is not honoured there by we are compromising on availability (CP). Also the premise here is the system is distributed in nature and a Partition (P )has occured for reasons like network failure or node failure . Finally in case of no partition that is system completely healthy then we get both Consistency and Availability (CA)

[@sudocode]

I would appreciate if you can mark the time stamp where wrong information is shared from my end. It will help me learn and also publish errata if something is actually wrong.

[@laterlater8348]

@@sudocode I have replied to your other comment , you can get the time stamp there

[@onepercentswe]

We can't achieve perfect CA system which is scalable. It won't be scalable. Hence, most of the systems, achieve Availability with eventual consistency. Means, we prefer to keep the system available all the time and bring the system to consistent state eventually as the time passes by synchronising the data in background. Not immediately.

[@gouravkumar-or6jy]

Consistency can only achieved by RDBMS and they rarely goes down so they can also provide availability and High availability can only provided by Cassandra cause of its ring partitioning of the nodes and For Partition i preferred MongoDB, DynamoDB etc.

[@lovewithcompetitiveprogramming]

CAP is possible. Let's start to take a order based on decided range of order id by person1 & person2. Whenever customer will ask the status then redirect request to corresponding person's server.

[@NegiWorld24]

Great explanation Yogita

[@PankajVerma-ct8ho]

I really admire how clearly you explained, by taking a very practical human-level example, that one could relate to and follow. Keep up the good work. Thank you so much!

[@UECAshutoshKumar]

Thank you

[@lalatenduguru8121]

I read many content on CAP, but my doubt got clarified in this video. Thanks for all effort!

[@amadousallah8916]

Thank you.

[@maherbhavsar7420]

what if the order id is based on customer data like the phone number of customer, then both bar and foo will have consistent data, and if a customer places multiple orders, then we append the order id with _1, _2,... both at foo and bar ? can you please let me know

[@shridhar_rao]

♥️🌠🎆

[@easylearning2401]

Simple and easy to learn! Great job

[@saumyasundaram2768]

check for CA (not very high because there could be network issues) AP CP

[@nimishgupta8357]

What if we keep a pub sub model, do we still need to care about partition tolerence?

[@himanshushekhardas1730]

no cap

[@harshsonigra3040]

Nice❤️❤️

[@shubhamdeshmukh9287]

W O W

[@vaibhavvasani2401]

very good explanation.. Thanks :)

[@myLifeSk98]

#50k soon ❤️ sudocode thanks

[@vasanthdhage]

Was looking for this... Thank you

[@sudocode]

😀

[@mystiqkc]

Great example of partner and orders. Thank you!

[@Quenchedfooty]

Firstrate explanation 👌👌

[@channaveerhakari7319]

Hi, awesome video as always. If it's a monolithic database application then can we call that it's Highly Consistent and Highly Available?

[@rohankulkarni6029]

If it's monolithic architecture, then I feel the system will be highly consistent, but there's also a risk of a single point of failure. If the database fails, the system is no longer available.

[@shreyasbhargava55]

very well explained

[@VijaySharma-hw4kv]

C and A is possible in case of non-distributed(single instance of system handling everything) systems but not in distributed.

[@sudocode]

That's absolutely correct. CAP theorem talks about distributed systems. Refer page 4 in these slides - www.researchgate.net/profile/Eric_Brewer3/publication/221343719_Towards_robust_distributed_systems/links/09e41511a6f75c93ff000000.pdf

[@adityagarg9806]

how to handle that scenario if before replicating a particular data channel breaks?eg after writing to A then a will sync it to b but before that a fails?

[@sudocode]

The whole write is failed/rejected in that case.

[@risaromana7545]

this was a great explanation! thank you :)

[@dangidelta]

Beautifully explained... Loved it 👍🏻

[@reshusingh6005]

Thank you so much for this Mam!!

[@tanveer.shaikh]

nicely explained

[@shivamojha7999]

Ma'am please make videos of system design of facebook, twitter etc.

[@sudocode]

We will in the future Shivam. 🙂

[@jinalpatel9154]

Good explanation...specially analogy with ordering system. In context of CAP , there is one another concept going around PACELC. Hopefully going to see more detail in coming videos. :)

[@sudocode]

Thanks for sharing Jinal :)

[@yogeshdixit8111]

maam where have you read such things !! i also want to read please suggest!

[@sudocode]

A video on books on system design is coming soon Yogesh!

[@harshthakkar7503]

Really mam this example of foo and bar is nicely explained.

[@sudocode]

Thanks Harsh. Glad you found it useful.

[@jay-rathod-01]

As requests increases in DS the load increases in which A with P is not possible , C without P is possible . C and A without P will only be a good choice.

[@sudocode]

C and A without P will only be a good choice. and if we have single server then there is no problem of consistency and partitioning - then it won't be a case of CAP theorem since it is applicable to only distributed systems. Besides, availability means a node being available in the system. It does not entails the capacity at which requests are being served. Refer these slides: www.researchgate.net/publication/221343719_Towards_robust_distributed_systems

[@jay-rathod-01]

Oh my bad.🤦

[@amantiwari9275]

Thanks a lot for such quality content

[@sohitbhaskarwar3546]

Thanks for the both videos. It was really good. Thank You

[@sudocode]

Glad you enjoyed it!

[@alishachhabra7995]

This is the best tutorial on CAP ever. Really appreciate your efforts. Looking forward to learn from you.

[@sushmitagoswami7320]

I think only possible combinations would be 1. CA - partially. 2. AP 3. CP

[@RajVerma-mc2pj]

Awesome work .Pls start distributed systems series

[@sudocode]

We will, very soon :)

[@ganeshaditya3042]

Thank you for your videos

[@sudocode]

Glad you like them!

[@ChandraShekhar-by3cd]

Thanks a lot for such an amazing video. Request you to please upload video on like Facebook messager, New Feed , Uber , etc full system design video. Thanks a lot for your effort and hard work to deliver this kind of informative contents. :)

[@sudocode]

Sure I will. Thanks a lot for appreciation!

[@sanathmugeraya6061]

Great work Madam . Pls start low level design in java from scratch . Like ticket booking system etc .

[@sudocode]

Thanks Sanath. Will try!

[@vanessacarly5140]

This is really helpful! Explained it better than any of the explanations I've seen so far as I've been trying to learn this to prepare for interviews. Thank you!

[@sudocode]

You're very welcome!

[@kunalthorat3290]

Ma'm after this video my curiosity levels are going very please kindly upload the next part of the video asap. Giving gratitude for ur excellent explaination skills that made me fall in love with system designs.

[@sudocode]

Thanks a lot Kunal. Next video will be up soon!

[@shyamsundervr5749]

As easy as it can get - this explanation is really the most easily understood one for CAP. Thanks.

[@sudocode]

You're very welcome Shyam :)

[@asjjain191183]

You are so beautiful, its really difficult to concentrate on the technical thinks :)