CAP | Consistency, Availability and Partitioning | System Design Tutorials | Lecture 19 | 2020

[@sudocode]

Correction at 11:42: When one partner stops taking order, the system is consistent but not available because one of the nodes is not available. For a system to have property of availability, all the nodes(number of partners here) of the system should be available.

[@laterlater8348]

this is wrong. When one node is not present for consistency no request is honoured. But for availability even if one node is available then it is available but the serving may not be consistent. As per your example if one partner does not take order then by virtue of CAP that order is served by the other person , end result is the client will not get something but may not be what he ordered. So for property of availability even if one node is present then it is available but may not be consistent , here we are comprimising consistency. For consistency all nodes must be present , that is the request is honoured only when all nodes are available, so even if one node is not present the request is not honoured there by we are compromising on availability.

[@datta3268]

@@laterlater8348 Exactly! Availability means at least one node should be available to process the request, whereas Consistency requires all the nodes to be available.

[@girishanker3796]

But if one of the nodes goes down, the system can still be consistent right. Now instead of N nodes we have an N-1 node which is serving the requests and serving the same data throughout. And a system is said to be highly available when we have multiple systems spread across which helps in serving more requests. I think what she said is right here

[@prathameshlakhpati9970]

The story telling is really fantastic. I was able to recreate the same situation while being interviewed. Thanks alot

[@ShubhamSharma-ii9jn]

Great that someone is not just click-baiting and teaching something that's useful. Keep up the good work.

[@sudocode]

I appreciate your honest comment. This is one of the qualities we are focusing on for our channel. Thanks for appreciating.

[@princechhabra9342]

Thankyou Sister for making such wonderful playlist.. Your videos were so helpful in preparing intrws.. I think Interviewer asked everything from this playlist, at one point I thought he had also watched your videos.. lol He asked CAP theorm and I vent out everything that I learnt here and at the end he said”very good”.. that was such a lovely feeling

[@santa676neot8]

Lovely playlist ❤❤

[@lalatenduguru8121]

I read many content on CAP, but my doubt got clarified in this video. Thanks for all effort!

[@Manojshankaraj]

This is the best video on CAP theorem in KZbin. Thanks for the clear explanation. Looking forward to more of your videos on System Design!

[@sudocode]

More to come!

[@laterlater8348]

lol

[@yogeshpathak]

after watching more then 7-8 videos on CAP theorem found this video easy to understand. Thanks for the Food and Bar example

[@PankajVerma-ct8ho]

I really admire how clearly you explained, by taking a very practical human-level example, that one could relate to and follow. Keep up the good work. Thank you so much!

[@Abcnews18]

Nice examples, theories are available on internet but the examples you provided for CAP & MQs are awesome, that's the quality of goood teacher. Thank you.

[@sudocode]

Glad you like them!

[@alishachhabra7995]

Having high availabilty and high consistency at the same time might be possible or not. But in real life scenarios I think, partition tolerance cannot be ignored. Hence, we have to choose either between the availability or consistency with partition tolerance, i.e., AP OR CP.

[@gomzysharma]

That's correct

[@pathakutkarsh05]

A superb example that makes good sense. Kudos

[@ishankagarwal7798]

All your videos are great Yogita. Thanks for sharing your knowledge.

[@sudocode]

Thanks Ishank.

[@jaytube277]

Excellent, Great explanation. Thank you very much Yogita

[@aditatu4762]

Another brilliant video!!! - Thanks for making and sharing them.

[@alishachhabra7995]

This is the best tutorial on CAP ever. Really appreciate your efforts. Looking forward to learn from you.

[@garikinarajesh1999]

Crystal clear explanation

[@deadpool4098]

Awesome. Very clean video with clear understanding

[@easylearning2401]

Simple and easy to learn! Great job

[@Quenchedfooty]

Firstrate explanation 👌👌

[@NegiWorld24]

Great explanation Yogita

[@lovewithcompetitiveprogramming]

CAP is possible. Let's start to take a order based on decided range of order id by person1 & person2. Whenever customer will ask the status then redirect request to corresponding person's server.

[@mystiqkc]

Great example of partner and orders. Thank you!

[@thecloudbaba8668]

Partition P is must in real world for sure... now you have trade off between A & C .. Example of of AP - Social Media like Instagram, Facebook, LinkedIn etc. Where Consistence can be given off for eventual consistence or delayed consistency but Availability and Partiotion are preferred. for example, its okay to wait to read the comments or post r like with some eventual consistency. Example of CP - Banking, Finance where transaction needs consistence following the ACID properties. Where A transaction like sending and receiving funds needs constancy. hence C is must else your core service will have no purpose... now you have trade off between P and A but P is must... so most financial sectors follows PC CAP is itself a huge topic

[@RKARAN-zs5zn]

ONe of the Best explanation

[@shyamsundervr5749]

As easy as it can get - this explanation is really the most easily understood one for CAP. Thanks.

[@sudocode]

You're very welcome Shyam :)

[@eddanapudir]

Very well explained. 🙏

[@harshthakkar7503]

Really mam this example of foo and bar is nicely explained.

[@sudocode]

Thanks Harsh. Glad you found it useful.

[@msrabea]

CAP is not possible, CA is RDBMS, AP such as Cascanda or CouchDB, CP such as MongoDB, HBase, BigTable

[@kapilrules]

Hi Yogita thank you for the video. In case of active active we have high availability but it may not be consistent and in case of active stand by it will be consistent but not HA even though active can failover to standby any case of disaster. By consistency do we mean when A receives data copy of It is sent to B and then acknowledgement is sent back to client? I feel consistency and partioning are mutually exclusive Please correct me if I am wrong

[@vanessacarly5140]

This is really helpful! Explained it better than any of the explanations I've seen so far as I've been trying to learn this to prepare for interviews. Thank you!

[@sudocode]

You're very welcome!

[@dangidelta]

Beautifully explained... Loved it 👍🏻

[@nimishgupta8357]

What if we keep a pub sub model, do we still need to care about partition tolerence?

[@shameerbasha-vo8gw]

Hello mam , u said that cap theorem do not support all 3 attributes when we look up on to our modern applications like FB , YT , In shorts where we can see consistency - up to date information , Availability we can use 24/7 , Partition tolerance - Even any one of our node crash it shows us the response due to its replication ..( My question was ) : if any one of the database like cassandra , mongodb , sql , orcale ,do not support how the Application work ??? it will helpful for my Btech project Thank you ...

[@jinalpatel9154]

Good explanation...specially analogy with ordering system. In context of CAP , there is one another concept going around PACELC. Hopefully going to see more detail in coming videos. :)

[@sudocode]

Thanks for sharing Jinal :)

[@gouravkumar-or6jy]

Consistency can only achieved by RDBMS and they rarely goes down so they can also provide availability and High availability can only provided by Cassandra cause of its ring partitioning of the nodes and For Partition i preferred MongoDB, DynamoDB etc.

[@vasanthdhage]

Was looking for this... Thank you

[@sudocode]

😀

[@maherbhavsar7420]

what if the order id is based on customer data like the phone number of customer, then both bar and foo will have consistent data, and if a customer places multiple orders, then we append the order id with _1, _2,... both at foo and bar ? can you please let me know

[@NeerajPahuja]

Network partitions are unavoidable . I don’t think you have a choice of not having partition . The only choice you can make is between consistency and availability, that is, CP or AP

[@sudocode]

That's almost correct. More details in next video!

[@sohitbhaskarwar3546]

Thanks for the both videos. It was really good. Thank You

[@sudocode]

Glad you enjoyed it!

[@sureshiva4605]

HI...Why don't we hash the CustomerphoneNO or CustomerId while receiving order and send it to specific service for that hash...SO that data don't have to be consistent while taking orders...We can sync data in parallel if we need to be...

[@sureshiva4605]

We have to start with more services...We would be estimating in advance how many customers we would be serving(based on survery's) ....While customer's about to increase...we replicate data to more services with hashfunction( parllely with original system) a month in advance to more machines...and reboot Loadbalancer with new services+old service info...

[@amantiwari9275]

Thanks a lot for such quality content

[@reshusingh6005]

Thank you so much for this Mam!!

[@vaibhavvasani2401]

very good explanation.. Thanks :)

[@onepercentswe]

We can't achieve perfect CA system which is scalable. It won't be scalable. Hence, most of the systems, achieve Availability with eventual consistency. Means, we prefer to keep the system available all the time and bring the system to consistent state eventually as the time passes by synchronising the data in background. Not immediately.

[@alivation3409]

having partitioning tolerance should always be assumed, so really the only combos would be CP or AP, because partitions will always happen at some point, and having a point of failure due to them is a flaw in design imo

[@SujeetYadav-pu9nb]

same example is share by SCALER Academy by Ansuman..

[@kartiksood5508]

Having consistency without partition tolerance is not possible because if there is no partition tolerance, that means there is no communication between nodes, there is no way for system to be consistent.

[@risaromana7545]

this was a great explanation! thank you :)

[@kunalthorat3290]

Ma'm after this video my curiosity levels are going very please kindly upload the next part of the video asap. Giving gratitude for ur excellent explaination skills that made me fall in love with system designs.

[@sudocode]

Thanks a lot Kunal. Next video will be up soon!

[@harshsonigra3040]

Nice❤️❤️

[@adityagarg9806]

how to handle that scenario if before replicating a particular data channel breaks?eg after writing to A then a will sync it to b but before that a fails?

[@sudocode]

The whole write is failed/rejected in that case.

[@subee128]

Thank you very much

[@amadousallah8916]

Thank you.

[@saumyasundaram2768]

check for CA (not very high because there could be network issues) AP CP

[@jay-rathod-01]

As requests increases in DS the load increases in which A with P is not possible , C without P is possible . C and A without P will only be a good choice.

[@sudocode]

C and A without P will only be a good choice. and if we have single server then there is no problem of consistency and partitioning - then it won't be a case of CAP theorem since it is applicable to only distributed systems. Besides, availability means a node being available in the system. It does not entails the capacity at which requests are being served. Refer these slides: www.researchgate.net/publication/221343719_Towards_robust_distributed_systems

[@jay-rathod-01]

Oh my bad.🤦

[@RajVerma-mc2pj]

Awesome work .Pls start distributed systems series

[@sudocode]

We will, very soon :)

[@VijaySharma-hw4kv]

C and A is possible in case of non-distributed(single instance of system handling everything) systems but not in distributed.

[@sudocode]

That's absolutely correct. CAP theorem talks about distributed systems. Refer page 4 in these slides - www.researchgate.net/profile/Eric_Brewer3/publication/221343719_Towards_robust_distributed_systems/links/09e41511a6f75c93ff000000.pdf

[@santoshr4212]

Why we need partition?

[@pawankrishna8918]

possibility is CAP, CP, AP is it true??

[@namahshrestha3226]

Understood the concepts properly 1. System needs to be Consistent as well as Available. Partitions cannot be tolerated because that would make the system inconsistent. Availability does not get affected in this case. This case we have CA. 2. System needs to Tolerate Partition and be Consistent. Then sacrifice one of the partitions by shutting that partition down. The system is Consistent as we chose one of the two partitions. No communication errors will occur. Also the system has just tolerated paritition. But it is not Available since we just shut down an entire partition. This case we have PC. 3. The system needs to Tolerate Paritition and be Available. Do not shut down one of the paritions. This means we do not shut down any partition and keep being available to the customer. But now Consistency is an issue since the partitions have a broken communication channel between them. This case we have PA. So you can have either: CA, PC, PA

[@shreyasbhargava55]

very well explained

[@ganeshaditya3042]

Thank you for your videos

[@sudocode]

Glad you like them!

[@kartikayagrawal5825]

The example for explaining CAP was too good. I think I should use this example to explain it to others. By the way one can only achieve two properties at a time in CAP. All the three can't be achieved at the same time.

[@channaveerhakari7319]

Hi, awesome video as always. If it's a monolithic database application then can we call that it's Highly Consistent and Highly Available?

[@rohankulkarni6029]

If it's monolithic architecture, then I feel the system will be highly consistent, but there's also a risk of a single point of failure. If the database fails, the system is no longer available.

[@yogeshdixit8111]

maam where have you read such things !! i also want to read please suggest!

[@sudocode]

A video on books on system design is coming soon Yogesh!

[@bilaladoui5866]

I would say, it can be implemented but it's not guarantee , there will be always a possibility for a failure.

[@ChandraShekhar-by3cd]

Thanks a lot for such an amazing video. Request you to please upload video on like Facebook messager, New Feed , Uber , etc full system design video. Thanks a lot for your effort and hard work to deliver this kind of informative contents. :)

[@sudocode]

Sure I will. Thanks a lot for appreciation!

[@laterlater8348]

Writing this for those who are new to this topic and the video is a bit misleading. When one node is not present for consistency no request is honoured. But for availability even if one node is available then it available but the serving may not be consistent. As per your example if one partner does not take order then by virtue of CAP that order is served by the other person , end result is the client will get something but may not be what he ordered. So for property of availability even if one node is present then it is available but may not be consistent , here we are comprimising consistency (AP). For consistency all nodes must be present , that is the request is honoured only when all nodes are available, so even if one node is not present the request is not honoured there by we are compromising on availability (CP). Also the premise here is the system is distributed in nature and a Partition (P )has occured for reasons like network failure or node failure . Finally in case of no partition that is system completely healthy then we get both Consistency and Availability (CA)

[@sudocode]

I would appreciate if you can mark the time stamp where wrong information is shared from my end. It will help me learn and also publish errata if something is actually wrong.

[@laterlater8348]

@@sudocode I have replied to your other comment , you can get the time stamp there

[@sanathmugeraya6061]

Great work Madam . Pls start low level design in java from scratch . Like ticket booking system etc .

[@sudocode]

Thanks Sanath. Will try!

[@buzzword4433]

We can achieve only two of them at a time.

[@shivamojha7999]

Ma'am please make videos of system design of facebook, twitter etc.

[@sudocode]

We will in the future Shivam. 🙂

[@shubhamdeshmukh9287]

W O W

[@shridhar_rao]

♥️🌠🎆

[@sushmitagoswami7320]

I think only possible combinations would be 1. CA - partially. 2. AP 3. CP

[@himanshushekhardas1730]

no cap

[@myLifeSk98]

#50k soon ❤️ sudocode thanks

[@asjjain191183]

You are so beautiful, its really difficult to concentrate on the technical thinks :)

[@UECAshutoshKumar]

Thank you

[@DevOL-001]

Thank you 🌹

[@tanveer.shaikh]

nicely explained