Fail-over and High-Availability (Explained by Example)

Failover is the technique of switching to a redundant backup machine when a certain node goes down. This is a very common implementation for achieving high availability and it is often mixed with different load balancing techniques such as layer 4 and layer 7 balancing.
In this video i want to go through following
Chapters
0:00 Intro
1:47 What is Failover?
3:00 ARP - Address Resolution Protocol
5:40 VIP & VRRP
12:12 High-availability Example
#KeepAlived
🏭 Software Architecture Videos
• Software Architecture
💾 Database Engineering Videos
• Database Engineering
🛰 Network Engineering Videos
• Network Engineering
🏰 Load Balancing and Proxies Videos
• Proxies
🐘 Postgres Videos
• PostgresSQL
🚢Docker
• Docker
🧮 Programming Pattern Videos
• Programming Patterns
🛡 Web Security Videos
• Web Security
🦠 HTTP Videos
• HTTP
🐍 Python Videos
• Python by Example
🔆 Javascript Videos
• Javascript by Example
👾Discord Server / discord
Support me on PayPal
bit.ly/33ENps4
Become a Patreon
/ hnasr
Stay Awesome,
Hussein

[@hnasr]

* What is Failover? 1:47 * ARP - Address Resolution Protocol 3:00 * VIP & VRRP 5:40 * High-availability Example 12:12

[@_SoundByte_]

Can’t sleep without watching one of the backend video of yours

[@mohammedhewedy8231]

Your channel is awesome, you are hitting very important topics for the developer to understand how the parts underneath them are working, and most of the time you find developers just ignore it. I Like your channel and your choice of topics ALOT

[@hnasr]

Thank you Mohamed! Appreciate your kind words bro 🙏

[@psstream889]

KZbin is going to make good and quality engineers

[@ryandeckard1479]

Thank you so much for all the work you put into your videos, these examples are extremely helpful and well-made. I subscribed and hope to learn a lot from you!

[@hnasr]

Appreciate it thanks Ryan! I hope you enjoy the content

[@ice194]

Thanks for the explanation! It’s really good. Subbed!

[@mr.daniish]

Excellent videos Hussein, you are an excellent and fun techie! Quick question, do you have a DraaS video as well, if not please do so.

[@jamiekim5430]

So succinct and easy to understand!

[@wh264]

love the examples, subscribed.

[@joechen9498]

Great example. I love it. Thank you for the video. i subscribed.

[@sureshk763]

Thanks for taking your time to share your knowledge with us....

[@hnasr]

Suresh K Thanks Suresh!!! I am so happy I could share my knowledge . It gives me great pleasure.. let me know what should i talk about next

[@sureshk763]

@@hnasr thanks for the reply, Could you create a video for Cloud servers connectivity with Storage devices

[@himanshupatel4264]

Great video man.. haven’t seen some content in while

[@snowman7480]

This is what we call passionate. Good luck buddy ;)

[@ArpanDasS]

Dope content! 🔥

[@ial9809]

glad i found ur channel gotta learn a lot from u

[@AnkitSingh9787]

great explanation

[@pramodpatil3925]

Superb !

[@jamesking890]

i like this guy. great stuff

[@stutteringcris468]

i learned a litte, here. you're funny as hell!

[@anvitasingh5]

who can dislike this lovely video !!! 13 people are legit crazy.

[@yahyaensa]

Thank you Hussein for you GREAT VIDEO, i'm in love with your channel ! Quick question, what happen when the 2 haproxy are not in the same LAN (ARP requests won't work right) ? What if we want to do High-Availibility but accross multiple geolocation ?

[@hnasr]

DNS load balancing combined with this technique is one idea

[@AmitKumar-qg5mu]

Nicely explained

[@hnasr]

Thanks Amit! 😊

[@truthadjustr]

Can you describe a little bit what is happening in the background between a primary and secondary to ensure that their file systems are in sync. Does it need running services within the node for the fail-over to be successful? If I can ssh into an Ubuntu machine, how can I tell that it is being configured for fail-over?

[@sagartyagi2450]

Just curious to know, can we have the fail over at any level?? Like even after having it at the top of load balancer, we can add one more fail over at the backend server levels right?

[@tanumoymajumdar874]

Wonderful explanation. Just one doubt. We are still with one single database instance so single point failure. How can we achieve high available system with single instance of database ?

[@douglasmakey]

Hi bro thanks for taking your time to share your knowledge with us, I have a question, I know each computer has a local ARP table to store each know mac addresses, the first time when the machine ask "who has the ip *.100", it will add the response to this table 10.0.0.100 == AAA, so how is the machine know that the first machine with mac AAA has failed, and it need to ask again "who has the ip *.100" ? Sorry for my English, thanks

[@hnasr]

Douglas Mendez good question!! When the client asks who has the ip address 10.0.0.100 it will be received by all machines and machine AAA will respond. So the client will know update its ARP table that 10.0.0.100 is AAA. When AAA dies BBB takes over (because BBB knows that AAA died since it didn’t receive heartbeat 💓) BBB now sends a broadcast to all clients to flush their ARP table so that 10.0.0.100 is now BBB. Its a reverse operation

[@zahrasadeghibogar]

First of all, I would like to thank you for your positive energy and your interesting way of teaching. Thank you:-) Second, I want you to help me. or maybe to make a video about the structure of the network, in terms of languages, for example, I am a beginner in learning networks, firewall, security and the languages that are related to this area, in fact, I always confused what languages are for server-side and which are for client-side, I did search about it but I am still confused and have no clear image of the structure. I hope, I explained my purpose clearly:-( I also want you to help in in learning IPsec VPN and SSL VPN and their differences and also, whats the meaning of this statement that IPsec is a layer 3 protocol which is better then SSL which is a layer 4 protocol, I do not know what it is mean Thanks a lot

[@hnasr]

Hi Zahra, Thank you for your comment! Almost any language can be server side or client side check out my latest video. Check out my network engineering playlist, the more you learn the more questions you will have and that is when you know you are beginning to understand kzbin.info/aero/PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr

[@vibekdutta6539]

Sir could you please make a video on RAFT, its a protocol by which nodes in a distributed system agree upon who is master and whos follower, please add some example also :}

[@soufianta8374]

Hi. Stupid question maybe: from which ip does the browser send a packet in case of multiple IP interfaces (1 physical and x-virtual)? I know that every trafic can only travel through the physical one but how does it actually work? I’m trying to understand how I can go on internet, communicate with local machines, remote machines from the intranet (through vpn) etc. with only 1 physical interface !!! Thanks

[@edufree565]

Nice video, thanks for the simple explanation. I have a question, slightly off-topic but pls excuse me. I want to know what happens in real prod scenarios, say both of the front end servers(or app or db) go down, how is the third one introduced? I assume the newly introduced server will need some configuration before it can handle requests. How is that handled automatically?

[@hnasr]

edufree good question, I don’t have a straight answer unfortunately. The originations I worked for always used Passive Active model just two servers and when the main server goes down the new server takes over (fail over) clients will have to close and reopen. The probability where both servers go down at the same time is low in enterprise environment. And if it does you will basically go offline. Here it depends on your SLA. Do you want to cater for that case? You have just add another server to the cluster to fail over to. In high active environment such as google, twitter this is done through containers and orchestrating layer (kubernetes) if something fails another container is always available.. Databases are tricky to scale as containers people are still figuring this out.. essentially making the DBMS stateless and decoupled from storage. Hope that helps or did u have something else in mind?

[@edufree565]

@@hnasr Thanks for the reply, yeah I guess containers provide more bandwidth for failures in HA environments

[@MayankAggarwal]

Awesome video! Can you please let me know from where you prefer to read about all this?

[@hnasr]

I start from Wikipedia then scroll all the way down to resources to read original resources. The more I read the more questions I have, I search for answers , sometimes with luck sometimes without.. so I put the pieces together here is where most of my own work come to place and prior knowledge of the topic. hope that helps :)

[@MayankAggarwal]

@@hnasr Thanks! You have done an awesome work in these videos.

[@d.s.5157]

Hi how would this still work if laptop was at another site and the 2 servers were in two separate locations ?

[@dawahtv7942]

hello please give me some advice to solve my problem sir. i have load balancer TPlink TL-ER6200 and i setup a fail-over i have two ISP sometimes my 1 ISP goes wrong no internet but modem is active just no internet. the load balancer not outomatic switch the backup isp in the back up wan. they switch only the backup wan when main wan not connected in ISP 1

[@siddharthdeshpande4086]

Does Application running on Microsoft Cluster works on same ARP/ VIP concept? For example we need to deploy IIS/Web server as HA using Microsoft Cluster Service... I always used to think may be DNS is getting updated every failover happens but never thought about ARP... Great Video !!

[@hnasr]

Yeah MS uses NLB as a layer 4 load balancer. Thanks!

[@adityasethi9794]

if communication b/w master n backup has an issue. backup thinks master is down but its not really down. in this case both will reply yes for the ARP. how does in work now?

[@hassanawodi5888]

Thanks for this. What other software will you recommend for high availability on Linux?

[@hnasr]

Definitely keepalived for high availability mixed with either nginx or haproxy for layer 7 load balancer and layer 4 load balancing ..

[@hassanawodi5888]

@@hnasr thanks. Will give it a go.

[@omaramine8710]

good job!

[@omaramine8710]

Hussein , what about keepalived master and slave ! ?

[@hnasr]

I explain the concept in the video, I just call it master backup instead. Ill make a dedicated keepalived practical video . This one was to explain fail over concept

[@omaramine8710]

@@hnasr Any update here bro :-) thank you !

[@husseinnasser]

omar amine not yet, still busy with other projects. Stay tuned :)

[@sunshineheavens6473]

How to actually set up a failover IP? before it can be used with keepalived or other setup. Thanks in advance.

[@vivekdey1827]

Hello Sir, is it what the heartbeat protocol does?? Thanks.

[@PerfectBonzo]

Look at me! Look at me! I’m the master now

[@amitarora0623]

At 13:01, you mentioned "if you are still doing REST". What do you recommend "if not doing REST" ?

[@hnasr]

Nothing really wrong with REST if it works

[@stealtime]

@@hnasr What do you use if you're not using REST?

[@ammarel-shabrawy6275]

Could explain example using Message queue example instead of HTTP server ?

[@krishnakondapalli]

IP failover is good, but what if want a certain filesystem to be failedover as well along the IP?

[@hnasr]

Krishna Kondapalli hmm like Raid cluster?

[@krishnakondapalli]

@@hnasr yeah some clustering system that can failover IP and a filesystem for app to write its data on. Also have to say this - you're doing a great job sharing you knowledge. thank you!

[@joeyalfaro2323]

Reminds me of tunnel rats of Vietnam. 2 man teams number 1 and number 2 guy. If number 1 guy dies number 2 guy becomes number 1 guy gets fresh recruit he becomes number 2.

[@atomicluis]

I’m the OG now... lol

[@agrawalbansal88]

its good video, but try to avoid voice modulations, its irritating. btw content is quite good

[@hnasr]

ankur agrawal I will do better in future videos (with audio quality) apologies

[@agrawalbansal88]

@@hnasr thanks for quick response :)

[@gam3955]

Good explanation but too long !

[@vigneshkini9489]

It could have been better.. didn't like the way you sing and dance with the words to explain the concepts..

[@johnfernandes455]

irritiating

[@hnasr]

❤️

[@naziaqureshi2563]

Hi hussein , i wanted to connect with you regarding some queries related to fail over scenarios. Please share your email id