Proxmox Virtual Environment Complete Course Part 16 - High Availability

[@Rickety3263]

It's worth talking about creating groups for HA as well. By assigning your VM to a group, you can add more control to it's HA behavior - You can restrict nodes for migration, prioritize nodes for migration, and control how fast machines will migrate back to the original node when the downed server comes back online.

[@snowballeffects]

Absolutely Brilliant Course! - For FREE!!! - Saved thousands of dollars and have implemented 5 nodes with full HA - Finally kicked VMware into touch!

[@kegmysta]

am interested , IF it was really financially viable with all effort to migrate form VMware to proxmox? I really like the look of proxmox, BUT does it stack up price wise in your experience? @snowballeffects

[@maestrx]

Thanks for the great series! Just a note on the "did not loose a ping". If you check again at time 12:26, you will see that seq numbers 86-88 are missing. So 3 pings lost :)

[@cbw56]

I spent most of my day with this course series and have learned so much, had a lot of fun with my home lab and this is just the start. Thank you so much.

[@demandelz]

Thank you for your series on Proxmox. I have been migrating my personal vm's to a homemade, baling-wire Proxmox cluster and found your video series on Proxmox very useful. I want to add that I like your work and often recommend your videos in community college classes I teach.

[@hnic29]

Thank you for pumping these out this week. Thanks Jay!!

[@arunkhan4951]

Thank you for a wonderful series on proxmox. Learned quite a lot about the various aspects of virtualization platforms. I do have 3 1U servers, and I was planning on using them to bring up such a setup for upping my skills. I struggled with ovirt. With proxmox, I feel more confident about building a working cluster with HA.

[@notbadforasparky4791]

Enjoyed the series Jay, thank you. This was my first dip at learning the use of ProxMox. Don't think I could have started anywhere better.

[@balloth]

At 12:25, you say it didn't lost any ping but it did, the loss is almost silent but we can see the sequence number going from 85 to 89, so 3 pings (which is not a lot) have been lost

[@Rickety3263]

Ha. I was just going to comment about that. I saw that too... but a 3-second downtime compared to a 3-minute downtime is the comparison being made here. Migrating machines on my cluster have ZERO ping loss, but that's on a 10gig network

[@jafrujafru]

yes, you are right. (ping -O X.X.X.X) should do the job😀

[@DaveBukowski]

I was going to say the same thing It lost 86,87,88 in the sequence.

[@DaveBukowski]

@@jafrujafru -O doesn't exist as an option in Windows which is what the pings were originating from. Could use the shell or another linux system to do the pings with the -O switch.

[@timtjtim]

@@DaveBukowski The pings were originating from Pop OS, a Linux distro

[@shadynit]

The watching your Video is like addiction. Very smooth learning and I miss your upcoming videos. Thanks again from my heart. I learn a lot from you specially you have explained the enterprise things and production environment.

[@seanwoods1526]

Would love to see a bit more detail on VLANs and assigning them to a single port. Great work

[@jenniferw8963]

Thanks for this video. I managed to get HA setup with the single NVME installed in each node -- no other drives. When I installed Proxmox, I set it the small 512GB NVME as a ZFS RAID 0 (but single drive RAID 0). Then after say creating a container, I set it up to replicate to the ZFS pools (rpool) on the other 2 nodes.. I set the replication frequency to be like every 5 minutes during this experiment. I went to HA and added the container to HA. Unplugged the node it was on (after it had replicated already) and sure enough the HA started up the CT ID on another node using the replicated copy :) So you can do HA with ZFS Replication. (It takes a snapshot and sends teh snapshot differences to the other host when replicating.. it's pretty fast).

[@dudemena]

Awesome tutorial Jay. Thank you so much for sharing your knowledge.

[@caasberg]

I love your videos. Great work! Membership added. 😀

[@kegmysta]

BRILLIANT COURSE !!!

[@hazoumsoussi4678]

Thanks , i'm a cloud system engineer architect and this is really usefull

[@lespinoz]

I really enjoyed all chapters, was able to follow along and was able to implement my home lab! Thank you very much! This is something I will for sure recommend!

[@tokoiaoben3842]

Thank you very much Jay. Finally I finished the whole series. I will definitely come back and watch the series when I have 3 servers to setup clusters and HA. Also a special request if you can do full series on SAMBA active directory on Ubuntu. Anyway thanks again.

[@NatureEU]

Thanks a lot for this wonderful tutorial

[@aqeelabbas8264]

thank you so much for this curse, it helped a lot now I'm able to setup a high end web-server using Proxmox. Great Job !

[@eyes2bj]

I really appreciate this course. Very easy to understand. Thanks.

[@NajibAsmaty]

Excellent training content.

[@applemodus]

Thanks for the video

[@レフです]

Thank you for the good lecture, It helped me a lot.

[@shadynit]

You are awesome man. The way your explanation is superb awesome. I learn with you and its really a nice fun with you. Feels like this is live class instead of video. So much thanks to you.

[@brianhayward8240]

Given the status of VMWare -> Broadcom and license model changes, there should be one more video in your series: Migrating VM's from VMWare to Proxmox. And BTW, you did actually lose 3 pings during that migration. Notice the icmp_seq went from 85 to 89, ping just didn't immediately report the error. Pretty sure as soon as you killed that ping, it would have reported some amount of packet loss.

[@muhammadabidsaleem7048]

Please make a video with Ceph HA cluster.

[@iscariotproject]

watched the entire series i wish you went more into the details on why you do certain things like when you had to remove the cloud init you never explained why just that you had to do it.great series thank you for doing it,it makes info visual and easier to understand.

[@markdownsouth1500]

What you simulated was actually a host isolation caused by a network failure. What about HA when a host with a bunch of VMs has a kernel panic or both power supplies burn up and it takes down all the running VMs on a shared storage environment? I'm actually having a hard time seeing anyone demonstrate actual host failures with Proxmox. What about prioritization of VM restart on remaining nodes? Or VMs starting in a specified order?

[@IamKanuKingsley]

This is so beautiful. Thank you very much🎉

[@RP-rs6ky]

Thank you Jay. Appreciate your work. Keep the good work 👏 going.

[@atomboy83]

Thank you for this great full course, Jay! I learnt a lot with the videos and I will try to rethink the structure of my servers at my workplace.

[@TejasSameerDeshmukh]

This is aweesome !

[@T313COmun1s7]

The servers don't vote for themselves. So in the case of 2 servers, they would always vote for the other server and a quorum would never be reached. If every server voted for itself, it would never matter how many you had, a quorum would never be reached. You don't necessarily need shared storage (or CEPH) for HA. You can also do ZFS replication. Each has its own positives and negatives. It is easy to find people talking about how great CEPH is, but note that if you go that route you lose not only ZFS performance, but features like replication and snapshots, and basically every other advantage you gain from Copy on Write. On the HA live migration you said it didn't actually lose a ping, it just took longer. It seems you are not looking at the sequence numbers. It jumped from 85 to 89, so you actually lost 3 pings.

[@robertsretrogaming]

Thanks for the great series. I'd like to see you present information about the HA topics of "groups" and "fencing". Info on these things is present in the documentation, but it's a bit sparse.

[@MarkConstable]

A video about how best to manage regular non-VM/CT storage would be great. Also, using Storage Replication to speed up migrations for those of us who do not have 10GbE backed shared NFS storage systems would help too. Keep in mind, the target audience for this series is probably folks that do NOT have "real" servers but first time homelab'ers cobbling together minimal systems to follow along. Folks with serious racks of hardware already know most of what you are demonstrating.

[@udayarpandey3937]

Thank you jay. You are great

[@skytree21]

Great tutorial , thank you so much

[@2Blucas]

Hi, thanks for your great content, simple well explained. "Regarding Proxmox VE's High Availability features, if I have a critical Microsoft SQL Server VM, will the system effectively handle a scenario where one PVE node crashes or if there's a need to migrate the VM to another PVE? Specifically, I'm concerned about the risk of losing transactions during such events. How does Proxmox ensure data integrity and continuity for database applications like SQL Server in high availability setups?"

[@waynein_uk9551]

Hi, I'm wondering if you progressed with this to a stage where SQL in HA on Proxmox VE is viable and reliable? I'm wanting to get HA working for Progress OpenEdge RDBMS with DB sizes between 250GB and 2TB, I wondered if you have any pointers after your eork with SQL?

[@udaykitty]

a very good course really appreciate the time and effort.

[@johngrabner]

Excellent series.

[@banhonghosp9677]

Thank you Jay Request to ceph storage cluster proxmox make a video about Can you give it to me as well?.

[@aptdep7860]

Thank you, the course is very good. However, as in many similar courses, there is no information about configuring storage, especially when creating a cluster on different storage configurations. In fact, it is a very important topic, because many people have difficulties with this.

[@camerontgore]

Truly the end of an era

[@dimitristsoutsouras2712]

i think its worth to be mentioned that if someone sets up a cluster and then decides to shut down one of the servers , his syslog will flood with error messages and he wll need to de-activate the service that is responsible for the (I dont recall now) replication between the two servers. Same goes for Proxmox backup server. If you shut it down then syslog floods again.

[@ardatun]

How does Proxmox keep track of servers, with vrrp, ping? And when does proxmox decide and start to move VMs to another node? What if only 1 ping is lost on a node? Will this trigger the moving of all VMs on it?

[@foxale08]

You can do HA with two nodes but it requires special configuration.

[@ripvanwinkle2741]

Thank you for everything you have done. Do you think you will do a stand alone video on the difference between local and local lvm? I’ve seen people remove local lvm, when is this recommended if ever. Also would love a video that discusses recommended settings when using SSDs.

[@CemKavuklu]

Thank you for the series. It is much appreciated. Also, Your KZbin Play Button on the wall is kinda crooked :)

[@willielemaitre3854]

Great video course thank you! One question: How is DNS haldled please? Dies the HA edit the DNS host files etc, to let the nework know where to find the migrated server?

[@videofeed99]

Absolutely A+ Your videos are so good. I have a question: If I deploy 3 ProxMox Servers in 3 Different regions of the country, and I want High Availability for my VMs, how would I go about it? Do you incorporate Load Balancers/Proxy Servers?... etc..

[@MrPDC-jr5yl]

Nice video Jay. So do you have to move all containers to another node on every update? and then move back ...and do same for each node?

[@LeXXai]

Note. If backup method is 'stop' instead of 'snapshot' for VM, HA not working for this VM. Stop method need for correct flush PgSQL server.

[@fuseteam]

isn't the shared storage a single point in failure in this HA setup? or can that too be HA?

[@fatal3713]

But what about adding a "q-device" to be the tie breaker in a even node cluster?

[@MAD20248]

thank you so much , one question please, should all of my 3 server have similar or close hardware specs " ram and specs" , I'm planning to use a high ned nuc for my home lab iot project while the other 2 will have lower specs, idk if this a doable options or not ?

[@ebiscaia]

Maybe a bit late Jay, but what about a chapter about passing through hardware components to virtual machines? I installed Proxmox to a computer with built-in DVD reader but I am not able to use it. The identifies the DVD reader differently than Proxmox. Also, some users would like to use their graphic or network cards for their applications. Thanks, Eduardo

[@MrTrever1969]

I have add 2 bonded 1G nics and configured the switch. How do I move the ceph cluster network to those bonded devices?

[@anis5709]

Great job! Thank you. Can you pls make a video about "REPLICATION" it would be really usefull and nice.

[@BrianThomas]

I was finally able to create a high available VM running Open Media vault which is pretty cool. Thanks to these videos. My question is. Now that my VM is redundant with multiple bare metal machines running VMs. How can I create a high available storage cluster? Right now I only have one NFS share. What if that machine running the NFS goes down? How can I make NFS high available?

[@MrLexhoya]

Is there also an option in Proxmox to do load balancing? Is this more part of the HA or Clustering then? Your series has been a huge help for me understanding this platform. I moved away from Hyper-V (mem hog) and ESXi (not supporting my hardware). With a Freenas (2TB and 2 i3 NUCs with 16GB RAM you can get things running in under a day!

[@AlexX-hl4fi]

Wrong!!!, big mistake in the measurement method, you can see on 12:10 to 12:13 and from the indicator icmp_seq=85 the next sequential icmp_seq=89. 89-85 = 4 icmp ping is lost(3 or 4 second). It is very wrong to say that the connection is not lost at all, this is misleading.

[@neail5466]

Is the move disk is a disk to disk copy (if delete source is unchecked) ? Is it over rsync?

[@buildfordev]

What if we use a mirroring approach in every cluster? Is it possible?

[@andreasantini4357]

Hello, thanks for the wonderful lessons. Could you explain something to me? Do I also need to use fence to use HA? I have three clustered HP ML 350 servers but I don't understand how I should use fence. Thank you.

[@junaidij3683]

my problem is every time the vm move to another server in HA, the vm OS reboot, how to solve this

[@B20C0]

Thanks very much for the tutorial. I got a question though: What about fixed IP addresses, will those create problems for you? Also what is the best approach for them? Better to put the MAC of the VM into the DHCP server to give it a fixed address or just set it manually on the VM?

[@isaacfl]

Does Proxmox not support DNS? I notice you are always static ipv4 addresses and always looking in the console to "find" the ip?

[@kegmysta]

would you run this in enterprise?

[@johngrabner]

An interesting application for home would be moving kids plus wife's computer to a promox vm with pcie passthrough. When they are at school/work, stop their windows vm and launch Ubuntu with distributed deep learning app. This way I can train across multiple computers and quickly switch back when they need their pc back. Is this practical?

[@praca1736]

the 101 does not automatically switch to pve2? do I have to do it manually?

[@Sheyk871]

I have a problem with my PVE Server, it doesn't have internet access, so I can't even update it. The Containers and Virtual Machines do have access to the Internet without problem

[@Eli0569]

Noob question: in order to do a good cluster and be able to do HA properly, do all computers need to be the same setup (hardware wise) or can they all be different configurations (hardware) providing they all can run the VM with the correct VM setup on each of the nodes??? TIA

[@lepsycho3691]

You don't have to, but there are caveats! In my setup I have 3 nodes with all different hardware and amount of ram. So I have to setup HA in a manner that a VM doesn't exceed core or ram amount of the weakest node. And if you plan on doing pcie passthrough you have to have the same hardware (motherboard, maybe cpu?) otherwise the pcie address would differ from server to server.

[@djmrlee76]

Do you a video on resetting the root password (for Pam and PVE) and get back into the cluster? I’m testing a 3 node proxmox cluster and the scenario is that we have 1x user, root, and forgotten the password. And I know, disable root and add users but this is only a few VMs on my laptop until I figure out how to get past potential lock outs. Anyway, the test client LAN is accessible but I can’t manage the hypervisor. I can reset the Debian password. Besides that, I’m stuck.

[@b00m3rh4nd_sol]

you went from ping sequence 85 --> 89 so you lost four pings

[@spyderxs7000]

When I setup a cluster with a windows machine and migrated it over without having it replication job in it killed my vm.

[@saad5891]

You never made it clear are these 3 servers all part of the virtualisation or they are on different physical sservers I mena pve1, 2 and 3?

[@davidciprys7811]

pve1,2,3 are physical servers and webserver-1 and webserver-2 are VMs

[@systecservicos6275]

Hello I am new to proxmox I am trying to change the name of the Container via command line .. Unfortunately I am not succeeding ... I need to do it via command line and not via the web interface ... Thanks

[@Mr.E_Days]

Maybe you can backup the Container and restore it using a new name.

[@mridulranjan1069]

It missed 4 pings at 12:37 🙂

[@Superspace8]

Jay, does HA work without shared storage?

[@pgoof78]

One way or another you have to have shared storage for HA. There are a few ways to share storage between the nodes without an extra storage system.

[@Superspace8]

@@pgoof78 hi, that's more what I meant. What ways can I do HA without a dedicated storage system? Would I need to use something like Ceph?

[@pgoof78]

Ceph or I forget what it is called but basically mirroring the storage between all the nodes. It'll will create a copy on every node.

[@pgoof78]

@@Superspace8 I would like to point out that Ceph is pretty intensive. I always recommend 10G backbone at least on storage but with Ceph you really want to. Good processor and ram is also helpful. SSD's I think are recommended for Proxmox+Ceph but I have done it without. Couldn't tell you the performance. I'm rebuilding my cluster now

[@LeXXai]

I use ZFS on every Proxmox server's, and replication every 15 minutes, it very quick so transfer only snapshot difference. In this case HA working without external servers.

[@ramonkawa]

I'm not sure how you could have said that you needed 3 servers minimum for High Availability so many times in so many ways. I thinnk you invented a handful while at it.

[@Bergeronwebdesign]

If you have your VM storage on a shared drive and the storage for your second (HA) server on the same shared storage. then this is not fault tolerant. if the shared storage goes down the backup server cant boot because the disk is gone. For true HA the compute power and storage of the 1st VM and its copy should be on different Physical severs. i am not taking shots a Proxmox when i say this but VMware doesn't this much better. wen you add HA to a VM, VSphere immediately copies the VM to another server and storage medium. its literally running a shadow copy. whatever changes are done on the original VM they are copied to the shadow copy. every cpu calculation i mean everything. and when the original goes down you dont know it. you can be in a SQL database and when the shadow VM comes online you literally cant tell the difference. but the icing on the cake is that VSphere starts another shadow copy of the now online VM that use to be the shadow copy. so the process starts all over again. So you still have HA and you never even new anything happen. this works great with virtualize firewalls. if the firewall fails the shadow copy pics right up and even you VPN clients never lose connection and it is completely statefull, zero connections dropped. not even VoIP calls

[@lepsycho3691]

This demonstration is basic HA, I think a CEPH cluster would be more similar the kind of ha you are referring to. Now to be fair these kind of setup are not really for the home labber, because they requires identical machine with high performance.

[@owieOne]

Jay you like trains?

[@HDFoxra]

prooobably should have redid this... would have been nicer to see it 'actually' happening, and not just a cut to 'oh hey it worked after some fiddling'. . . also would have been nicer if you had left all the vms up, and just had them on the pve2 so that we could see how multiple vms get migrated.. because we might have gotten a glimpse of the actual quorum at work, where it'd split the load across pve1 and pve3.

[@ewenchan1239]

Is the clustering really only for live migrations and HA? (i.e. not really used for load balancing type applications?)

[@greob]

Nice demonstration.