Proxmox Cluster QDevice Raspberry Pi

[@TechTutorialsDavidMcKone]

If you want to learn more about Proxmox VE, this series will help you out kzbin.info/www/bejne/qXm6ioiqZbtgmZo

[@kymnyth]

This was very helpful. I have come back to this video a number of times. Thank you!

[@TechTutorialsDavidMcKone]

Good to hear the video was helpful. Thanks for the feedback

[@discovery_af]

Thanks for the help! I have a mini pc along with a server build that I clustered. Didn't realize that quorum was a thing until someone pointed it out. Thankfully this video exists.

[@TechTutorialsDavidMcKone]

It's definitely useful in a small network So good to know the video was helpful

[@Farang_Lifestyles]

very usefull tutorial David,,, I have been running with two bare-metal Proxmox VE using it to migrate VM back and forth when doing maintenance, etc... was wondering why sometimes things would not function .. you explanation of the quorum and votes makes sense and having a RasPi to be the 3rd party to vote is a perfect for a low energy solution

[@TechTutorialsDavidMcKone]

Good to know the video was helpful I only need two servers myself, so it was great to find you could use a Pi as a qdevice

[@905jay]

great content as usual - thanks for posting this it's exactly what I needed :)

[@TechTutorialsDavidMcKone]

Good to know it was useful I found it's a very useful way to save money

[@jsg9582]

Thank you for the clear explanation and instructions! Bookmarked for future reference :)

[@TechTutorialsDavidMcKone]

Glad to hear the video was helpful

[@sinisterpisces]

This is an awesome video. I was completely unaware that I could set up a third device to provide a quorum for my cluster. I was about to go through the trouble of setting up a whole third Proxmox node. :) You saved me a complex and apparently unnecessary adventure. I did have a few questions: 1) Are there any stability issues with installing on top of the full Raspberry Pi OS with GUI (as opposed to the minimal/no GUI version)? 2) I noticed that it copied SSH keys when you ran through the setup, but I didn't see you set those up. Does that happen automatically, or did you configure your Proxmox node to use SSH keys in another video? 3) Would you feel comfortable running any other services on a Pi 4b 8GB acting as your Qdevice? (I'm assuming Corosync doesn't use a ton of CPU.) Thanks again!

[@TechTutorialsDavidMcKone]

Good to know the video was helpful The desktop OS might be stable enough, as long as the Pi has the resources, but I can't be certain as I no longer use it You don't have to create any keys yourself for this setup process as it's copying over a public key for the root user and I didn't supply that If you install anything else on the qdevice you could introduce a security risk; Any other software probably means other devices will need access to the Pi and that could then be used as a possible jump point to a hypervisor Having extra software means a computer is more likely to have a bug and crash or misbehave And you could run into the problem of having too much network traffic on the NIC, or in the network it's connected to, which results in the corosync traffic being dropped and that leads to clustering problems I just had a Pi lying around and as it's low power I'm quite happy to let it just be a qdevice This way it sits in an isolated network and aside from management access, only the hypervisors can talk to it and vice versa

[@ShoruKen]

since it is just a home lab and not a live/production environment, the qdevice allows me to take down a server in the cluster without killing the whole cluster!

[@TechTutorialsDavidMcKone]

I've found it to be very useful A 3rd server would just be a voting machine in a home/lab environment so I'd rather have a Pi do that

[@Neo8019]

Thanks, great video. I was getting "Host key verification failed" when trying to setup the QDevice on the first PVE. After some search I found that I had to run the following command on the PVE ( I run it on both) pvecm updatecerts

[@TechTutorialsDavidMcKone]

I've never ran into that issue myself, but good to know Thanks for sharing

[@bangan9570]

First, Thanks! However, i got errors on starting the qdevice, and spent a couple of hours researching. I could change user to root and get it to run, but that is bad practice from what i learned. :D running '''chown -R coroqnetd:coroqnetd /etc/corosync/qnetd /var/run/corosync-qnetd''' solved my problem, as the owners of both directories was root. Is it something missing in the corosync-qnetd package, or did i do something the wrong way?

[@TechTutorialsDavidMcKone]

Out of curiosity what did you install this on as I used a Pi and it's still running the older Bullseye code? The software was simply installed as shown and it took care of the folders and permissions The user in the service is showing as coroqnetd so what you've run into seems odd Not sure if something has changed, but for me, the ownership for the folder /etc/corosync/qnetd/nssdb/ is root:coroqnetd So the user account has access to it via the group Ownership for the parent folders is root:root I'm seeing a symbolic link for /var/run pointing to /run I do have a folder /run/corosync-qnetd and the ownership for that is coroqnetd:coroqnetd Makes me think there may have been a bug in the installation package It makes no sense to run this software as coroqnetd but not allow access for the user

[@bangan9570]

@@TechTutorialsDavidMcKone I had a fresh install of latest version Raspberry Pi OS 64bit. For testing, i just tried with Raspberry Pi OS (Legacy, 64-bit) (Bullseye) and it worked as soon as i readded the qdevice to proxmox. I am too new to this to know if i made a mistake or if there are differences. :) Again, huge thanks for making these video tutorials for us 'newbies'. Next up I'll try to get mail notifications (gmail) working.

[@GaryInnerarity]

Can we have multiple qdevices - to act as redundancy to the 1 qdevice? Knowing raspberry pi's, it would be nice to have an extra as a backup.

[@TechTutorialsDavidMcKone]

Normally you don't have extra redundancy in IT, you design to handle a single failure Proxmox's recommendation is to only use a qdevice when you have 2 nodes It makes sense for a small cluster because if a node fails, the qdevice allows changes and so on to still go ahead And if the qdevice fails, well you still have your two nodes Having 2 qdevices would also result in a cluster of 4 voting machines and you need an odd number in normal conditions to avoid a tie breaker

[@squareeyesproducer]

Tried to install on a docker container on the Pi but even with 5405 forwarded the remote root execution fails. Seems part of the script provisions things on the Pi instead of the container (maybe?). Then it occurred to me. Why not install Proxmox on the Pi and have it natively participate in the quorum? Or even have a CT on Proxmox on the Pi that runs barebones corosync?

[@TechTutorialsDavidMcKone]

I hadn't considered a container as the Pi I had was just gathering dust, so I just put it to use as a qdevice Presumably there must be something it needs access to and maybe AppArmor is blocking access to it? I'm aware of a version of Proxmox for the Pi, but it's not official and I think it's still on v7, using 32-bit Debian Mind you, the devs of Pimox suggest you could mix a Pi running Pimox with x86 servers running Proxmox So that might be a better idea for a lab as in theory you'd have 3 nodes in a mixed cluster You wouldn't need a qdevice, but you'd be limited to 32-bit VMs on the Pi I wouldn't run a qdevice as a container on a node though, or even a VM The qdevice is only recommended when you have only 2 nodes and if the node running it fails, the qdevice is lost and HA won't work

[@8bitkid408]

Great video. Can I run pihole and QDevice on a PizeroW?

[@TechTutorialsDavidMcKone]

I wouldn't recommend it as it uses Wi-Fi The cluster needs fast i.e. low latency responses so you need a wired computer

[@richardfrancis9620]

@@TechTutorialsDavidMcKone so if i was to have an usb to ethernet adapter a pi zero would work?

[@TechTutorialsDavidMcKone]

@@richardfrancis9620 It could be worth trying in a lab But I don't know if the software will be available as the OS is different or if the hardware could keep up

[@fatal3713]

I set a password and user in Pi imager before installing into Pi Zero 2. I went and changed the root password as you demonstrated. When I go to setup device it says that root password is wrong.

[@TechTutorialsDavidMcKone]

That's puzzling Other than mistyping a character, and it is possible if you type too fast for instance, I wonder if the issue is keymapping If the password contains symbols that can cause a problem if the keyboard map doesn't match the keyboard layout If that's the case, you could try pasting in the password to see if that resolves the issue or experiment with different keys to find the correct symbol Unfortunately you can't change the password without knowing it and then you have to reinstall the OS If you have access to the CLI you can type things out as well to what symbol you get back versus what the keyboard shows

[@fatal3713]

@@TechTutorialsDavidMcKone thanks for responding, will try your suggestions

[@fatal3713]

@@TechTutorialsDavidMcKone I finally got it to work, don't ask me how because I don't remember. Thank you