Simplify Your Proxmox VE Tasks: Ansible Automation Made Easy

[@jimscomments]

Thanks for taking the time to do this video. I have heard about Ansible and watching this video you show a nice concise overview on how to get it going.

[@TechTutorialsDavidMcKone]

Thank you for the feedback and good to know the video was useful

[@uniXlyTV]

Great video Dave, thank you. I have been wanting to learn ansible for a bit and proxmox is a great place to start. Really liked how you used a playbook to install the ssh keys on all the nodes too, very cool! Everything was pretty smooth except on the final part: In the last section "Testing" deploying VM I was getting a message under "TASK":"The proxmox_default_behavior option will change its default value from "compatibility" to "no_defaults" in community.general 4.0.0" which was causing an error: "FAILED! => {"changed": false, "msg": "creation of qemu VM vmtest with vmid 100 failed with exception=400 Bad Request: Parameter verification failed."}" I found I needed to add in the pve_create_vm.yml under proxmox_kvm: proxmox_default_behavior: "no_defaults"

[@TechTutorialsDavidMcKone]

I haven't run into that error myself but thanks for sharing Maybe it's to do with the version

[@ChrisValcke]

experienced same (ansible version 2.10.8). Thanks for the solution! works great now.

[@larrymelone5456]

Great tutorial and the right level of detail and explanation, thank you! The only small thing that didn't work as shown by you was the proxmoxer installation via apt-get, for me it would only install version 1.0.2X which then resulted in proxmoxer not supporting authentication via token_id which needs > v1.10. However, installing proxmoxer using pip install proxmoxer instead worked for me. Thanks again

[@TechTutorialsDavidMcKone]

That's interesting to know, thanks for sharing I guess something must have changed since the video was done

[@pappyman179]

Nice content, well delivered. Also, appreciate the wall art. Us gray-hairs have to stick together.

[@TechTutorialsDavidMcKone]

Thanks for the feedback The Golden Age of Hollywood...different times

[@nigelnovelo279]

very very good content well explained and understood, also it kept me captivated.

[@TechTutorialsDavidMcKone]

Thanks for the feedback, much appreciated

[@Zygersaf]

The topics you are covering and when are lining up perfectly with what I'm trying to do at the moment, thanks! Question though, regarding ansible, would it be recommended to run ansible from a small server that is not on the proxmox cluster you are trying to manage? I imagine running it from a VM that is on the cluster works fine, but just "feels" wrong somehow! How is your structure setup here?

[@TechTutorialsDavidMcKone]

I'm running Ansible from a VM Starting from scratch though, Ansible would be run from a physical computer Then once the cluster is up and running, the role is handed over to a VM If everything broke I could start over, beginning with a physical computer again, because the files get backed up and Ansible can be installed and run from any Linux computer

[@warpmonkey]

A great video for someone new to Ansible, thanks! Is your 'control-node' also in this proxmox cluster, or do you run it outside the cluster so that you can manage/rebuild the cluster without the dependency of it running?

[@TechTutorialsDavidMcKone]

It runs within the cluster It was built from another computer, which has Ansible installed It's like the early dilemma with Active Directory as you have to have a physical computer to start that initial build process But after that everything becomes a virtual computer One of the things I really like about Ansible is that it's easy to install and as long as you have the code you could rebuild everything from a laptop for instance and be back up and running relatively quickly

[@iKn-ft2bc]

i was searching for this for months.

[@TechTutorialsDavidMcKone]

Good to know the video was useful Thanks for taking the time to comment

[@ENO-group]

Ty very much, more of these, very good video.

[@TechTutorialsDavidMcKone]

Good to know the video was useful

[@monsterhuntfreak2011]

Great video, thanks for all of this information :) Just a little suggestion: maybe provide the files you used in the video e.g. in a Github Repository.

[@TechTutorialsDavidMcKone]

I think this has been asked before but It's something I'll need to look into

[@michaelrichardson7277]

Great tutorial!! If I may, as you may have some automation or Ansible noobs watching, it might serve those viewers well if you would have touched on good practices like encrypting your playbooks with secrets and other sensitive information with Ansible Vault. Just my 2 cents but otherwise I loved the info. 😉

[@michaelrichardson7277]

Never mind. I just saw that you have a video on that already. Forgive me.

[@TechTutorialsDavidMcKone]

I was just about to mention the vault video But that's an interesting idea about good practices in general

[@ChrisValcke]

great video! proxmox without automation is indeed treacherous path. After setting up the API credentials, I'm confused on why the user/key is still required on the command line. It throws an error without it but I expected the playbook API info to suffice?

[@TechTutorialsDavidMcKone]

Not sure what you mean But different connections will need to supply the API credentials if you run a playbook for instance it will have to supply them And even if you run the same one again, it will have to supply them again As far as I'm aware there won't be any caching of credentials and every connection will be new, even if it's from the same computer

[@ChrisValcke]

@@TechTutorialsDavidMcKone sorry about confusion. question was really, if you already have all the API credentials in the playbook, why the need to supply the command line with the user/key info.

[@TechTutorialsDavidMcKone]

@@ChrisValcke If you're running a playbook that knows about the API details or at least knows where to find them, then you won't need to supply them at the command line when you run the playbook

[@mariobader2152]

Ansible is one of my favorite tools :-) I clone using Ansible and Proxmoxer, the desired VM, it works great! I just haven't found a way to change the IP and hostnmame during the cloning process,because I don't use Cloudinit (I have pre-made templates). That's what I've been doing with a second playbook so far but maybe there is a better way

[@TechTutorialsDavidMcKone]

Yeah, endless possibilities with Ansible If you want to use cloud-init you need a cloud-init image and attach a cloud-init drive as well That image is basically a pre-built OS that gets answers to complete the install through the cloud-init drive If you want to use it to build a template to clone from, don't boot the VM, because that completes the installation Instead, turn it into a template that is basically just the hardware and an OS waiting to be finalised Any VM can then be cloned from that, given its name through the cloning process and a static IP could be defined in its cloud-init settings You can add software packages to install through the cloud-init process but it won't suit all cases My goal is to also maintain VMs through Ansible after they're built, so I deal with application installation and other maintenance in a separate playbook

[@mariobader2152]

@@TechTutorialsDavidMcKone yes, that's right, but I often need it. e.g. 2 web servers and a load balancer with Ngnix, which I built as a template and can clone in seconds using Ansible and Proxmoxer. I then configure this via Ansible: 1. the ssh key you need (via sshpass): --- - name: Copy SSH keys to server hosts: new become: true # If sudo privileges are required tasks: - name: Make sure the ~/.ssh directory exists file: path: ~/.ssh state: directory owner: torsten group: torsten mode: '0700' - name: Copy the SSH public key to the server authorized_key: user: torsten key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}" # Path to the public key on your local machine state: present - name: Update the package list apt: update_cache: yes tags: update - name: install updates tags: update apt: upgrade: yes update_cache: yes tags: update - name: Perform the system upgrade apt: upgrade: dist autoremove: yes autoclean: yes tags: upgrade Then I configure the computers --- - name: Change IP and Hostname hosts: new become: yes vars_prompt: - name: IP prompt: Bitte IP-Adresse eingeben private: false - name: HN prompt: und hostname eingeben private: false tasks: - name: Replace IP in Netplan configuration ansible.builtin.replace: path: /etc/netplan/00-installer-config.yaml # Der tatsächliche Pfad zu Ihrer Netplan-Konfigurationsdatei regexp: '(addresses:\s*-\s*)\d+\.\d+\.\d+\.\d+/24' replace: '\g{{ IP }}/24' become: yes - name: Change Hostname ansible.builtin.hostname: name: "{{ HN }}" # Replace new_hostname with the new hostname - name: Update /etc/hosts ansible.builtin.lineinfile: path: /etc/hosts line: "{{ IP }} {{ HN }}" # Replace new_ip and new_hostname with the new values - name: Update DNS A Record nsupdate: server: "192.168.0.35" zone: "vreden.home" record: "{{HN}}" value: "{{IP}}" - name: Update PTR Record nsupdate: server: "192.168.0.35" zone: "0.168.192.in-addr.arpa" # Die korrekte rDNS-Zone für das Subnetz record: "{{ IP.split('.')[-1] }}" # Die letzten Oktette der IP-Adresse ohne Punkte type: PTR value: "{{ HN + '.vreden.home' }}" - name: Reload BIND configuration become: true command: rndc reload - name: Neustart des BIND-Dienstes systemd: name: named # Der Name des BIND-Dienstes kann variieren, bitte überprüfe den Namen auf deinem System state: restarted - name: Restart networking service ansible.builtin.service: name: systemd-networkd state: restarted become: yes - name: Reboot the system ansible.builtin.reboot: reboot_timeout: 300 # Optional: Timeout in Sekunden, bevor der Neustart als fehlgeschlagen gilt test_command: "uptime" # Optional: Befehl, um zu überprüfen, ob das System wieder online ist become: yes async: 0 poll: 0 For me, this is faster than configuring cloud images with software, But there are definitely better ways :-)

[@mariobader2152]

@@TechTutorialsDavidMcKone But yes, I have to learn any better about cloud images

[@TechTutorialsDavidMcKone]

I used to create specific templates but switched to cloud-init images to save time I only need that initial OS up and running and after that Ansible can do the sort of work you've suggested so that the VM does what I want it to do

[@mariobader2152]

@@TechTutorialsDavidMcKone yes, I think I'll try cloud-init too, I've always had problems initializing the German keyboard and with an English keyboard, I can't remember where the brackets and special characters are and God knows we need them 🙂🙂🙂

[@razu732]

I get an error after the command at 13:58 saying FAILED! => {"msg": "Missing sudo password"}. All previous steps worked fine. What can I check to trouble shoot? Thanks

[@TechTutorialsDavidMcKone]

It sounds like the Ansible user account requires a password and that needs to be supplied to gain sudo rights Add this parameter at the end when you run the playbook so you'll be prompted for the sudo password -K The alternative is to add this --ask-become-pass

[@hamidouz31]

I think for provisionnig it is much easier to use terraform even for automating package install or other stuf

[@TechTutorialsDavidMcKone]

I wasn't impressed by terraform The main benefit I find with ansible is it's as flexible as you want it to be and easy to install, at least if you don't mind doing things from the cli Currently I'm in the process of migrating vms to podman containers so I can save compute resources and I'm using ansible to do that The end goal is to tear everything down and then have ansible build it all from scratch Well, you have to do the initial install of pve first, but that can be automated, and then it would be handed over to ansible "One tool to manage them all"

[@hamidouz31]

@@TechTutorialsDavidMcKone excelent, can you please share with us on a videos or blog this awsome experience. And please CAN y tel me more about automating frech pve install. Thanks a lot for your support

[@TechTutorialsDavidMcKone]

@@hamidouz31 It's still a work in progress Podman for instance has a newer version I wasn't aware of and it changes things a lot So for now everything is its own mini project that I'll release videos about Unattended installs for pve though was improved in 8.2

[@jorgegomez374]

I was starting to make research on this. I have a cluster 3 machines and what to shutdown at night an turn morning. Make backups to my unRAID server and other thing. Thanks

[@TechTutorialsDavidMcKone]

That would be a useful way to save energy Computers still consume quite a bit of power when powered down so I started using smart switches on some of mine so I can turn them off at the wall And with the BIOS set to turn on when power is restored they'll boot straight up

[@egysnet]

Great video thank you

[@TechTutorialsDavidMcKone]

Glad to hear the video was helpful Thanks for taking the time to comment

[@shyuhei]

Is there anywhere I can download this playbook?

[@TechTutorialsDavidMcKone]

You can find the details on my blog www.techtutorials.tv/sections//ansible/ansible-gui-semaphore/

[@zxmzc14]

there is something about pam which not allowing me to get the "sudo" rights added to the "ansible" I installed (root@pve:~# apt install python3-proxmoxer) on the command prompt because the ansible script was hanging at "gathering facts". the pve_create_vm.yml ----"FAILED! => {"changed": false, "msg": "creation of qemu VM vmtest with vmid 100 failed with exception=400 Bad Request: Parameter verification failed."}"

[@TechTutorialsDavidMcKone]

You could login as root and manually add the Ansible account to the sudo group If that fixes things then the problem is getting Ansible to grant sudo rights and it needs looking into It can help to run a playbook with the -v flag to get more information about what's going on Sometimes I have playbooks jumping between user accounts for instance and I find the wrong account is trying to do something or the task does/doesn't need sudo rights, so that extra information helps pinpoint the cause

[@Mavo1989]

Its a good video, but definately easier utilizing vscode and terraform provider for proxmox with ansible i found this video harder verse utilizing vscode for the work

[@TechTutorialsDavidMcKone]

Thanks for the feedback I did look into tools like terraform but it's just an extra tool to learn I've found I can get everything done just using ansible, and my main goal has been to be able to rebuild everything from scratch with less effort I do use vss but sometimes it gets in the way, messing up file structures

[@AdrianuX1985]

+1