Automate Homelab Deployment With Terraform & Proxmox

[@AT-lp8iz]

Thanks Jim, your clear and concise explanations of complex subject matter are some of the best - I’ve learnt so much from your videos !

[@Jims-Garage]

Thanks, appreciate your feedback. Join Discord if you have any problems.

[@simonbritton2677]

Thanks Jim, you were right about things moving. Learned a lot based on some changes they have made, couple of things for anyone trying this as of Jun 14th. can now use the version in providers without having to make anything. terraform { required_providers { proxmox = { source = "Telmate/proxmox" version = "3.0.1-rc3" } } } The cloudinit_cdrom_storage = "local-lvm" seems to have been deprecated and moved to the disks block, my tweaks as per below worked for me. Adjust for your use of course. disks { scsi { scsi0 { disk { storage = "local-lvm" size = 12 } } } ide { ide3 { cloudinit { storage = "local-lvm" } } } } Again, thanks Jim, keep them coming.

[@TheRealClutch1010]

Thank you for this series. I've used terraform on cloud providers and I have found the Telmate provider documentation to be lacking. Using your video and your example repo has helped me get my systems off the ground!

[@Jims-Garage]

Great, thanks for the feedback. Good job

[@jwderoode]

Thanks Jim! Great video, never had started with Terraform if it wasn't for your info. Was a little bit of updating needed: UPDATE: Oct-24. For those getting stuck on the line: cloudinit_cdrom_storage = "nvme" That is no longer in use, change the disk definition: old: cloudinit_cdrom_storage = "nvme" disks { scsi { scsi0 { disk { storage = "nvme" size = 12 } } } } into: disks { ide { ide3 { cloudinit { storage = "nvme" } } } scsi { scsi0 { disk { storage = "nvme" size = 12 } } } } Now you can terraform plan again.

[@elements88xyz]

Again, fantastic job Jim! Your channel is a pure gold mine! Thank you thank you and thank you! :)

[@Jims-Garage]

Very welcome, appreciate the feedback.

[@MarcMcMillin]

This is awesome! I'm looking forward to using it. Thanks Jim!

[@ijustwanttoeatcookie]

Awesome video! I am actually working on the exact same project and I’m using your script as input to provision the cluster with a few tweaks The only thing I added into the process is packer. I preinstall packages for my K3s-ansible controller and I export the public SSH key onto a share during building the cloud init image. I also use the ansible server as my terraform and packer builder server. Just a bit of a chicken and egg problem. For my K3s nodes I inject this public ssh key and create the ansible user in the packer image. Once all done my ansible machine can immediately connect to all K3s nodes and start building the cluster. Still a work in progress but it’s super nice you are doing exactly what I’m working on and your videos are extremely useful

[@Jims-Garage]

Thanks, I'm going to focus on packer later 🙂 good luck

[@DMBrownlee]

I was using Packer to build VM templates in Proxmox that I could then clone with the Telmate Terraform plugin. I started looking into using cloud-init as an alternative just before Proxmox 8.1 broke Telmate for me. One of the cloud-init configuration options is adding your own keys (and admin username, if you like) so there is no need to use packer to build your own image, just to insert keys. I'm now using the stock cloud-init image from Debian for my project and I have stopped using Packer for now. I've seen examples of Terraform managing the generation of key pairs which would be a good prerequisite to add to this video.

[@ijustwanttoeatcookie]

@@DMBrownlee you're right, but I'm not only inserting keys! I use an ubuntu server 22.04 cloud init image and I create 2 users. My own admin user and the ansible user. I generate the ansible priv/pub keypair while building the image and it then stores the public key on my nfs share. I do this so that I can inject the public key in the image for my k3s nodes. In addition, I also install some software packages and set-up some repos with packer. for my Ansible master server I install: ansible, packer, terraform and kubectl so that I have a dedicated image that can fully control and configure my k3s cluster. For my K3s nodes I intent to do the same by baking k3sup into the image using packer. Will just use ansible to basically connect the nodes together as a cluster. I also don't use the telmate provider for terraform anymore since the update cycle is way to slow. Instead I opted for thegameprofi's provider for proxmox. It is similar to bpg's provider and it's rock solid so far on proxmox 8.1

[@DMBrownlee]

@@ijustwanttoeatcookie cloud-init also lets you configure the default user as well. If the default user is your configuration management user, you can use ansible post-creation to manage the rest of the system configuration including adding additional admin users if needed. Still not seeing anything that would require building an image with Packer. Not saying Packer is bad. I used it for years and it works great. But it only makes sense if you cannot use the vendor's stock cloud-init.

[@ijustwanttoeatcookie]

​@@DMBrownlee you're right I just realised I can perfectly well use ansible to copy its public ssh key to all my k3s nodes. Just adjusted my code :) Bit offtopic here on the homelab, but wouldn't you say that having a golden image for specific server types (web servers, data base servers etc) can be quite conveniant for enterprise environments? Ansible can provision with the required packages for each server as soon as the newly created vm's / containers come up of course. However, having that stuff baked into the image can save quite some time and bandwith when deploying at scale or am I overlooking something?

[@jhanssenfavaro]

Thank you so much by the Help! Very good !!!

[@Jims-Garage]

You're welcome 😁

[@ninja2807]

As always great tutorial. Thanks 💯

[@Jims-Garage]

Thanks for the feedback

[@rexeus]

Great video! video suggestion- how to secure VMs/K8s clusters and use of tools like Wazuh etc...

[@ISBayHudson]

This is similar to something I've done - but I use proxmox containers nearly exclusively. I found the telmate provider and ran with it but later found bgp and realise now how better it is but switching seems like such a pain... I'll get around to it. Main comment refers back to containers and credentials. FYI connecting via API credentials has it's limitations as you can't created privileged containers (something I required to connect NFS shares from my storage device). With either of these modules you will need to switch to your root user credentials if you wanted to create privileged LXC containers. FYI I use terrafom for provisioning and ansible for config/ setup - keeping secrets in bitwarden and use the bitwarden ansible plugin to retrieve the secret

[@hushandnoise]

Not all heroes wear capes. Thank you Jim!

[@Jims-Garage]

Haha, thanks. You're welcome 😁

[@MrIvaroki]

Great video! Next step to deploy from git+cd ;)

[@Jims-Garage]

That's the plan! All in good time... soon hah!

[@moejoe4654]

Thanks Jim, I appreciate your videos a lot. btw it would be interesting to look into Ansible dynamic inventories for your next videos.

[@Jims-Garage]

Thanks for the feedback and pointer. I'll take a look.

[@georgelza]

question, how has this changed over time... have you completed the balance of this IaC deployment, aka the ansible scripts to stand up the entire K8S cluster. you Master and worker nodes, are they bare bones os deployment or have you "prepped" them with settings and packages... or would that sit in your ansible scripting... ?

[@Jims-Garage]

@@georgelza I haven't combined them yet. But it should be a case of writing a bash script to run terraform then Ansible. Perhaps something for a future video

[@georgelza]

@@Jims-Garage bummer... was hoping.

[@Jims-Garage]

@georgelza it's a double edged sword. Great the first time it works then very likely in 6 months time when you need it something has changed 😂

[@georgelza]

@@Jims-Garage ye... sounds like Tf and AWS/EKS also... whats the newest... do I go with this plugin or having played with it and the others do I go with bpg?

[@csd4ni3l]

Didnt you leak your api tokens at 9:00 and a bit before when viewing providers config?

[@Jims-Garage]

I did :) They're also in the example config on GitHub. Takes about 10s to swap them.

[@niceitguy2444]

@@Jims-Garage still you should consider using terraform variables for specifying the value as sensitive to not be display/be ofuscated at runtime. And declare your variables in a separate tfvars file so you can retain the logic and manage multiple environments

[@ivanaevsky]

Perfect!!! Thx a lot!

[@jdturner2813]

my OCD would not handle the non-sequential VM IDs vs node names...great work as always

[@dromer1967]

Thanks for this video! I struggled a bit to see the link between the cloud-init tempate I created in proxmox and how to specify that parameters of it in terraform so that the vm is properly created (my vm would never boot because the hdd definitions were wrong or something). That link is now much more clear to me so thanks very much for that! One think I was wondering however, why does your use of the plugin seems so complicated, building it yourself and all? Am I missing something there, perhaps because you are using a liniux(/wsl2) prompt and I am simply using the windows binaries? Because in main.tf I simply have this terraform { required_version = ">=1.7.2" required_providers { proxmox = { source = "telmate/proxmox" version = "3.0.1-rc1" } } } And this works fine without having to compile any binaries or plugins myself 😊

[@Justin_Jay]

nice, will have a look at the repo

[@andrewwarner3969]

I'm running the script exactly as you have shown in your video (with key variables changed of course) but some vms are not being created. For example, it will create master 1 and 3 but not 2. The Terraform process also never seems to finish even though the vms it has created are up. ??

[@Jims-Garage]

I sporadically witnessed this issue. Ctrl c and rerun it, should complete

[@imtiazali1628]

Thanks James. I tried telmate but it did not worked. I using now bpg provider and test the same. It will provision the required number of machines and create the host file for anisble then ansible will provision the k3s cluster with metallb. testing now longhorn and other apps provisioning. Thanks again for such informative videos

[@jhmc93]

Do u have a link of that video where you created the clone machine (23.04-non-KVM), Good video

[@Jims-Garage]

No, but follow the cloud init video and just swap the image

[@jhmc93]

@@Jims-Garage did u make a cloud init vid?

[@Jims-Garage]

@@jhmc93 yes, part of the Kubernetes series

[@georgelza]

any chance you have a Tf script that takes a controller image and a node image and deploys a K8S cluster... ? I have the commands to take the base ubuntu 22.04 based template (want to script it as ansible) and then add the various packages settings to prep it as either a controller or a worker node.

[@andrewivanof2940]

Thanks Jim. This doesn't work anymore with cloudinit_cdrom_storage :( maybe you can release short update video with repo update.

[@TheJollyRogerPirate]

i have this set up in my homelab, however any time i deploy a change to a single vm, all vms restart.. its getting annoying. starting to think its just a bug in the tf provider... does anyone know a way aroudn this

[@thomaspetit4312]

There is a `proxmox_cloud_init_disk` resource available. Is there a specific reason you didn't use this for the tutorial?

[@Jims-Garage]

Thanks, I wasn't aware of that from the documentation on the provider. I'll test it

[@iKn-ft2bc]

Jim, i wanted to do a school project with proxmox but im failing since months. I can get API Acess and Generate a testCloudinit but i cant remove it or manage it. i think i have problems getting the API keys. i know what they are for but i cant get it working. i dont really know how to get them theres something i didnt understood well. My school project is Automating proxmox with Terraform. then i can manage vms and containers- with this i want to get basicly an appstore running with little knowlage of linux and such for the end user. basicly a solution for 1 admin at a school to install many fancy things without reading much into them

[@luisenrique2138]

Hi Jim's great work thanks! i followed this guide and work fine to me... so how i can add soft packages to vm deployed? i need add some package after vm deployed but no add in image template.

[@hyperprotagonist]

Speaking for terraformation, if you moved out of the garage to, say, a loft studio… would you then be known as Jim’s Loft?

[@Jims-Garage]

Jim's Cellar coming soon 🔜

[@hyperprotagonist]

@@Jims-Garage😂

[@JohnWeland]

I hope that Talmate project gets legs under it and grows. It would be nice in a Proxmox cluster to be able to assign these VMs to various nodes in the deploy. eg three Proxmox nodes, this would then build a master and x workers per Proxmox node. I wonder if there is a provider for iDRAC7, could get real close to metal then. get three new dell servers, use Terraform to update them, provision disks, install Proxmox, setup the cluster, and CEPH; this run this to provision the VMs

[@ernestoditerribile]

Why haven't you switched over to VSCodium yet?

[@Jims-Garage]

It's a great project. I feel there's little privacy win when I'm still using Windows.

[@motionthings]

Just an fyi. Terraform went closed source. The open source fork is called opentofu

[@Jims-Garage]

Good to know, thanks for sharing

[@tonyc2227]

Opentu is the open source version of terraform now.

[@Pasukaru0]

I don't know why my comments keep getting deleted, so I'll try once more. We also use terraform at work. Will you also make a video on Pulumi? It's open source and more flexible imho.

[@Jims-Garage]

Thanks, if you're putting links it often filters it. I will take a look, appreciate the feedback.

[@PCMagikHomeLab]

And maybe the time packer from hasicorp?

[@Patterner]

to be fair, Terraform and Terrifying sound very similar.

[@mattiavadala7870]

the same for Ansible and Anxiety xD

[@rubenkhachaturov3309]

You need to define emulatessd = true in disks to get ssd=1

[@Jims-Garage]

Thanks 👍

[@rubenkhachaturov3309]

@@Jims-Garage I have posted my configuration example in your Disqord channel

[@Jims-Garage]

@@rubenkhachaturov3309 thanks 👍

[@ltonchis1245]

I was thought this infrastructure as code could redeploy Proxmox OS With my configurations And vm's and containers

[@MarkConstable]

Wake me up when you have an OpenTofu tutorial available for LXC containers.

[@AljoschaKrui]

You can Just as easy use ansible playbooks to spin up multiple vm's.

[@javisartdesign]

Terraform bad OpenTofu cool!

[@-rm-rf]

Tofu +1 😂

[@manuelschmidt570]

Very simpel view on the matter, but hey we are on the Internet so what do I expect. 😂

[@chfmrf9605]

Great video!

[@Jims-Garage]

Thanks

[@1988marksie]

I find the bpg proxmox provider is much better personally

[@ernestoditerribile]

I wanted to visit your website, but I'm blocked by your cloud flare.

[@Jims-Garage]

Thanks, but I don't have one.