Getting Started with cloud-init

cloud-init is an awesome technology that can be used to customize Linux images for deployment, that lets you do all kinds of neat things such as automatically creating users, installing packages, resetting SSH keys, and more. However, it's often shrouded in mystery. In this video, I'll walk you through using it to create a user, set the hostname, and install some packages.
☁️ Support LearnLinuxTV and Set up your own cloud server with Akamai Connected Cloud:
➡️ learnlinux.link/akamai
cloud-init links
🐧 Official page:
➡️ cloud-init.io/
🐧 Modules documentation:
➡️ cloudinit.readthedocs.io/en/l...
🕖 Timezone list:
➡️ https//manpages.ubuntu.com/manpages/bionic/man3/DateTime::TimeZone::Catalog.3pm.html#available%20zones
LearnLinuxTV Sites:
🐧 Main site:
➡️ www.learnlinux.tv
🐧 LearnLinuxTV Community:
➡️ community.learnlinux.tv
Support LearnLinuxTV
📖 Check out Jay's latest book, Mastering Ubuntu Server 4th Edition. Covers Ubuntu 22.04!
➡️ ubuntuserverbook.com
🙌 Support me on Patreon and get early access to new content!
➡️ learnlinux.link/patron
🛒 Affiliate store for Linux compatible hardware/accessories:
➡️ learnlinux.link/amazon
💻 Check out the Tiny Pilot KVM for your Homelab:
➡️ learnlinux.link/tinypilot
About Me
🐦 Follow me on Twitter!
➡️ learnlinux.link/twitter
👨 More about me:
➡️ www.jaylacroix.com
➡️ www.learnlinux.tv
Recommended evergreen videos:
💽 How to create a bootable flash drive for installing Linux
➡️ linux.video/flash-usb
🐧 Understanding Linux permissions
➡️ linux.video/perms
🐧 OpenSSH Guide
➡️ linux.video/ssh
📖 LVM Deep-dive:
➡️ linux.video/lvm
🔐 How to better secure OpenSSH:
➡️ linux.video/secure-ssh
☁️ How to create a cloud Linux server with Linode:
➡️ learnlinux.link/create-linode
FAQ
🐧 Which distro do I use?
➡️ learnlinux.link/mydistro
💽 My recording gear (commissions earned):
➡️ learnlinux.link/recording-stuff
#DevOps #Linux #cloud-init

[@shashikantovhal1769]

Thank you so much for creating this Video. Learning new things everyday . 👍🏻

[@michelangelop3923]

Nano tip: you can exit nano even faster, just hold down the ctrl key, then press once the S key, and then the x key without taking your finger from the ctrl, this way with just 3 moves you have saved your work and exited the editor, so ctrl+s, ctrl+x

[@bluesquare23]

Nano tip: Use vim instead!

[@AnzanHoshinRoshi]

Thank you, Jay. Saved for reference.

[@johncrunk8038]

I have been toying with cloud-init for some time and this helped answer some questions. I'm looking forward to your "in-depth" tutorial. LOL

[@lucavercellotti9959]

Real professional quality video.Very interesting argument, very good explanations. Thanks a lot

[@dorinxtg]

Great video. just one small thing, if you'll explain cloud-init: the qemu-guest-agent can be added using libguestfs-tools with the command virt-customize, so the template can have the agent ready and running when the user creates a new VM from the template.

[@lhxperimental]

Your video quality is really good. The text and other visual elements are sharp and clear. What do you use for capturing your videos? Any special codec/settings/bitrate?

[@ChuckNorris-lf6vo]

It must be taking advantage of 4k. KZbin is serving it as av01 (h264) it must be high bitrate capture.

[@verynice5258]

Thank you for everything

[@gdvissch]

Love the “silly” birth certificate!

[@abdouabdou1089]

Thanks Jay that's was helpful

[@mukto2004]

Love your contents

[@R3nj7]

Beau boulot !

[@aljeffcoat1327]

It would have been nice to know HOW to get to your starting point from "scratch", say with just an ISO and a clean Proxmox install. Found it elsewhere, but otherwise a good video.

[@GoGirlWTF]

Can you recommend the video/blog you used?

[@ArunJayapal]

When I followed these steps on ubuntu live server (non-cloud) image installed on virtualbox on I noticed the user folders didn't get created. Anyone else with this issue?! How to see what went wrong? Why it didn't create?!

[@Glatze603]

Thanks Jay, but isn't it faster to create a fresh system with all deinstallations, installations and configurations (for example with ansible) and then convert it to a proxmox-template? Then I only have to create a full clone, change the hostname and upgrade repopsitory or integrate it to ansible-inventory :-) A new system takes me 1-2 min. Thanks very much for your ansible-videos!!!

[@LearnLinuxTV]

Possibly - depends on the overall situation. You can get cloud-init down to being as simple and copying a single file into your install, and that’s as simple as it gets.

[@NickDrone]

So is cloud-init an alternative to ansible? Or more of a complimentary tool?

[@daniels1924]

Both fullfill a different purpose and can be combined together. Cloud-Init is meant for "machine creation". So basically, everything that happens on the first boot. Ansible is an orchestrator and configuration management, but cannot change settings of a VM it cannot access. Very often you will see something like: 1. terraform/ansible -> trigger instance creation in aws/azure/kvm/gcp 2. cloud-init/kickstart/preseed -> do the initial instance configuration (initial user/password/pubkey) 3. ansible/chef/puppet -> do further configuration management Furthermore, you can use cloud-init for bare metal servers, too. You will basically boot a prepared image with cloud-init in it and during the initial boot cloud-init takes care of the initial network configuration, users, filesystem, etc

[@wilkoazkijaromir9749]

cloud-init allows one to bootstrap mostly short lived VM's. Ansible allows one to effectively manage relatively long lived machines.

[@Rickety3263]

cloud-init and ansible are completely different. cloud-init is like a template for spinning up NEW servers (usernames, keys, etc) and used once ansible is for the ongoing management of existing servers and used often

[@giridharanr7803]

How do we keep a command to update Ubuntu for all software packages (“apt-get update” and “apt upgrade” ) in cloud-init

[@TunifyBasic]

nano tip: ctrl + s to save then ctrl + x to exit and you could select a part then just do ESC-3 to comment it.

[@dellis67]

I enjoyed this video, but I was wondering if there’s a difference between cloud.init and ansible?

[@sammagnum]

They are similar in that they both manage configuration, bu not much else. Cloud.init is for images, you install it on a running vm, configure it and create an image. Your configuration is set on first run of a new image. Ansible is has much larger scope. It use yaml files to manage running servers ... Generally. So Big Picture Cloud init is better suited for image configuration, where ansible is better suited for live server configuration. Ansible is also more of a swiss army knife. Where Cloud.init is very specific to image prep.

[@RetiredVet1]

How about a video on creating a boot_command line for subiquity? Ubuntu’s new server installation system. Creating this is one of the most arcane thing I have ever seen in the world of server management. Starting an auto installation should not be so hard.

[@vhoover3609]

What about using cloud-init with ansible-pull? Would that be counter productive, beneficial, ... Main reason why I ask is are there things cloud-init can't do that ansible-pull can do? Or visa versa

[@michaelventarola7100]

How do you integrate on the ISO for new builds?

[@mikechurcher9320]

So, now that you have it configured, how is it used? Can you show with examples etc.

[@conghuayu]

In my test, if the root partition is followed by a swap partition(debian 10 with default partitioning scheme), then the cloud-utils-growpart can not resize the root partition on boot automatically.

[@jeschinstad]

What's the point in using swap partitions on Linux in the 2020s? Back in the days, there were performance issues with using swap files, but these days, I wouldn't use a swap partition for anything.

[@mikemilkshake205]

Would you pls do a video about ltsp5 ?

[@brightedward7783]

This is awesome But need a video on "qm" command in Linux

[@maherkhalil007]

Great, How can I run script using cloud-init?

[@MarkConstable]

As usual, I am late to the party. Jay, you mentioned a wiki page with copy n paste details, but I can see a link to it? I guess I will just have to watch the video again at normal speed and take special note of the parts I need to modify.

[@cgarafulic]

Hello Jay, I listed all my groups with the command cat /etc/group and there is not netdev group. I am running Ubuntu server 22.04. I read that Debian has this group. Does the Ubuntu version I am using has netdev. If yes, why is not listed in groups? Thansk!

[@bjornarsimonsen7592]

Can't you just put the users and packages already in the image and wouldn't you want the hostname to be something different each time? Really curious.

[@nixeverything]

I think the advantage here is that cloud-init can work on other distros with the single file instead of a image of one particular distro? I am only in this mess of learning cloudinit because I ran into problems of cloud-init resetting my ansible changes >.

[@stephenhookings1985]

This is what I am hoping to find. Ability to set a template the for hostname in X do boot machine and apply cloud init to make hostname. Seems easier to have two images: master and worker. Each new master is a create master then rename to master(n+1). Similarly for each new worker. Still looking for this workflow

[@chromerims]

Interesting

[@flesz_]

So cloud init is something like sysprep for windows?

[@magetaaaaaa]

That's what I thought when I came across these videos.

[@ambig1]

after seeing this i am puzzled on why dont we just use ansible to do this on 1000s of servers at once ?!

[@lvbio]

How are you going to connect to those systems with Ansible? If you don't have an SSH public key on them, you're not going to be able to connect. Cloud-init comes *before* Ansible. It's what you use for *initial* setup. You can use this to create a template, that has your pre-populated key (for example) and then make your 1000s of VMs that you then use Ansible to perform actions on. Make sense?

[@HubbertSmith]

hi - my cloud-init clean failed Error: Failure during run-parts of /etc/cloud/clean.d: Runparts: 1 failures (99-installer-use-networkmanager) in 2 attempted commands (yes I checked /etc/systemd/network/99-default.link doesnt exist) any ideas?

[@Just__habiba892]

after I flowing all steps I found this error 2024-05-06 11:44:29,991 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 9 column 4: "while parsing a block collection in "", line 9, column 4: - name: sichost ^ expected , but found '?' in "", line 10, column 4: lock_passwd: false ^" I need your help plz

[@tolpacourt]

You lost me with the hostname business. My server (qemu-kvm host) already has a hostname. Are you saying we could or should set a hostname for the cloud-init _service_ ?

[@alexkaouris6755]

dpkg -get-selections? Why not dpkg -l ? :)

[@aungjee]

" yooo Neymar sunny init ? "

[@HenryBiglin]

Ugh this config does not work anymore!

[@johncrunk8038]

a good starr, but there is a lot of detail to learn.

[@atexnik]

Well, I think it doesn't get any serious attention because there are already plenty of provisioning tools available, and developers are already overloaded with trying to remember all this stuff.

[@emilne83]

Yes, but those other provisioning tools all require some baseline network connectivity to be in place in order to complete their provisioning tasks. Cloud-init is very useful if you have an image that needs to be deployed as multiple instances or in multiple enviromments. Cloud-init can load a simple bootstrap set of network, ssh and user config so that some other tools like ansible or terraform can then provision the remaining, more complex settings. Without cloud-init, you'd either need to have unique images for each instance you wish to deploy, or else you'd need to setup that initial connectivity config manually using a console.

[@rs-tarxvfz]

I appreciate your effort but this is NOT the "getting started". As absolute beginner, it is not much helpful beyond typing some commands.

[@priit7777]

Could have told us wtf this cloud-init is in the first place and why we even want it at all and why we should keep it installed. Discovered it in my default Ubuntu Server installation, came to look for information what it is -- still none the wiser, except that it's a bloat I probably can just remove and forget I ever had it.

[@TecraTube]

"no, I'm not creating a video to just RTFM". Proceeds to talk to us as if we're children. I wish I could tell you, anyone watching this (mostly), should understand some things right off the bat, like running sudo and root. Purging the app to reinstall it, really? And another thing- Ubuntu is lame AF

[@LearnLinuxTV]

For Ubuntu being so “lame”, it sure has quite a few downloads and acts as the base for a number of popular distributions. If you’ve done case studies on what there intended audience should understand, I’d love to read your notes. 🙂

[@KnightRiderOfVoid]

SPOILER ALERT: *VERY* LONG COMMENT with lots of constructive criticism and some hopefully helpful advise. @@LearnLinuxTV I mean Jay, the guy has a harsh and wrongly oriented way with words, but you can't deny he has some truth to it. I should be writing a novel or a blog post instead this but here it goes, let's hope for the best. I know you want to be as thorough and explicit as possible, but I tell you one thing. I like your videos, like a lot! But man are you slow paced and sometimes repetitive. Don't get me wrong, your audience needs to be clear and I get your point, but you really talk to your audience as some children who can't follow a regular paced conversation or a list of steps. I have (for a long time) refrain myself from commenting this, as I am aware it may seem toxic and be not very welcome from your side. But hear me out, you are trying to teach advanced procedures to your audience, you want them to succeed. You need to take them seriously and not assume they can't just because some people want everything done to them. You have to be clear, explicit, but you get repetitive and condescending, and you immediately start to received hate and ingratitude from the same people you are helping. Been there, done that. I know it's not your purpose, but that's the truth whether we like it or not. Now, I *really* don't want to just sound like a douche and give you all the negative feedback without thanking you for your work and sharing a lot of valuable info. I DO appreciate you and this is why i trll you. I watch a lot of your videos, and *every single one of them* I watch at 1.5x speed, man you have to listen to yourself, at default speed I often get lost in your repetitiveness and deviation from the topic, your pace is so slow that I get bored and close the video to find someone else who gives me the info I need in the least amount of time possible. So my advise about that would be that you speed up a little the pace at which you speak, it gets troublesome to follow just by the slowness of it, even if the info you're giving is gold and your data is correct. When I watch at 1.5x, I can follow you WAY easier than at your usual speed. Another thing is something OP referenced. You can't be showing peoplehow to use sudo or clean packages and files in every video, may be a little explanation here and there is totally fine, as long as it is something needed for the tutorial or process, otherwise you're better off making a dedicated video on the tools and reference people to that video to clarity anything out of scope. You have to pick a topic, and leave the rest out. If you don't, you just repeat yourself and get that condescending vibe that most people hate. Nobody wants to be treated like an idiot, and that's what it feels like when you do that, even though I know it's not your intention. Just try to be a little more assertive and precise, no need to go on every detail all the time. If you point people to other little videos or yours where you explain specific things like sudo, package cleaning, etc. You may even get more views, more videos and variety to expand your channel, more time to focus on new topics, etc. I'm not saying you should just ignore peoples need for knowledge and the considerations needed for your tutorial. It's not about leaving them out completely, it's about focusing and giving precise info without distractions and repetitiveness. PS. I really hope you get the right idea of my feedback and advice, hopefully you don't take it personally and it helps you do an even better job than you are already doing. Thank you for sharing your knowledge and keep the good work. Bye Jay! Edit: KZbin's markdown messed up some parts, I think it's fixed now.

[@wsstory313]

Thank you for all of your videos on Proxmox! I learned a LOT quickly that would have taken me some time to figure out on my own. In this video, I tried it with Rocky Linux 8.7. cloud-init init resulted in errors that seem to be related to the colon. My distro doesn't have the newer cloud-init. It comes with 22.1-5.el8.0.1. I get errors like the following: Is there any way to make it work correctly? 2023-01-28 01:15:49,644 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 13 column 7: "while scanning a simple key in "", line 13, column 7: sudo ["ALL=(ALL) NOPASSWD:ALL"] ^ could not find expected ':' in "", line 14, column 7: shell: /bin/bash ^" 2023-01-28 01:15:49,682 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 13 column 7: "while scanning a simple key in "", line 13, column 7: sudo ["ALL=(ALL) NOPASSWD:ALL"] ^ could not find expected ':' in "", line 14, column 7: shell: /bin/bash ^" 2023-01-28 01:15:50,196 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 13 column 7: "while scanning a simple key in "", line 13, column 7: sudo ["ALL=(ALL) NOPASSWD:ALL"] ^ could not find expected ':' in "", line 14, column 7: shell: /bin/bash ^"

[@emaayan]

15:32 actually according to cloudinit.readthedocs.io/en/latest/topics/examples.html#run-commands-on-first-boot it will actually run on EVERY boot.. you can configure it using cloud-init-per but using runcmd section is for first foot i've been having a very very hard time trying to make autoinstall with ubuntu, and there things there which are very confusing