My NEW HomeLab automation platform // Kestra

In this video, I will introduce you to the new automation platform, Kestra, and show you how it can help you automate, schedule, and orchestrate your IT stack. We'll go through the features and capabilities of Kestra, including its ability to run scripts, query APIs, deploy infrastructure, and seamlessly integrate with popular tools like Terraform, Ansible, and scripting languages such as Python, Node.js, and PowerShell. Join me as we install the free open-source community edition of Kestra and explore some interesting use cases and examples for automating your HomeLab. Special thanks to Kestra for supporting this video and making this comprehensive tutorial guide possible. Let's dive in and automate your HomeLab!
References
- Check out Kestra's Github: github.com/kestra-io/kestra?u...
- Join Kestra's slack: kestra.io/slack?Yo...
- Traefik Tutorial: • Is this the BEST Rever...
- Authentik Tutorial: • Secure authentication ... a
________________
💜 Support me and become a Fan!
→ christianlempa.de/patreon
💬 Join our Community!
→ christianlempa.de/discord
________________
Read my Tech Documentation
christianlempa.de/docs
My Gear and Equipment-*
christianlempa.de/kit
________________
Timestamps:
00:00 Introduction
01:14 Kestra Overview
03:20 Install Kestra OSS
06:39 Kestra UI
07:53 Create a first Flow
15:37 How to find examples?
17:41 Automate Python and Discord
37:29 Final thoughts
________________
All links with `*` are and/or include affiliate links.

[@BenThatOneGuy]

Much prefer these sponsored product demos over generic "Sponsored By:" preroll ads. Preroll ads are just pictures and marketing. This is actually useful, because it shows what the product does and i can see if this is even something i would want to consider spend my time or money on. A good demonstration of the workflow, some features, and what it's like to actually use and configure the product. Much better representation of what the sponsor is trying to promote. You need to be supported to make content. Companies need to pay their developers. We use this content to make decisions for our own labs, or even our businesses. for once in the advertisement funded internet economy, it feels like everyone wins. Always love your content, sponsored or not. Keep up the good work :) If you are planning on more Kestra videos, I would personally love to see more on the ansible, Teraform, and Git plugins.

[@christianlempa]

Thank you so much for this feedback! It makes me so happy to see y'all in the community enjoy these videos. :) I think what it comes down to is the right balance, between sponsored content and vendor independent content

[@mxc2007]

@@christianlempa Also thank you for the ADVERTISEMENT banner. Makes it really easy to know if you're doing sponsored content or not.

[@filmcarbon]

Exactly, all I'm seeing here is a reinvented Gitlab CI

[@laurentiusjudhianto6631]

This is something Ive tried before and I am still contemplating between this and Rundeck. Definitely a great video to get started with it!

[@christianlempa]

Thanks!

[@FoX84tac022]

I was just about to ask how it compares to Rundeck. We use Rundeck at work, so naturally I stood up my own instance to play with and learn. But, I'm always open to learn a better tool.

[@5ilentB0B]

😱 I just finished the Zitadel/Netbird/Traefik setup based on your Netbird introduction. Thanks god is Sunday and thanks for showing Kestra.👍🏼

[@christianlempa]

Cool! Hope you enjoyed it :D

[@donbeave]

Kestra is amazing. Thanks for this video. We decided to migrate from AirFlow to Kestra mostly because of its Cloud Native design.

[@nigelpa74]

Thanks for this kind of content. 🎉🎉 Your help is always good for new projects in server life

[@christianlempa]

Thank you so much :D

[@bubi352]

in before anyone complains again: have a look in the upper right corner: ADVERTISEMENT

[@christianlempa]

😁☝️, hope y'all find it still useful, just like me!

[@ThatNateGuy]

Food, electricity, and equipment all cost money!

[@marcoschirrmeister]

Why would someone complain? We all know the most videos of the know KZbinrs are advertising in one form or another.

[@timderks5960]

IMO, advertisements should be clearly identified by text on screen that can't be missed (aka a pause in the video, only text on screen), and a clear audio announcement that clearly says it's an ad and in which way it is an ad. Why both? Because some may be watching without audio, others may only be listening. Plus deaf and blind people obviously. A small, translucent text that says advertisement and a "this video is supported by X" is not enough when your title is "My new homelab automation platform". Again, my opinion. Especially when your channel is known for tutorials and mostly unbiased "hey look at this cool stuff" videos, a title like "My new homelab automation platform" is really not cool, especially since it's very unlikely that it actually is. Obviously, content creators can and should do these kinds of videos, especially if that enables them to also do unsponsored videos just because they think we like it or because they like it. However, clearly stating that this is an ad doesn't change anything regarding the income form the video. It does however show that you respect your audience, and aren't trying to scam them. I know that's not what Christian is trying to do, but that doesn't mean he can't improve.

[@cheebadigga4092]

so? whats the problem?!

[@DanZimmerli]

This looks really impressive and it was pretty easy to deploy on my docker server but if I’m not a coder I’m not sure what I would use it for. Great video!

[@christianlempa]

Thank you! Yes I agree as a non developer just like us, it will take a bit time to learn it

[@augustindaligaut8478]

Great video! Will definitely try Kestra's product! Seems easier / more powerfull than Airflow / dagster

[@christianlempa]

Awesome! Glad you like it

[@dennisk4813]

Great Video 🎉 I‘ll give it a try in my Homelab

[@christianlempa]

Thank you! 😊

[@RABWA333]

looks quite cool took , thanks a lot

[@christianlempa]

Glad you like it!

[@Ivan-ny2dq]

Been waiting for this video since you announced the Kestra sponsorship previously, great work! My collegue has the idea of automating his foodplanning from the Mealie docker container to export shopping lists and prepare online food orders from the store Since none of us are code-ninja developers we might get a struggle but I bet Kestra will give us a good boost on the way to a solution

[@christianlempa]

Thank you! That sounds awesome :) I will do another video about Kestra at some point and show you more examples, so stay tuned

[@theTrigant]

Or just use tandoor with a built in shopping planner

[@Ivan-ny2dq]

@@theTrigant Thanks! Will check that out :)

[@jus4027]

"of course I was suprised when it worked as i didn;t test it" proper made me laugh at the end of the tute, another great howto Christian

[@christianlempa]

:D thank you bro

[@ricardoalcantara5846]

Awesome project.

[@christianlempa]

Thanks 🙏

[@joelv4495]

Very cool. The flows remind me of CI scripting in GitLab. Is there a way to maintain the flows and other configs in a git repo?

[@christianlempa]

I haven't seen it - yet. That's a good question for the kestra devs ;)

[@tenThs-ni6is]

Were you able to get this to work with traefik labels?

[@Sapop3rro]

Reinventing the wheel.

[@cheebadigga4092]

thank you very much!!!!

[@christianlempa]

Thank you 😊

[@tronskywalker3633]

O: love the tool! ...can you have in one workflow multiple (scheduled) triggers? probably yes. will play around with it!

[@christianlempa]

Awesome! Yes that's possible, the triggers: can held a list of different types of triggers as well

[@dawnrazor]

How does kestra compare to ansible? Do they do the same thing? Is kestra a newer version of ansible? I have a big todo item on my list which was to learn to use ansible to help me automate my dev setup, but I’m thinking it could be superseded by kestra. Luckily I haven’t started yet so I have no attachment to ansible

[@christianlempa]

Kestra is more like an orchestrator of automation pipelines, it can run several tasks, even execute Ansible playbooks. Ansible is a configuration automation engine.

[@h.selcukbeyhan8252]

looks pretty useful

[@freestudymusic550]

Awesome video ❤

[@christianlempa]

Thank you 😊

[@haraldhacker]

Yo Christian. I need I mini server which I can use for some VMs i would like to create via proxmox and offering a storage pool of 20tb plus. what do you recommend? it should not consume much power. Thanks man!

[@christianlempa]

Maybe look for more modern hardware such as mini PCs, gives you good performance and low energy cost

[@90DaysOfDevOps]

Nice video. I have an idea for this that involves creating workflows of spinning up multiple demo environments in cloud but also within vSphere and other platforms. The Kubernetes plug in doesn’t look to advanced just yet either. Be good to see a helm trigger maybe.

[@christianlempa]

Thanks! :) Alternatively, you could just spin up a new python script using docker, install the requirements for kubectl and helm, inject the secrets, etc. And you could probably script it.

[@nahakuu]

I tried to go to their website but it is not loading :{

[@raughboy188]

I've look into it and you get a lot just for opensource version, entreprise on the other hand adds convenience.

[@senaddizdarevic7849]

Thank you, Christian. That was a very informative video. I am unsure about the purpose of this software. It seems like just another process to run, and everything it does can be achieved with more mature technologies that it is using anyway. For example, when it comes to monitoring like in your example, there are much easier ways to do that using Grafana, Lambda, GitHub Actions, and so on. Why should I use this instead of those solutions? I does not make sense to use one more wrapper around the existing solutions and maintain one more software in order to get a "nice" UI interface.

[@anantmishra6783]

7:37 lol :'D

[@christianlempa]

Who likes light mode? :D

[@SB-qm5wg]

This is amazing.

[@christianlempa]

It is yeah

[@Pray4Tre]

Can someone help me understand the local storage? I tried commenting out the volumes on top and hardcoding the volumes such as ./postgres-data and ./kestra-data but it wouldnt start up then. I would like to keep all this data inside my /dockerdata/kestra/ folder where the docker compose is. I've never really messed with driver: local volumes before. Thank you!

[@Pray4Tre]

I got it up and running the way I wanted, I was missing the ./tmp/kestra-wd. That with commenting out the top volume driver: local part worked for me. If this is a wrong/bad way to go about this though please let me know!

[@christianlempa]

Good to know, I'm not sure why this is the case for though

[@pablojefcobar]

Does it support ARM?

[@halllo54321]

In your opinion, Can IT replace ci/cd Pipelines ?

[@christianlempa]

It can, question is if it fits your needs :)

[@Glatze603]

Hi Christian and thanks for the video. Kestra seems interesting but (maybe because I am not a coder) I do not understand anything 😞 I automate many tasks (linux, docker and proxmox commands and scripts) in my homelab with "chronicle task scheduler", what is open source too and has a nice web gui, where you can config everything.

[@christianlempa]

Thank you so much! I feel you, since I also don't have a dev background, it first was hard for me to adapt DevOps processes and thoughts, however, with enough patience and effort I feel I'm slowly getting there :D

[@user-tw2kd5wb5i]

Really wanted to see a whole showcase of the features before the "how to deploy it" part

[@ratage]

This! I really wanted to know WHY I should try it before spending time learning how to deploy it. I'm 8 minutes in and I know that I can change between dark/light mode and how to change the font, but still have no idea what this software does!

[@drbyte2009]

Hello Christian, great video again! What do you like the most, Ansible or Kestra? I just started recently with ansible, so i still have a lot to learn 🙂

[@christianlempa]

Thank you! :) To me, Kestra is more an orchestrator to run automation scripts, it can even run Ansible Playbooks as part of a more complex pipeline logic. Think of it like Ansible as a wrench or hammer, and Kestra is your workbench to use all these tools.

[@drbyte2009]

@@christianlempa Thanks for the reply, i also start with Kestra to see the learning curve !

[@drbyte2009]

@@christianlempa I tried to run some ansible playbooks, but i'am stuck on the ssh key. I have been searching but i can not find the instructions on how to use an existing ssh key.

[@fredrik354]

Without finding any relevant examples I've managed to successfully run the Ansible plugin with sshkeys. Slowly but surely enjoying the process of learning Kestra :)

[@christianlempa]

Awesome! Yeah it’s a bit tricky, I’ve gone through some trial error myself :D

[@janiel471]

How do u think between this and portainer?

[@christianlempa]

It's hard to compare those tools, as they serve different purposes. But I don't use Portainer much these days, and switched more to automation tools like Kestra.

[@FaiT90]

This reminds me of a luxury version of cronjobs. I'm going to play more with this then my wife! My wife says "Thanks Christian!"

[@christianlempa]

😂

[@mikrom]

I just played with an ActivePieces recently. Which is more like IFTTT replacement.

[@christianlempa]

Sounds nice as well

[@8eck]

Useful only for long running workflows.

[@ChrisRyder-de]

would be great to see 2 things. 1) how to install this SECURELY in Azure with ingress and authentication and 2) how to use this with Terraform and Ansible to make a VM in Azure. Everyone avoids Azure like the plague but it's used a lot in corporates.

[@gizlas]

SSO as Enterprise only features feels bad. I get that you have to place the barrier somewhere, but SSO should be included in the free version, especially since it can be solved with a proxy.

[@dkryptonut]

I finally sit down to try and get this up and running, and do you think I can find the Kestra documentation that tells you how to run this in docker-compose without the user set to root....like it says to check the documentation in the comment above it? 🤣🤦‍♂

[@dev-akeel]

I can't understand this video properly, can you guess out what is missing in me? Being honest I don't know what this video is all about but I feel I need this technology it is going to make my work easier. I am frontend developer with experience of 2+ years, I recently moved on to backend, I have decent knowledge of docker and basic knowledge of docker compose.

[@metalmasterlp]

@7:35 my eeeeeeeeeeyyyyyyyyyyyyessssssssssss :D

[@christianlempa]

:D

[@jakobkristensen2390]

Is this like a self hosted Zapier?

[@christianlempa]

Not quite, because it’s a not so easy, but also more flexible and customizable

[@neckfireteam6115]

7:36 warning flashbang

[@christianlempa]

:D

[@greendsnow]

You mean `podman compose`... Podman is the new docker now

[@ArtemMelanich]

Kinda a Github Actions but different flavor

[@christianlempa]

Yeah, it can serve the same purpose, however you got a bit more control over the execution process and variables, also the UI is a huge benefit in my opinion.

[@1974best]

you gotta take a break with these videos... i cant keep up... :D

[@christianlempa]

Haha yeah I know the feeling, take it easy buddy ☺️

[@CharlesFlippet]

This looks like jenkins on steroids

[@kelvinkwong6365]

try n8n

[@christianlempa]

I'll do at some point ;)

[@Sarahbolesofficial]

I would love a video where the product the video is not the sponsor of the video.

[@christianlempa]

I hear you! I definitely want to do much more vendor-independent videos again in the future. In the case of Kestra, I found it to be such a great fit because I is super valuable for me and (hopefully) you all as well 😁 But don't worry! In the future, I can promise there will be more videos where the sponsor isn't the main topic of the video (with just a few exceptions where it just makes a lot of sense) Hope you're all good with it. ❤️

[@neolinux]

n8n ;)

[@christianlempa]

Maybe I'm going to check it out ;)

[@toniramirez6784]

Sounds and looks like …. Cloudslang 😅

[@patriciadelimadelemos6392]

First!!!!!

[@Br4mVAL]

so sad that every video is now becoming an advertisement.

[@christianlempa]

it will change soon, don't worry ;)

[@f4ture]

Why is there another sponsored video? Since you were paid for it, I don't trust anything you say. I had hoped for a regular video without being advertised.

[@Glatze603]

If you do not trust what Christian says, stop watching his videos! Christian introduces us to a topic where at the end of the video you can judge for yourself to what extent you find the content good or useful or not. Questioning everything just because the manufacturer of the product is also the sponsor of the video is a bit too far for my taste.

[@DudeItsDallyBoy]

Yes but he is right it's a sponsored video. Take it with a grain of salt. Besides, we have already had apps like this. N8N does everything this does and more, and it's open source

[@Glatze603]

@@DudeItsDallyBoy I don't think he's right in what he said. I think it's wrong to question someone's complete credibility just because they get paid for the work they put into a video. That doesn't mean that everything that is shown as shiny in the video has to be gold 🙂

[@christianlempa]

@f4ture I can understand your concerns. To be honest, I get a lot of requests about paid videos, and I only accept deals where I trust the brand and the product to be valuable for you. However, I have to admit in the recent past it was a bit too much, so I’m planning to to more videos where the sponsor is not affiliated with the main topic of the video. I still hope you find those paid deals valuable and maybe learn something new, even though I’ve been compensated 😊

[@bubi352]

well, then watch another channel, i would say - easy as that!

[@Ruslan-UA]

don't understand why so many negative comments over the sponsored videos. The presentation discovers really well the initial ways the product can be used for and then it's your decision to dive deep into or skip it depends on whether the free features cover your needs. It's free content for you, cmon!

[@christianlempa]

Thank you so much for this feedback, bro! It makes me happy that most of y'all enjoy the content and find it valuable :)

[@Ruslan-UA]

@@christianlempa thank you for your effort to make the homelabers life easier. Waiting for your next top-notch videos.