Docker vs Kubernetes, what's better in a Homelab?

[@cantthinkofabettername7016]

Docker is the way way way easier, manageable and overall better solution for a homelab. This is why i run kubernetes in mine.

[@durgeshkshirsagar5160]

Highly laughable 😄😄

[@0xN1nja]

Had us in the first half ngl

[@syrus3k]

Well, what's the point of a homelab? To learn right.

[@kingcobraDE]

Team K8S - not because I need it, but wanted to learn it. Now it became my first choice when deploying containers.

[@christianlempa]

Glad to have you in the team! :D

[@apolloeosphoros4345]

Same here. I stared into the Kube madness and it stared back into me... now I'm hooked on how easy it makes certain things.

[@majorgear1021]

is it better than k3s? I found k3s to be a nightmare to keep running. container would be stuck in “starting” and Longhorn storage “volumes” would be degraded, also for no apparent reason. I threw my hands up and went back to docker compose and gitops for running apps.

[@sergeygr]

Personally I still use Docker in Swarm mode with multiple master and worker nodes (total of 6 nodes). With Traefik as LB and Portainer for management. Also a HAProxy in front of it. Also I use a K3s cluster, created with Techno Tim's ansible playbook. For a learning purpose. As for now, I dont see a need to switch to the Kubernetes as Swarm does everything I need.

[@kareemschultz]

Can you provide some information on how you did this I was really hard today to get this done with nginx proxy manager and Keepalived

[@B20C0]

Another pro for k8s in a Homelab: You learn it. It's great that Christian made this point: If k8s is something you may come into contact with in your professional career, use it in your homelab.

[@hansaya]

I bang my head against a wall for months over k3s but after I figure out the basics it was the most rewarding platform I ever used. I got everything automated to point I can start up my home services in minutes even if the house burns down. K3S makes managing my home services lot easier and makes it super easy to play with new projects

[@christianlempa]

Haha similar experience like me :D

[@lewismalcolm3536]

Wonderfully done video Christian. I've been reading up on Kubernetes in the last month or so and had thought that I'd port all of my applications to a kubernetes cluster some time this year, but after doing more research I started to realize just how deep the Kubernetes rabbit hole goes and started setting up things on Portainer instead about a week ago. I remember seeing a video from another creator targeted at new home-labers that said "If you think you need Kubernetes you don't" that felt very condescending since it didn't explain why that was the case, thus I ended up researching it more. If I had seen your video back then instead, I'm sure I would have picked up Portainer instead and it would have saved me a lot of trouble - I'll be sure to mention your video if these conversations come up. Thanks!

[@christianlempa]

Thank you so much! :)

[@javiersp]

Same wagon here. I have tried k8s on a proof of concept production environment on the cloud and after much struggle and headache I am setting all up on a swarm. I have no doubt that k8s offers more advanced clustering capabilities but if you properly set up your nodes on different availability and fault domains, you actually don’t need anything more than a swarm if you are going to run just a couple dozens of containers. K8s seriously needs an easy, small clustered version if they intend to target small and medium sized business.

[@Droiduxx]

What about docker swarm mode ? People ignore this most of the time in the homelabbing community, but to me it seems much more relevant in a homelabbing context...

[@christianlempa]

Good point, honestly this is something I should have done from the beginning, :D but I will take some time next year to learn it and then I will probably add it to my stack.

[@DavidChad]

Thanks! Super helpful content!

[@christianlempa]

Thank you so much for the support! I'm glad it was helpful :)

[@kareemschultz]

I would like to see a video on docker swarm with shared storage such as glusterfs for high availability for docker containers

[@christianlempa]

Hmm I still haven't found the time to look into swarm :(

[@itssoaztek4592]

Great summary, very useful for people like me who have not yet worked with docker or kubernetes. Much appreciated!

[@christianlempa]

Glad it was helpful! :)

[@dandogamer]

Nomad might be a reasonable alternative to k8s, its architecture is a lot simpler and it works great with the consul service mesh

[@MrWadezz]

nomad is indeed a lot simpler than kube, and for 95% of homelabs (and organizations), it does everything you could possibly want. It is also imo way more flexible since the integration with consul and/or vault is optional, and can be done later if you're just getting started ! Glad to see a fellow nomad user in the comments !

[@christianlempa]

Still didn't have time to look at nomad :(

[@MrWadezz]

@@christianlempa it's pretty cool ! especially once you couple it with consul and vault for the service mesh and secret management features :)

[@andrebalsa203]

In one word: both ! Very good question and nice discussion, thank you.

[@kektozaur]

I tried k8s a couple of years ago, but there weren't many charts available for the stuff I was running and having to make them from scratch is tedious. Docker compose saved me from alot of headaches.

[@yifeiren8004]

Kompose can automatically translate docker compose file, except volumes.

[@EllisWhitehead]

I kind of despise kubernetes because its too complex and unintuitive to manage on the side. I used it for two years for both home and work clusters, but after my initial enthusiasm, I concluded that the struggle wasn't worth it. I decided to switch to Docker Swarm (with glusterfs shared drives) two years ago, and I've been much happier with the simplicity of it all.

[@procheeseburger_2]

Docker is easy.. K8s is amazing. Like I CAN do it in docker.. but if I spend the time in K8s it can be much easier. You also bring up a good point around 8:00 when something breaks it can be almost impossible to correct it and the whole cluster is dorked.

[@dandogamer]

Yeah sometimes you can fuck up the cluster real good, but if you have a good fundamental understanding of the components that make up the cluster and how they interact with each other along with a basic level of linux / networking you can debug it pretty good

[@arimil.]

I've ran into this after migrating my old server to TrueNAS which pushes you towards Kubernetes, many things that I felt were easy to setup with Docker became much more complicated. TrueCharts has been really helpful with this though. I do believe that Kubernetes is overkill for my use case, since I don't see me ever needing more than 1 replica for anything.

[@alvsanand]

Great video. Here one guy running everything in k3s. As Cristian said, It is 100% worthy my time spent making it just working!

[@christianlempa]

Awesome 👏

[@Ecker00]

Went with Docker Swarm for latest setup, get most the benefits without too much complexity. 👍

[@christianlempa]

Also a nice alternative!

[@blackpaw29]

I spent days trying to get a k8s cluster running, got there eventually, but never did manage to get it to recognise my ceph cluster for storage. Eventually got up and running with Nomad, wouldn't say it was simple, but it made a lot more sense than Kubernetes, a lot less fragile moving parts and well documented. Has been running my DB's and web apps for weeks now, without a hiccup. Using Fabio for load balancing.

[@mrb180]

they're not mutually exclusive, I think they both have their place, Docker for running containers and Kubernetes for orchestrating them. But AFAIK Docker Swarm can take the place of Kubernetes in a small environment, like a homelab, providing similar functions.

[@sgt_mate]

yes, thats how its somewhat supposed to be done. kubernetes as an orchestrator does not run the container itself. under it there is the container runtime, which does that. comparing swarm mode and kubernetes is far more interesting, as both tools have a similar goal but are made for different usergroups.

[@IAmNumber4000]

Using managed kubernetes (DigitalOcean) was very cool and actually simplified some things from my previous docker-compose setup, but for a homelab, there is no way I would want to deal with setting up a cluster from scratch in a homelab tbh. Docker-compose is more than enough for me. This would probably be different if I had a server rack with multiple computers in it. But I don't. I have like 1 NAS and 1 server.

[@mickolesmana5899]

i will give my 2cents here, there are 2(3 to be fair) type of homelabs. 1. Media type 2. Calculation type 3ish. Learning type. Most of us will fall into category 3, learning homelab for the sake of learning, but when it come to actual usage, most of the hobbyist will fall into type 1, media server, home automation, or maybe web serve. Where HA and distribution architecture maybe good but not a priority. Hell even an overkill in my opinion. And the second type, where i fall into this category, Using homelab to train AI on Torch, deploy spark job, and data analytic. Where distributed system is a first class in a most of out of shelf products (Kubeflow, YARN, MESOS, Horovod), and i mean why won't you not parallelize your calculation. K8S spin up , manage, and spin down my spark and horovod very nicely. People are shocked when i told i only has 8 core Ryzen and k8s-ing my system. I am not using K8S as a HA but instead as a distribute compute unit

[@mr_wormhole]

I have suffered 2 years in k8s, kind, minikube, microk8s, I have learnt nothing until I started self-hosting my k3s non-HA setup on a small cloud provider with k3sup It is hard but noone said it is going to be harder than learning locked-in provider like AWS/GCP/Azure, it is also even debatable how much people don't know about systemd and container internals when people start learning docker, learn the core not the tools, kubernetes gives you all the aspects when you know the core, freedom is a hard road but enjoyable once you learn fundamentals / basics + charts + operators, you will be unstoppable

[@dandogamer]

yeah exactly this! Learn basic linux, networking, containers, processes, ssh etc. K8s and docker are simply abstractions on this that make it possible

[@keithwolfe1090]

Docker for my infrastructure services, 5-node Microk8s cluster for projects, and a single-node OpenShift cluster (work-related). I can do just about anything with this setup :)

[@piperjohn_3]

You are so good at this, thanks.

[@christianlempa]

Thanks ;)

[@sgt_mate]

Interesting video, currently working on a comparison of swarm mode and kubernetes. But im doing it for buisness usecases only (we are using kubernetes in production, where i work). Also maybe check out swarm mode, it is a far easier orchestrator, integrated in docker hosts, combinded with compose its a pretty good alternitive for smaller deployments. Sory for my english, as Im a german native speaker.

[@mrwalter1049]

Your English is fine. No need to apologize. BR from fellow English as second language user from Finland

[@jbauson1]

It should be Docker Swarm vs Kubernetes

[@arshamalh]

What do you think about Nomad? It would be great if you make a video about it and tell us your opinions ✌🏼

[@christianlempa]

Haven't tried it yet

[@robynrox]

Okay... so I think for me it has to be k8s. I don't need the power at home, but I am a software engineer, and developing that knowledge would help up my game. Thanks!

[@Glitch_860]

How does Kubernetes compare to Docker Swarm? I currently run a Docker Swarm with multiple separate nodes, running portainer to use for easy management.

[@christianlempa]

No idea, haven't tried out Swarm, yet.

[@Glitch_860]

@@christianlempa When you do. Would be a cool topic for a video ;-) In the mean time setup of a kubernetes cluster is in my future.

[@christianlempa]

@@Glitch_860 Yeah agreed, but that's not happening in this year I suppose :(

[@dmckrk]

Good comparison, but once you get into k8s you will never look back. Managing etcd, backups, storage and whole resilience is some difficulty but still best way. Also there are many great tools that allows to manage and debug cluster. Its not that bad :)

[@christianlempa]

Thank you! :)

[@tba77]

I just started using talos to spin kubernetes clusters and I would say that it's really a great solution it's secured ready for production high available and with the minimum needs for kubernetes so no shell no ssh connection everything is managed through api which makes it simpler to maintain. And like k3s and other tools you can spin a cluster in 5mn it can be the subject of a new video :) it's worth to try

[@christianlempa]

I recently made one about k3s, maybe that's already helping you :)

[@tba77]

@@christianlempa Yes I saw your video about k3s and it's interesting but for production, I think talos is more practical and useful as it's designed to be simple to deploy secured and works with almost everything.

[@StephenCunningham1]

1 docker compose yaml with 700 lines runs my lab. When I had kube gitops set up they were constantly making changes to stuff and helm charts that would break everything, plus storage was a bear.

[@TantissTheEmperor]

I use OKD, which is also pain in the ass to install, config and work with, but it's more for learning and mastering purposes than real needs. It allowed me to apply to new jobs and make this my everyday work so even if it's not something 'wise' to use in a little homelab, it could be interesting for professional skills to learn and master it at home. Now if you don't plan to work professionally with it, don't waste your time and hairs, docker swarm or docker is enough.

[@Bathi666]

Excellent video as always mate . Cheers

[@christianlempa]

Thank you so much! :)

[@theangelofspace155]

You can run docker in multiple servers and yiu can use traefik in docker. And there is docker swarm as well for passwords (secrets) or not?

[@christianlempa]

I'm not sure about swarm tbh, that's still on my backlog to learn

[@febriansasi]

After watching your videos, I install portainer and dockerize my homeserver. I use old notebook so I only activate container that I need using androtainer

[@christianlempa]

Nice!

[@febriansasi]

@@christianlempa did you have tutorial how to migrate my existing portainer and container if someday I want to upgrade my homeserver?

[@jonisapp]

No mention of Docker Swarm, why... I think it's a perfect fit for a homelab.

[@spaceboy978]

in dont need k8s because i only have 1 server at home.

[@seethruhead7119]

maybe docker in a VM in a proxmox cluster would make more sense with ceph or nfs storage if you need to take down a node then you can migrate the entire docker vm to a different server.

[@maxnatamo]

There seems to be some noise in the video, when viewing in 4K. It's really bad at 2:59, for example.

[@christianlempa]

Hm can't hear it, maybe that's part of the music?

[@maxnatamo]

@@christianlempa I meant some visual noise on the diagram. My fault

[@christianlempa]

@@maxnatamo aaah okay, thanks for letting me know. Yeah sometimes that happens when the resolution isn’t good because I zoomed in too much. I’m sry :(

[@maxnatamo]

@@christianlempa You make up for it with great videos, editing and diagrams. Don't worry, it's a nitpick if anything

[@christianlempa]

@@maxnatamo no worries, I'm happy when you report it to me :D I always try to improve the quality. Thank's again!

[@AD-qs9sb]

Ive been using docker swarm and it works well. It allows replicas and some HA features and can also be managed with portainer. I tried k3s a couple of years ago and after fighting with it for a couple weeks finally realized its never going to work well in a home lab. Its designed for cloud providers that handle all the external stuff (networking, storage, LB) for you in their own proprietary way. There are solutions that kind of make it qork in a home lab but it's a constant struggle.

[@charlesduncan9250]

Great video!! What about lxc containers in proxmox vs docker??

[@LampJustin]

Docker in LXC containers on Proxmox is what you want ;)

[@christianlempa]

Haven't used them yet so I can't really say much here :/

[@MichaelSmith-fg8xh]

LXC/Proxmox gives containersation with much less human overhead.. also with Proxmox it's great to have the fallback of being able to do VMs on the same server. I'd be nervous using it at work though.

[@th3r3v92]

I'm using k8s during work, but I don't think there is any reason to use it at home. For homelab, swarm is much better IMHO. Yeah, ppl always talk about "swarm is dead", but actually, it is soooo much better in a small environment... Just init a swarm, join a worker or two additional managers, and use network-ed volumes. Also, you can use the same compose files as in "normal" mode, you can use portainer to manage the swarm and you have overlay networks. For a homelab, IMHO, it is the best, because it is almost the same as plain old simple docker.

[@frankniethardt1813]

Would you recommend getting started with Kubernetes using TrueNAS SCALE or with separate VMs?

[@christianlempa]

If you want to learn Kubernetes you have to use a different system than TrueNAS Scale, as it hides all the important stuff behind graphical menus and abstraction

[@dl2085]

Why do you use k3s instead of k8s? With kubeadm the installation of k8s ist pretty simple and I've read that the performance of both is pretty similar.

[@christianlempa]

K3s just seems easier to me

[@stanislavtrifan96]

Kubernetes is awesome tool for production. However for something like homelab podman desktop, nomad from hashicorp or portainer may make more sense

[@christianlempa]

great alternatives as well, but I think if you're using your homelab for learning, you can't get around using what's used in companies production systems.

[@MrTmorton77]

​@@christianlempa There is a much larger community behind k8s vs other solutions because so many commercial companies use k8s in production. This also means there are more options for a given solution and it will typically have better long term support. Not to mention the recent statements that were later retracted by the folks at Docker. Not hating on Docker but they realized they lost the container battle and now are scrambling to find relevance and a revenue model.

[@therealslimaddy]

Debian Vm plus docker is the one for me.

[@Stuffynes]

I use truenas scale for running all my containers. It’s much easier than setting k3s on my own and so far has been reliable.

[@christianlempa]

That's a good alternative as well!

[@rebelavie7772]

Hey.. Christian... Do you have some experience using jsonnet? ( data templating language) in kubernetes... If yes Can do a video about it..? I heard it could be a good alternativ to kustomize or helm

[@Trains-With-Shane]

For me Docker, even without Swarm, is more than sufficient for my home lab services. I may eventually work up a Swarm or switch to K3s but for me the juice isn't worth the squeeze other than for learning experience which I have spun up K3s deployments for in the past on my home lab. But again those were just for labs and not "production" workloads.

[@Sincer1te]

1:28 What's this?

[@Jimmy_Jones]

ArgoCD. He mentions it later on.

[@Sincer1te]

@@Jimmy_Jones Thanks. Missed that

[@PaulMetalhero]

Can Portainer work with podman instead of docker?

[@Clarence-Homelab]

not sure about that have a look at "yacht" for podman container management if you don't mind trying something new

[@christianlempa]

I'm not sure, I don't think so because it connects to the docker engine.

[@closerlookcrime]

Ahh the old Matrix screen saver. Thanks for the video.

[@christianlempa]

You're welcome!

[@AlexanderGarzon]

Being using k8s since last year, and I regret the headaches, Like I read in a meme, I don't remember where 😂: - Do we need kubernetes for this project? - No. - But... - No I was able to replicate the same "features" the client wanted by using only docker swarm and traefik. He is happy, I'm happier.

[@christianlempa]

When you're happy, that's the most important thing! :D

[@streambarhoum4464]

Hey Christian Could you please deploy some home server apps with the kubernettes approch in a commig video ? As you wish for the appropriate solution Kube-compose, k3s etc....?

[@christianlempa]

Good suggestions, I'll do a few other topics first, but in the second half of this year I might do more project focused videos about Kubernetes Deployments!

[@sgt_mate]

And as for if kubernetes will be the standard in the future. It is now the standard! If you look up statistics, most container orchestration is done with Kuberenetes or a product build on it.

[@michealchoudhary9861]

I love Kubernetes and find it complex , even tho i am CKA and CKAD certified.

[@christianlempa]

Oh yeah it is complex :/

[@adammontgomery7980]

Don't get me wrong, this is cool. But, what are you guys doing at home that you need a cluster running container orchestration software?

[@christianlempa]

Just for fun and learning! :)

[@tehmessiah5617]

Docker Swarm works across multiple servers doesn't it?

[@steverhysjenks]

The bit I'm struggling is how to migrate to k3s....... how do it translate my docker compose scripts into K3s - its really put me off kubenetes

[@travisschneider3011]

Why not migrate to Docker Swarm? Easier migration, you also get a bunch of the benefits of Kurbernetes.

[@christianlempa]

You might have to rewrite a lot manually with learning the kubernetes specific processes, I know it can be challenging, but it's better for learning and understanding kubernetes, because many things are handled a lot different from docker.

[@kanarie93]

traefik work on dockerstacks too so thats not k3s specific...

[@dougmoore5823]

and what is that background/desktop on your mac to your right?

[@christianlempa]

What do you mean?

[@StevePrior]

I'm looking for an online community/forum to ask questions and share information on k3s in a home lab but haven't found a good one yet. Can anyone suggest some? I'm not looking for paid tutorials or communities.

[@christianlempa]

Join our Discord :)

[@StevePrior]

@@christianlempa Can't join your Discord - I only use my Google Voice number, not even my work has my device cell# and your Discord won't accept that.

[@StevePrior]

Do you have an alternate way to verify me so I can join?

[@ronm6585]

Thanks Christian.

[@christianlempa]

You're welcome :)

[@cristianku75]

and what about Proxmox ?

[@christianlempa]

That is in my opinion a different thing, I use them all btw :D

[@takehaver]

“The conversation around Kubernetes vs. Docker is often framed as either-or: should I use Kubernetes or Docker? This is like comparing apples to apple pie, and it’s a common misconception that you must choose one or the other. The difference between Kubernetes and Docker is more easily understood when framed as a “both-and” question. The fact is, you don’t have to choose-Kubernetes and Docker are fundamentally different technologies that work well together for building, delivering, and scaling containerized apps.”

[@christianlempa]

So, you agree with what I said in the video? :D

[@takehaver]

@@christianlempa Great video and love your content in general, but I don’t agree with “vs” in the title :). Usually kubernetes is something you use for or with docker containers. Thank you for all hard work you put in your labs and presentations!

[@christianlempa]

@@takehaver you sometimes need to do a little clickbait and keep the titles SEO friendly xD but yeah technically you’re right

[@CodyakaLamer]

Thanks Christian for the informal video. Keep up with the good work I like listening these videos while I'm at work

[@christianlempa]

Thank you! :) Glad you like it

[@Janemayank]

this solved my long pending doubt ✌

[@christianlempa]

Glad it helped :)

[@g.s.3389]

why didn't you use Rancher?

[@LampJustin]

Because Rancher has tons of problems and completely digs into your cluster. It's not worth it imho. If you're uncomfortable using lens, k9s or the Kubernetes dashboard, you should rather the using docker. I don't mean it rudely at all, but there's so much inherent complexity with Kubernetes that needs to be understood in order to run it properly.

[@g.s.3389]

@@LampJustin thx

[@weluvmusicz]

Use Docker Swarm with Portainer.

[@majorgear1021]

I found Kubernetes k3s to be fragile and it broke for no apparent reason at random time. I would delete and reinstall nodes so often that I knew the commands by heart ! Also , Cluster storage would become degraded for no apparent reason and I would tear my hair out trying to fix it. I have found gitops , which is managing my containers with git and github actions, to be a happy compromise. I store configuration and secrets in github, and github actions deploy and update my containers.

[@vzetanet]

do you use kubernetes? why not try rancher?

[@alevsk]

Rancher is a UI for kubernetes

[@christianlempa]

^- this here!

[@vzetanet]

@@alevsk I could also be wrong, but they write on the site that it manages multiple Kubertenes clusters providing security, their orchestration through any infrastructure, it seems to me much more than a simple UI for Kubertenes.

[@zeal514]

your discord link is dead fyi. tried on multiple videos from 2 years ago to this 1. no bueno.

[@christianlempa]

just checked it, works on my machine _-oO-_

[@AD-nm7ne]

On-premise K8s is honestly overly complicated. It’s easier to just use Docker or podman Of course, it's okay if you use it as a portfolio. BGP design access control for load balancer ACL, calico DNS, SSL, SSO, etc. Load balancing is also more complicated than expected when using Kubernetes on-premise. It's a good choice if you're an engineer who wants to learn Kubernetes. Otherwise, just use a solution like Cloud EKS. Of course, experts will have to design it as a hybrid, but Kubernetes is difficult to learn and there are many things to learn. Especially when it comes to on-premise, the number increases.

[@twoleftfeet90]

What about nomad for home labs? Anyone using that?

[@christianlempa]

Still on my backlog and I can't find the time to do it :D

[@Larkc]

Getting started with kubernetes is easy. Finishing is the hard part.

[@christianlempa]

True :D

[@DanielBrzanov]

No Mastodon account? 🤔

[@christianlempa]

No, I don't like these platforms :/

[@tcurdt]

The overhead (even of k3s) is just absurd - at least for a homelab.

[@soubinan]

If you want containers + a good and simple, efficient and reliable orchestrator.. Keep an eye on NOMAD, really ;)

[@christianlempa]

Thanks! At some day, I need to :)

Maybe future year...for now docker is for me.

[@christianlempa]

Go for it next year :D

[@pazuso]

Kubernetes is the ultimate spaghetti. Setting it up from scratch on Ubuntu servers is wild! With the work involved, might as well watch physical servers with your eyeballs and enter adhoc commands to manually do failover and load balancing lol for real I hate it

[@benoitheylens4802]

Fabrizio

[@jfbeam]

The word you're looking for is "bloated". Kubernetes is quite bloated, and overly complex. I've built several k8s's. I'm pissed every time by the pure amount of crap that's running before it's even to the point I can put _my_ containers on it. In a homelab, unless you need to learn k8s, there's very little reason to step in that mess. I'd say vmware is bloated, too. (modern vcenter certainly is) But the actual hypervisor (esxi) is very small, and really simple, but it's not a container platform.

[@emilpeychev8714]

Yes, buddy kubectl all the way. You need to keep cursing at is at a consistant level, just for good measure. Especially in the beginning - lot's of moving parts.

[@HaraldRingvold]

HashiStack.

[@berndeckenfels]

Remeber that Kubernetes does not protect the s3crets

[@christianlempa]

Fair point, but it's easier to manage!

[@ArifKamaruzaman]

Wah, i'm the 2nd comment here.

[@CrazyMineCuber]

NixOS is the answer if you take the Nix pill.

[@nixxblikka]

ERSTER