What an amazing video! You inspired me to finally get my own home-lab after thinking about it on and off for a couple of years, so thank you for being such a great inspiration.
@christianlempa15 күн бұрын
Oh, thank you so much for the kind words! And welcome on board :D
@GianpaoloMacario2 ай бұрын
That's probably one of the most amazing videos which I have viewed so far! Thanks a lot Chris for explaining so many things in detail!
@christianlempa2 ай бұрын
Thank you so much for the kind words 🙏
@52Pi_Maker_Education2 ай бұрын
Thank you for your sharing. You are using our 52pi N04 expansion board, your idea is fantastic. I also want to try to build a similar environment on our Rack mate T1 and give K8S a try, it looks very promising! You are absolutely right, everyone should learn Linux!
@christianlempa2 ай бұрын
Thank you 😊 great feedback!
@tofmoa1847Ай бұрын
Hi, thanks for the tutorial, little tips from my side to disable traefik + servicelb you can use --no-extras. it work every time
@christianlempaАй бұрын
Oh I have to try that! Thank you 🙏
@tofmoa1847Ай бұрын
@@christianlempa Another things the --no-extras should be add on the join command as mention on the github.
@freedo2012 ай бұрын
Great video, i installed my cluster without kube vip because i was installing it via my own install script instead of k3sup and it wasnt easy, will need to give this a shot
@christianlempa2 ай бұрын
Nice! Hope the video helps :)
@ElTebe2 ай бұрын
Racknex is awesome. Thank u for this info!
@christianlempa2 ай бұрын
It really is!
@questionablecommands94232 ай бұрын
3:33 oh my goodness. This is a MASSIVE improvement to the k3s documentation since the last time I deployed a net-new cluster.
@christianlempa2 ай бұрын
True!
@guillaumearchambault9052 ай бұрын
It's pretty cool, but what do you use for persistent data storage? Longhorn or NFS. I find that the storage part is too often set aside when this is the most important thing. The distributed storage is top, except that on raspberry with a single ethernet it is normally not enough. So I wouldn't be interested to know what you're putting in place
@dmckrk2 ай бұрын
Completely agree. Im homelab you need to think about computing power as well as electricity bills, especially in middle Europe. I just kicked longhorn out because it failed on idle power consumption as well as with rebuild time on segmentation issues. Resiliency is not everything.
@allards2 ай бұрын
@@guillaumearchambault905 I also agree 100% persistent data storage seems to be ignored often in video’s and guides; without it a Kubernetes cluster is nothing but a bunch of clustered standalone nodes!
@tomekem34732 ай бұрын
@dmcrkrk And what have you used instead of Longhorn?
@dmckrk2 ай бұрын
@@tomekem3473 for now openebs, but I'm still testing few things. Longhorn is about to release 1.7, Ill check it again if it can handle idle state correctly, this was broken about v1.4.
@christianlempa2 ай бұрын
Im currently testing longhorn and give it a try again! Follow up video is coming at some point ;)
@Waveshare_Ruan2 ай бұрын
Thank Lempa for putting so much effort into creating and sharing this video with everyone. This video provides a wealth of information for Raspberry Pi cluster project users and offers strong support for Waveshare POE HAT customers in their practical applications. I believe Waveshare should sponsor you with free products. If you have any ideas about incorporating other new Waveshare products in your future videos, I will certainly provide you with the greatest support!
@christianlempa2 ай бұрын
Thank you so much 😊 but I’m fine to pay for stuff when I got a use for it, so it’s okay for me
@Waveshare_Ruan2 ай бұрын
@@christianlempa Thank you for letting us know! We believe your independent educational videos will provide our enthusiasts with increasingly objective and valuable information! We all really need you!
@christianlempa2 ай бұрын
@@Waveshare_Ruan thank you! :)
@krzysztofkrolikowski2 ай бұрын
wow, that's really good raspberry pi setup on steroids! Compared to my cluster your's is super awesome! I own 4 raspberry pi's on my cluster, but they are 4'th gen version. I'm using uctronics rackmount for raspberry pi. It was only reasonable rack solution at that time ;-) Every node is installed on SD 64GB cards. I've built it 2 or 3 years ago. What i can share with you Christian, consider adding one or two amd64 vm's to this cluster. Not every app is working well on arm. Kubernetes with proper nodeSelector configuration will do the job, when you'll encounter some problems with ARM architecture ;-) Many thanks for tip with kube-vip solution. It's one of the crucial missing pieces in my homelab setup.
@christianlempa2 ай бұрын
Thank you so much! Also great tip, I will think about that :)
@svenАй бұрын
Got a racknex mount for the Vigor 165 DSL Modem. Was the only one but its great
@christianlempaАй бұрын
Nice!
@TobiasEhlert2 ай бұрын
I think that you don't need to use kube-vip if you have opted for the cilium cni instead of flannel, since that is quite rich in feature when it comes to l2, service mesh, kube-proxy replacement and more. Maybe perfect for a follow up video to make that switch ^_^ other than that thanks, a great video as usual!
@christianlempa2 ай бұрын
Thanks! I don’t have much experience with cilium but I agree it would be another interesting topic!
@mattiavadala78702 ай бұрын
R.I.P. srv-prod-3... you've done well. xD
@christianlempa2 ай бұрын
😂🤫
@ChadE10202 ай бұрын
thank you been waiting for this.
@christianlempa2 ай бұрын
🥰🙏
@tkdlove022 ай бұрын
Hey Christian, love your work and all the great content you make. 1 nitpicky thing though, the font on those diagrams in the beginning and elsewhere in the video is horrendous. Completely unreachable without zooming in (using mobile to watch this). Obviously, nothing overly important just a suggestion to use a standard font in the future. PLEASE keep making this content, it is a huge motivation to me and others to explore areas outside our respective fields and there is always great stuff to try out. Thank you for all the hard work!
@christianlempa2 ай бұрын
Thank you! I’ll keep this in mind for the next one 😊☝️
@wstrater2 ай бұрын
Very nice. I like the idea and simplicity of building your cluster on bare metal. Feels more approachable then starting with a hypervisor. I hope you take the same approach with storage and keep the cluster completely self-contained so you could take that RackNex unit with you and have a portable cluster. Are you looking at file systems like Gluster or Ceph?
@christianlempa2 ай бұрын
Im currently looking at longhorn again, maybe it’s working better this time! :)
@ralumbur2 ай бұрын
Are you sure you bought the sbc-213 model? Because it says sbc-202 on your front panel?
@MuchamadFatihurrahman2 ай бұрын
Instead, metal lb can also provide a virtual IP for the load balancer claster k8s
@scrumcoach29 күн бұрын
For me CNIs worked very well too with onPrem Setups. Very interesting tutorial and so well made!!
@sergioromerobarra70582 ай бұрын
Hello! Great video. I love K3s too. Are your Fitwok Nvme still running ok? My Pi5 completely destroyed my SD card after running fro 15 days with Ubuntu server and K3s and I'm thinking of moving my single node k3s to nvme instead of wasting money with a better SD.
@christianlempa2 ай бұрын
So far it's running fine :)
@esc-sh2 ай бұрын
Great video. May I ask what advantage does running k3s on three Pis have over running on Proxmox? If it is power efficiency, wouldn't Proxmox on Mini PCs be better? While I was looking at building a k3s cluster, getting a pi, the hats and ssd all combined became more expensive and more finicky than getting a used Lenovo/HP/Dell mini PCs. I ended up going with Mini PCs + Proxmox and that is very power efficient and is a lot more flexible than the PIs
@christianlempa2 ай бұрын
Thank you! :) It was just a fun experiment to tinker, and I wanted to have an environment that is a bit more power efficient than my bigger proxmox server, so I can shut it down when I don't need it and still run my prod services on my kube cluster
@jucosorinАй бұрын
Great video, Christian! One question, if the kube-vip address is virtual then how do you reserve it on your dhcp server so it doesn't get allocated to another machine in your network?
@christianlempaАй бұрын
You can use different ip ranges for dynamic and static ip addresses. Most dhcp servers have settings to exclude specific IPs from the dynamic range as well
@rzabcio32 ай бұрын
Damn, that's a lot of work with the hardware... But, it could be useful. Thanks for the detailed video!
@christianlempa2 ай бұрын
Thank you so much 🥰
@Noxcrunor2 ай бұрын
Hey Cristian, great video! Wanted to try out Kubernetes myself but never really get the hang on it but I am eager to try it out with mini PCs I have lying around. :D One side question, which is not dedicated to the video itself: In some other videos you uploaded, I saw a metal plate you screwed on the wall and hung your (for example) Zima Board onto it - I never found such a metal plate; what's this thing called, where did you get it? You did not mention it anywhere on your Kit page, either. Keep up the great work, you helped me and my projects many times! Best.
@christianlempa2 ай бұрын
Hey thank you! Good luck with your project :) honestly I don’t remember that plate anymore it was just a random piece I found on Amazon :D
@Noxcrunor2 ай бұрын
@@christianlempa That is sad to hear. ;D Thanks for the quick answer. :)
@kevinolvera11092 ай бұрын
Great content! It would be nice if you could review some technologies like OKD, is the community edition of Openshift.
@christianlempa2 ай бұрын
Thank you 🙏
@DustinGrady2 ай бұрын
Great video
@christianlempa2 ай бұрын
Thanks!
@DavidC-rt3or2 ай бұрын
Trying to find with kube-vip can/does it do any kind of health checks of the endpoints? For example, with keepalived and haproxy (or nginx as a load balancer) and configure health checks to know if x failures occur remove the load from the lb unless y successes occur? From what I've seen so far with kube-vip that if say the k3s on a master died but is still on, would the request then just fail if that node has the vip? or would it move to another node? or if the vip is on node2, but k3s svc on node1 is dead would traffic get sent there?
@christianlempa2 ай бұрын
I think does but you might need to check in the docs how exactly that’s working
@magicmanj322 ай бұрын
Hi Christian, any chance of doing an install howto for teleport (community edition) for a homelab, I feel this is more advanced than I would like, thank you, love your videos very insightful
@christianlempa2 ай бұрын
Maybe this video helps: Installing Teleport + Traefik (Letsencrypt TLS certs) kzbin.info/www/bejne/hKu2lYGlh5J9htE :)
@dandogamer2 ай бұрын
I attempted this last year but docs werent great may have to try again. Im still unsure what youre using for storage. I need something thats not gonna go down
@christianlempa2 ай бұрын
Do you mean kube-vip? or k3sup?
@JohnWeland2 ай бұрын
So I am 90% there, I tried with k3sup and just plain old k3s docs and `--disable servicelb` doesn't seem to work because the first time I create a deployment (nginx) and expose it with an lb, I get 6 pods beginning with `svclb-nginx -` all running in my cluster with an ip address of the host node.
@christianlempa2 ай бұрын
Maybe we can help you on the discord, we have a help forum :)
@Ramhound2 ай бұрын
Can you post where you got the PCIe cables? Definitely interested in doing something like this.
@christianlempa2 ай бұрын
They’re on my kit page: christianlempa.de/kit
@woife07052 ай бұрын
This is nice but unfortunately some images do not run on the raspi architecture. I do not remember by hard which was it in my case exactle I believe timescaleDB.
@scrumcoach29 күн бұрын
How do you renew the certificate when its failing on you, ie after one year?
@christianlempa27 күн бұрын
I just download it from the k3s kubeconfig directory.
@dark4igi2 ай бұрын
only .kube/config was updated on your workstation/laptop but not only your kubectl communicate with kube api, another nodes also communicate with kube api, what still on all another confs ip of first node. 27:09 you add second node using ip of first node, not vip I bet, if you turnoff first node, you will got problems with your cluster
@christianlempa2 ай бұрын
Hmm that’s right! Thanks for the heads-up! I might need to change the config later at some point
@6bBeVaEq5AhEG62 ай бұрын
The Raspberry Pi uses arm, so some applications and services cannot be deployed... You have to write your own yaml /docker compose file for deployment, which is a bit troublesome. Thank you boss for sharing. By the way, why didn’t you consider using Argon NEO? 🤣
@DavidC-rt3or2 ай бұрын
Keep in mind that is the control plane, worker nodes (agents) could be deployed on other pi's (with same or different os), x86 or x64 physical hardware (with various os'es), and/or proxmox vm's. Then using labels (for example on the nodes) could then define which apps would go where based on the definition in the yaml (by os, platform or other label) . Also, if it's your own app(s) then you could do multi platform builds.
@pdx22402 ай бұрын
Its such a long time ago, I have encountered an application that I could not run in my K3S arm64 cluster. Basically support for arm is great now
@dmckrk2 ай бұрын
ARM is not problematic now. Risc-V is coming, and still you can mix all of them together.
@niklaswe26862 ай бұрын
Why not use servicelb shipped with k3s?
@dmckrk2 ай бұрын
If you need virtual address instead of fixed port numbers then You need something like this or metallb.
@crispyreddis70512 ай бұрын
Hi! Is it possible to verify my account in Christian discord without phone? Discord still does not send me verification sms. Tried 5 times.
@christianlempa2 ай бұрын
Unfortunately not, that hold spam and scam away from the channel
@longuinni2 ай бұрын
does gitlab support arm64 processos?
@christianlempa2 ай бұрын
Good question, I hope it does, but if it doesn't, I could add more x86 nodes to the cluster, we'll see :)
@allards2 ай бұрын
Talos OS en Talos Omni, a Video about that would be cool anyway!
@christianlempa2 ай бұрын
Maybe somewhere next year I’ll give it a try again :)
@SergeyToroshchin2 ай бұрын
I still do not truly don't understand why ppl are choosing PI5 instead of mini PC (for eg on N100) ... mini PC have much faster CPU ... more options about memory ... more options about storage ... nower days you can connect nvme without any problems ... where you need some extra super puper adapter to connect any hdd or ssd to PI5
@itssoaztek45922 ай бұрын
For some applications small size, low power draw and the already integrated IO options for cameras and other stuff are important enough to make the RPi a favorable choice. There is a reason why Raspberry Pi's are produced (and sold) in fairly large numbers, i.e. it's not just hobbyists who buy this stuff. In the past at least, price was also a major reason why the RPI was so popular. Obviously, there are also many other applications where a RPI is not the best solution (like you said).
@SergeyToroshchin2 ай бұрын
@@itssoaztek4592 the key is here "for some" ... if we speak about regular serving like kuba ... its not about RPi ... yes we can up some kind of "hello world" web site but we still will have problems with storage, cooling and etc. Many years I tried to serve Home Assistant on RPi and use the same RPi to host GUI on external touch screen ... an it was awful time ... Ive finally switched to mini PC from GMK on Intel cpu and it works much better and flawlessly if we compare it with RPi. I also have some exp with multiple mini PC from Asrock to test cluster and it was muuuch better than RPi setup
@christophschug92952 ай бұрын
Can you name a "mini PC" which runs with PoE?
@kettusnuhveli2 ай бұрын
@@christophschug9295Minisforum S100, Intel n100 based miniPC designed to run off of PoE
@guillaumearchambault9052 ай бұрын
@@christophschug9295 Some mini pcs can have several power outputs via the same main unit
@tomklein65402 ай бұрын
Use Talos 😉
@_vr2 ай бұрын
Agree
@dmckrk2 ай бұрын
Talos is just next level, talos over iPXE :)
@allards2 ай бұрын
@@tomklein6540 There is nothing more straightforward building an K8s cluster than with Talos Omni!
@tomklein65402 ай бұрын
@@allards Amen to that 🥳
@erikhellman39742 ай бұрын
Talos is amazing
@dmckrk2 ай бұрын
This is short story how to start with tiny pi and convert it into big and expensive, problematic build still limited to just 1Gbit :/ You are able to stick 16x pi on 2u case. You could use RK3588 alternatives to get about 2x more cpu power, 2.5x faster ethernet as well as 4x faster nvme (or 10Gbit ethernet) and up to 4x more RAM. Same with any n100 board. This is expensive lesson. Raspberry lost its fame for beeing affordable and powerfull choice. Have You mentioned final price for unit/whole build? Have You measured power consumption for all units on idle and full load?
@PeteMaverickMitchell872 ай бұрын
I was thinking nearly the same thing.
@christianlempa2 ай бұрын
The main goal was a bare metal kubernetes and play around with rpi. Sure you might find some better options but as this is the first project of this kind, I wanted to go with something that is well supported and where you find enough content and help
@oh_zoinkers2 ай бұрын
First?
@christianlempa2 ай бұрын
Second :)
@floriankapfenberger2 ай бұрын
I would be interested in IPv6 Dual Stack and how to expose a service using IPv6 Loadbalancer. Can kube-vip do that? Not sure if it is a Talos problem, but I failed to get IPv6 communication between nodes working using ULAs. I used the default Flannel CNI.
@christianlempa2 ай бұрын
No idea! IPv6 is a topic I’d like to tackle next year
@floriankapfenberger2 ай бұрын
@@christianlempa There is definitely a shortage of IPv6 content out there and more content on it would definitely help! I changed the CNI to Cilium and then IPv6 with ULAs started working. I could also replace MetalLB with Cilium IP Pools and BGP control plane.