Monitor Kubernetes in 10 minutes

Subscribe to show your support! goo.gl/1Ty1Q2 .
Patreon 👉🏽 / marceldempers
Today we're going to speed run and showcase how quickly and easily we can setup monitoring on Kubernetes using open source tooling like Prometheus.
Remember that I do have a full monitoring guide where I go into the components you see in the video in much greater detail. This video is more of a summary of putting it all together.
Check out my monitoring playlist below.
If you want to support the channel further, become a member 😎
marceldempers....
Checkout "That DevOps Community" too
marceldempers....
If you are new to Kubernetes, check out my getting started playlist on Kubernetes below :)
Checkout the source code below and follow along :)
Kubernetes Guide for Beginners:
---------------------------------------------------
• Kubernetes development...
Kubernetes Monitoring Guide:
-----------------------------------------------
• Kubernetes Monitoring ...
Kubernetes Secret Management Guide:
--------------------------------------------------------------
• Kubernetes Secret Mana...
Like and Subscribe for more :)
Source Code
--------------------------------------------------------------
github.com/mar...
Follow me on socials!
marceldempers.dev
Twitter | / marceldempers
GitHub | github.com/mar...
Facebook | thatdevop...
LinkedIn | / marceldempers
Instagram | / thatdevopsguy
Music:
Track: Amine Maxwell - Amine Maxwell & Soyb - You & I | is licensed under a Creative Commons Attribution licence (creativecommon...)
Listen: / amine-maxwell-soyb-you-i
Track: Amine Maxwell - Night And The City | is licensed under a Creative Commons Attribution licence (creativecommon...)
Listen: / night-and-the-city
Track: J3bii - camel toe [ Prod. J3bi ] | is licensed under a Creative Commons Attribution licence (creativecommon...)
Listen: / camel-toe-prod-j3bi
Track: souKo - souKo - Parallel | is licensed under a Creative Commons Attribution licence (creativecommon...)
Listen: / parallel

[@markhorrocks7278]

I really enjoyed the kubernetes series. I wanted to install Prometheus and Grafana without using CLI's and your source code does exactly that. My k8s server version is 1.16.5 (AWS) so I installed the code from your prometheus monitoring 1.15-1.17 repo. It works out of the box but the node exporter full dashboard meters were broken. I substituted the grafana folder from the 1.18.4 repo and the dashboard works fine now. I'd really like to see a video about keeping Prometheus and Grafana up to date. Also one about Prometheus and horizontal auto scaling.

[@tal1296]

Great video! can you add a video on EFK or logging k8 in general to complement the monitoring videos?

[@rafaelsantos236]

Excellent

[@RodrigoMocca]

Great video!, I have a question: What's the cost of running this extra pods, deployments, services in a production server? I mean, it wouldn't consume lot of CPU and memory?

[@majmicky]

What an amazing series of vidoes. I have a question about node exporter. In our setup I have access to few namespace based on the capability I joined. Can I export metrics based on namespaces instead of all the pods as we have one huge cluster for everyone Thanks for making amazing content, love it everyone bit of it

[@kunchalavikram]

Great video. Can you also make 1 on EFK from scratch and not using helm

[@josephbatish9476]

Nice guidance please do tutorials about alert manager

[@MarcelDempers]

Gotcha! 💪🏽kzbin.info/www/bejne/joTKqqKta7GBetU

[@janituma9769]

Thank you v m for this video👏 🔥🔥 can you pls also make a vid regarding operators and controllers

[@oraculotube]

Amazing videos. How can I create a outside VM from the cluster to monitor the real k8s cluster? thinking if everything crash, I can still monitoring and check the logs etc. thxx

[@SwarupDasCaz]

This video series of Kube monitoring with prometheus is indeed very helpful for a guy like me who is just getting started in kube monitoring space. Few queries: 1. As I'm using EKS Fargate cluster, although I could up all other components (operator, kube-state-metrics and prom) out of the box from you 1.15-1.17 folder, the Node exporter pods are getting stuck in Pending state. Any guidance you can give? 2. Port forwarding seems to not work for me. I'm using bastion to connect to Kube env which could be a reason. Any alternate ways to this?

[@chlks4286]

Absolute Chad!

[@ottoayoub]

Hello, is there a way to monitor the size/volume of files on a k8s pod ?

[@helders]

This video and tutorial are amazing! I was looking for something exactly like that. One thing i didn't understand, since this setup is not using any persistent volume where is the data stored? Can i loose that data somehow? What happens if a pod dies, would i be able see that pod data later on?

[@MarcelDempers]

Prometheus keeps the data in memory by default and retention is not high for that reason. Folks generally feed the data to other systems for longer term storage. So yes if Prometheus pod dies, the data restarts. I would look into feeding the data to an external system if you wish to retain it like ElasticSearch. It has "remote_write" capability to send the data out

[@pjakash3017]

Hey Marcel, great video, I just wanted to ask where can I set the scrapeInterval for prometheus?

[@MarcelDempers]

Sure you can, see the Servicemonitor's endpoint "interval" property github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#endpoint

[@marcoibarra1777]

Hello....How can i expose, in this example, prometheus via NodePort???...port-forward has a time-out.....thanks in advance.

[@Kvaasten]

The best way of exposing web services in Kubernetes is via ingress. You need to first deploy an ingress-controller for example nginx or traefik and then you can make an ingress manifest to expose it.

[@CristobalJofre]

Hello, I was doing the tutorial and at the time of doing it "kubectl -n monitoring apply -f grafana" , it sends me this error "The ConfigMap "grafana-dashboard-nodeexporter" is invalid: metadata.annotations: Too long: must have at most 262144 bytes", and Grafana cant start. Do you know how i can fix this?

[@MarcelDempers]

Could you try `kubectl create` instead of `apply` ?

[@CristobalJofre]

@@MarcelDempersDo you know how I can configure users, groups and organizations within Grafana, but that they are persistent over time? I mean, is there a yaml file to be able to configure this?

[@tomsiby]

I tried doing it, I have my cluster on AKS. It seems that all the kubelet monitors are showing 401 no authorised error. I have to edit the kubelet config and set --authentication-token-webhook=true and --authorization-mode=Webhook. Can you help me and find a way to do this without disturbing any other clusters. Thanks in Advance.

[@MarcelDempers]

401 Permission denied mostly relates to RBAC. This works on AKS and you should not have to modify kubelet settings at all. The service account that prometheus runs as (sometimes its misconfigured and does not run as one) needs to have access to scrape the kubelet endpoint. I have 1.15-1.17 that should cover AKS correctly github.com/marcel-dempers/docker-development-youtube-series/tree/master/prometheus-monitoring/kubernetes This cluster role allows prometheus-k8s service account to access it github.com/marcel-dempers/docker-development-youtube-series/blob/master/prometheus-monitoring/kubernetes/1.15-1.17/prometheus-cluster-monitoring/cluster-role.yaml Notice I do not have namespace restrictions on it so it should be able to scrape kube-system too. Also ensure the service account is set on the prometheus instance: github.com/marcel-dempers/docker-development-youtube-series/blob/master/prometheus-monitoring/kubernetes/1.15-1.17/prometheus-cluster-monitoring/prometheus.yaml#L30 Hope this helps

[@godwin7063]

Hi I watched you videos installed docker on Windows and enabled kubernates but it just keeps staring forever, ive tried all I could. Pls any advice on how to get through this

[@MarcelDempers]

Oh no! double check your Windows version make sure you have latest updates. Perhaps test your Hyper-V and make sure you can create virtual machines. Also ensure your docker windows version is up to date. Maybe uninstall and do a fresh re-install. If all else fails, I'd open a issue on the Docker repo

[@godwin7063]

Thanks I've fixed it It was an issue with Windows hosts file format, it was utf-8-bom instead of utf-8 or ANSI. So the IP's listed where not detected. It too so much time to figure out. Your videos are awesome, informative and straight to the point. 🦾

[@dorsetdroneguy5804]

Mate, not sure what you are doing different with your encoding but I’ve started having problems playing your recent vids on chrome cast. It’s juddery. Nothing I do fixes it. Just letting you know In case you changed your rendering process recently.

[@dorsetdroneguy5804]

I think it may just take YT a while to render all the versions

[@MarcelDempers]

Strange one, I upload wit the standard YT presets since over a year ago :) Also I leave the video uploaded for 1-2 days for YT to do its magic (HD\SD processing), so it should be good. Works on all my devices.