Setup Prometheus & Grafana Monitoring On Kubernetes Using Helm

#prometheusmonitoring #kubernetes #devops
In this video we are going to learn how to setup Monitoring Kubernetes with Prometheus & Grafana.
----------------------------------------------------------------------------------
Timestamp
-----------------------------------------------------------------------------------
0:00 - Intro
0:57 - Agenda
2:01 - What is Prometheus
3:14 - What is Grafana
4:31 - Prometheus Architecture
8:03 - Different ways to setup Promethus & Grafana on Kubernetes
11:09 - Prometheus and Grafana playlist
12:39 - Hands On setup of Prometheus and Grafana
--------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Grafana playlist: • Grafana
Prometheus Playlist: • Prometheus
-------------------------------------------------------------------------------
------------------------------------------------------------------------------
Steps for Helm Installation:
github.com/shazforiot/Monitor...
----------------------------------------------------------------------------
=======================================
Follow me @:
/ thetips4you
/ thetips4you
/ thetipsforyou
www.thetips4you.com
=======================================

[@theBrightman]

I've been spending days trying to get to this point and your tutorial is the one that finally got me there. Thank you.

[@Thetips4you]

That is really great to hear

[@polishetti8]

Hi this is Ravi Kumar I am also same way but when trying to access from browser getting site can't be reached

[@taujago]

thank you brother. as a new comer to be Devops engineer, I really appreciate this video.

[@Thetips4you]

You are welcome brother. Wishing you all the best for your devops journey.

[@dudyalaganesh6998]

Thank you so much for the detailed information. Enjoyed watching the whole video. Thank you!!

[@Thetips4you]

You are welcome my friend.

[@safari433_]

Amazing. Im studying this concepts to my apply in my new work. That was helpful. Thanks

[@Thetips4you]

Glad it was helpful!.

[@uzairbagwan4117]

Understood the concept real quick ,excellent teaching

[@Thetips4you]

Thank you🙏

[@thomaswongcoding3761]

man you made all this very easy, thank you for the video

[@Thetips4you]

Thank you 🙏

[@zongzaili9701]

Thank you for sharing excellent vedio

[@Thetips4you]

Happy to hear. You are welcome.

[@adityatelang6739]

your videos are to the point and information dense. Kudos, keep it up. #freelearning #opensource

[@Thetips4you]

Thank you🙏 ❤

[@jaysanmi]

Thanks for this!

[@Thetips4you]

You are welcome

[@michaeleluz9445]

amazing!! thank you so much.

[@Thetips4you]

You are welcome.

[@venisree1732]

Thank you!

[@Thetips4you]

You are welcome.

[@georgeradu3759]

Amazing explained

[@Thetips4you]

Thank you

[@SoumyaranjanMohanty-nn6pk]

Awesome explained 😇

[@Thetips4you]

Thank you. Happy to hear this from you.

[@Imran_kpk]

Very very informative ❤❤❤

[@Thetips4you]

Thank you

[@larsvontrierpung9337]

Thanx a lot. Something that would be cool is to have a guide howto setup loki,mimir,tempo with grafana and prometheus in k8. Thanks!

[@Thetips4you]

Thanks for your feedback

[@sayedafzalkhurseed2622]

Amazing

[@Thetips4you]

Thank you

[@rampanwar1316]

Amazing content

[@Thetips4you]

You are welcome. Thanks for your feedback.

[@hamidullahmuslih6301]

great videos! can you please tell which chart should I use? - (kube-prometheus-stack): include prometheus operator which I believe is more powerful and easy to manage - or (prometheus). currently running on our EKS cluster. My concern is about future: - We will be using this as centeral monitoring point for for both production and staging env. So I am thinking which chart will be good for ease of management and scalability? Need to mention, we are deploying this with terraform helm provider. thanks!

[@Thetips4you]

Kube prometheus stack will be easier one to deploy and use, you will have to go through the details of their roadmap plan as your future management of the system will be based on that.

[@hamidullahmuslih6301]

@@Thetips4you thanks! got it

[@rishabhjain2940]

Dear Author, thank you for this amazing video. Also, I want to point out one error that you have made at 15:54 pod/prometheus-alertmanager READY 2/2 (Does not mean it have two pods or the count of pod is 2) (It means the running pod is having two containers and both are running.) or we can say that (number of running containers/Total number of containers)

[@Thetips4you]

Thanks

[@TECStudy]

Thank you

[@Thetips4you]

You are welcome.

[@user-ts7cx7ks6q]

Thanks for the great video. I have one doubt, you have added Prometheus as source in Grafana UI, what happens if the grafana pod is recreated, do we need to add it again after pod recreation? or can we give a source to the Grafana directly in its helm chart ?

[@Thetips4you]

You can add the datasource config file in iac.

[@user-js7ud9du2y]

i was having issue doing the echo part but after using an online decoder i guess that worked too

[@Thetips4you]

Ok. thanks

[@iyin_dev]

Clear and insightful video. Thanks. Would it be possible to have prometheus/grafana setup monitor both kubernetes cluster and Kafka services at the same time? Thanks

[@Thetips4you]

Thank you for your feedback. Yes it would be possible.

[@iyin_dev]

@@Thetips4you Thanks for your response. Do you have anytips4that? :) Or any documentation for monitoring both? Much appreciated

[@rajualapati8742]

Can you please help me with how to do it on multiple AWS accounts without VPC peering? I need to get metrics from private IP subnets using any exporter and IAM role ... Please help

[@solaiman15]

"helm install grafana stable/grafana" is depricated. you should use : helm install grafana grafana/grafana" instead. otherwise great video. thank you for your help!

[@Thetips4you]

Thanks for the information. Unfortunately I cannot edit the existing video.

[@coopmech2546]

thanks sir, you're solve my problem

[@Thetips4you]

@@coopmech2546 You are welcome

[@laskyb4275]

now the repo stable/grafana got completely removed helm install grafana stable/grafana Error: INSTALLATION FAILED: repo stable not found

[@inhtung4380]

hi, if I want to have 2 replicas of Prometheus on 1 cluster, how do I edit the helm chart?

[@Thetips4you]

Pull chart from helm repository and make your modifications.

[@bhavikvegad3449]

nice !! make tutorials on teamcity too please !!

[@Thetips4you]

Will check on it.

[@aliiraza4064]

I’ve created a server on which I’ve configured Prometheus and Grafana. On prometheus I’m monitoring my 10 VMs. I also want to monitor my AKS cluster and 2 Microk8s cluster on same Prometheus. How can I achieve my goal?

[@kanikagola9613]

Hey, I want the mtrics of my pods which i have created (bascially deployments), so where is the prometheus configuration file, so that i can alter it according to me?

[@Thetips4you]

Yes. You can configure the Prometheus file.

[@vikaspandey1051]

I am not able to access. The link from minikube service --all of grafana and prometheus is not open in browser. What's the issue????????????

[@Thetips4you]

Are you getting some error?

[@AdeolaAlabi-hl3ex]

Hi, After exposing the grafana clusterIP service as a Nodeport, I get ERR_CONNECTION_TIMED_OUT when I type x.x.x.x:32578 in my browser. I run a 3 node k8s cluster with 1 controlplane and 2 worker nodes. Can you please help?

[@Thetips4you]

Did you opened the port in firewall ?

[@AdeolaAlabi-hl3ex]

@@Thetips4you No, but it works now. I was using the controllplane IP thats was why it failed. thanks

[@brahmanaiduaelem]

How to configure Prometheus and grafana using ingress for secure login.

[@myronz1137]

How to add alerts in grafana values.yaml by using helm?

[@Thetips4you]

Do you mean package your grafana using helm?

[@sairajexpaws4323]

when i entered this minikube service prometheus-server-ext it doesn't showing anything such URL or ip and prometheus also not exposed on browser

[@Thetips4you]

Did you checked the running services?

[@sairajexpaws4323]

@@Thetips4you services are running

[@sairajexpaws4323]

Everything gone well but except exposed on browser with port

[@pankajchaurasiya8481]

Will it help to monitor the the kubernetes cert expiration

[@Thetips4you]

You will have to configure the respective stanza for the monitor.

[@nnmbnmbnmnm]

as it seems to keep the history it should have to use Persistent volume. since we haven't configured it does it mean it will wipe out the history on the next restart? if not then how it is possible where it is keeping the data.

[@Thetips4you]

Can you be more clear on history of what?

[@dheerajyadav6981]

I checked they are using PV and PVC using hostpath, which means the data would be stored on the host. Once the pods are recreated they will have the data back from there. I hope this answers your query.

[@diegodesantiagoruiz2950]

Great video, I have an interview Right Now and was great content, when can I donate or sponsor?

[@Thetips4you]

Thank you for your support. Currently I am not collecting any donations or sponsor.

[@systechadmin8368]

Thanks,Where we can download custom dashboard template ?Please share link if any

[@Thetips4you]

grafana.com/grafana/dashboards/

[@sanghoang.]

i have pod error default prometheus-server-7d57c9697f-hqngs 1/2 Error 3 (30s ago) 74s

[@gulamgaushkhan218]

After installing prometheus helm chart, when i go to check the pods or resources status it show resources in unhealthy state

[@Thetips4you]

Check the logs of the pod to see why it's failing.

[@allenfitness105]

for the grafana installation as of 9th August, 2023 they're saying the chat is deprecated, pls what do we do?

[@Thetips4you]

What do you mean by chat? Did you installed latest version of Grafana?

[@michaeltaxsian4712]

helm install grafana grafana/grafana

[@RavindraRaivlogs]

My dear friend.. where is configured node_expoter in Prometheus.yml file. With port 9100 and .how can you recived metrics without configured of node_exporter in prometheus.yl file...... please reply

[@Thetips4you]

If you see the running pods, node exporter is also running as pod.

[@gulamgaushkhan218]

@@Thetips4you But in my case node exporter pod is pending state showing error msg " 1 node(s) didn't have free ports for the requested pod ports. "

[@pankajthakur6319]

I have configured the same but unable the see the data for CPU, Memory and IO same like your dashboard I see the N/A. Could you please share the steps to fix this

[@Thetips4you]

Are you facing any error?

[@pankajthakur6319]

@@Thetips4you no errors only metric data is not showing. I can see the metrics only for pods, deployments, containers.

[@user-dt1ug2rj5c]

I installed Prometheus and grafana as per this video using helm on Azure Ubuntu VM And exposed the svc using nodeport but not able to access the server Please help

[@Thetips4you]

Not able to access means?

[@tharindusamarakoon8951]

helm install grafana stable/grafana Error: INSTALLATION FAILED: repo stable not found Why is this not installing ? any idea ?

[@Thetips4you]

Check the helm repo Or connectivity

[@michaeltaxsian4712]

try helm install grafana grafana/grafana

[@polishetti8]

I have installed minikube with docker driver on Ubuntu 20.04 and instlled helm and install Prometheus based on your video but iam not able to accessible from browser getting site can't be reached

[@Thetips4you]

Did you check the ports are not blocked in the firewall?

[@polishetti8]

@@Thetips4you I have given all traffic in security groups may be iam thinking any compatibility issues

[@Thetips4you]

Are you using a cloud system?

[@polishetti8]

@@Thetips4you yes AWS cloud

[@polishetti8]

Finally I have successfully setup Prometheus and grafana using Ubuntu 20.04 in AWS cloud

[@naturevibezz]

If the cluster itself goes down then how can we monitor?

[@Thetips4you]

You cannot.

[@naturevibezz]

@@Thetips4you so how we can avoid such issues?

[@Thetips4you]

Keep a high availability setup

[@user-if2oc8jl4n]

unable to install grafana from stable

[@Thetips4you]

What is the error?

[@michaeltaxsian4712]

try helm install grafana grafana/grafana

[@IjonasKisselbach]

Grafana install at kzbin.info/www/bejne/mpyVXpJ6l7yVbpI

[@Thetips4you]

Ok

[@AdeolaAlabi-hl3ex]

Hello can you explain why I'm getting this error when installing prometheus? helm install prometheus-release prometheus-community/prometheus Error: template: prometheus/templates/NOTES.txt:86:46: executing "prometheus/templates/NOTES.txt" at : error calling index: index of untyped nil

[@Thetips4you]

Did you tried the installation again.

[@famouscloud1292]

Grafana chart name has been updated btw. you need to run -> helm install grafana grafana/grafana

[@Thetips4you]

Thank you for sharing the info.

[@mohitmishra3091]

oooh bhai asli player tu nikla, thank you man !!

[@rajivbhatia1197]

Hi, I was installing Prometheus. After "helm install prometheus prometheus-community/prometheus", I noticed that alertmanager and prometheus-server pods are in Pending state for more than 30 minutes. Appreciate any help. I have tried it twice - appreciate some help. Thx.

[@Thetips4you]

use the command kubectl describe and check the status of the pods. it should show you the reason why it is in pending state.

[@rajivbhatia1197]

@@Thetips4you Hi there. I sent you an email. Can we connect for 5 mins to discuss an immediate engagement?

[@rajivbhatia1197]

This is what I get from describing the pod towards the end. Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 18m default-scheduler 0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims. Warning FailedScheduling 18m default-scheduler 0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims. Warning FailedScheduling 13m default-scheduler 0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims. arjun@k8s-controller-01-hp:~$

[@rajivbhatia1197]

Looks like there are two pvc's but no pv's. arjun@k8s-controller-01-hp:~$ kubectl get pv No resources found arjun@k8s-controller-01-hp:~$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE my-prometheus-alertmanager Pending 45m my-prometheus-server Pending 45m arjun@k8s-controller-01-hp:~$

[@Thetips4you]

There is some issue creating the persistent volume. I would suggest you to clean the release and delete the PV, PVC if anything there manually. Restart the docker service and then try again.

[@sanghoang.]

i check logs pod [root@k8smaster2 ~]# kubectl logs prometheus-server-7d57c9697f-hqngs Defaulted container "prometheus-server-configmap-reload" out of: prometheus-server-configmap-reload, prometheus-server 2022/06/29 14:18:46 Watching directory: "/etc/config" please help to check.